Visualizing F&O telemetry data with Grafana

 

Following up on my previous post, I explored alternative ways to visualize telemetry emitted by Finance & Operations environments. Azure Data Explorer is a powerful option, but I wanted something more oriented toward dashboards. Azure Managed Grafana integrates well with Application Insights and turned out to be a good fit.

Below is a step-by-step breakdown of how I connected Application Insights to Grafana to visualize batch job errors using a custom metric.

Prerequisites

  • To follow these steps, make sure you have:
    • An Application Insights instance receiving telemetry from your F&O environment
    • An Azure Managed Grafana workspace in your Azure subscription
    • Sufficient permissions to query logs and create dashboards

Step 1: Build the KQL query in Application Insights

Start in the Logs blade of your Application Insights resource. The telemetry I wanted to visualize was coming through a custom metric named BatchJobErrorCount. The metric contains a JSON array of batch job error objects.

Here’s the query I used:

customMetrics
| where name == "BatchJobErrorCount"
| extend customDimensions = todynamic(customDimensions)
| extend batchesRaw = tostring(customDimensions.BatchesInError)
| extend batchesInError = parse_json(batchesRaw)
| mv-expand batch = batchesInError
| where batch.Company <> ""
| project timestamp, BatchJobId = tostring(batch.BatchJobId), Status = tostring(batch.Status), BatchJobName = tostring(batch.BatchJobName), Company = tolower(tostring(batch.Company))
| summarize Count = count() by bin(timestamp, 1h), Company

This expands the custom payload, filters the entries, and aggregates error counts per company per hour.

After confirming the results in the table view, I used the Chart tab to preview the output using a stacked column chart. This provided a quick way to verify that data was correctly grouped and visually aligned with what I expected.


Step 2: Export the query to Azure Managed Grafana

Once the query was ready and producing expected results, I used the built-in integration to export it to Grafana.

From the Logs view:

  • Click the Save dropdown above the query editor
  • Select “Pin to Grafana dashboard”
  • Choose your Azure Managed Grafana workspace
  • Specify a dashboard name, and a folder (it is also possible to use an existing one)


Grafana will receive the query and automatically configure the data source using Azure Monitor.

For more details on this integration, you can refer to the official documentation: https://learn.microsoft.com/en-us/azure/azure-monitor/visualize/grafana-plugin

Step 3: Adjust the query for Grafana and configure the panel

Once the query is pinned, open the corresponding dashboard in Azure Managed Grafana. You’ll find the new panel created with the query already applied, but by default, Grafana may render it as a table or show a warning related to time series formatting.


Grafana expects the data to be:
  • Aggregated over time (bin(timestamp, ...))
  • Numeric (Count)
  • Sorted by time (recommended)
To support this, I made a small modification to the original query by adding an explicit sort operation: 
customMetrics
| where name == "BatchJobErrorCount"
| extend customDimensions = todynamic(customDimensions)
| extend batchesRaw = tostring(customDimensions.BatchesInError)
| extend batchesInError = parse_json(batchesRaw)
| mv-expand batch = batchesInError
| where batch.Company <> ""
| project timestamp, BatchJobId = tostring(batch.BatchJobId), Status = tostring(batch.Status), BatchJobName = tostring(batch.BatchJobName), Company = tolower(tostring(batch.Company))
| summarize Count = count() by bin(timestamp, 1h), Company
| order by timestamp asc

Why this matters:

Grafana uses the time column as a reference for aligning data points. Adding order by timestamp asc avoids warnings and ensures the chart renders correctly, especially when displaying long time series.

In the panel settings:

  • Change the visualization type to Time series
  • Use timestamp as the X-axis
  • Use Count as the value
  • Split series by Company
  • Optionally enable stacking to group total error volume by hour

Once adjusted, the panel should reflect the correct time-based distribution of batch job errors, split by legal entity.


Conclusion

This setup provides a simple way to visualize structured telemetry from F&O without needing to move or transform data externally. The custom metric we used contains structured JSON, which we parsed and aggregated directly using KQL inside Application Insights.

By exporting the query to Azure Managed Grafana, we were able to create a live dashboard without writing any backend code or setting up a data pipeline. There’s no ETL, no exports, and no duplication—just querying the telemetry where it already lives, and visualizing it in Grafana.

Comments

Post a Comment

Popular posts from this blog

Monitoring Batch Job Errors in D365FO using Application Insights

Image
While Dynamics 365 Finance and Operations (D365FO) includes native integration with Application Insights , the out-of-the-box capabilities still have limitations—especially when it comes to monitoring the real-time status of internal processes like batch jobs. Although batch job task exceptions are tracked , they don’t always reflect the current status of a batch job. Tasks may have failed and later been retried successfully, meaning the job is actually healthy by the time we query it—yet the historical exceptions remain in telemetry. In this post, I’ll show an alternative technique that extends the telemetry pipeline by programmatically sending custom metrics and structured data to App Insights . We’ll walk through a lightweight batch class that scans for failed batch jobs across all companies and pushes that data into App Insights as both a metric and JSON payload , enabling Kusto-based analysis, dashboards, and alerts. This approach is flexible and can easily be adapte...
Read more

Infamous "The natural key for some table was not found" solved once and for all

Image
Data entities in Dynamics 365 for Finance and Operations (D365FO) are awesome, as long as you can use the wizard to create them. Quite often you will find that some tables throw the infamous "The natural key was not found". This is because many tables actually don't have unique indexes - for many reason - but here I'll present a general solution to create any entity - read-only entities - in very simple steps.  Capture when trying to create an entity for InventTrans table In this example I'll create an entity for the InventTrans table as this table doesn't have an unique index.  Step 1: Use the wizard to create an entity but select any know table that happens to have unique indexes. In this case I'll select InventTable.  In the next screen Deselect all the fields and click Finish. You will end up with an entity with a single datasource Step 2:  In this step we need to create a View that is based on the table that we want to expose as an entity - in this ca...
Read more