Metrics collection

EDOT Cloud Forwarder for Azure collects Azure platform metrics and forwards them to Elasticsearch as OpenTelemetry-native metrics. Platform metrics can be collected from any Azure resource type that supports metric export.

Azure offers two mechanisms to stream platform metrics to an Event Hub: Diagnostic Settings and Data Collection Rules. Both methods send metrics to the same metrics Event Hub, and EDOT Cloud Forwarder processes them using the same pipeline.

Supported Azure Monitor metric namespaces can vary by resource type and collection method; if a namespace is not available through Diagnostic Settings, use a Data Collection Rule.

Data Collection Rules include dimensions, which are key-value pairs that provide additional context for each metric (for example, the Kubernetes pod name, namespace, or readiness condition). This provides richer filtering and grouping when exploring metrics in Kibana. Diagnostic Settings are simpler to configure but produce flatter metrics without this dimensional breakdown.

Use Diagnostic Settings when:

• You want the quickest setup with minimal configuration.
• Aggregate totals per resource are sufficient for your monitoring needs (for example, overall CPU or memory usage for a cluster).
• You do not need to break down metrics by individual components such as pods, nodes, or namespaces.

Use Data Collection Rules when:

• You need dimensional breakdowns (for example, per-pod CPU usage, per-namespace memory, or per-condition readiness status).
• You need metric namespaces that are not available through Diagnostic Settings for your resource type.
• You plan to build dashboards or alerts that filter or group by specific dimensions.

You can also use both methods together. For example, use Diagnostic Settings for resources where aggregate metrics are sufficient, and add a Data Collection Rule for resources where you need dimensional detail.

For setup instructions, refer to Deploy EDOT Cloud Forwarder for Azure.

When an Azure platform metric arrives at the metrics Event Hub, EDOT Cloud Forwarder for Azure processes it through the following pipeline:

1. The Azure Functions runtime receives the Event Hub message and forwards it to the EDOT Cloud Forwarder collector as an HTTP request.
2. The collector's azure_encoding extension decodes and transforms the Azure metric JSON into OpenTelemetry metric format.
3. Each Azure metric record produces five gauge data points, one for each aggregation: Total, Count, Minimum, Maximum, and Average.
4. The resulting OTel metrics are exported to Elasticsearch through the OTLP endpoint.

To browse the platform metrics streamed to Elasticsearch:

1. Go to Discover in your Elastic Cloud deployment or Serverless project.
2. Select the metrics-* data view.
3. Filter by data_stream.dataset to narrow down to the metrics of interest.
4. Use the following fields to explore the data:
   • name: the Azure metric name (for example, kube_pod_status_ready, node_disk_usage_percentage)
   • resource.attributes.cloud.resource_id: the Azure resource that emitted the metric
   • attributes.*: dimensional attributes (when using Data Collection Rules)

This section shows the raw Azure metric format for each collection method. Both formats are transformed into OTel-native metrics by the azure_encoding extension.

Diagnostic Settings send metrics in a JSON envelope with a records array. Each record contains the aggregated metric values, the source resource ID, and the metric name.

{
  "count": 1,
  "total": 6,
  "minimum": 6,
  "maximum": 6,
  "average": 6,
  "resourceId": "/SUBSCRIPTIONS/12CABCB4-86E8-404F-A3D2-1DC9982F45CA/RESOURCEGROUPS/MKATSOULIS/PROVIDERS/MICROSOFT.CONTAINERSERVICE/MANAGEDCLUSTERS/MKATSOULIS-CLUSTER",
  "time": "2025-11-18T11:39:00.0000000Z",
  "metricName": "apiserver_cpu_usage_percentage",
  "timeGrain": "PT1M"
}
		

Data Collection Rules send metrics as NDJSON (newline-delimited JSON). Each line contains the same aggregation fields as Diagnostic Settings, plus a unit field and a dimension object with key-value pairs.

{
  "Total": 0,
  "Count": 1,
  "Minimum": 0,
  "Maximum": 0,
  "Average": 0,
  "resourceId": "/subscriptions/12cabcb4-86e8-404f-a3d2-1dc9982f45ca/resourceGroups/mkatsoulis/providers/Microsoft.ContainerService/managedClusters/mkatsoulis-cluster",
  "time": "2025-11-18T12:47:00.0000000Z",
  "unit": "Count",
  "metricName": "kube_pod_status_ready",
  "timeGrain": "PT1M",
  "dimension": {
    "condition": "unknown",
    "namespace": "kube-system",
    "pod": "prometheus-kube-state-metrics-55f8b5d87b-rv5ts"
  }
}
		

The azure_encoding extension transforms each Azure metric record into five OTel gauge metrics — one per aggregation. The metric name is derived from the original metricName field, lowercased and suffixed with the aggregation type (for example, _total, _count, _minimum, _maximum, _average).

The following example shows how the Diagnostic Settings sample above (apiserver_cpu_usage_percentage) appears in Elasticsearch after transformation. Only one of the five resulting metrics is shown:

{
  "@timestamp": "2025-11-18T11:39:00.000Z",
  "data_stream": {
    "type": "metrics",
    "dataset": "generic.otel",
    "namespace": "default"
  },
  "name": "apiserver_cpu_usage_percentage_total",
  "gauge": {
    "value": 6.0
  },
  "resource": {
    "attributes": {
      "cloud.provider": "azure",
      "cloud.resource_id": "/SUBSCRIPTIONS/12CABCB4-86E8-404F-A3D2-1DC9982F45CA/RESOURCEGROUPS/MKATSOULIS/PROVIDERS/MICROSOFT.CONTAINERSERVICE/MANAGEDCLUSTERS/MKATSOULIS-CLUSTER",
      "azure.eventhub.name": "metrics",
      "azure.eventhub.consumer.group": "ecf",
      "azure.eventhub.namespace": "mbranca-2025-11-04.servicebus.windows.net"
    }
  }
}
		

The following example shows how a DCR metric with dimensions appears in Elasticsearch. Dimensions are mapped to data point attributes:

{
  "@timestamp": "2025-11-18T12:47:00.000Z",
  "data_stream": {
    "type": "metrics",
    "dataset": "generic.otel",
    "namespace": "default"
  },
  "name": "kube_pod_status_ready_count",
  "unit": "Count",
  "gauge": {
    "value": 1.0
  },
  "attributes": {
    "condition": "unknown",
    "namespace": "kube-system",
    "pod": "prometheus-kube-state-metrics-55f8b5d87b-rv5ts"
  },
  "resource": {
    "attributes": {
      "cloud.provider": "azure",
      "cloud.resource_id": "/subscriptions/12cabcb4-86e8-404f-a3d2-1dc9982f45ca/resourceGroups/mkatsoulis/providers/Microsoft.ContainerService/managedClusters/mkatsoulis-cluster",
      "azure.eventhub.name": "metrics",
      "azure.eventhub.consumer.group": "ecf",
      "azure.eventhub.namespace": "mbranca-2025-11-04.servicebus.windows.net"
    }
  }
}