Four clicks from alert to root cause: how Elastic Observability links APM services to Kubernetes infrastructure
Check service dependencies and compare per-pod CPU, memory and network trends on the Infrastructure tab to find which instance is causing trouble, all without leaving the alert investigation.
Elastic Observability links your OTel-instrumented services to the Kubernetes hosts, containers, and pods they run on. The Infrastructure tab in APM puts per-instance CPU, memory and network trends a few clicks away, so when a service degrades you can spot which pod lines up with when the problem started, all from inside the investigation. This walkthrough follows a latency alert on a recommendation service from notification to the problematic pod in four steps.
Availability
This is available in Elastic Observability serverless today and is coming to Elastic Cloud Hosted and self-managed deployments in 9.5.
Prerequisites for linking APM services to Kubernetes infrastructure
You need application traces and Kubernetes infrastructure metrics in the same Elastic Observability project.
- Application instrumentation: EDOT-instrumented services sending traces via the EDOT Collector or an upstream OpenTelemetry Collector with both the
elasticapmconnector and theelasticapmprocessor. The EDOT Collector includes both by default; for a custom upstream pipeline, see the upstream collector setup. - Kubernetes observation: the cluster observed via OpenTelemetry with host and Kubernetes metrics from the EDOT Collector. See the Kubernetes quickstarts and Kubernetes observability with EDOT for setup.
- Backend: Observability serverless today, or Elastic Stack 9.5 on Elastic Cloud Hosted and self-managed when 9.5 releases.
APM alert triage: from notification to problematic pod
Step 1: Confirm the service degradation on the APM alert detail page
The redesigned alert detail page in Elastic Observability shows the impacted service, environment, endpoint and RED metrics in one view. Open it from the alert notification.
You can clearly see which service is impacted, which environment it runs in, what endpoint is being affected and easily look for correlations in their RED metrics. In this case, we can immediately rule out a spike in traffic as the throughput is clearly stable.
Step 2: Rule out service dependencies with the embedded service map
The newly embedded service map preview on the alert detail page shows the health and RED metrics of every dependent service, so you can rule out upstream causes without navigating away. In this case, we have been able to quickly rule out problems with other services causing the symptom with the symptomatic service:
Step 3: Review Kubernetes infrastructure metrics per pod, container and host
After ruling out service dependencies, open the service's updated Infrastructure tab in Elastic Observability to check for infrastructure-level patterns.
Step 4: Compare per-instance metric trends to find the root cause
The Infrastructure tab shows the average metric values over the specified time period. To really understand whether there is a problem with the infrastructure, we need to compare the pod, container and host metrics over time. This allows us to easily spot differences between different entities that may correlate with when the service started showing symptoms. In our example, we can clearly see a difference between some of the metrics between the pods that correlates with when the service symptoms began. So we know there is something going on with the infrastructure that needs investigating:
Summary: from APM alert to root cause in four clicks
In just a few clicks from an alert in Elastic Observability, you can rule out healthy dependent services without leaving the alert detail page, then compare per-pod infrastructure metrics to see which instance correlates with when the symptoms started.
Frequently Asked Questions
What setup does this workflow require?
Your application service instrumented with EDOT, and the Kubernetes cluster it runs on observed via OpenTelemetry. Both send telemetry to Elastic Observability. APM uses the shared OpenTelemetry resource attributes to link the service to the pods, containers, and hosts underneath.
Can I use this workflow for hosts, containers, and pods?
Yes. On Kubernetes observed via OTel, the Infrastructure tab in Elastic Observability surfaces pods, containers, and underlying hosts depending on what metrics your collectors gather. The Infrastructure tab shows average metrics per instance for the selected time period. Comparison view plots trends over time for every metric on each instance.
When should I check the service map before infrastructure metrics?
Check dependencies first whenever a service shows a symptom but the cause is unclear. If dependent services also show elevated latency or errors, investigate the dependency chain before comparing infrastructure metrics on the symptomatic service.
What is the difference between reviewing infrastructure and analysing trends?
Step 3 shows average metric values per instance over the selected time period. That is a summary for the window: which host, container, or pod ran hotter on CPU, network, or memory relative to its peers. Step 4 shows trends over time per instance for every infrastructure metric. Each instance gets its own time series so you can correlate the service symptom with when a metric changed.
Do I need separate infrastructure monitoring tooling for this flow?
No for the initial narrowing. Elastic Observability shows infrastructure metrics inside the APM service investigation. You may still pivot to Kubernetes, your cloud provider, or host tooling for remediation, but the comparison step tells you which instance and time range to focus on.
