Kubernetes monitoring with ES|QL: crash loops and memory

This post has nine ES|QL queries for diagnosing Kubernetes problems in Elasticsearch. They cover crash-looping pods, memory pressure before an OOM kill, saturated nodes, and error spikes by namespace. Every query runs against Kubernetes data collected with the Elastic Distributions of OpenTelemetry (EDOT) and pastes into Discover with little to no editing. Turn any of them into a dashboard panel or an alert once you've found the one you need. Jump straight to the query that matches what you're seeing or read through the whole set to get a feel for your cluster.

What you need before running these Kubernetes ES|QL queries

To follow the queries in this article, you need:

Elasticsearch 9.2 or later.
EDOT Collectors running in your cluster and shipping data to Elasticsearch, with the kubeletstats, k8s_cluster, and filelog receivers enabled.

Why ES|QL for Kubernetes monitoring

Elasticsearch ES|QL is a piped query language that lets you start from a data source and then add one operation per line: filter, compute, aggregate, sort. That structure fits investigation work well because you refine a query step by step as you narrow down a problem.

When to use the ES|QL TS command for Kubernetes metrics

The TS command understands time series. This matters because most Kubernetes metrics are counters or gauges sampled over time. Using TS for metrics avoids the common mistake of summing cumulative counters across pods and getting a meaningless number.

EDOT receivers these Kubernetes queries depend on

The queries below assume EDOT Collectors are running in your cluster and shipping data to Elasticsearch. A typical setup uses:

The kubeletstats receiver for pod, container, and node resource metrics.
The k8s_cluster receiver for object state such as pod phase and container restarts.
The filelog receiver with the k8sattributes processor for container logs.

Resource attributes follow the OpenTelemetry semantic conventions. Pod, namespace, and node identifiers appear as k8s.pod.name, k8s.namespace.name, and k8s.node.name. Metric names are defined by the receiver that emits them, not by the semconv spec itself: k8s.pod.phase comes from the k8s_cluster receiver, while utilization metrics like k8s.container.memory_limit_utilization come from kubeletstats. EDOT preserves all of these names natively in Elasticsearch. The exact fields you have depend on which receivers you enabled, so treat the queries as templates and adjust field names if a metric is missing.

Exploring the cluster

Start broad. Before investigating a specific symptom, it helps to see what the cluster looks like in your data.

This query counts the pods reporting metrics in each namespace over the past hour.

FROM metrics-*
| WHERE @timestamp >= NOW() - 1 hour
| STATS pod_count = COUNT_DISTINCT(k8s.pod.name) BY k8s.namespace.name
| SORT pod_count DESC

COUNT_DISTINCT collapses the many metric samples per pod into a single count per namespace. The result is a quick inventory: which namespaces are busy and whether anything you expected to be running is missing.

Finding pod restarts and crash loops

Restarts are usually the first signal that something is wrong. The k8s.container.restarts metric is a gauge that reports the current restart count for each container.

This query surfaces the containers that have restarted the most in the last 24 hours.

FROM metrics-*
| WHERE @timestamp >= NOW() - 24 hours AND k8s.container.restarts IS NOT NULL
| STATS restarts = MAX(k8s.container.restarts)
    BY k8s.namespace.name, k8s.pod.name, k8s.container.name
| WHERE restarts > 0
| SORT restarts DESC
| LIMIT 20

MAX takes the highest restart count seen in the window, which reflects the latest value of the gauge. A container with a high and climbing restart count is the textbook sign of a crash loop. Once you have the pod name, you can pivot straight to its logs with the queries further down.

Spotting pods that are not running

A restart count tells you a pod recovered. The pod phase tells you whether it is healthy right now. The k8s.pod.phase metric encodes the phase as a number:

Value	Phase
1	Pending
2	Running
3	Succeeded
4	Failed
5	Unknown

This query uses TS to read the latest phase per pod and keeps anything that is not Running.

TS metrics-*
| WHERE TRANGE(15m)
| STATS phase = MAX(LAST_OVER_TIME(k8s.pod.phase))
    BY k8s.namespace.name, k8s.pod.name
| WHERE phase != 2
| EVAL phase_name = CASE(
    phase == 1, "Pending",
    phase == 3, "Succeeded",
    phase == 4, "Failed",
    phase == 5, "Unknown",
    "Other")
| SORT phase_name

LAST_OVER_TIME picks the most recent sample for each pod's time series, so you compare the current state rather than an average. Pods stuck in Pending often point to scheduling problems, such as insufficient CPU or memory on the nodes. Pods in Failed or Unknown are worth an immediate look.

Catching memory pressure before the OOM kill

Out-of-memory kills are one of the most common Kubernetes failures, and they are easier to prevent than to debug after the fact. When you enable limit metadata on the kubeletstats receiver, EDOT reports k8s.container.memory_limit_utilization as a fraction between 0 and 1 of the container's memory limit.

This query finds containers that ran close to their limit in the last hour.

TS metrics-*
| WHERE TRANGE(1h)
| STATS peak_mem_pct = MAX(MAX_OVER_TIME(k8s.container.memory_limit_utilization))
    BY k8s.namespace.name, k8s.pod.name, k8s.container.name
| EVAL peak_mem_pct = ROUND(peak_mem_pct * 100, 1)
| WHERE peak_mem_pct > 85
| SORT peak_mem_pct DESC

MAX_OVER_TIME finds the peak within each container's series, and the outer MAX keeps that peak per container. A container that repeatedly touches 95% or higher is a strong candidate for the next OOM kill. Pair this with the restart query above: a container with both a rising restart count and high memory utilization was very likely OOM killed.

Tracking CPU usage and node pressure

CPU problems show up as throttling and slow response times rather than crashes. The k8s.pod.cpu.node.utilization metric reports each pod's CPU use as a fraction of total node capacity.

This query charts the busiest pods over the last hour in five-minute buckets.

TS metrics-*
  | WHERE TRANGE(1h)
  | STATS avg_cpu = AVG(AVG_OVER_TIME(k8s.pod.cpu.node.utilization))
      BY k8s.pod.name, TBUCKET(5m)
  | SORT avg_cpu DESC

AVG_OVER_TIME averages the samples inside each pod's series for the bucket, and the outer AVG combines series that share a pod name. TBUCKET(5m) produces one point every five minutes, which renders cleanly as a time series chart.

To check whether the nodes themselves are saturated, query node utilization directly.

TS metrics-*
  | WHERE TRANGE(1h)
  | STATS cpu = AVG(AVG_OVER_TIME(k8s.node.cpu.usage)),
          mem = AVG(LAST_OVER_TIME(k8s.node.memory.usage))
      BY k8s.node.name, TBUCKET(5m)
  | SORT cpu DESC

A node sitting near full CPU explains throttled pods across many namespaces at once, which is easy to misread as an application bug when you only look at a single pod.

Investigating container logs

Once metrics point you at a pod, logs explain what it was doing. EDOT stores the log message in body.text and the level in severity_text, alongside the same k8s.* fields as the metrics.

This query ranks namespaces and pods by error volume in the last hour.

FROM logs-*
| WHERE @timestamp >= NOW() - 1 hour
  AND severity_text IN ("ERROR", "FATAL")
| STATS errors = COUNT(*)
    BY k8s.namespace.name, k8s.pod.name
| SORT errors DESC
| LIMIT 20

Counting by pod tells you whether errors are concentrated in one workload or spread across the cluster. A single noisy pod and a cluster-wide spike call for very different responses.

To read what a specific pod is logging, filter by pod name and search the message text.

FROM logs-*
| WHERE @timestamp >= NOW() - 1 hour
  AND k8s.pod.name == "checkout-<your-hash>"
  AND body.text LIKE "*timeout*"
| KEEP @timestamp, severity_text, body.text
| SORT @timestamp DESC
| LIMIT 50

LIKE "*timeout*" does a simple wildcard match on the message. For full-text relevance instead of wildcards, swap it for MATCH(body.text, "timeout").

Chaining ES|QL queries to investigate a Kubernetes incident

Real Kubernetes investigations chain multiple ES|QL queries together instead of running one in isolation.

A useful loop looks like this:

Count errors by pod to find the noisy workload.
Check that pod's restart count and memory utilization to see if it is crashing or starved.
Read its recent logs to find the specific failure.

Because every query uses the same k8s.namespace.name and k8s.pod.name fields, you can carry a pod name straight from one query to the next. The same fields let you build a single dashboard where a metrics panel and a logs panel filter together as you click through namespaces.

Turning queries into alerts and dashboards

Any ES|QL query in this post that produces an aggregated value can back an alert, not just support ad hoc investigation.

For example, the restart query becomes an alert when you keep only containers above a threshold and trigger on a non-empty result.

FROM metrics-*
| WHERE @timestamp >= NOW() - 15 minutes AND k8s.container.restarts IS NOT NULL
| STATS restarts = MAX(k8s.container.restarts)
    BY k8s.namespace.name, k8s.pod.name, k8s.container.name
| WHERE restarts >= 5

Wire this into an Elasticsearch query rule and you get notified the moment a container crosses five restarts in fifteen minutes, instead of finding out when a user does. The same pattern applies to memory utilization, node saturation, and error counts.

Building a Kubernetes monitoring toolkit with ES|QL

ES|QL gives you one language for every Kubernetes signal, from object state to resource metrics to container logs. Start with the exploration query to understand your cluster's shape, then keep the restart, phase, memory, CPU, and log queries close for the next incident. Use TS when you need time series functions like MAX_OVER_TIME or TBUCKET to aggregate correctly within each pod or container series. For counting distinct values or taking a simple MAX on a gauge, FROM is enough.

From here, you can adapt the field names to your own receivers, save the most useful queries as dashboard panels, and promote the critical ones to alerts.

To go deeper, see the ES|QL reference, the TS command documentation, and the EDOT Kubernetes guide.

Common ES|QL queries for Kubernetes monitoring