How to correlate Kubernetes audit logs with container runtime data

If you already ship Kubernetes (K8s) audit logs and Defend for Containers (D4C) into Elastic, you still have to join them by hand, and neither source is complete on its own. In our lab, a compromised workload service account ran discovery, read secrets, minted a token, created a privileged pod, and execed into it to attempt a container escape. The escape wrappers, nsenter and chroot, never appeared in the runtime process events. Kubernetes audit logs recorded them in the decoded requestURI.

Real incidents cross both planes, which is why our write-up of the Hugging Face intrusion shows Kubernetes and runtime abuse in the same campaign. The two data sources answer different questions:

This post makes that connection practical in Elastic. We cover the fields that join the two data sources and two ways to join across them, depending on how the API call is attributed. We also cover the one-sided context fields that help you read the activity. Plus, we walk through each of those with queries and result views from our Amazon Elastic Kubernetes Service (EKS) lab.

This continues the control-plane correlation thread from the TeamPCP container attack scenario. For D4C setup and policy basics, see Getting started with Defend for Containers.

How Kubernetes audit logs and container runtime data connect

Two joins cover most cases. Which one you use depends on how Kubernetes attributed the API call.

Join keys shared by Kubernetes audit logs and container runtime

These fields carry the same meaning on both data sources. Pair them with a time window around the activity you care about (often 15 to 60 minutes).

One of the most useful pod joins for tracking threat behavior is create and then use. K8s audit records the create (or an exec) and names the pod. We then use that same pod name on the D4C side to see what ran inside it a few minutes later. Once you know the namespace and pod name, look for matching orchestrator.namespace and orchestrator.resource.name, and then scan process.name and process.args for what ran in that pod.

Joining on pod caller identity

Some Elastic prebuilt rules correlate D4C and K8s audit in one detection using Event Query Language (EQL) sequences across both data sources. Service Account Token or Certificate Access Followed by Kubernetes API Request is a useful example of the pod (caller identity) join from the table above. Within 60 seconds, it joins a service account token or digital certificate file open on orchestrator.resource.name to a Kubernetes audit API event on kubernetes.audit.user.extra.authentication.kubernetes.io/pod-name.

sequence with maxspan=60s
 [file where host.os.type == "linux" and event.type == "change" and event.action == "open" and
  file.path in (
    "/var/run/secrets/kubernetes.io/serviceaccount/token",
    "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
  ) and
  process.interactive == true and container.id like "*"] by orchestrator.resource.name
 [any where data_stream.dataset == "kubernetes.audit_logs" and
  kubernetes.audit.stage in ("ResponseComplete", "ResponseStarted")]
  by `kubernetes.audit.user.extra.authentication.kubernetes.io/pod-name`

That pod-name extra on a Kubernetes audit event is often present when a request uses the pod’s bound service account (SA) identity, including controllers and agents that call the API from inside the pod. When it’s present, you can line it up with the D4C field orchestrator.resource.name, including in sequence rules like the one above. Some API traffic is attributed only as a service account in user.name, without the pod-name extra. That requires a different join; start from the SA, collect pod names from objectRef, and carry those into D4C with the namespace and a shared time window. That path is shown next. 

Joining on service account and objectRef pod names

Caller identity joins on the pod that made the API call. This section joins on the service account principal and the pods it touched. Both are valid ways to move between Kubernetes audit and D4C; which one fits depends on how the audit event is attributed.

This join assumes a compromised or suspicious service account is driving API activity and that the pods it creates or execs into are where we should look next in D4C.

The join runs in three steps:

1. Timeline the service account in Kubernetes audit logs. Filter on user.name matching system:serviceaccount:<namespace>:<sa> to see every API call that principal made and which objects it touched.

2. Collect pod names from kubernetes.audit.objectRef.name. Keep the rows where objectRef.resource is pods, which covers both creates and execs, and note the namespace from orchestrator.namespace.

3. Query Defend for Containers with those pod names and namespace. Match orchestrator.resource.name against the pod names and orchestrator.namespace against the namespace, using the same time window as the audit activity.

Each step is expanded below with the queries we ran.

Kubernetes audit carries the API principal in user.name (and kubernetes.audit.user.username), for example system:serviceaccount:<namespace>:<sa>. On a D4C process event, that K8s principal isn’t present as user.name, and user.id is the Linux user ID inside the container (for example, 0 when the process runs as root). There are different mappings from different integrations, so even when both documents expose a field by the same name, the value reflects that data source’s schema, which isn’t always a 1:1 correlation.

A good approach is to timeline the suspicious service account in audit. This shows what that principal did and which objects it touched:

user.name: "system:serviceaccount:<namespace>:<sa>"

Or as Elasticsearch Query Language (ES|QL), keeping the fields that matter for the next hop:

FROM logs-kubernetes.audit_logs-*
| WHERE @timestamp > NOW() - 6 hours
  AND user.name == "system:serviceaccount:<namespace>:<sa>"
| KEEP @timestamp,
       kubernetes.audit.verb,
       kubernetes.audit.objectRef.resource,
       kubernetes.audit.objectRef.subresource,
       orchestrator.namespace,
       kubernetes.audit.objectRef.name,
       user_agent.original,
       `kubernetes.audit.annotations.authorization_k8s_io/decision`,
       event.outcome
| SORT @timestamp ASC

In the service account timeline results, look at kubernetes.audit.verb, objectRef.resource, objectRef.subresource, orchestrator.namespace, and objectRef.name for what was touched.

Collect pod names from the objects that SA touched. For pod creates and execs, the pod sits in kubernetes.audit.objectRef.name, the namespace in orchestrator.namespace, and the verb in kubernetes.audit.verb, and exec shows up as kubernetes.audit.objectRef.subresource: exec (with objectRef.resource: pods). Filter the same SA timeline to create and exec activity when you only want pod targets:

data_stream.dataset: "kubernetes.audit_logs" and 
user.name: "system:serviceaccount:<namespace>:<sa>" and
kubernetes.audit.objectRef.resource: pods and
(
  kubernetes.audit.verb: create or
  kubernetes.audit.objectRef.subresource: exec
)

Query D4C with those pod names and the shared namespace. Use the namespace and pod names you collected (and the same time window as the audit activity):

data_stream.dataset: "cloud_defend.*" and 
orchestrator.namespace: "<namespace>" and 
orchestrator.resource.name: ("<pod-1>" or "<pod-2>")

If more than one service account creates or execs into the same pod in that window, D4C on that pod name shows shared runtime context, not a clean attribution back to a single SA.

Namespace and pod name work in the other direction too. Starting from a Defend for Containers process event, take orchestrator.namespace and orchestrator.resource.name and search Kubernetes audit logs for that pod in objectRef to recover create, exec, and related API activity.

Context fields that make the joined events readable

Join keys get you onto the right documents. Context fields usually exist on only one data source and add the detail you need to read those documents. We use several of them in the lab that follows.

Context fields in Kubernetes audit logs

Context fields in Defend for Containers

Lab scenario: Tracing a compromised service account from audit logs to runtime

In our EKS lab, we ran a compromised workload service account through discovery, secret access, a privileged breakout create, exec, and cloud metadata probes. That traffic is attributed as an SA without the caller pod-name extra, so the following walkthrough uses the service-account / objectRef join and the context fields from above to move between Kubernetes audit and D4C.

Finding a suspicious service account in Kubernetes audit logs

We started on the K8s audit side, looking for workload identities whose clients didn’t look like normal controllers or app traffic. Most legitimate SA requests come from in-cluster components with familiar user agents. When a system:serviceaccount:… principal shows up with something like curl or python-requests, that can mean someone is driving the API with a stolen or projected token from a shell or a short script, rather than through the application’s usual client. That pattern doesn’t prove compromise on its own, but it gives us an identity and namespace to expand on. The same hunt works for other HTTP clients you may see on a stolen token (axios, undici, and similar). Treat user_agent.original as a lead, not proof. It’s easy to spoof. A curl session can send a legitimate-looking kubectl user agent, so a “normal” user agent doesn’t rule the identity out.

Using the audit context fields user.name and user_agent.original, we looked for SA principals (system:serviceaccount:…) paired with clients like python-requests/… or curl/….

user.name: system\:serviceaccount\:* and
(user_agent.original: python-requests* or user_agent.original: curl/*)

We kept that service account and namespace for the rest of the example.

Timelining a suspicious service account's API activity

We timelined the SA and kept verb, object, user agent, authorization decision, and outcome in view:

FROM logs-kubernetes.audit_logs-*
| WHERE @timestamp > NOW() - 6 hours
  AND user.name == "system:serviceaccount:emu-satoken-0297d6:worker-sa"
| KEEP @timestamp,
       kubernetes.audit.verb,
       kubernetes.audit.objectRef.resource,
       kubernetes.audit.objectRef.subresource,
       kubernetes.audit.objectRef.namespace,
       kubernetes.audit.objectRef.name,
       user_agent.original,
       `kubernetes.audit.annotations.authorization_k8s_io/decision`,
       event.outcome
| SORT @timestamp ASC

Context fields did most of the work here. user_agent.original separated python-requests discovery from curl harvest. event.outcome separated forbidden recon from allowed secret gets. objectRef.resource / subresource showed TokenRequest (serviceaccounts + token) and later pods / exec.

From those same rows, we also collected pod names where objectRef.resource was pods. One name was the original workload pod already running with that SA (the foothold). A later create event named a second pod, which is the breakout that we explore next.

Spotting the privileged pod create behind a container escape

In the same timeline, a kubernetes.audit.verb: create on objectRef.resource: pods named a new breakout pod. The creation alone is interesting; the audit-only fields on requestObject.spec are what showed how sensitive it was. That document had hostPID, hostIPC, a privileged security context, and a hostPath mount to /, still attributed to the same workload SA and curl user agent that had been reading secrets.

That same document also carried the pod create → use join into D4C. objectRef.name and objectRef.namespace are the pod and namespace we query on the runtime side next.

Querying container runtime data with pod names from the audit timeline

The service account timeline gave us two pod names: the foothold pod and the breakout pod. We queried Defend for Containers for both:

data_stream.dataset: "cloud_defend.process" and
orchestrator.namespace: "emu-satoken-0297d6" and
orchestrator.resource.name: ("worker-0297d6" or "breakout-0297d6")

On the foothold pod, process context showed a projected-token read:

On the breakout pod, process context showed cloud instance-metadata clients were queried:

Why nsenter and chroot never reach container runtime events

On the breakout pod, we also ran a lab-safe host breakout attempt through pods/exec: nsenter -t1 -m -u -n -i true and chroot /host true. The true processes are no-ops, so the session doesn’t do anything destructive. In a real breakout, the same wrappers usually target a shell or follow-on tool, for example nsenter … /bin/bash or chroot /host /bin/sh.

We went back to the D4C process events for that breakout pod looking for nsenter and chroot next to the metadata activity. On runtime events, we read process.name and process.args together. Those wrapper names weren’t there. D4C showed process.name: true and process.args: true instead, matching the true targets in our lab commands. nsenter and chroot only set up the breakout and then hand off to that target. After the handoff, runtime records that target command, which here was true. Even with both fields, the wrappers never made it into D4C, so we turn to Kubernetes audit next.

K8s audit and D4C answer different questions on the same breakout. The same SA’s pods/exec events carry the full command list in kubernetes.audit.requestURI. Decoding that URI showed nsenter and chroot against the breakout pod (the API-level breakout tools), while D4C showed what actually ran in the container after those wrappers and what that session did next.

FROM logs-kubernetes.audit_logs-*
| WHERE @timestamp > NOW() - 6 hours
  AND user.name == "system:serviceaccount:emu-satoken-0297d6:worker-sa"
  AND kubernetes.audit.objectRef.subresource == "exec"
  AND kubernetes.audit.objectRef.name == "breakout-0297d6"
| EVAL uri = URL_DECODE(kubernetes.audit.requestURI)
| WHERE uri LIKE "*nsenter*" OR uri LIKE "*chroot*"
| WHERE kubernetes.audit.stage == "ResponseComplete"
| KEEP @timestamp, kubernetes.audit.objectRef.name, uri
| SORT @timestamp ASC

Conclusion

We’ve covered the core of unified container visibility: bridging Kubernetes audit logs with Defend for Containers telemetry. Plus, we’ve explored the two primary ways to join these datasets; that is, using pod-caller identity for direct attribution or service account and objectRef mapping for broader investigations. And we’ve demonstrated how context fields like user_agent.original or decoded requestURI parameters turn those joined events into clear, actionable attack timelines.

Kubernetes audit and D4C remain two halves of the same picture: the API on one side, what ran inside the container on the other. Know which fields actually join them and which one-sided context fields still change how you read an investigation. Two joins cover most cases: caller identity when the pod-name extra is on the audit event, and service account plus objectRef pod names when the call is attributed only as a service account. Fields such as requestObject.spec, a decoded exec requestURI, and user_agent.original won’t join the two data sources by themselves, but they’re often what makes the joined documents actionable.

What next?

Now that you have the mapping foundation, the best way to operationalize these techniques is to test them against your own environment's telemetry. Start by implementing the queries we walked through in our lab scenario to identify suspicious service account activity, and then pivot to D4C to verify the actual runtime impact. For a shortcut, explore our prebuilt rules tagged Data Source: Elastic Defend for Containers and Data Source: Kubernetes, which serve as ready-to-use examples of these join patterns in action.

You can browse the same detection content in the detection-rules repository or our rules explorer page.