Kubernetes API Server Proxying Request to Kubelet

edit
IMPORTANT: This documentation is no longer updated. Refer to Elastic's version policy and the latest documentation.

Kubernetes API Server Proxying Request to Kubelet

edit

Detects non-system identities using the Kubernetes nodes/proxy API to proxy requests through the API server directly to a node’s Kubelet. The nodes/proxy subresource allows any principal with this RBAC permission to reach the Kubelet API on any worker node without needing direct network access or Kubelet TLS certificates. Through this proxy path, an attacker can list all pod specifications including environment variable secrets, read Kubelet configuration and PKI material, retrieve container logs, and access running pod metadata across all workloads on the target node. Monitoring and health check endpoints such as /metrics, /healthz, and /stats are excluded to reduce noise from legitimate observability tooling.

Rule type: query

Rule indices:

  • logs-kubernetes.audit_logs-*

Severity: medium

Risk score: 47

Runs every: 5m

Searches indices from: now-9m (Date Math format, see also Additional look-back time)

Maximum alerts per execution: 100

References:

Tags:

  • Data Source: Kubernetes
  • Domain: Kubernetes
  • Use Case: Threat Detection
  • Tactic: Privilege Escalation
  • Tactic: Lateral Movement
  • Tactic: Discovery
  • Resources: Investigation Guide

Version: 1

Rule authors:

  • Elastic

Rule license: Elastic License v2

Investigation guide

edit

Triage and analysis

Investigating Kubernetes API Server Proxying Request to Kubelet

Review the user.name, source.ip, and user_agent.original fields to determine who initiated the proxy request and from where. Examine kubernetes.audit.requestURI to identify which Kubelet endpoint was proxied — the path after /proxy/ maps directly to the Kubelet API path the attacker accessed.

Possible investigation steps

  • Check the proxied Kubelet path in requestURI to determine attacker intent:
  • /proxy/pods — pod spec enumeration including environment variable secrets
  • /proxy/exec or /proxy/run — command execution inside containers on that node
  • /proxy/configz — Kubelet configuration and authentication settings
  • /proxy/runningpods — active workload enumeration
  • /proxy/containerLogs — log harvesting for leaked credentials
  • Identify how the principal obtained nodes/proxy permission by reviewing RBAC bindings
  • Check if a ServiceAccount token was created shortly before via the TokenRequest API - this indicates the attacker minted a token specifically for this access.
  • Review whether the same principal accessed multiple nodes via proxy in a short window, which indicates systematic lateral movement across the cluster.

False positive analysis

  • Prometheus, Datadog, and other monitoring agents scrape /metrics and /stats via nodes/proxy. These endpoints are excluded by default. If additional monitoring paths generate noise, add them to the requestURI exclusion.
  • Cluster administration tools that inspect node health via the proxy API can match. Correlate with change management windows and verify the source identity.

Response and remediation

  • Immediately review the RBAC role granting nodes/proxy permission and determine if the binding is authorized. Remove unauthorized bindings.
  • If /proxy/pods was accessed, assume all environment variable secrets on that node are compromised. Rotate affected credentials, API keys, and database passwords.
  • If /proxy/exec or /proxy/run was accessed, treat the target node as compromised. Isolate the node, review running containers for unauthorized modifications, and check for persistence mechanisms.
  • Audit all ClusterRoles for nodes/proxy permission — this is a powerful privilege that should be restricted to infrastructure automation only, never granted to application service accounts.

Rule query

edit
kubernetes.audit.objectRef.subresource:"proxy" and
kubernetes.audit.objectRef.resource:"nodes" and
not kubernetes.audit.requestURI:(*metrics* or *healthz* or *stats/summary* or *elastic-agent* or *configz*) and
not user.name:(
  system\:kube-controller-manager or
  system\:kube-scheduler or
  system\:serviceaccount\:kube-system\:* or
  system\:node\:* or
  eks\:* or aksService
)

Framework: MITRE ATT&CKTM