Kubernetes Secret Access via Unusual User Agent
editKubernetes Secret Access via Unusual User Agent
editThis rule detects when secrets are accessed via an unusual user agent, user name and source IP. Attackers may attempt to access secrets in a Kubernetes cluster to gain access to sensitive information after gaining access to the cluster.
Rule type: new_terms
Rule indices:
- logs-kubernetes.audit_logs-*
Severity: low
Risk score: 21
Runs every: 5m
Searches indices from: None (Date Math format, see also Additional look-back time)
Maximum alerts per execution: 100
References: None
Tags:
- Data Source: Kubernetes
- Domain: Kubernetes
- Domain: Cloud
- Use Case: Threat Detection
- Tactic: Credential Access
- Resources: Investigation Guide
Version: 2
Rule authors:
- Elastic
Rule license: Elastic License v2
Investigation guide
editTriage and analysis
Disclaimer: This investigation guide was created using generative AI technology and has been reviewed to improve its accuracy and relevance. While every effort has been made to ensure its quality, we recommend validating the content and adapting it to suit your specific environment and operational needs.
Investigating Kubernetes Secret Access via Unusual User Agent
This rule flags requests to read or enumerate Kubernetes secrets when they come from an uncommon client profile, account, and network source, which can expose tokens, keys, and passwords that unlock wider cluster or cloud access. A common attacker pattern is to compromise a pod or steal a kubeconfig, then use curl or a custom script from a new host to pull service-account tokens, registry credentials, or application secrets for follow-on movement.
Possible investigation steps
- Determine whether the requesting identity, source IP, and user agent align with a known administrator workstation, CI/CD runner, controller, or approved automation path by validating change records, VPN or proxy logs, and asset ownership.
- Review the exact secret names and namespaces accessed to assess impact, prioritizing service-account tokens, registry credentials, cloud keys, kubeconfigs, and secrets tied to production or highly privileged workloads.
- Compare the event to the identity’s historical Kubernetes activity to confirm whether the client pattern, originating network, targeted namespaces, or access volume are new or unusually broad for that account.
- Correlate nearby cluster and cloud activity from the same identity or source for signs of follow-on actions such as pod exec, token creation, role binding changes, API discovery bursts, or authentication attempts using newly exposed credentials.
- If the access is not clearly authorized, contain by revoking or rotating the exposed secrets and linked credentials, then inspect the originating host or pod and its RBAC permissions for evidence of compromise or misuse.
False positive analysis
- A cluster administrator may legitimately use curl, a browser, or a custom script from a newly assigned workstation or bastion host to inspect a secret during troubleshooting, so verify the activity against approved maintenance records and confirm the source IP and user identity map to that authorized host and user.
- A workload or internal automation can access secrets through a nonstandard Kubernetes API client after a deployment, restart, or credential rotation, so confirm the service account, namespace, and RBAC scope match the application’s expected behavior and correlate the timing with recent operational changes.
Response and remediation
- Isolate the source of the secret access by quarantining the implicated workstation or pod, cordoning the hosting node if needed, and temporarily blocking its network path to the Kubernetes API server and other sensitive services.
- Revoke the attacker’s access by disabling the abused user or service account, deleting exposed or suspicious kubeconfigs, API tokens, CronJobs, DaemonSets, backdoor pods, and any newly created RoleBindings or ClusterRoleBindings tied to the activity.
- Rotate every secret that was read or listed and all downstream credentials it protects, including service-account tokens, registry passwords, cloud IAM keys, database credentials, and application secrets, then restart affected workloads so they load the new values from trusted sources.
- Restore the cluster to a known-good state by redeploying affected workloads from trusted images and manifests, validating namespaces and mounts against approved baselines, and removing any unauthorized containers or helper utilities left behind for persistence.
- Escalate to incident response immediately if the access touched production namespaces, cluster-admin or cloud-linked credentials, multiple namespaces, or was followed by pod exec, token creation, or lateral movement from the same host or account.
- Harden against recurrence by reducing secret-read permissions to only required service accounts, forcing administrative access through approved bastions, enforcing short-lived credentials with regular rotation, and alerting on nonstandard clients or sudden secret enumeration activity.
Rule query
editevent.dataset:"kubernetes.audit_logs" and kubernetes.audit.objectRef.resource:"secrets" and
kubernetes.audit.verb:("get" or "list") and user_agent.original:(* and not (*kubernetes/$Format))
Framework: MITRE ATT&CKTM
-
Tactic:
- Name: Credential Access
- ID: TA0006
- Reference URL: https://attack.mitre.org/tactics/TA0006/
-
Technique:
- Name: Unsecured Credentials
- ID: T1552
- Reference URL: https://attack.mitre.org/techniques/T1552/
-
Sub-technique:
- Name: Container API
- ID: T1552.007
- Reference URL: https://attack.mitre.org/techniques/T1552/007/