The security attack that hid inside your observability data

When organizations run separate tools for observability and security, the split feels logical on paper. Ops teams care about uptime and performance. Security teams care about threats and compliance. Why force them to share a platform?

Attackers, though, don't stay in their lane.

Modern threats are designed to look like infrastructure noise. A cryptominer masquerades as a kernel process. A data exfiltration job looks like normal egress traffic. A compromised web server generates the same memory pattern as a memory leak. When observability and security data live in separate systems, those threats stay invisible because neither team has the complete picture, and the gap between platforms is exactly where attackers hide.

But the platform separation doesn't just create a visibility problem. It quietly duplicates costs.

The cost is duplicated (and easily missed) because two separate teams are storing and processing the same underlying events twice: once for observability and once for security. Infrastructure logs ingested into a SIEM are often the same logs feeding your APM or metrics platform. Network flow data queried by your SOC analysts is the same flow data your SRE team uses to debug latency. When those platforms are separate, you pay to ingest that data twice and store it twice. And your teams can only ever see half the picture at any given moment.

The case for a single platform isn't about forcing ops and security teams to share a UI. It's about giving them access to the same underlying data so that when an alert fires, the full context is already there.

Consider what changes in the scenario above if observability metrics and security events live in the same Elasticsearch cluster. The process table that shows kworker/u8:2 consuming 97% CPU sits next to the security event that shows that same process was spawned from /tmp/.x11-unix/.kw8 six hours ago, running as www-data, not as root. The DNS query to d0.pool.minexmr.com is in the same cluster as the CPU chart. The 15-minute outbound TCP connections are one query away from the Kibana alert that paged the on-call engineer.

The data was never missing. It was in a different box.

When those boxes become one, an AI agent can reason across both domains in a single conversation, not by querying a separate SIEM and then querying Elastic and trying to cross-reference the results manually, but by using search across both observability and security indices as a unified investigation capability. Ask about the host's CPU history, then ask about its recent process creation events. The agent doesn't switch platforms; it just searches.

This is the architecture that turns an observability alert into a security investigation automatically before anyone has had to open a second tool.

One of the most uncomfortable truths for security teams is how much signal is already flowing through their organization's observability data. Host metrics include running process lists. APM traces capture service-to-service calls that can reveal lateral movement. Infrastructure logs contain DNS queries, network connections, and authentication events that map directly onto MITRE ATT&CK techniques.

The signal is there. What's missing is the index.

Organizations that ingest observability data into Elastic already have the raw material for security detection. Process telemetry collected by Elastic Agent for infrastructure monitoring is the same telemetry that powers Elastic Security's host-based detection rules. Logs ingested for SRE workflows are the same logs that feed SIEM correlation. The question isn't whether to collect more data; it's whether the data you're already collecting is being used for both purposes.

This is where ingesting via search becomes a force multiplier. When data from any source — cloud infrastructure, on-premises hosts, network devices, and third-party SaaS — is brought into Elastic through a unified ingest layer, it becomes queryable by every team the moment it lands. An observability engineer searching for slow queries and a SOC analyst hunting for unusual process behaviour are both running against the same indices. Neither team needs a separate copy of the data. Neither team double pays for the data.

Add an AI agent on top of that unified layer — one that can hold tools over multiple indices and reason across them in a single chain of thought — and the correlation that used to take a human analyst an hour to perform manually happens in seconds automatically every time an alert fires.

A general-purpose language model knows what a cryptominer is. It can describe containment best practices in reasonable terms. What it doesn't know is that your organization's IRP-004 requires dual approval from the SOC lead and payment operations before any payment host is isolated. It doesn't know that your PCI-DSS obligations require the compliance team to be notified within four hours of confirmation, not 24 hours, not "as soon as possible," but four hours documented in your own policy. It doesn't know that your network quarantine procedure uses VLAN PAYMENT-QUARANTINE or that your forensics team needs a disk snapshot tagged with the incident reference before any remediation begins.

That specificity is the difference between advice you can act on and advice you have to reinterpret before you can act on it. In a regulated industry, reinterpretation introduces risks like the wrong call, the missed notification window, or the undocumented step.

The answer isn't to train a model on your internal documentation (which changes, is version-controlled, and may be restricted). The answer is to put your incident response playbooks into Elasticsearch and give your AI agent a search tool over that index.

This is search in its most practical form: not a UI for humans to browse, but a retrieval mechanism for an AI agent to ground its reasoning in your actual procedures at the moment they're needed. When the agent confirms a cryptominer on a PCI-DSS-scoped host, it doesn't reach for general knowledge. It searches the playbooks, retrieves IRP-001 and IRP-004, and derives its recommendations directly from the steps your security and payment teams wrote and approved. "Isolate the host" becomes "apply the PAYMENT-QUARANTINE security group, allow inbound from the forensics VLAN only, get sign-off from SOC lead and payment operations, and notify the compliance team within four hours." Every recommendation is traceable to a specific section in a specific document.

This matters beyond compliance. It makes the AI's recommendations auditable. When a regulator asks how a decision was made, the answer isn't "the model suggested it." Instead, it's "the agent retrieved IRP-004 Section 3.2 and followed it." That's a chain of evidence you can defend.

The same principle extends beyond security playbooks. Product runbooks, escalation matrices, vendor SLAs, regulatory obligations, and change procedures shape how your organization responds to events. Ingesting them into Elasticsearch makes that guidance available to the agent as grounded, retrievable context. You're not training the model; you're giving it a library and teaching it to search.

In the case of observability and security, cross-domain AI investigation is what makes a unified platform meaningfully different from two separate ones.

An AI agent with search tools over both an observability index and a security index can discover that the kernel worker thread consuming 97% CPU is a cryptominer because it can query both the metrics and the security events in the same reasoning chain. That capability isn't achievable by running two separate AI agents on two separate platforms and hoping they communicate. It requires the data to be co-located, queryable together, and under a single context window.

The ops engineer who gets paged at 3:00 a.m. shouldn't have to know which platform to check. The verdict should arrive before they've opened a second tool, and the full audit trail from CPU alert to confirmed threat to case creation should exist in one place and be ready for every compliance requirement the organization needs to satisfy.

Don’t pay twice for data; let your data do double duty.

If your team is running separate platforms for observability and security, the threat that hides in the gap between them isn't a matter of if; it's a matter of when. Reach out to learn how Elastic can help you unify both worlds on a single platform.