Gauntlet: What happens when your agent's tools fight back

Instead of building a sandbox, Gauntlet uses a mocking agent that intercepts your primary agent's tool calls and finds creative ways to break it. When your agent calls search_emails, the mocking agent sees the result and decides whether to mutate it, injecting a prompt injection into an email body, returning subtly wrong data, or feeding false information to see if the primary agent catches it. The primary agent never knows it's in a simulation.

The interface is two decorators:

First, the mocking agent needs to maintain a coherent model of the world throughout the conversation. If it told the primary agent that an email was from Alice, it can't later contradict that. A mutated email that's obviously fake teaches you nothing. Plausibility is the whole game.

Second, the mocking agent needs to find novel bugs. Rediscovering the same prompt injection pattern 50 times isn't useful. It needs to remember what it has already found and explore in new directions while staying grounded in what the tools actually do.

Both of these are search problems. And that's where Elasticsearch becomes the backbone of the system.

The mocking agent runs on two memory circuits, both living in Elasticsearch.

Short-term memory tracks everything within the current session: every tool call intercepted, the original result, what it was mutated to, and what the primary agent did in response. This is the coherence layer. The mocking agent can query its own recent decisions and stay internally consistent while still being adversarial. Balancing creativity with coherence was the hardest design problem in the entire project.

Long-term memory is where the creativity compounds. It stores confirmed bugs with embeddings for similarity search, full tool implementations so the agent can reason about failure modes, and historical results from past runs. When the mocking agent needs a new attack idea, it searches long-term memory for what's been tried before, finds gaps, and hypothesizes something new.

These feed into a closed cycle: hypothesize what bugs might exist, create circumstances to prove them, and store confirmed bugs back into the index. The inventory grows. The attacks get more creative. The gap between Gauntlet and manual sandbox setup widens over time.

The entire mocking agent is built inside Elastic Agent Builder — instructions, tool bindings, and multi-turn conversation state via the Amazon Bedrock Converse API; no external orchestration needed.

The tool I'm most proud of is generate-hypothesis. It's a single ES|QL statement that samples existing bugs, aggregates them with MV_CONCAT, and calls COMPLETION inline to propose a novel attack hypothesis. It handles sampling, aggregation, LLM reasoning, and result generation all in one query, never leaving the ES|QL pipeline. I went in expecting I'd need to shuttle data between Elasticsearch and an external script. I didn't.

ES|QL's COMPLETION function was the biggest surprise. Between COMPLETION, STATS, MV_CONCAT, and SAMPLE, I could build entire reasoning pipelines as single queries. Bug storage uses Kibana Workflows, and a programmatically created Kibana Dashboard gives real-time visibility into bug counts, severity breakdowns, and attack pattern heatmaps.

The Converse API solved another problem I'd been dreading. The mocking agent needs to remember what it's already told the primary agent within a single run. I assumed I'd have to fetch conversation histories from indices and reload them into the agent on every call. But it turns out that the Converse API handles multi-turn state natively. I didn't write any conversation management logic. Just keep calling converse, and it stays coherent.

Manual adversarial sandbox setup takes roughly an hour per scenario. With Gauntlet, the same process takes 2–10 minutes, and its long-term memory means each run is informed by every previous run. The more you use it, the more it learns about your agent’s weak points and the harder it tries to find new ones.

Right now Gauntlet is a 1v1: one mocking agent versus one primary agent. But the problem is embarrassingly parallel. 20 attack sessions could run simultaneously on separate sessions without any architectural changes. Scaling is the obvious next step.

The more interesting open question is exploration versus exploitation in the long-term memory. The mocking agent needs to balance trying variations of known successful attacks (exploitation) against completely novel hypotheses (exploration). This is a well-studied problem in other domains, but applying it to adversarial agent testing feels unexplored. There might be something worth pursuing beyond this project entirely.

I also keep thinking about Rehearse. Gauntlet is a special case: fuzz-testing works because failure in simulation implies possible failure in reality. But there are other domains where the environment is stable enough between rehearsal and execution that the original Rehearse concept could work. I haven't found them yet, but I'm looking.

If you're building agents with access to real-world tools, test what happens when those tools fight back. Not just once manually, but continuously, with a system that remembers what it's tried and gets more creative over time. That's what Gauntlet does.