Elastic's AI Year in Review
One million messages later: Lessons from building AI agents on the Elasticsearch Platform
Building AI agents that actually work in production requires critical development decisions and a true feedback loop.
In 2025, Elastic's Field Technology team ran five AI agents across customer support and sales workflows. Over one year, those tools processed more than one million messages from customers and Elastic employees worldwide.
This report is built entirely from that data. By routing every interaction through a centralized observability layer, we transformed raw conversation logs into a structured dataset covering 209,220 threads to understand conversation quality, user sentiment, retrieval relevance, and adoption patterns.
The result is a ground-level view of what production AI actually looks like: where it excels, where it quietly fails, and what engineering teams need to do differently to move from prototype to mission critical engine.
Download nowHighlights
Dive deeper into the most significant lessons from our journey building AI agents at Elastic.
- Why your interaction logs are the most underutilized asset for measuring AI performance
- Why partial context produces worse answers than no context at all and how a single relevance threshold fixes it
- What 209,000 conversation threads reveal about the power law of AI adoption and who your most important users really are
- Why your highest token sessions are likely your highest value interactions, not a cost problem to optimize away
- How zero result queries become a demand driven roadmap for your knowledge base
