Large language models (LLMs) are powerful, but they possess a fundamental limitation. An LLM's performance is not solely a function of its static internal knowledge, which is frozen at its last training date.

Its practical success critically depends on the external information and tools provided to it at the moment of inference. Without a native ability to access your live proprietary data, models will generate plausible but factually incorrect information.

This is where context engineering becomes essential. For models to pull the most relevant context, teams need access to advanced search techniques that match user query intent with relevant context from source systems.

Without accurate context, an agent will fail by hallucinating, selecting the wrong tool, or drifting away from its original objective. Context poisoning occurs when these errors compound across subsequent interactions.

Build a comprehensive context engineering practice by:

Implementing retrieval augmented generation (RAG): This involves the AI retrieving external data "just in time" from a knowledge base, such as internal company documents or public websites. RAG enables the AI to answer questions using information it was not originally trained on, thereby ensuring its responses are both current and accurate.
Managing memory strategically:
- Short-term: Use checkpointers for current conversation state.
- Long-term: Persist cross-conversation information to appropriate stores.
- Implement trimming, summarization, and relevance filtering.
Optimizing tool availability: Minimize tool count while maintaining coverage. Consider RAG-based tool selection to combat "tool confusion" where too many tools reduce accuracy.
Ranking outputs for smarter content surfacing: Tools like Jina Reranker rescore retrieved content by how closely it actually matches a query, replacing rough similarity matches with precise relevance ranking. This ensures a reliable way to control what material gets surfaced and prioritized.

Context engineering

Vector database

Search powered applications

Logs

Threat protection

Workflows

Elasticsearch

Kibana (Discover, Dashboards)

Elastic Agent Builder

AutoOps

Piped query language

Jina AI search models

Elastic Cloud Serverless

Elastic Cloud Hosted

Self-managed Elasticsearch

Ecommerce search

Customer support search

Search-driven apps

Log analytics

Infrastructure monitoring

Digital experience monitoring

App performance monitoring

AIOps

LLM observability

Next-gen SIEM

Workflows for security

XDR and endpoint security

AI for security

10x your data's value

Cloud providers

Elastic AI Ecosystem

Search AI Partner Program

AV-Comparatives

Forrester Wave™ Leader

Gartner Magic Quadrant Leader

IDC MarketScape Leader

Search

Security

Observability

Get started

Demo gallery

Downloads

Integrations

Docs

Elasticsearch Labs

Elastic Security Labs

Elastic Observability Labs

Blog

Community

Events

Webinars

Discuss

Training

Support

Consulting

Author

Articles by Rusty Searle

Senior Director, Enterprise Technology and Innovation

5 data foundation and technology stack gaps stalling your AI agents

Sign up for Elastic Cloud free trial