NASA Aviation Safety Reporting System (ASRS) reports describe unusual events during flights, such as missed altitudes, confusing clearances, runway issues, or mechanical problems. Each report already has an official category, like altitude deviation, course deviation, or ground encounter. In this article, we ask a different question: What does this report reveal about the pilot who wrote it? The idea is to ask a model to infer a schema grounded on the data to classify the report based on criteria that help us figure out information about the report writers. Then ask a second model to apply the labels.

You can find the full workflow definitions and helper scripts here.

What you need to run this LLM pipeline

• Elastic Stack 9.4+ or Elastic Cloud Serverless. Elastic Workflows has been generally available (GA) since 9.4.

• Elastic Agent Builder enabled in your deployment.

• A Kibana generative AI (GenAI) connector pointing at Claude Sonnet (or an equivalent reasoning model). This is the planner.

• A Mistral API key. We’ll use it to register an Elasticsearch inference endpoint.

• Python 3.10+ with elasticsearch>=9.0 and pandas. Used by the dataset loader.

How two-tier LLM orchestration works

The workflow has two jobs: Decide what labels should exist, and then apply those labels to every report.

The first job is open-ended. A large model reads a varied sample of reports and proposes a small schema of categorical fields. A field is one way to describe the writer, such as attribution_style or procedure_orientation. Each field has a few allowed values, such as self_critical, system_attributing, or balanced.

The second job is repeatable. After a human approves the schema, a smaller model reads each report and chooses one value for each field.

We use Elastic Workflows because the steps are known ahead of time: sample reports, propose labels, wait for approval, classify every document, and store the results. Writing those steps in YAML makes the process reproducible, observable, and cheaper to rerun.

Why split LLM work across two model tiers?

A small model could handle classification, but schema discovery is a different shape of problem. It requires reading a diverse sample, spotting latent patterns, and proposing complex structures. In practice, smaller models over-anchor on surface keywords and produce redundant or nonexclusive fields.

Classification is simpler, the schema exists, the values are enumerated, and the task is to pick one per field. A smaller model handles this reliably and at a fraction of the cost, since it runs once per document across the entire corpus.

Large and small here mean reasoning capability. In this article, Claude Sonnet plays the planner and Mistral Small 3.1 plays the executor.

Classifying NASA pilot reports with a two-tier LLM pipeline

We’ll use the NASA ASRS database, which collects voluntary, anonymous incident reports from pilots, controllers, and mechanics. The dataset is public, and the reports are written as free-text narratives.

What we want to ask is:

What does this report reveal about the pilot who wrote it?

The planner reads a varied sample of reports and decides which distinctions are meaningful based on how the reports are actually written.

Step

Role

Model tier

sample

Pull a diverse subset of reports from the corpus.

(no LLM)

discover

Read the question and the sample, propose a schema of fields with enum values.

Large

approve

Human reviews the proposed schema and approves or edits it.

(Human via waitForInput)

apply

Iterate over the corpus, assign one value per field to each report.

Small

store

Write the schema and the per-document field values to Elasticsearch.

(No LLM)

Registering Mistral and Claude as Elasticsearch inference endpoints

The small model will be registered as an Elasticsearch inference endpoint using the native mistral service integration. 

The alias mistral-small-latest resolves to Mistral Small 3.1. It has a 128k context window and supports JSON-mode output.

The large model will be an AI connector pointing at Claude Sonnet. The Agent Builder UI walks you through creating the connector. Take a note of the connector ID since we’ll reference it from the workflow.

Indexing NASA ASRS incident reports into Elasticsearch

The ASRS dataset is indexed with keyword mappings for aggregation fields and text mappings for the narratives the models will read.

Download the ASRS CSV (the database publishes quarterly extracts at the ASRS Database Online page), and index it. The mappings are:

The mapping types follow how each field is used. flight_phase and anomaly are mapped as keyword because we’ll run terms aggregations on them to build the sample, and aggregations need exact, non-analyzed values. narrative and synopsis are mapped as text because they hold free-form prose that the models will read. The companion notebook has the full loader script that reads the CSV and bulk-indexes the documents.

Building a stratified sample for the planning LLM

The YAML snippets in this and the following sections are steps of the workflow definition that the notebook registers via the Workflows API. The first two steps generate a representative sample: They aggregate by flight phase and by anomaly and pull a few documents per bucket with top_hits.

How the large LLM discovers a classification schema from the data

The prompt needs both the question and the sample. A question alone may produce generic labels disconnected from the corpus, and a sample alone produces descriptive clusters that ignore the angle of the question. 

When both are present and the output is structured, the model produces labels that are grounded in the data and oriented to the task: a schema of categorical fields, each with two to four mutually exclusive value options backed by evidence from the sample.

Here’s the planner step from the workflow:

The structured output schema enforces the shape of the response:

• name: Identifier for the categorical field.

• definition: What this field measures, in one sentence.

• why_useful: How this field serves the question; this also helps the downstream classifier understand the intent.

• values: Two to four mutually exclusive options. Each has a value and a definition.

Here’s an example of the produced schema. We can see how the writer is being classified and the reasons why the model decided to create the category. definitionand why_useful fields are used by the second model to classify the documents.

Human-in-the-loop schema approval with waitForInput

The proposed schema is now passed to a person for approval. Elastic Workflows has a waitForInput step that pauses the workflow with a schema, exposes a form, and resumes when the input is submitted.

waitForInput has no timeout of its own, so if nobody responds, the execution waits indefinitely. To put a limit on that, set a workflow-level settings.timeout; if it elapses before the reviewer submits the form, the execution is canceled.

When the workflow reaches this step, the execution pauses and the Kibana UI shows an "Action is required" badge. Clicking Provide action opens a form where the reviewer can paste or edit the schema JSON. Since waitForInput cannot be prepopulated from a previous step, the code polls the discover step output and prints a paste-ready JSON block that can be copied directly into this form.

The step_output helper (in the notebook) polls the execution via GET /api/workflows/executions/{id} until the discover step completes and then returns its output.

Code JSON output pasted on Kibana:

The reviewer can keep useful fields, rewrite unclear ones, merge overlapping values, and add notes. After approval, the workflow resumes and sends the final schema to the executor step.

For a new corpus, keep this human gate in place. Once the schema is stable, you can auto-approve and only fall back to review when it’s worth it: Route just the low-confidence extractions to a person, or compare a new discovery run against the schema stored in the schemas index and trigger review only when fields or values change beyond a threshold.

Classifying the full corpus with a smaller LLM

By the time the workflow reaches this step, the open-ended part of the job is over. From here, the small model takes over and classifies each report against the approved schema.

Note: The classification step uses ai.agent instead of ai.prompt because ai.agent accepts an inference-id, which lets it call the Elasticsearch _inference endpoint directly, while ai.prompt only accepts a connector-id.

The fetch_corpus step is the third elasticsearch.request in the workflow, so it’s worth saying why we read from the index again. The first two (by_phase and by_anomaly) only pulled a small stratified sample for the planner to reason over, not the data to label. Now that the schema is approved, fetch_corpus pulls the documents we actually want to classify. We cap it at 100 with match_all to keep the demo fast; this is where you would page through the full corpus.

For every field, it returns a value (or null), a confidence, and a short quote. Setting additionalProperties: true in the JSON schema lets the step return one entry per field without the workflow having to know the field names ahead of time. A stored extraction looks like this:

Here, review_required is true because procedure_orientation came back at 0.44 confidence, below our 0.5 threshold, which is the signal a confidence-based quality gate would act on.

The fetch_corpus step pulls the documents to classify. The foreachstep iterates over them sequentially, and iteration-on-failure handles the errors: retry covers transient API errors from the inference endpoint, and, if all attempts fail, the fallback step posts to Slack so the failure doesn’t pass silently. (An email connector works the same way.) continue: true then lets the loop move on to the next document instead of failing the whole run. 

For production-scale corpora, consider using executeAsync, which is the fan-out version of execute.

Writing schemas and extractions back to Elasticsearch

The workflow produces two things: the approved schema and the per-document field values. The store_schema step runs right after the human gate, before the classification step fans out:

Each extraction is written inside the foreach loop, so results are persisted as they’re produced rather than batched at the end.

The schemas index holds one document per discovery run (question, approved fields, reviewer notes). The extractions index holds one document per report per schema version. 

What this two-tier LLM orchestration pattern gives you

We built one Elastic workflow that pulls a stratified sample from an incident report index, sends it with a question to a large reasoning model to generate a classification schema based on the data and a user-defined angle, pauses for human approval, and then iterates over the full corpus with a small Mistral model that assigns one value per field. 

The approved schema and per-document field values are written back to Elasticsearch as structured data.

The point of the exercise is that two different shapes of work, schema discovery, and schema application can use two different model tiers and that a workflow lets you write the routing decision down.

Next steps for your own LLM pipeline

• Try it on a corpus of your own. The pattern doesn’t care whether the input is incident reports, customer feedback, weekly status updates, or property listings.

• Promote the foreach step to workflow.executeAsync once you’re comfortable for parallel fan-out at scale.

• Schedule the rediscovery workflow on a cron trigger so you can discover different schema variations based on the data that comes in.

• Read the Elastic Workflows documentation for the full step catalog.

Related reading

• Build AI agents with Elastic Inference Service (EIS) covers the broader multi-model wiring pattern via EIS, complementary to the Workflows-orchestrated split shown here.

• How to build AI agentic workflows with Elasticsearch is a higher-level survey of how Agent Builder and Workflows fit together.

• LangExtract and Elasticsearch tutorial explores a different extraction pattern using a hand-authored schema; useful for contrast with the discover-then-apply approach above.

Prerequisites

• Elasticsearch 9.x with jina-embeddings-v5-text-small inference endpoint available.

• Python 3.10+,

• Elasticsearch API key,

What is quantization?

An embedding is a list of numbers. By default, each number is a float32, which uses 4 bytes. Quantization stores each number with fewer bits, trading precision for space.

Like a JPEG, a quantized vector is a smaller, lower-fidelity copy of the original that still gets the job done.

Name

Bytes / dim

1024-d vector

Compression

`Float` (Baseline)

4

4096 B

1x

`int8`

1

1024 B

4x

`int4`

0.5

512 B

8x

`bbq`

~0.14

142 B

~29x

What is BBQ?

Better Binary Quantization (BBQ) is Elasticsearch's 1-bit quantization mode for dense vectors. Each dimension of the vector is stored as a single bit, plus a few corrective bytes per vector. Then, a rescoring step is applied at query time. This keeps the final retrieval quality close to a full precision search.

For the math behind each level, see Scalar quantization 101, Optimized Scalar Quantization, and the BBQ deep dive.

How does BBQ preserve search accuracy?

Plain 1-bit quantization leads to too high a search quality degradation on its own. BBQ maintains high retrieval quality through three mechanisms:

1. Asymmetric precision: Stored vectors use 1 bit per dimension.

2. Corrective factors: A few floats per vector record the rounding error and correct distances at scoring time.

3. Oversample and rescore: BBQ scans candidates with the bits and then reranks the top ones with higher precision. Fetching the top 10 means scanning about 30 candidates.

The result is the vectors that are roughly 32x smaller, with retrieval quality close to full precision. In the next section of the article, we’ll measure the memory savings and the recall on a real corpus.

How Jina embeddings v5 works

Jina embeddings v5 is a multilingual embedding model with quantization-aware training, which makes it a natural fit for BBQ in Elasticsearch: The 1024-dimensional vectors from jina-embeddings-v5-text-small sit above the dimensional floor where binary quantization stays accurate, and the model is trained so that 1-bit quantization loses little quality. Its main features are:

• One model for many tasks: v5 uses small Low-Rank Adaptation (LoRA) adapters on top of a single base model, one for each task: retrieval, text-matching, clustering, and classification. Elasticsearch picks the right adapter automatically at index and query time.

• Matryoshka dimensions: v5 is trained so you can truncate the vector (1024, 512 to 256) and minimize search quality reduction. This is another way to shrink vectors, independent of quantization.

• Quantization-aware training: v5 is trained to work with BBQ, so its 1-bit vectors lose little accuracy.

We use jina-embeddings-v5-text-small. This model is available through Elastic Inference Service (EIS) and outputs 1024 dimensions with 32k token context and is multilingual across 93 languages. That puts it above the 384-dimension threshold, below which Elasticsearch no longer defaults to bbq_hnsw.

Full model details are in the Jina v5 article on Search Labs.

Setting up the BBQ vs. float32 comparison

We’ll create two indices: Both share mappings, and what changes is the index_options.type parameter, which tells Elasticsearch how to store the dense vector field (as raw float32 HNSW or as 1-bit BBQ):

Index

`index_options`

Loaded into memory

`vectors-float32`

`hnsw`

Raw `float32` with no quantization (baseline)

`vectors-bbq`

`bbq_hnsw`

1-bit BBQ quantization + corrective factors

We then embed the corpus once with Jina v5, index those same vectors into both, and compare them on disk usage, memory footprint, and recall. You can follow along with the full supporting blog content notebook.

Connect to Elasticsearch

Create the two indices

Note: In production, you can use semantic_text to let Elasticsearch manage the mapping and inference endpoint automatically.

Point at the Jina v5 inference endpoint

We call the model jina-embeddings-v5-text-small directly (no need to create an inference endpoint) to turn text into vectors.

As result of the test, we got:

Load a multilingual news dataset

We stream real news articles from hotchpotch/multilingual_cc_news, a parquet mirror of CC-News. We take about 1,000 articles from five languages (around 3,000 docs total), plus a small held-out set of headlines to use as search queries. Using multiple languages also lets Jina v5 show its multilingual strength.

Generate the embeddings and bulk index

We embed the corpus a single time and feed those exact vectors into both indices.

We force-merge to a single segment so the storage numbers are stable and comparable.

Results: Disk versus memory

The disk usage API reports how many bytes each index spends on vectors (knn_vectors).

Result:

On disk, the two indices are about the same size. A quantized index still keeps the raw float32 vectors (needed for rescoring and requantization during merges) and adds the 1-bit vectors on top, so BBQ ends up slightly larger on disk.

The real savings is in memory. The HNSW scan only needs the 1-bit vectors in RAM, while the raw floats are read from disk to rescore the top candidates. We size that footprint using the documented kNN memory formulas: float uses num_vectors × dims × 4 and bbq uses num_vectors × (dims/8 + 14).

BBQ's extra bytes on disk should match the 1-bit payload we computed for memory. Here, that’s 13.25 - 12.80 = 0.45 MB versus the computed 0.44 MB. They line up.

Results: Recall

To check whether the quantized index returns results similar to the float baseline, we use recall:

recall@k = | BBQ top-k ∩ float32 top-k | / k, averaged over all queries.

We vary the oversampling factor (num_candidates / k) that’s the number of candidates BBQ scans with 1-bit vectors before reranking the top ones against the original floats to find the lowest value that still matches float32.

As a result, we have:

BBQ starts at 0.994 recall@10 at 1x oversampling, holds there up to 3x, and then settles at 0.989 at higher factors, meaning it returns at least 98.9% of the same top-10 documents as float32 across all oversampling values. For more on how recall varies across datasets under quantization, see Fast vs. accurate: Measuring the recall of quantized vector search.

BBQ quantization results summary

The same vectors, two storage formats, and one experiment:

• Disk: Roughly the same (12.80 MB versus 13.25 MB). BBQ keeps the raw floats around for rescoring and merging.

• Memory: 29x smaller (12.71 MB versus 0.44 MB). This is the number that decides whether your cluster fits the corpus.

• Recall@10: 0.994 at 1x oversampling. Quantization-aware training pays off.

When to enable BBQ: If your dimension count is above the 384-dim floor, if your vectors are the dominant memory cost, and if you can afford a few extra candidates to rescore. For Jina v5 specifically, the model is trained for it, so the recall hit on most corpora is small.

Further reading on BBQ and vector quantization

• Run the full notebook from this article in the supporting blog content repo.

• For the math behind BBQ, see Better Binary Quantization in Lucene and Elasticsearch.

• For more on Jina v5's architecture, see Jina embeddings v5 on Search Labs.

• For a broader walkthrough on adopting BBQ, see How to implement BBQ into your use case.

• For the original research behind BBQ, see the paper RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Search.

Prerequisites

• Elastic Cloud deployment running Elasticsearch 9.3+ or start a free trial

• Workflows enabled (Advanced Settings)

• Python 3.9+

• A Mistral API key

How LLM query routing works in this system

We're going to build a two-stage system: a router that decides how to answer, and an answering model that produces the response.

The router looks at the query and the metadata of the top search hits, like scores, categories, and complexity labels. From that, it picks one of two strategies: Answer directly from the top FAQ article, or synthesize across multiple articles with citations. That decision can be made from structured signals alone, so a small, fast model handles it.

The answering step varies. A single-article answer is bounded work that a small model does well and returns quickly. A multisource synthesis with citations benefits from a more capable model, and the extra time is worth it. Matching each query to the model that fits keeps simple answers fast and complex answers good.

Why use a small model for routing instead of the large model?

Because the router runs on every query, including the simple ones. A slow router makes every answer slow, even the ones a small model could have produced in a fraction of the time.

The key design choice is that the routing step only sees metadata, not full documents. A query like "my OTG isn't heating evenly" only needs to know that the top hits are in the "Product Troubleshooting - Appliances" category with issue_complexity: medium, not the full conversation transcripts. This keeps the classification prompt tiny (a few hundred tokens) and cheap. The full article content is only loaded in the response step once.

Set up AI connectors

We use two AI connectors for the workflow:

Connector

Model

Type

Role

Mistral Small

mistral-small-latest

Custom (OpenAI-compatible)

Classify query complexity from metadata, answer simple FAQ-style questions

Anthropic Claude Sonnet 4.6

Claude Sonnet

Elastic Managed LLM

Synthesize complex answers from multiple articles, with citations

Both connectors are billed per million tokens, with the smaller model costing significantly less. Routing simple queries to it saves money on top of the latency win. To learn more about Elastic Managed LLM, see this documentation.

The Claude Sonnet connector is already available as an Elastic Managed large language model (LLM). We only need to create a custom connector for Mistral using the .gen-ai connector type, which supports any OpenAI-compatible API. You can also create it through the Kibana UI.

All the setup code in this article is available in the companion notebook. You can run each section there as you follow along.

The connector ID is auto-generated by Kibana. We let the platform handle this instead of trying to set it manually.

Once created, the connector appears in the Connectors UI:

Load and index the dataset

We use the e-commerce-customer-support-qa dataset from Hugging Face. It contains 1,000 real customer support interactions from an ecommerce platform (BrownBox) with customer questions, agent solutions, issue categories, complexity levels, and customer sentiment.

The index mapping uses semantic_text with the .jina-embeddings-v5-text-small model from Elastic Inference Service. This field handles semantic search end-to-end: embedding generation, chunking, and querying. We use copy_to to aggregate the conversation and QA summary into a single searchable field:

Defining the query routing workflow in Elastic Workflows YAML

The routing workflow has four steps: semantic search, metadata-only classification, conditional branching, and a model-appropriate response step.

We use Elastic Workflows to encapsulate this routing logic. Workflows let us:

1. Expose the triaging as a tool in Elastic Agent Builder, so a conversational agent can call it.

2. Trigger it directly via manual execution, schedules, or alerts.

This flexibility means the same logic serves both programmatic and conversational interfaces without duplicating code.

Workflows are defined in YAML and configured directly in the Workflow UI (Elasticsearch > Workflows > Create a New Workflow). Each step can query Elasticsearch, call Kibana APIs, or prompt an LLM.

Here’s the complete workflow definition:

The workflow has four key parts:

1. search_es uses elasticsearch.search with a semantic query to find the five most relevant articles.

2. classify_querysends the customer query plus only metadata from the search results to Mistral Small. The prompt includes scores, categories, complexity labels, and product categories. This keeps the classification step cheap, preventing the use of large amounts of tokens.

3. route_by_complexity uses an if step to branch based on the classifier's output.

4. The response step depends on the route. For simple queries, Mistral Small gets the top FAQ article and rephrases it. For complex queries, Claude Sonnet gets all five articles and synthesizes a detailed response with citations. This is the only step where full document content is loaded.

Using the workflow as a tool in Agent Builder

Beyond the default triggers (manual, schedule, alerts), workflows can also be exposed as tools in Agent Builder. This adds a conversational layer where users interact through a chat interface, and the agent decides when to call the workflow.

We use the Agent Builder APIs to create the tool and the agent. After creating the workflow in the Kibana UI, copy its ID and use it to register the workflow as a tool:

Then create an agent that uses the tool:

The agent is now available in the Agent Builder UI in Kibana. You can also create the agent and its tools directly through the Agent Builder UI.

Once created, the agent appears in the Agent Builder UI with the workflow tool assigned:

Testing simple vs. complex query routing

Simple query

The workflow searches the knowledge base, finds a direct match in the FAQ articles about order tracking, classifies it as simple, and routes to Mistral Small. The response is concise and drawn directly from the matched article: instructions for using the "My Orders" section or the tracking number from the confirmation email.

Complex query

This query involves three distinct issues (damaged product, duplicate charge, address update) across different support categories. The workflow classifies it as complex and routes to Claude Sonnet, which synthesizes information from multiple knowledge base articles, addresses each issue separately, cites the relevant articles, and provides clear resolution steps for each.

Conclusion

Routing LLM queries by complexity in Elasticsearch reduces latency and cost for simple queries without sacrificing quality on complex ones. The small model answers FAQ-style queries in a fraction of the time the larger model would take, and the larger model is reserved for the queries that actually benefit from its capabilities. Cost savings come along for the ride: Simple queries routed to the smaller model are cheaper, too.

The pattern that makes this work is searching the knowledge base before routing. Without that context, the router is guessing based on surface-level cues. With it, the structure of the search results, like scores, categories, and complexity labels, tells the router whether the answer lives in a single article or needs synthesis across several. That's the actual signal for how to handle the query.

Elastic Workflows makes this possible without writing orchestration code. The entire routing logic lives in YAML inside Kibana, using native steps for search, LLM prompts, and conditional branching. Combined with Agent Builder, the same workflow serves programmatic triggers and conversational interfaces.

Next steps

• Try the notebook with the complete implementation.

• Add LLM monitoring with OpenRouter to track cost per routing tier.

• Explore Elastic Workflows for other automation patterns.

• Learn more about Agent Builder and how to expose workflows as conversational tools.

• Read about building AI agentic workflows with Elastic Agent Builder.

What is recall?

Recall measures on a scale from 0 to 1 how many of the documents that your users actually want appear somewhere in your search results. If a query should surface three products and your search returns only two of them in the top 10, recall@10 = 0.67 for that query. It’s a set-based metric: It doesn’t care about the position of the relevant documents within those k results. A relevant document in position 10 counts the same as one in position 1. Having a high recall means that you’re not losing relevant results.

The diagram shows two sets: all relevant documents (left) and what BM25 actually retrieved (top 10, right). Only the intersection counts toward recall, prod_1 and prod_2 were found, while prod_3, prod_4, and prod_6 were missed entirely. Result: Recall@10 = 2/5 = 0.40.

Prerequisites

Let's get down to business to better understand how recall works. This demonstration uses Python. You can follow along with it on the companion notebook (notebook.ipynb), where every code block is a cell ready to run.

The code provided uses the following:

• Elasticsearch 9.3+

• Python 3.10+

• A .env file with your Elasticsearch credentials

The dataset

We’ll use a product catalog of 1,000 products, spanning categories such as footwear, electronics, tools, and more.

Each document has four fields:

Field

Type

`title`

text

`description`

text

`brand`

keyword

`category`

keyword

The dataset is loaded from dataset.csv.

The power and limits of lexical search

BM25 is the default ranking algorithm in Elasticsearch and most search engines. It scores documents by how often your query terms appear in them, adjusted for document length and the frequency of those terms across the entire index. You get analyzers on top: lowercase normalization, stemming, and stopword removal. A query for "running shoes" will match "Running Shoes" and likely "run" as well.

This works well for a large class of queries:

• "running shoes" immediately matches products with those exact tokens in the title.

• "bluetooth speaker" surfaces portable audio products because the tokens appear verbatim.

The results are deterministic and explainable: A document ranks highly because the query terms appear in it. Debugging relevance is straightforward.

Where it breaks

Now let’s try these queries against the same catalog:

• "skincare routine": The word "routine" doesn’t appear in any product title. BM25 can partially match on "skincare," but face serums, body oils, and moisturizers are described using terms like "vitamin C," "retinol," or "brightening," none of which overlap with the query. Products that form a complete skincare routine are scattered across the index with no shared token to anchor them.

• "pet travel accessories": This is a use-case grouping, not a product category. A dog sling carrier, a pet car seat, and a travel crate are all relevant, but their descriptions talk about portability, safety, and comfort rather than "travel accessories." BM25 matches "pet" broadly but has no signal to distinguish travel-specific products from the rest of the pet catalog.

This is a recall problem. The relevant documents exist in your index. BM25 just cannot find them because the user's words and the document's words do not match closely enough.

Adding synonyms helps for known cases. But you cannot enumerate every way a user might express an intent. That is where vectors come in.

Why you should measure recall

Before fixing a problem, you need to quantify it.

Recall@k measures how many of the documents that your users actually want appear somewhere in your search results. Formally:

Precision@k measures the top k results and how many are actually relevant:

High precision means that the results you do return are good. In ecommerce, missing a relevant product (low recall) is often worse than showing a slightly imperfect result (lower precision), because a hidden product is a lost sale.

Elasticsearch's rank_eval API lets you measure both systematically. You provide a list of queries, each with a set of rated documents, and Elasticsearch computes the metrics for you across all queries.

Setting up the evaluation

The rank_eval API needs a ratings dataset: a mapping of queries to the documents that are relevant for each one, along with a relevance grade (0 = not relevant, 1 = relevant, 2 = highly relevant).

In the notebook, this is the judgments list:

The mix is intentional: q1 is a query that BM25 handles well (exact tokens in product titles), while q2, q3, and q4 are intent-based queries where the user's intent is expressed as a concept rather than specific product keywords.

Measuring BM25 baseline recall

First, set up the Elasticsearch client and index the raw text data:

Now build the rank_eval request for BM25. Each request in the list combines a query with its ratings:

Result:

0.43 means that across all four queries, BM25 finds only 43% of the documents it should find. The shortfall is concentrated in the intent-based queries: "skincare routine" misses face serums and body oils because "routine" never appears in product titles, and "pet travel accessories" retrieves off-topic pet products while missing carriers and crates described in terms of portability and safety rather than "travel accessories."

This is our baseline. Now we have a number to beat.

Adding vector search with Jina embeddings

Vector search encodes documents and queries as high-dimensional vectors, a type of vector made up of hundreds or thousands of numerical values, each encoding a specific feature of the data it represents. Documents with similar meaning end up close together in vector space, even if they share no words. "Gym equipment" and "dumbbell set" will be nearby because the concepts are related. I chose Elasticsearch as my vector database because it supports hybrid search, giving me both semantic understanding and keyword precision out of the box.

EIS includes out-of-the-box support for embedding models through its inference API.

Step 1: Using Jina embeddings v5 as an inference endpoint

If your cluster has GPU resources (available in Elastic Cloud and Elasticsearch 9.3+), the embeddings are generated on GPU, which is significantly faster than CPU inference and removes the performance trade-off that historically made vectors expensive at scale.

Why Jina embeddings specifically? jina-embeddings-v5-text is a multilingual model (119+ languages) with a 32,000-token context window and support for task-specific Low-Rank Adaptation (LoRA) adapters. It works well for short product descriptions out of the box. Read more about jina-embeddings-v5-text model here.

Step 2: Create the index with a semantic field

The semantic_text field type is the key here. It’s a higher-level abstraction over dense_vector: You point it at an inference endpoint, and Elasticsearch takes care of generating embeddings automatically.

The copy_to property on title and description means content from both fields flows into semantic_field for embedding, so a single vector captures the full product representation.

Step 3: Index the products

At index time, Elasticsearch calls the inference endpoint for each document and stores the resulting embedding in semantic_field. No extra code on your side.

Hybrid search: Combining BM25 and vectors with RRF

Adding vectors improves recall, but using vectors alone risks losing precision on exact-match queries; "running shoes" should still rank verbatim matches first. Hybrid search retains the lexical component specifically to preserve that precision.

Hybrid search with Reciprocal Rank Fusion (RRF) keeps the best of both:

• BM25 handles exact and near-exact queries with high precision.

• Semantic search handles intent-based and multilingual queries with high recall.

• RRF combines the two ranked lists into a single ranking.

The RRF formula assigns each document a score based on its rank in each result list:

A document that ranks highly in both lists gets a higher combined score. The rank_constant controls how much weight lower-ranked documents receive.

Result:

Hybrid improves substantially over BM25 (0.43) and preserves precision for exact-match queries like "running shoes."

Results: Before and after

Here’s the full comparison across all three approaches:

Result:

Method

Recall@10

BM25 (Lexical)

0.43

Hybrid (BM25 + Vectors)

0.75

Breaking it down by query:

Conclusion

Throughout this post, we saw that BM25 lexical search is reliable when users type exact queries, but it loses recall when they search by intent rather than keywords. Using rank_eval, we established a reproducible baseline to measure that gap with real numbers. From there, we added a semantic_text field powered by Jina embeddings and ran the evaluation again. The result: Hybrid search improved recall from 0.43 to 0.75 while preserving precision on exact-match queries, though the actual margin will depend on your query mix.

The pattern scales beyond this example: Collect judgments from your users' actual queries, run rank_eval as a baseline, add semantic_text, and measure again. You'll know exactly what improved and by how much.

Next steps

• Dive deeper into recall and vector search: Recall and vector search quantization by Jeff Vestal

• Add reranking for even better precision on the top results

• Explore Elasticsearch hybrid search documentation

• Read more about the rank_eval API

Prerequisites

• Elasticsearch 8.15+ (for :: cast operator support; core ES|QL features available from 8.11)

Runtime fields versus ES|QL

Runtime fields were introduced in Elasticsearch 7.11 as a way to define fields at query time. Instead of reindexing data, you could write a Painless script that computes values on the fly:

This works, but comes with trade-offs:

• Painless scripting overhead: Every runtime field requires scripting knowledge, and the syntax is Java-like, not query-like.

• Performance cost: Runtime fields evaluate per document at query time. Elasticsearch classifies them as "expensive queries" that can be rejected by cluster settings.

• Isolated computation: Each runtime field computes independently. There’s no way to chain transforms or use the output of one field in another within the same query.

ES|QL changes the equation. It has its own execution engine (not translated to Query DSL), runs queries concurrently across nodes, and provides a complete toolkit for field computation: EVAL, GROK, DISSECT, type casting, and pipeline chaining.

Let's see how each runtime field pattern maps to ES|QL.

Setting up the example data

All the code snippets in this article can be executed in the Kibana Dev Tools console.

To follow along, create a sample index with data that exercises all five patterns. This simulates a server logs scenario with mixed field types, raw messages, and some intentional data quality issues:

Now index some sample documents:

Notice that response_time is stored as a keyword (a common real-world mistake), and the last document has "US-EAST" instead of "us-east" (a data quality issue we’ll fix later).

Pattern 1: Field concatenation

A common runtime field use case is combining two fields into one. For example, creating a host:port identifier.

The runtime field approach

You can define it inline at query time. Query-time approach avoids modifying the mapping, but you still need Painless scripting, scoping it to a single search request:

The ES|QL approach

You can run ES|QL queries using the _query API endpoint:

Response:

CONCAT accepts two or more arguments and always returns a keyword.

Note: For brevity, the remaining ES|QL examples in this article show just the query. Wrap them in POST _query { "query": "..." } to run them in Kibana Dev Tools.

When to use

If you need endpoint to persist across all queries and be available in Kibana dashboards, use a mapping-level runtime field. If you need it for a single search request within Query DSL, use a query-time runtime field. If you need it for ad-hoc analysis or exploratory work, ES|QL is simpler.

Pattern 2: Data extraction from unstructured text

Extracting structured data from raw log messages is another classic runtime field pattern.

The runtime field approach

Painless uses Java's regex Matcher class:

This is verbose. You need to know Painless regex syntax, handle the Matcher object, and call emit() correctly.

The ES|QL approach: GROK

ES|QL provides two purpose-built commands for text extraction. GROK uses regex-based patterns:

Response:

GROK uses the %{SYNTAX:SEMANTIC} pattern format. It extracts multiple fields in a single and readable command.

The ES|QL approach: DISSECT

For structured data with consistent delimiters, DISSECT is faster because it doesn’t use regular expressions:

The syntax is nearly identical to GROK, but DISSECT works by splitting on delimiters rather than matching regex patterns. This makes it faster for data that follows a consistent format.

When to use GROK vs DISSECT

Use DISSECT when your data has a predictable structure (same delimiters, same field order). Use GROK when you need regex flexibility, for example when fields may be optional or formats vary.

Pattern 3: Dynamic type conversion

When a field is mapped as keyword but contains numeric data (a surprisingly common scenario), runtime fields can cast it at query time.

The runtime field approach

You need to handle parsing exceptions manually. If Long.parseLong fails on an unexpected value, the script throws an error.

The ES|QL approach

ES|QL provides explicit conversion functions and a shorthand cast operator:

Or with the :: cast operator (available since 8.15):

Response:

Both produce the same result. The key difference from Painless: Failed conversions return null instead of throwing exceptions. The document with "not_available" simply gets null for response_ms, and ES|QL emits a warning.

Common conversion functions include:

Function

Converts to

`TO_LONG()`

Long integer

`TO_INTEGER()`

Integer

`TO_DOUBLE()`

Double

`TO_DATETIME()`

Date

`TO_BOOLEAN()`

Boolean

`TO_IP()`

IP address

`TO_VERSION()`

Version

The :: operator works with all these types (for example, field::double, field::datetime).

When to use

ES|QL's graceful null handling makes it safer for dirty data. Runtime fields with Painless give you fine-grained control over error handling but require more code. For type conversion specifically, ES|QL is almost always the better choice.

Pattern 4: Dynamic field handling

Runtime fields support "dynamic": "runtime" in mappings, which prevents mapping explosion by creating all new fields as runtime fields instead of indexed fields:

Any new field sent to this index becomes a runtime field automatically. This is useful when you ingest semi-structured data with unpredictable field names.

Where ES|QL fits

ES|QL provides query-time flexibility, but it still needs fields to be visible in the mapping. This is where runtime fields and ES|QL complement each other rather than compete.

If a field exists in _source but isn’t mapped, ES|QL cannot access it directly. The current workaround is to define a runtime field to make the unmapped field visible:

Once defined, ES|QL can query it:

This is one scenario where runtime fields remain essential. They act as a bridge, making unmapped data accessible to ES|QL.

Pattern 5: Field shadowing for error correction

Runtime fields can shadow (override) indexed fields by defining a runtime field with the same name as an existing field. This is useful for correcting data without reindexing.

The runtime field approach

Remember our data quality issue, where region has inconsistent casing ("US-EAST" versus "us-east")?

This overrides the indexed region field for all queries. Every search, aggregation, and Kibana visualization will see the lowercase version.

When you use EVAL with an existing column name, ES|QL drops the original column and replaces it with the computed value. This is the exact equivalent of field shadowing, but scoped to the current query.

You can also chain multiple corrections in a pipeline:

When to use

If the correction should apply to all queries and Kibana dashboards, use runtime field shadowing. If you need to correct data for a specific analysis, ES|QL is more flexible since you can apply different transformations in different queries without modifying the mapping.

The ES|QL pipeline advantage: Going beyond runtime fields

This is where ES|QL fundamentally surpasses runtime fields. Runtime fields are isolated: each one computes independently, and you cannot use the output of one runtime field as input for another in the same query.

ES|QL pipelines chain transforms. Here’s a single query that combines multiple patterns:

This single query:

• Extracts fields from raw text (GROK).

• Converts the duration to a number (EVAL with cast).

• Normalizes region casing (EVAL with TO_LOWER).

• Filters for errors with high duration (WHERE).

• Aggregates by region (STATS).

To achieve the same result with runtime fields, you would need to define at least three separate runtime fields (for extraction, conversion, and normalization) and then write a Query DSL query with filters and aggregations. The ES|QL version is a single, readable pipeline.

You can even use expressions directly inside aggregations:

Conclusion

What we covered:

• ES|QL provides a full toolkit (EVAL, GROK, DISSECT, type casting with ::) that replaces most runtime field patterns without any Painless scripting.

• Failed type conversions in ES|QL return null instead of throwing exceptions, making it safer for real-world data.

• Pipeline processing (chaining GROK into EVAL into WHERE into STATS) goes beyond what runtime fields can do in isolation.

• Runtime fields remain valuable for persistent computed fields, field shadowing across all queries, and as a bridge for unmapped data in ES|QL.

One important caveat: Both runtime fields and ES|QL compute values at query time, which means they pay the cost on every query. If you find yourself applying the same transformation repeatedly (type corrections, field extraction, data normalization), consider using ingest pipelines to fix the data at index time instead. Ingest pipelines let you parse, enrich, and transform documents before they’re stored, so queries can work with clean, properly typed fields directly. Runtime fields and ES|QL are great for exploration and ad-hoc analysis, but for production workloads, indexing the right data from the start is almost always the better choice.

The key takeaway: Runtime fields aren’t deprecated, and they aren’t going away. But for most query-time computation patterns, ES|QL offers a simpler, more powerful, and more performant approach. And when the transformation is known up front, an ingest pipeline is the most efficient option of all.

Next steps

• ES|QL documentation

• Runtime fields reference

• ES|QL timeline of improvements

• Getting started with runtime fields

• ES|QL processing data with DISSECT and GROK

In this article, we’ll explore the benefits of building a custom Elasticsearch MCP server and show how to create one in TypeScript that connects Elasticsearch to LLM-powered applications.

Why build a custom Elasticsearch MCP server?

Elastic provides some alternatives for MCP servers:

• Elastic Agent Builder MCP server for Elasticsearch 9.2+

• Elasticsearch MCP server for older versions (Python)

If you need more control over how your MCP server interacts with Elasticsearch, building your own custom server gives you the flexibility to tailor it exactly to your needs. For example, Agent Builder's MCP endpoint is limited to Elasticsearch Query Language (ES|QL) queries, while a custom server allows you to use the full Query DSL. You also gain control over how results are formatted before being passed to the LLM and can integrate additional processing steps, like the OpenAI-powered summarization we'll implement in this tutorial.

By the end of this article, you’ll have an MCP server in TypeScript that searches for information stored in an Elasticsearch index, summarizes it, and provides citations. We'll use Elasticsearch for retrieval, OpenAI's gpt-4o-mini model to summarize and generate citations, and Claude Desktop as the MCP client and UI to take in user queries and give responses. The end result is an internal knowledge assistant that helps engineers discover and synthesize best practices across their organization’s technical docs.

Prerequisites:

• Node.js 20 +

• Elasticsearch

• OpenAI API key

• Claude Desktop

What is MCP?

MCP is an open standard, created by Anthropic, that provides secure, bidirectional connections between LLMs and external systems, like Elasticsearch. You can read more about the current state of MCP in this article.

The MCP landscape is evolving every day, with servers available for a wide range of use cases. On top of that, it’s easy to build your own custom MCP server, as we’ll show in this article.

MCP clients

There’s a long list of available MCP clients, each with its own characteristics and limitations. For simplicity and popularity, we’ll use Claude Desktop as our MCP client. It will serve as the chat interface where users can ask questions in natural language, and it will automatically invoke the tools exposed by our MCP server to search documents and generate summaries.

Creating an Elasticsearch MCP server

Using the TypeScript SDK, we can easily create a server that understands how to query our Elasticsearch data based on a user query input.

Here are the steps in this article to integrate the Elasticsearch MCP server with the Claude Desktop client:

1. Configure MCP server for Elasticsearch.

2. Load the MCP server into Claude Desktop.

3. Test it out.

Configure MCP server for Elasticsearch

To begin, let's initialize a node application:

This will create a package.json file, and with it, we can start installing the necessary dependencies for this application.

• @elastic/elasticsearch will give us access to the Elasticsearch Node.js library.

• @modelcontextprotocol/sdk provides the core tools to create and manage an MCP server, register tools, and handle communication with MCP clients.

• openai allows interaction with OpenAI models to generate summaries or natural language responses.

• zodhelps define and validate structured schemas for input and output data in each tool.

ts-node, @types/node, and typescript will be used during development to type the code and compile the scripts.

Set up the dataset

To provide the data that Claude Desktop can query using our MCP server, we’ll use a mock internal knowledge base dataset. Here’s what a document from this dataset will look like:

To ingest the data, we prepared a script that creates an index in Elasticsearch and loads the dataset into it. You can find it here.

MCP server

Create a file named index.ts and add the following code to import the dependencies and handle environment variables:

Also, let’s initialize the clients to handle the Elasticsearch and OpenAI calls:

To make our implementation more robust and ensure structured input and output, we'll define schemas using zod. This allows us to validate data at runtime, catch errors early, and make the tool responses easier to process programmatically:

Learn more about structured outputs here.

Now let’s initialize the MCP server:

Defining the MCP tools

With everything configured, we can start writing the tools that will be exposed by our MCP server. This server exposes two tools:

• search_docs: Searches for documents in Elasticsearch using full-text search.

• summarize_and_cite: Summarizes and synthesizes information from previously retrieved documents to answer a user question. This tool also adds citations referencing the source documents.

Together, these tools form a simple “retrieve-then-summarize” workflow, where one tool fetches relevant documents and the other uses those documents to generate a summarized, cited response.

Tool response format

Each tool can accept arbitrary input parameters, but it must respond with the following structure:

• Content: This is the response of the tool in an unstructured format. This field is usually used to return text, images, audio, links, or embeddings. For this application, it will be used to return formatted text with the information generated by the tools.

• structuredContent: This is an optional return used to provide the results of each tool in a structured format. This is useful for programmatic purposes. Although it isn't used in this MCP server, it can be useful if you want to develop other tools or process the results programmatically.

With that structure in mind, let’s dive into each tool in detail.

Search_docs tool

This tool performs a full-text search in the Elasticsearch index to retrieve the most relevant documents based on the user query. It highlights key matches and provides a quick overview with relevance scores.

We configure fuzziness: “AUTO” to have a variable typo tolerance based on the length of the token that’s being analyzed. We also set title^2 to increase the score of the documents where the match happens on the title field.

summarize_and_cite tool

This tool generates a summary based on documents retrieved in the previous search. It uses OpenAI’s gpt-4o-mini model to synthesize the most relevant information to answer the user’s question, providing responses derived directly from the search results. In addition to the summary, it also returns citation metadata for the source documents used.

Finally, we need to start the server using stdio. This means the MCP client will communicate with our server by reading and writing to its standard input and output streams. stdio is the simplest transport option and works well for local MCP servers launched as subprocesses by the client. Add the following code at the end of the file:

Now compile the project using the following command:

This will create a dist folder, and inside it, an index.js file.

Load the MCP server into Claude Desktop

Follow this guide to configure the MCP server with Claude Desktop. In the Claude configuration file, we need to set the following values:

The args value should point to the compiled file in the dist folder. You also need to set the environment variables in the configuration file with the exact same names defined in the code.

Test it out

Before executing each tool, click on Search and Tools to make sure that the tools are enabled. Here you can also enable or disable each one:

Finally, let’s test the MCP server from the Claude Desktop chat and start asking questions:

For the question “Search for documents about authentication methods and role-based access control”, the search_docs tool is executed and returns the following results:

The response is, “Great! I found 5 relevant documents about authentication methods and role-based access control. Here's what was found:”

The tool call returns the source documents as part of its response payload, which are later used to generate citations.

It’s also possible to chain multiple tools in a single interaction. In this case, Claude Desktop analyzes the user's question and determines that it needs to first call search_docs to retrieve relevant documents and to then pass those results to summarize_and_cite to generate the final answer, all without requiring separate prompts from the user:

In this case, for the query “What are the main recommendations to improve authentication and access control across our systems? Include references.”, we obtained the following results:

As in the previous step, we can see the response from each tool for this question:

Note: If a submenu appears asking whether you approve the use of each tool, select Always allow or Allow once.

Conclusion

MCP servers represent a significant step toward standardizing LLM tools for both local and remote applications. Though full compatibility is still in the works, we’re moving fast in that direction.

In this article, we learned how to build a custom MCP server in TypeScript that connects Elasticsearch to LLM-powered applications. Our server exposes two tools: search_docs for retrieving relevant documents using Query DSL; and summarize_and_cite for generating summaries with citations via OpenAI models and Claude Desktop as client UI.

The future of compatibility between different client and server providers looks promising. Next steps include adding more functionalities and flexibility to your agent. There’s a practical article on how you can parameterize your queries using search templates to gain precision and flexibility.

Prerequisites

• Elasticsearch 9.3 or Elastic Cloud Serverless: You can create a cloud deployment following these instructions, or you can use the start-local quickstart instead.

• Python 3.12: Download Python here.

• Hugging Face access token.

Chat completions using a Hugging Face inference endpoint

First, we’ll build a practical example that connects Elasticsearch to a Hugging Face inference endpoint to generate AI-powered recommendations from a collection of blog posts. For the app knowledge base, we’ll use a dataset of company blog articles, which contains valuable but often hard-to-navigate information.

With this endpoint, semantic search retrieves the most relevant articles for a given query, and a Hugging Face LLM generates short, contextual recommendations based on those results.

Let’s take a look at a high-level overview of the information flow we’re going to build:

In this article, we’ll test SmolLM3-3B capacity tocombine its compact size with strong multilingual reasoning and tool-calling capabilities. Based on a search query, we’ll send all the matching content (in English and Spanish) to the LLM to generate a list of recommended articles with a custom-made description based on the search query and results.

Here’s what the UI of an article site with an AI recommendations generation system could look like.

You can find the full implementation of this application in the linked notebook.

Configuring Elasticsearch inference endpoints

To use the Elasticsearch Hugging Face inference endpoint, we need two important elements: a Hugging Face API key and a running Hugging Face endpoint URL. It should look like this:

The Hugging Face inference endpoint in Elasticsearch supports different task types: text_embedding, completion, chat_completion, and rerank. In this blog post, we use chat_completion because we need the model to generate conversational recommendations based on the search results and a system prompt.This endpoint allows us to perform chat completions directly from Elasticsearch in a simple way using the Elasticsearch API:

This will serve as the core of the application, receiving the prompt and the search results that will pass through the model. With the theory covered, let’s start implementing the application.

Setting up ​​inference endpoint on Hugging Face

To deploy the Hugging Face model, we’re going to use Hugging Face one-click deployments, an easy and fast service for deploying model endpoints. Keep in mind that this is a paid service, and using it may incur additional costs. This step will create the model instance that will be used to generate the recommendations of the articles.

You can pick a model from the one-click catalog:

Let’s pick the SmolLM3-3B model:

From here, grab the Hugging Face endpoint URL:

As mentioned in the Elasticsearch Hugging Face inference endpoints documentation, text generation requires a model that’s compatible with the OpenAI API. For that reason, we need to append the /v1/chat/completions subpath to the Hugging Face endpoint URL. The final result will look like this:

With this in place, we can start coding in a Python notebook.

Generating Hugging Face API key

Create a Hugging Face account, and obtain an API token by following these instructions. You can choose between three token types: fine-grained (recommended for production, as it provides access only to specific resources); read (for read-only access); or write (for read and write access). For this tutorial, a read token is sufficient, since we only need to call the inference endpoint. Save this key for the next step.

Setting up Elasticsearch inference endpoint

First, let’s declare an Elasticsearch Python client:

Next, let’s create an Elasticsearch inference endpoint that uses the Hugging Face model. This endpoint will allow us to generate responses based on the blog posts and the prompt passed to the model.

Dataset

The dataset contains the blog posts that will be queried, representing a multilingual content set used throughout the workflow:

Elasticsearch mappings

With the dataset defined, we need to create a data schema that properly fits the blog post structure. The following index mappings will be used to store the data in Elasticsearch:

Here, we can see more clearly how the data is structured. We’ll use semantic search to retrieve results based on natural language, along with the copy_to property to copy the field contents into the semantic_text field. Additionally, the title field contains two subfields: the original subfield stores the title in either English or Spanish, depending on the original language of the article; and the translated_title subfield is present only for Spanish articles and contains the English translation of the original title.

Ingesting data

The following code snippet ingests the blog posts dataset into Elasticsearch using the bulk API:

Now that we have the articles ingested into Elasticsearch, we need to create a function capable of searching against the semantic_text field:

We also need a function that calls the inference endpoint. In this case, we’ll call the endpoint using the chat_completiontask type to get streaming responses:

Now we can write a function that calls the semantic search function, along with the chat_completions inference endpoint and the recommendations endpoint, to generate the data that will be allocated in the cards:

Finally, we need to extract the information and format it to be printed:

Let’s test this by asking a question about the security blog posts:

Here we can see the cards in the console generated by the workflow:

You can see the full results, including all hits and the LLM response, in this file.

We’re asking for articles related to: “Security and vulnerabilities.” This question is used as the search query against the documents stored in Elasticsearch. The retrieved results are then passed to the model, which generates recommendations based on their content. As we can see, the model did a great job generating engaging short text that can motivate the reader to click on it.

Conclusion

This example shows how Elasticsearch and Hugging Face can be combined to create a fast and efficient centralized system for AI applications. This approach reduces manual effort and provides flexibility, thanks to Hugging Face’s extensive model catalog. Using SmolLM3-3B, in particular, shows how compact, multilingual models can still deliver meaningful reasoning and content generation when paired with semantic search. Together, these tools offer a scalable and effective foundation for building intelligent content analysis and multilingual applications.

Model Context Protocol (MCP) is an open-source standard that enables applications and LLMs to communicate with external systems via MCP tools (programmatic capabilities), and LangGraph (an extension of LangChain) provides the orchestration framework for these agentic workflows.

We’ll implement an application that can search both internal knowledge (Elasticsearch stored data) and external sources (on the internet) to identify potential and known vulnerabilities related to a specific tool. The application will gather the information and generate a detailed summary of the findings.

Requirements

• Elasticsearch 9.2

• Python 3.1x

• OpenAI API Key

• Elasticsearch API Key

• Serper API Key

Elastic Agent Builder

Elastic Agent Builder is a set of AI-powered capabilities for developing and integrating agents that can interact with your Elasticsearch data. It provides a built-in agent that can be used for natural language conversations with your data or instance, and it also supports tool creation, Elastic APIs, A2A, and MCP. In this article, we will focus on using the MCP server for external access to the Elastic Agent Builder tools.

To know more about Agent Builder features, you can read this article.

Agent Builder MCP feature

The MCP server is available in the Agent Builder and can be accessed at:

The Agent Builder offers Built-in tools, and you can also create your custom tools.

Reference architecture

To get a complete overview of the elements used by an agentic application in an end-to-end workflow, let’s look at the following diagram:

Elasticsearch is at the center of this architecture, functioning as a vector store, providing the embeddings generation model, and also serving the MCP server to access the data via tools. To better explain the workflow, let’s look at the ingestion and the Agent Builder layer separately.

Here, the first element is the data that will be stored in Elasticsearch. The data passes through an ingest pipeline, where it is processed by the Elasticsearch ELSER model to generate embeddings and then stored in Elasticsearch.

Elastic Agent Builder layer

On this layer, the Agent Builder plays a central role by exposing the tools needed to interact with the Elasticsearch data. It manages the tools that operate over Elasticsearch indices and makes them available for consumption. Then LangChain handles the orchestration via the MCP client.

This architecture allows Agent Builder to work as one of many MCP servers available to the client so that the Elasticsearch agent builder can combine with other MCPs. This way, the MCP client can ask cross-source questions and then combine the answers.

Use case: Security vulnerability agent

The security vulnerability agent identifies potential risks based on a user’s question by combining three complementary layers:

First, it performs a semantic search with embeddings over an internal knowledge base of past incidents, configurations, and known vulnerabilities to retrieve relevant historical evidence.

Second, it searches the internet for newly published recommendations or threat intelligence that may not yet exist internally.

Finally, an LLM correlates and prioritizes both internal and external findings, evaluates their relevance to the user’s specific environment, and produces a clear explanation along with potential mitigation steps.

Developing the application

The application’s code can be found in the attached notebook.

You can see the setup for the Python application below:

We need to access Agent Builder and create one agent specialized in security queries and one tool to perform semantic search. You need to have the Agent Builder enabled for the next step. Once it’s on, we’ll use the tools API to create a tool that will perform a semantic search.

Configure your tools following the best practices defined by Elastic for developing Tools. Once created, this tool will be ready to use in the Kibana UI.

With the tool created, we can start writing the code for the ingestion workflow:

Ingest pipeline

To define the data structure, we need to have a dataset prepared for ingestion. Below is a sample document for this example:

For this type of document, we will use the following index mappings:

We are creating a semantic_text field to perform semantic search using the information from the fields marked with the copy_to property.

With that mapping definition, we can ingest the data using the bulk API.

LangChain MCP client

Here we’re going to create an MCP client using LangChain to consume the Agent Builder tools and build a workflow with LangGraph to orchestrate the client execution. The first step is to connect to the MCP server:

Next, we create an agent that selects the appropriate tool based on the user input:

We’ll use the GPT-5.2 model, which represents OpenAI’s state-of-the-art for agent management tasks. We configure it with low reasoning effort to achieve faster responses compared to the medium or high settings, while still delivering high-quality results by leveraging the full capabilities of the GPT-5 family. You can read more about the GPT 5.2 here.

Now that the initial setup is done, the next step is to define a workflow capable of making decisions, running tool calls, and summarizing results.

For this, we use LangGraph. We won’t cover LangGraph in depth here; this article provides a detailed overview of its functionality.

The following image shows a high-level view of the LangGraph application.

We need to define the application state:

To better understand how the workflow operates, here is a brief description of each function. For full implementation details, refer to the accompanying notebook.

• call_agent_builder_semantic_search: Queries internal documentation using the Agent Builder MCP server and also stores the retrieved messages in the state.

• decide_internet_search: Analyzes the internal results and determines whether an external search is required.

• perform_internet_search: Runs an external search using the Serper API when needed.

• generate_response: Correlates internal and external findings and produces a final, actionable cybersecurity analysis for the user.

With the workflow defined, we can now send a query:

In this example, we want to evaluate whether this specific version of Express is affected by known vulnerabilities.

Research results

See the complete response in this file.

This response clearly correlates internal and internet findings and provides actionable mitigation steps. It successfully highlights the severity of the vulnerability and offers a structured, security-oriented summary.

Extensions and future enhancements

This architecture is modular and allows us to extend its capabilities by replacing, improving, or adding components to the existing list. We could add another agent, consumed by the same MCP client. We can also use an automated ingestion workflow with tools such as Logstash, Kafka, or Elastic self-managed connectors. Feel free to change the LLM, the MCP client framework, or the embeddings model or add more tools depending on your needs.

Conclusion

This reference architecture shows a practical way to combine Elasticsearch, the Agent Builder, and MCP to build an AI-driven application. Its structure keeps each part independent, which makes the system easy to implement, maintain, and extend.

You can start with a simple setup (like the security use case in this article) and scale it by adding new tools, data sources, or agents as your needs grow. Overall, it provides a straightforward path for building flexible and reliable agentic workflows on top of Elasticsearch.