NASA Aviation Safety Reporting System (ASRS) reports describe unusual events during flights, such as missed altitudes, confusing clearances, runway issues, or mechanical problems. Each report already has an official category, like altitude deviation, course deviation, or ground encounter. In this article, we ask a different question: What does this report reveal about the pilot who wrote it? The idea is to ask a model to infer a schema grounded on the data to classify the report based on criteria that help us figure out information about the report writers. Then ask a second model to apply the labels.

You can find the full workflow definitions and helper scripts here.

What you need to run this LLM pipeline

• Elastic Stack 9.4+ or Elastic Cloud Serverless. Elastic Workflows has been generally available (GA) since 9.4.

• Elastic Agent Builder enabled in your deployment.

• A Kibana generative AI (GenAI) connector pointing at Claude Sonnet (or an equivalent reasoning model). This is the planner.

• A Mistral API key. We’ll use it to register an Elasticsearch inference endpoint.

• Python 3.10+ with elasticsearch>=9.0 and pandas. Used by the dataset loader.

How two-tier LLM orchestration works

The workflow has two jobs: Decide what labels should exist, and then apply those labels to every report.

The first job is open-ended. A large model reads a varied sample of reports and proposes a small schema of categorical fields. A field is one way to describe the writer, such as attribution_style or procedure_orientation. Each field has a few allowed values, such as self_critical, system_attributing, or balanced.

The second job is repeatable. After a human approves the schema, a smaller model reads each report and chooses one value for each field.

We use Elastic Workflows because the steps are known ahead of time: sample reports, propose labels, wait for approval, classify every document, and store the results. Writing those steps in YAML makes the process reproducible, observable, and cheaper to rerun.

Why split LLM work across two model tiers?

A small model could handle classification, but schema discovery is a different shape of problem. It requires reading a diverse sample, spotting latent patterns, and proposing complex structures. In practice, smaller models over-anchor on surface keywords and produce redundant or nonexclusive fields.

Classification is simpler, the schema exists, the values are enumerated, and the task is to pick one per field. A smaller model handles this reliably and at a fraction of the cost, since it runs once per document across the entire corpus.

Large and small here mean reasoning capability. In this article, Claude Sonnet plays the planner and Mistral Small 3.1 plays the executor.

Classifying NASA pilot reports with a two-tier LLM pipeline

We’ll use the NASA ASRS database, which collects voluntary, anonymous incident reports from pilots, controllers, and mechanics. The dataset is public, and the reports are written as free-text narratives.

What we want to ask is:

What does this report reveal about the pilot who wrote it?

The planner reads a varied sample of reports and decides which distinctions are meaningful based on how the reports are actually written.

Step

Role

Model tier

sample

Pull a diverse subset of reports from the corpus.

(no LLM)

discover

Read the question and the sample, propose a schema of fields with enum values.

Large

approve

Human reviews the proposed schema and approves or edits it.

(Human via waitForInput)

apply

Iterate over the corpus, assign one value per field to each report.

Small

store

Write the schema and the per-document field values to Elasticsearch.

(No LLM)

Registering Mistral and Claude as Elasticsearch inference endpoints

The small model will be registered as an Elasticsearch inference endpoint using the native mistral service integration. 

The alias mistral-small-latest resolves to Mistral Small 3.1. It has a 128k context window and supports JSON-mode output.

The large model will be an AI connector pointing at Claude Sonnet. The Agent Builder UI walks you through creating the connector. Take a note of the connector ID since we’ll reference it from the workflow.

Indexing NASA ASRS incident reports into Elasticsearch

The ASRS dataset is indexed with keyword mappings for aggregation fields and text mappings for the narratives the models will read.

Download the ASRS CSV (the database publishes quarterly extracts at the ASRS Database Online page), and index it. The mappings are:

The mapping types follow how each field is used. flight_phase and anomaly are mapped as keyword because we’ll run terms aggregations on them to build the sample, and aggregations need exact, non-analyzed values. narrative and synopsis are mapped as text because they hold free-form prose that the models will read. The companion notebook has the full loader script that reads the CSV and bulk-indexes the documents.

Building a stratified sample for the planning LLM

The YAML snippets in this and the following sections are steps of the workflow definition that the notebook registers via the Workflows API. The first two steps generate a representative sample: They aggregate by flight phase and by anomaly and pull a few documents per bucket with top_hits.

How the large LLM discovers a classification schema from the data

The prompt needs both the question and the sample. A question alone may produce generic labels disconnected from the corpus, and a sample alone produces descriptive clusters that ignore the angle of the question. 

When both are present and the output is structured, the model produces labels that are grounded in the data and oriented to the task: a schema of categorical fields, each with two to four mutually exclusive value options backed by evidence from the sample.

Here’s the planner step from the workflow:

The structured output schema enforces the shape of the response:

• name: Identifier for the categorical field.

• definition: What this field measures, in one sentence.

• why_useful: How this field serves the question; this also helps the downstream classifier understand the intent.

• values: Two to four mutually exclusive options. Each has a value and a definition.

Here’s an example of the produced schema. We can see how the writer is being classified and the reasons why the model decided to create the category. definitionand why_useful fields are used by the second model to classify the documents.

Human-in-the-loop schema approval with waitForInput

The proposed schema is now passed to a person for approval. Elastic Workflows has a waitForInput step that pauses the workflow with a schema, exposes a form, and resumes when the input is submitted.

waitForInput has no timeout of its own, so if nobody responds, the execution waits indefinitely. To put a limit on that, set a workflow-level settings.timeout; if it elapses before the reviewer submits the form, the execution is canceled.

When the workflow reaches this step, the execution pauses and the Kibana UI shows an "Action is required" badge. Clicking Provide action opens a form where the reviewer can paste or edit the schema JSON. Since waitForInput cannot be prepopulated from a previous step, the code polls the discover step output and prints a paste-ready JSON block that can be copied directly into this form.

The step_output helper (in the notebook) polls the execution via GET /api/workflows/executions/{id} until the discover step completes and then returns its output.

Code JSON output pasted on Kibana:

The reviewer can keep useful fields, rewrite unclear ones, merge overlapping values, and add notes. After approval, the workflow resumes and sends the final schema to the executor step.

For a new corpus, keep this human gate in place. Once the schema is stable, you can auto-approve and only fall back to review when it’s worth it: Route just the low-confidence extractions to a person, or compare a new discovery run against the schema stored in the schemas index and trigger review only when fields or values change beyond a threshold.

Classifying the full corpus with a smaller LLM

By the time the workflow reaches this step, the open-ended part of the job is over. From here, the small model takes over and classifies each report against the approved schema.

Note: The classification step uses ai.agent instead of ai.prompt because ai.agent accepts an inference-id, which lets it call the Elasticsearch _inference endpoint directly, while ai.prompt only accepts a connector-id.

The fetch_corpus step is the third elasticsearch.request in the workflow, so it’s worth saying why we read from the index again. The first two (by_phase and by_anomaly) only pulled a small stratified sample for the planner to reason over, not the data to label. Now that the schema is approved, fetch_corpus pulls the documents we actually want to classify. We cap it at 100 with match_all to keep the demo fast; this is where you would page through the full corpus.

For every field, it returns a value (or null), a confidence, and a short quote. Setting additionalProperties: true in the JSON schema lets the step return one entry per field without the workflow having to know the field names ahead of time. A stored extraction looks like this:

Here, review_required is true because procedure_orientation came back at 0.44 confidence, below our 0.5 threshold, which is the signal a confidence-based quality gate would act on.

The fetch_corpus step pulls the documents to classify. The foreachstep iterates over them sequentially, and iteration-on-failure handles the errors: retry covers transient API errors from the inference endpoint, and, if all attempts fail, the fallback step posts to Slack so the failure doesn’t pass silently. (An email connector works the same way.) continue: true then lets the loop move on to the next document instead of failing the whole run. 

For production-scale corpora, consider using executeAsync, which is the fan-out version of execute.

Writing schemas and extractions back to Elasticsearch

The workflow produces two things: the approved schema and the per-document field values. The store_schema step runs right after the human gate, before the classification step fans out:

Each extraction is written inside the foreach loop, so results are persisted as they’re produced rather than batched at the end.

The schemas index holds one document per discovery run (question, approved fields, reviewer notes). The extractions index holds one document per report per schema version. 

What this two-tier LLM orchestration pattern gives you

We built one Elastic workflow that pulls a stratified sample from an incident report index, sends it with a question to a large reasoning model to generate a classification schema based on the data and a user-defined angle, pauses for human approval, and then iterates over the full corpus with a small Mistral model that assigns one value per field. 

The approved schema and per-document field values are written back to Elasticsearch as structured data.

The point of the exercise is that two different shapes of work, schema discovery, and schema application can use two different model tiers and that a workflow lets you write the routing decision down.

Next steps for your own LLM pipeline

• Try it on a corpus of your own. The pattern doesn’t care whether the input is incident reports, customer feedback, weekly status updates, or property listings.

• Promote the foreach step to workflow.executeAsync once you’re comfortable for parallel fan-out at scale.

• Schedule the rediscovery workflow on a cron trigger so you can discover different schema variations based on the data that comes in.

• Read the Elastic Workflows documentation for the full step catalog.

Related reading

• Build AI agents with Elastic Inference Service (EIS) covers the broader multi-model wiring pattern via EIS, complementary to the Workflows-orchestrated split shown here.

• How to build AI agentic workflows with Elasticsearch is a higher-level survey of how Agent Builder and Workflows fit together.

• LangExtract and Elasticsearch tutorial explores a different extraction pattern using a hand-authored schema; useful for contrast with the discover-then-apply approach above.

What is metric temporality in OpenTelemetry?

Metrics stores usually receive client-side, pre-aggregated metrics. For example, if an application records request response times, it won’t send each individual response time as a single data point to your metrics back end. Instead, the application (or rather the OTel SDK) pre-aggregates those raw response times into counters or histograms. These pre-aggregated values are then exported at a periodic interval, dramatically reducing the number of data points. Temporality is about how this pre-aggregation works. There are two temporality models: cumulative and delta.

Cumulative temporality in OTel metrics

With cumulative temporality, each data point represents the totalamount of change in the metric value since the process started. Values monotonically increase, with occasional reset to 0 (for example, when the process restarts).

Take a counter tracking the total CPU time consumed by a Java Virtual Machine (JVM):

Timestamp

Value

Meaning

10:01

12.4s

12.4s total CPU time since start

10:02

13.1s

13.1s total CPU time since start

10:03

13.9s

13.9s total CPU time since start

To compute the rate of change between 10:01 and 10:02, we subtract: 13.1 - 12.4 = 0.7s of CPU time was consumed in that interval. Dividing by the time range of the interval gives us the rate. This is the default temporality for counters in both Prometheus and OTel.

Delta temporality in OTel metrics

With delta temporality, each data point represents the change since the last measurement. Values are independent of each other. In other words, after each export, the OTel SDK resets all values for all series.

The same raw observations from the cumulative example above would look as follows with delta temporality.

Timestamp

Value

Meaning

10:01

0.5s

0.5s of CPU time in this interval

10:02

0.7s

0.7s of CPU time in this interval

10:03

0.8s

0.8s of CPU time in this interval

To compute the rate or increase, we can use the value directly, without any subtraction.

Trade-offs between cumulative and delta OpenTelemetry metrics

Both temporalities have practical trade-offs:

• Resilience to data loss: Cumulative counters are self-describing: If you miss an export, the next data point still gives you the correct total. Delta values are incremental, so a lost data point means that the corresponding increase is lost.

• Metric producer memory footprint: For cumulative temporality, the OTel SDKs need to keep a state for every series in memory. For delta temporality, the footprint is much lower. There, the SDKs only need to keep track of counters or histograms which changed since the last export. If there are a lot of counters or histograms and many of them don’t increase each period, this difference can be quite substantial.

• Aggregation across restarts: Cumulative counters require reset detection logic, which in edge cases can fail: If the metric value decreases, it’s detected as a reset. We assume that the application was restarted and the counter started from 0 again. This can be missed if the first reported counter value after the restart is higher than before the restart. A concrete example:

• The service consumes 1 second CPU time and restarts.

• After the restart, the service performs a CPU-intensive task and consumes 2 seconds of CPU time before the metric is exported again.

• The metric back end just sees 1 followed by 2 as the metric value. It never observes a decrease and therefore misses the reset.

Delta values don't have this problem since each value is independent.

If you’re using histograms, the trade-offs have an even bigger effect:

Trade-off

Cumulative

Delta

Histogram size

Buckets accumulate across exports, consuming more storage

Buckets reset each export, producing smaller histograms

Min/max accuracy

Approximated from buckets for custom time ranges (tracked values represent extremes since process start)

Exact per-export minimum and maximum values

Query performance

Faster: only the first and last value in a time range plus resets are needed

Slower: all histograms in the queried range must be combined

OpenTelemetry supports both models and lets you choose per SDK via the OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE environment variable.

Why native temporality support eliminates OTel Collector workarounds

Prometheus and most other metrics back ends pick a side: All metrics have to be either cumulative or delta. Elasticsearch previously followed that pattern, too, with native storage of cumulative counters and delta histograms, and workarounds for everything else. Delta counters were stored as gauges, functional but without native counter semantics for rate queries. And cumulative histograms were unsupported.

One workaround for unsupported temporalities is to configure your metric producers (for example, OTel SDKs) to produce data with the temporality that your back end supports. In large-scale deployments, this can be a very challenging task. And sometimes this isn’t even possible (for example, if you consume OTLP metrics from third-party services).

Another workaround is to convert the temporality prior to ingestion. In the OTel Collector, you would typically use the cumulative-to-delta processor, which comes with a big warning sign about statefulness. The conversion is inherently stateful, requiring ordered delivery of metric series to the same collector and persisted state across restarts. In practice, it works, but at scale, it comes with a lot of deployment headaches.

With Elasticsearch 9.5, you can skip the conversion pipeline entirely. Elasticsearch natively stores and queries metric data with both temporalities. It doesn’t require any stateful conversion required or explicit configuration of your OTel SDKs.

Demo: ingesting cumulative and delta OTel metrics side by side

To demonstrate the temporality support, we'll reuse a demo setup from our OTel histogram metrics ES|QL blog post: a Java Renaissance benchmark instrumented with the OTel Java agent. The twist this time: We run two instances of the benchmark, each configured with a different temporality:

• renaissance-delta: Exports metrics with delta temporality.

• renaissance-cumulative: Exports metrics with cumulative temporality.

Both instances report the same metrics under the same service name renaissance, but with different service.instance.id values. Here’s the relevant section of the docker-compose.yml that can be found in the companion code:

To run the demo yourself, you'll also have to fill out the managed OTLP endpoint URL and the corresponding API key:

After starting the demo with docker compose up --build, both instances will start reporting metrics to Elasticsearch.

Querying OTel counter metrics with ES|QL and PromQL

Let's query the first few raw data points of jvm.cpu.time for both instances to see the different temporalities in action:

This gives us the first five data points for each service instance:

The benchmark consumes CPU at a nearly constant rate. This is directly visible based on the delta temporality data: The values are nearly constant between exports. In contrast, the cumulative temporality values grow over time, as they represent the total CPU usage of the benchmark instance.

Now let's have a look at how to properly query this metric using PromQL:

The screenshot shows that both benchmark instances consume a nearly constant of 1 to 1.2 number of CPU cores with some variance. This query works because we made our rate implementation respect the temporality: Every time series (so every service instance in our case) stores the temporality as a metric dimension. The rate implementation looks at this dimension and interprets the data accordingly: For delta temporality, values are summed up; for cumulative temporality, a difference computation is done. This all happens automatically in the background, without requiring any changes to your queries.

We’ve adapted rate, increase, and irate to work this way. The same applies when using those functions in ES|QL TS queries:

Because Elasticsearch tracks the temporality as a dimension, you can have multiple series with different temporalities for the same metric, just like in the demo use case. Aggregating across series also works as expected, because at that point rate, increase, or irate already took care of normalizing the data:

Querying OTel histogram metrics across temporalities

Metric temporality applies to histograms in the same way it applies to counters: histogram buckets are effectively a set of counters, each tracking values in a specific range.As in our histogram demo, we use exponential histograms, where bucket boundaries adapt automatically to minimize relative error.

Due to this similarity, histograms can also be cumulative or delta. Either the counter per bucket is reset after each metric export or the cumulative count carries over between exports.

Let's query the median major garbage collection (GC) duration for our benchmark instances, which is a histogram metric:

Or the equivalent ES|QL query:

Again, both queries will automatically load the temporality per series and interpret the histograms accordingly. In PromQL, this is handled by the increase function. Note that in ES|QL, you don't explicitly call increase on histograms. The TS command automatically handles the temporality-aware merging of histograms when you use aggregation functions, like PERCENTILE, MEDIAN, or AVG.

How Elasticsearch stores metric temporality in TSDB

Elasticsearch's time series database (TSDB) stores metric temporality in a dedicated dimension field on each document. The index.time_series.temporality_field index setting lets you specify which field carries the temporality information. The field must be a keyword field with time_series_dimension: true and the permissible values "delta" or "cumulative".

As soon as this setting is present on a time series index, ES|QL and PromQL will load the corresponding field when performing temporality-dependent aggregations. If the field isn’t present or has no value on a document, we fall back to defaults based on the type of the corresponding metric: counters default to cumulative, and histograms default to delta. This matches the historical behavior and ensures existing queries and existing data continue to work without changes.

When you ingest metrics via the OTLP endpoint, Elasticsearch automatically adds a temporality dimension field to each document, populated from the OTLP AggregationTemporality metadata. For custom (neither OTLP nor Prometheus remote write) ingestion, you’ll have to manually set up the index.time_series.temporality_field setting and populate your temporality dimension.

The temporality is also respected during downsampling: As it’s a dimension, it’s preserved automatically and used to compute the aggregate values.

Getting started with mixed-temporality OTel metrics in Elasticsearch

With Elasticsearch 9.5, cumulative versus delta is no longer a decision you have to get correct at the start. Ingest both temporalities side by side, even for the same metric name, and let ES|QL and PromQL handle the rest. You can switch between both without having to touch your queries. For more details, see the metric temporality documentation.

What backward compatibility means for the Kibana Dashboards API

During technical preview, the API shape could change between releases.[1] That's no longer the case. General availability (GA) means:

• Complete backward compatibility. New fields and panel types will be added over time, but existing fields and behavior remain unchanged. Any future breaking changes would be very carefully considered and would only be introduced in a new major stack version.

• Production-ready with full support. The API carries Elastic's full support guarantees. You can safely use it in production environments for automated deployments, environment promotion, and programmatic dashboard management.

New Kibana API endpoints for Tags, Markdown, and Links panels

Elastic 9.5 also introduces a new  standalone endpoint for Tags, which let you categorize and filter dashboards. Now you can manage them programmatically through dedicated CRUD endpoints, making it easier to organize dashboards at scale across environments.

New Markdown and Links panel endpoints are available now in Serverless and will land in the next stack release (9.6).

What panel types does the Kibana Dashboards API support?

The Dashboards API supports all by-value panels in 9.5 (those defined directly in a dashboard, as opposed to library panels saved for reuse). Every supported panel type has a typed, validated schema.

Panel type

Status

XY charts

Supported

Metrics

Supported

Pie

Supported

Gauge

Supported

Heatmap

Supported

Data tables

Supported

Treemap

Supported

Discover sessions

Supported

Controls

Supported

Markdown

Supported

Links

Supported

ML panels

Supported

Observability panels

Supported

Maps

Coming soon

Vega

Coming soon

How to manage Kibana dashboards as code

The Dashboards API enables a full dashboards-as-code workflow: Export a dashboard as clean, diffable JSON, commit it to Git as the source of truth, review changes in pull requests, and deploy the same definition across development, staging, and production. Once a dashboard is managed as code, treat Git as the single source of truth: Changes made directly in the UI are overwritten the next time you deploy.

The main challenge when moving a dashboard between spaces, clusters, or stages is that dashboards reference objects like data views and library visualizations by ID. Because these IDs are auto-generated and differ across environments, a dashboard exported from one environment can point at objects that don't exist in another. There are three ways to handle this, listed here from most to least automated:

• Use Terraform. The Elastic Stack Terraform provider tracks each resource and maps IDs per environment automatically, so references stay consistent as you promote a dashboard from development to production.

• Define by-value Elasticsearch Query Language (ES|QL) panels. The most portable way to build a panel is to define its visualization with ES|QL directly in the dashboard. An ES|QL query reads from the indices you name in it, so the panel carries no external references to data views or library objects. The result is a fully self-contained, portable dashboard.

• Assign matching IDs. If you reference saved objects, like data views or library visualizations, create them with a chosen ID using PUT (upsert) rather than POST (which auto-generates an ID). Use human-readable IDs, like logs-prod, so they're easy to reuse and recognize across environments.

For a detailed walkthrough of these portability patterns and the full dashboards-as-code workflow, see the Manage dashboards as code documentation.

Create a Kibana dashboard with the Dashboards API using PUT

Here's a quick example creating a dashboard with a metric panel using PUT instead of POST to assign a custom ID using the dashboard name (service-health-overview). The same logic works for creating standalone visualizations saved in the library.

Kibana Dashboards API roadmap: Maps, Vega, and standalone endpoints

We're actively expanding the API surface. Maps and Vega panel support is next, adding typed schemas for them. We're also building standalone CRUD endpoints for Discover sessions (beyond their existing support as dashboard panels), Vega, Maps, and Annotations, decoupled from the dashboard lifecycle.

For the full schema definitions, visit the Dashboards API documentation. For Terraform users, the Elastic Stack Terraform provider supports the GA Dashboards API.

Note

1. The core endpoints are unchanged from the technical preview. If you built integrations against 9.4, they work in 9.5. The only breaking changes are two minor ones affecting dashboard listing and duration unit formats, documented here.

Elastic Workflows now writes its own YAML. You type what you want automated in plain language, the Elastic AI Agent generates a complete workflow against a typed schema, and nothing runs until you've read it. YAML is why this works: it gives the model a constrained, well-typed target, so what comes back is actual building blocks you can edit and run.

What is new in Elastic Workflows 9.5:

• Natural language authoring is GA and on by default: describe an automation, and the Elastic AI Agent writes the workflow, you review and run it.

• Versioning with diff and one-click rollback is GA.

• Three new experimental previews (behind an advanced setting): a visual mode that renders a workflow as a graph, human-in-the-loop steps that reach people in Slack for input or approval, and parallel execution.

• More to build on: new connectors, event triggers that react to Cases activity, token metering for AI steps, and a queue strategy for concurrency.

Workflows is the automation engine built into the Elastic platform. It reached general availability in 9.4, enabled by default and running against your Elasticsearch data with the connectors and access controls you already have. This post walks through what 9.5 adds.

Why YAML makes AI workflow automation work

YAML is the authoring language for Elastic Workflows because it's declarative, version-controllable, diffable, and portable across environments. It reads the same in a pull request as it does in the editor.

It was also a bet. Large language models (LLMs) are very good at generating structured, well-typed content, and a workflow language is close to an ideal target for that. Ask for prose and a model can wander. Ask for a workflow against a typed schema, with named step types and validated inputs, and there is a right shape for the answer.

In 9.5 that bet pays off, and it is GA. Inside the workflow editor, you write what you want in plain language:

When a detection alert fires for a host, pull the last 24 hours of related logs, ask the AI step to summarize what happened, and post the summary to the on-call Slack channel.

The Elastic AI Agent generates the workflow: the trigger, the Elasticsearch query, the AI summarize step, the Slack step, wired together with the right inputs and outputs. You get inspectable, editable YAML back. Nothing runs until you read it, adjust it, and decide to run it. You can also point it at a workflow you already have and describe the change you want, and it edits in place.

The output is worth reading because there is a rich, well-typed language underneath it. A short prompt expands into real building blocks:

• foreach and while loops, with guardrails that stop runaway execution.

• switch for clean multi-way branching.

• data steps like data.filter and data.aggregate for in-flight transforms.

• on-failure handling on every step, so you can retry, continue, or abort.

• workflow.execute, so one workflow can call another and you assemble new automation from pieces you have already tested.

Natural language gets you the first draft fast; the language underneath is what makes that draft real.

Workflow versioning with diff and one-click rollback

Versioning is GA in 9.5. Every workflow now has version history: every change is tracked and diffable, and you can roll back to any prior version in one click. You see who changed what and when, compare any two versions side by side, and undo a bad edit without reconstructing it by hand. This is the change-control foundation teams asked for before they would run automation against production systems.

Versioning pairs with the production controls already in place: granular role-based access control (RBAC) over who creates, edits, runs, and views workflows, every management action written to the security audit log, and import/export that moves workflows between environments with their connector references intact.

Versioning in the product is the near end of a longer arc. A workflow is a declarative YAML definition, plain text with a well-defined schema, which means it already fits the tooling built for code: it can be diffed, reviewed, and version-controlled. Where we are headed is full, bidirectional integration with the version control systems you already use, so that a workflow could live in your repository, move through review, and deploy the same way the rest of your software does. That is coming, and the same bet that made natural language authoring work, a declarative and well-typed language, is what will let you manage workflows as code.

Visual mode, human-in-the-loop workflows, and parallel execution

Three of the newest 9.5 additions ship in Experimental. To try them, turn on Elastic Workflows: Experimental Features in Stack Management → Advanced Settings (it requires a page reload). Here is what each one does.

Visual workflow editor: see the logic as a graph

You can now switch a workflow between the YAML editor and a visual mode that renders the workflow as a graph. The graph lays out your steps, branches, and flow control, so you can see the logic and the paths a run can take at a glance, alongside the YAML. It is read-only in 9.5: you still author in YAML, and the graph stays in sync as you edit. It is the first step toward a full drag-and-drop builder, which is coming next.

Human-in-the-loop workflows with approval steps in Slack

Workflows could already pause for a person in 9.4 with waitForInput, which presents a schema-defined form and lets the response drive what happens next. 9.5 adds waitForApproval for the most common case of all, a binary approve or reject with labels you choose, and it extends both steps to their first external surface: Slack. Both pause the run until someone answers, with a timeout so a workflow never hangs forever. Not every decision should be fully automated, and this is how you put the human on exactly the steps that need one.

waitForInput is the flexible one: define a schema for the input you want back, and the response comes through typed. Reach for it when the choice is more than yes or no. Here, an Observability workflow has caught a service-level objective (SLO) burn-rate alert on payment-service and asks the on-call engineer which mitigation to run:

waitForApproval is the new one in 9.5, for a straight approve or reject. Here a security workflow has decided a host should be contained, but gates that destructive action on a human before it runs:

The new piece is the channels block. A wait step can now deliver to where people already are, and in 9.5 that means Slack: the workflow posts the request to a Slack channel, the person responds from Slack, and the workflow resumes with their answer. No one has to be sitting in Kibana for the automation to move. Slack is the first external surface, and richer delivery experiences across more channels are on the way.

Parallel execution: run independent workflow steps at once

By default a workflow runs one step after another, which is what you want when each step depends on the last. But plenty of work does not: enriching an alert from three sources, checking a file against several reputation services, investigating a handful of leads. Run those sequentially and the workflow is only as fast as the sum of its parts, when it could be as fast as the slowest one. The new parallel step lets you run independent work at the same time. It works two ways.

The first is when you know the work ahead of time. You define a fixed set of tasks, and they run at the same time, so you gather all the results in one step instead of waiting for each in turn. Enriching a security alert from two sources at once is the classic case:

The second is when you do not know the work ahead of time. You give the step a list, and it runs the same work once per item, all at the same time, up to a concurrency limit you set. Root cause analysis is a good example. An earlier AI step generates a set of hypotheses for why a service is degrading, and you do not know in advance how many there will be or what they are. Rather than investigate them one after another, you pass the list into a parallel step, and it investigates every hypothesis at once:

You control how many run at once with concurrency, and the engine caps both the concurrency and the total number of parallel tasks so a workflow cannot spawn unbounded work. All the results are available to the next step, so the workflow runs the parallel work, then continues once every task finishes.

More in Elastic Workflows: connectors, triggers, token metering and concurrency

Beyond the headline features, 9.5 widens what a workflow can reach and react to.

New connectors: BigQuery, Snowflake, HubSpot, Cortex XSOAR and more

The connector catalog keeps growing, with native connectors added in 9.5 for:

• BigQuery

• Snowflake

• Box

• Dropbox

• OneDrive

• Outlook

• Azure Blob

• Google Cloud Functions

• HubSpot

• Cortex XSOAR connector for security automation

More are on the way, and when there is not a dedicated connector for the system you need, the http step is the escape hatch: it can securely call any API endpoint, with credentials supplied by a connector rather than written into the YAML.

Event-driven triggers for Elastic Cases

A workflow starts from a trigger, and 9.5 widens what a workflow can respond to. Cases now emit events a workflow can subscribe to:

• A case is created.

• A case is updated.

• Its status changes.

• A comment is added.

• An attachment is added.

So a workflow can run the moment a case opens, to enrich it, tag it, or notify the right channel, or when its status flips to a state you care about, rather than polling for changes. Alert-triggered workflows also receive richer rule context now, including the rule's tags, type, and parameters, so the workflow has more to work with before it acts.

Token usage and cost tracking for AI workflow steps

Workflows can call AI steps: ai.prompt for a freeform prompt, ai.classify to sort something into categories, ai.agent to hand a task to an Agent Builder agent. In an automation that runs thousands of times a day, those calls add up. 9.5 now reports token usage for every AI step, input, output, cached, and total, both per step and for the whole run. You can see exactly what the AI in a workflow consumes, track it over time, and tune a prompt or a model choice with the numbers in front of you.

Workflow concurrency: cancel, drop or queue

A workflow's concurrency setting decides what happens when a new execution starts before the last one finishes. 9.5 adds a third strategy, so you can pick the behavior that fits the workflow:

Strategy

Use it when

Cancel-in-progress

Only the latest execution matters, like recomputing a current state

Drop

An execution already in flight covers the situation and extras are redundant

Queue (new in 9.5)

Every execution matters and order does, so they line up and run one after another, with a queue size and time-to-live you control

Queue is what you reach for when executions touch the same resource or should not overlap. Audit logging also covers more of the lifecycle in 9.5, including restoring a workflow from its version history.

Get started with Elastic Workflows

The fastest way to see this is to describe something you want automated. Open the workflow editor in 9.5, type it in plain language, and read the YAML that comes back. Natural language authoring is on by default. To try the visual mode, human-in-the-loop steps, and parallel execution, turn on Elastic Workflows: Experimental Features in Stack Management → Advanced Settings.

The theme across 9.5 is a shorter path from idea to running automation. You describe what you want and AI drafts it, you see it as a graph and version it as you go, you pause it for a person in Slack when a step needs judgment, and you run independent work in parallel. For the full details, see the Workflows documentation.

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.

The most powerful AI models run on remote cloud installations with access via a web API, meaning that you have to trust your AI service provider for security, service availability, and stable prices. You can’t easily align reasonable demands for reliability, privacy, manageable costs, and good data governance with increasingly powerful, sophisticated, and resource-intensive AI usage.

Government regulation, court rulings, and business considerations made in someone else’s interest have all recently resulted in restricting access to specific services. And even if you can switch to other services, AI models aren’t components that can just be swapped out whenever you want. Applications that use semantic embeddings depend on having access to the same models at query time as at data ingestion time. To lose access to your embedding model means your search system comes to a halt.

AI pricing models compound that risk. Recent financial disclosures from major AI vendors give customers good reason to be concerned about potential price hikes. Reliance on products with unpredictable costs adds more risk to capital-intensive AI investments that may not produce clear returns.

Jina On-Prem is Elastic’s answer to these challenges.

Who needs on-premises AI?

Local hosting and direct control over your AI models support a variety of technical demands, industry requirements, and business interests.

Local installation reduces what you pay your AI service providers, but it puts the cost of hardware and reliable access on your organization. Depending on your volume of use, it may simply be cheaper. But there are additional pressing reasons to consider running your own AI. If any of the issues described below concern your enterprise, consider a local AI solution like Jina On-Prem. This list is not exhaustive.

Use case

Why on-prem

Example

Air-gapped / high-security

No outbound data transmission; complete network isolation

Defence, intelligence, classified research

Regulatory compliance

Data sovereignty; no cross-border transmission or third-party exposure

Healthcare (Health Insurance Portability and Accountability Act [HIPAA]), finance, EU enterprises (General Data Protection Regulation [GDPR])

Latency-critical

Zero network dependency; no tolerance for connection failures

Robotics, edge computing, vehicles, ships

Cost predictability

Fixed infrastructure cost vs. per-token pricing with uncertain future rates

High-volume continuous inference workloads

Liability reduction

No third-party data exposure; maintains legal privilege and duty of care

Law firms, government agencies

Why air-gapped and firewalled systems need on-prem AI

Air-gapped and firewalled systems cannot use external AI APIs. Jina On-Prem runs entirely within your infrastructure with no outbound connections.

For organizations managing especially sensitive data, security and privacy considerations are paramount. It does little good to invest in protecting your sensitive data if you promptly turn it over to some remote third party that may have insufficient security in place or might be subject to the demands of a foreign government.

Employees in organizations that handle sensitive data often receive some training in secure data handling, but this isn’t very effective when they all have web browsers that may be open to any page on the internet while they handle that data. Isolation is the most effective security measure available, either through air-gapping or very restrictive firewalls, but that makes it difficult to use external services of any kind.

On-prem AI for latency-sensitive and high-availability systems

Software as a service and cloud computing represent a compromise between the cost of offering highly accessible, reliable services on your own computers and outsourcing the problem to someone else. But they come with variable latency, outages, and a complete loss of control when things go wrong. AI services aren’t the exception. If your search system goes offline when you can’t access your embedding model, it may no longer look like a good compromise.

Furthermore, relying on external AI will always involve risks that you can’t easily foresee or manage. Internet access and network latency can degrade without notice, as a result of political events, bad weather, or ships dragging their anchors over underwater fiber-optic cables. Governments can, and recently have, used export bans to suddenly block access to AI models. AI service providers sometimes withdraw models to induce you to switch to newer ones. The flexibility and managed costs of external services have to be balanced against the risks of dependency.

On-prem AI for GDPR, HIPAA, and data sovereignty compliance

Organizations that collect personal data are subject to increasingly stringent regulations which often differ between jurisdictions and may have contradictory requirements. Notably, HIPAA rules place very strict data protections on American healthcare providers, and strong general data protection laws in Canada, the European Union, and many Asian jurisdictions require all enterprises that handle personal information to do so securely and to limit the transmission of that data to other parties or other jurisdictions. These rules can even impose obligations on foreign entities if they have any customers in those jurisdictions. Financial institutions are frequently subject to even stricter rules and bear the same direct liability for information security that they have to protect against other forms of criminal activity.

Regulatory compliance can be incompatible with third-party AI services, especially if using them involves cross-border data transmission.

Furthermore, recent events show that rules restricting the physical location of data stores may not be a reliable source of protection when international cloud operators are subject to pressure from foreign governments. Local laws may conflict between jurisdictions, requiring local data storage and processing and making third-party services impossible to use. In some cases, the only solution is to take all the parts of your processes in house, including your AI systems.

AI liability risks from third-party data transmission

Data protection laws and recognized duties of care toward sensitive data routinely have liability implications, sometimes very severe ones. You can be liable for third-party service providers’ handling of your data. While courts and legal procedures might provide some retrospective protections from insecure service providers, those remedies are not available nor generally effective against national security actors, law enforcement, or criminal hackers.

For governments, there have already been instances of cross-border cloud service providers releasing sensitive state information to foreign actors.

But even if you don’t worry about foreign governments or hackers, and if your external AI service providers are themselves secure, just the fact that they’re external can create liabilities.

For example, in most jurisdictions, lawyers’ communications with their clients enjoy special legal protections, and law offices have strict liabilities when recording or storing this information. In the United States, this “attorney-client privilege” is so famous, it’s central to movie and TV plots. But one of the ways that privilege can be lost is by communicating information with someone who is not privileged, and recent developments suggest that external AI service providers might qualify.

It’s possible, at least in the United States, that just using third-party AI services over an internet API, like embedding models that provide indexing services, might violate critical confidentiality rules. A law firm might be sued, disciplined, or disbarred just for using externally hosted software, even if no security breach occurs.

On-prem AI for offline, edge, and physically isolated systems

Computer systems aren’t just isolated for security reasons. For example, moving vehicles cannot rely on internet access for any essential functions. Ships and aircraft have very extensive onboard computer systems that have to function without internet connections and therefore cannot use external AI services. Offshore platforms, remote facilities in wilderness areas, computer services in the Arctic, Antarctic and on small islands without adequate physical connections to global networks are all examples of installations that benefit from locally hosting all the services they need. As AI’s role in enterprise computing grows, these limitations become more important to address.

Emerging applications of AI to physical systems (robotics and other spatially confined or external-world–focused use cases, like logistics management systems or even supermarket checkouts) may be connected to the global internet, but they have no tolerance for connection failures or spikes in latency. If they rely on an AI system to operate, that AI system needs to be as local and reliable as possible.

Who doesn’t need on-premises AI?

Remote software services and off-site AI do have benefits. Running AI models can require expensive, power-hungry processors with notoriously short lifespans. Access to high-quality hardware is particularly difficult right now due to market factors and external economic shocks. Under the circumstances, it may make sense to pay by the token to use an external API instead of supporting the steep capital costs of local AI.

External APIs make the most sense for intermittent users. If you use AI models primarily to batch process data for analysis, rather than running a search system that has to be online all the time, it makes little sense to invest in capital-intensive hardware and local installations.

Furthermore, when your data processing is already cloud-based, for example, an ecommerce website hosted in the cloud for reliability and accessibility reasons, using AI services located in the same cloud infrastructure may provide a better value for money than introducing your own licensed AI model deployment. You’re already dependent on your cloud service provider, so being dependent on its AI services doesn’t add much risk.

If your use case sounds like it fits that description, Jina AI models are available on EIS, AWS Marketplace, and the Google Cloud Platform specifically to meet your needs.

The table below summarizes the key factors. Your answer depends on your data, infrastructure and usage pattern.

Factor

On-prem favored

Cloud API favored

Usage pattern

Continuous or high-volume inference

Intermittent or batch processing

Data sensitivity

Regulated, sovereign, or classified

No cross-border or third-party restrictions

Network environment

Air-gapped, firewalled, or unreliable

Stable, always-on internet

Existing infrastructure

Own or can procure GPU hardware

Already cloud-hosted with colocated AI

Cost model

Fixed hardware + license; predictable at scale

Per-token; lower up-front, variable long-term

Latency tolerance

None (robotics, edge, real-time)

Network variability is acceptable

Operational responsibility

Your team manages hardware and availability

Provider manages hardware and updates; you manage integration

You have to consider the costs and benefits in light of your particular circumstances and use cases, taking into account the issues highlighted in the previous section that apply to you. The cost-benefit analysis will doubtless change over time. We can’t predict the future of the AI industry or hardware prices even in the short term.

Introducing Jina On-Prem

For users who can benefit from local AI services, we’re introducing Jina On-Prem, a fully self-contained installation suite for Jina AI’s high-performance models.

Jina AI’s models match the accuracy of embedding models many times their size, reducing compute costs, memory footprints, and hardware requirements. This makes them an ideal choice for users who want or need to keep their AI on-premises. Commercial licenses are available with scalable, proportionately priced solutions for use cases of all sizes.

What API schemas does Jina On-Prem support?

• Available as a complete collection of dependencies for local installation or as a Docker container that you can install and run in minutes.

• Jina On-Prem installations do not call out to outside systems.

  • No call to Hugging Face Hub or any model registry (HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1 are baked in).

  • There’s no license server.

  • There are no telemetry or logging endpoints.

• Supports both CPU and GPU hardware, with GPU autodetection.

• All 28 Jina AI models available, including the latest jina-embeddings-v5-omni multimodal embedding models and jina-reranker-v3.

• Access via standard AI API schemas: Jina API, OpenAI, Cohere, Voyage AI, and Gemini. Jina On-Prem is a drop-in solution for applications built on those schemas.

• Drop-in replacement for models served by the EIS. Jina On-Prem integrates directly with air-gapped Elastic deployments.

Hardware requirements for Jina AI on-prem models

The hardware requirements vary for different Jina models. The table below shows the recommendations for the most recent models using GPU settings. You don’t need anything more powerful than an NVIDIA L4 GPU, although an A100 is recommended for the v5 embedding models. Our latest embedding model currently requires a minimum of 8 GB of VRAM.

Model

Minimum VRAM

Recommended GPU

jina-embeddings-v5-text-nano

2 GB

T4 / L4

jina-embeddings-v5-text-small

3 GB

L4 / A10G

jina-embeddings-v5-omni-small

8 GB

L4 / A10G / A100

jina-reranker-v3

3 GB

L4

jina-clip-v2

4 GB

L4

jina-code-embeddings-1.5b

4 GB

L4

ReaderLM-v2

4 GB

L4

If you use more than one model at a time, the VRAM requirements will increase. Please see the Sizing and Hardware page for more information.

How to install Jina On-Prem with Docker

The quickest way to get started is to install Docker (if you haven’t already) and follow the instructions on the Jina On-Prem Quick Start page.

There are pre-composed Docker containers for all 28 Jina models. Download one and transfer it to your installation target, and you can have Jina AI models running in under five minutes.

For multimodal or custom builds, or to download the complete dependency set for installation outside of a container, follow the steps outlined in the bundling guide.

Your Jina On-Prem installation supports all Jina API and EIS functionality and embedding generation via OpenAI, Cohere, Voyage AI, and Gemini APIs, so it can integrate into preexisting applications using standard interfaces. See the API documentation for more information.

Jina models, including models installed with Jina On-Prem, are available on various licensing terms, with the latest models free for noncommercial use under a CC BY-NC 4.0 license. To license Jina On-Prem for commercial use, please contact Elastic Sales.

Elasticsearch's inference API now connects directly to NVIDIA-hosted models. You get text embedding, completion, chat completion, and reranking, plus access to NVIDIA's catalog of NVIDIA Inference Microservices–optimized (NIM-optimized) retrieval and generative models, without writing any custom integration code.

In practice, that's vector search and retrieval augmented generation (RAG) applications built on NVIDIA-hosted embeddings. It's also multi-turn conversations through the chat completion API and reranking with NVIDIA's cross-encoder models to push relevance past keyword matching. All four task types run natively through the inference API, with support for both streaming and non-streaming responses. How do I get an NVIDIA API key?

NVIDIA offers a broad catalog of models designed for a wide range of use cases, all of which can be explored on the NVIDIA Build model catalog. Throughout this article, we provide specific examples of high-performance models optimized for each inference task type. After identifying the model that best aligns with your application requirements, choose the deployment approach that fits your infrastructure and operational needs. This could mean running it on-premises for greater control or using a serverless option for faster experimentation and simplified scaling.

To get started quickly, you’ll first need access to NVIDIA’s model catalog and APIs. Create an account or log in at https://www.build.nvidia.com/ to explore available models, evaluate their capabilities, and compare which ones best fit your use case before proceeding toward full-scale deployment. This site provides a web-based interface for testing models, which is useful during evaluation and experimentation. For production-level requirements, you can use NVIDIA NIM to deploy endpoints on your own infrastructure.

To access NVIDIA models, you need to generate an API key. This key will serve as the authorization mechanism when making calls to NVIDIA's endpoints. You can create, access, and manage your API keys at API keys. To create a new key, click the Generate API Key link in the top right, and then specify a name and expiration period for the key. After generating the API key, select the appropriate model for your task and set up the corresponding Elasticsearch inference endpoint.

Setting up Elasticsearch inference endpoints

Once you have set up your NVIDIA account and obtained the necessary API keys, you can create an Elasticsearch inference endpoint.

Endpoint setup can be done directly in Kibana using the console, which allows you to input the required steps into Elasticsearch even without using an API. The following sections provide examples and details on how to create and use endpoints for text embeddings, completion, chat completion, and reranking.For more examples and detailed information, please consult the Elasticsearch API reference documentation.

Creating and using a text embeddings inference endpoint

To create a text embedding inference endpoint, you first select an appropriate model that can perform embedding operations. NVIDIA lists its models in the NVIDIA Build model catalog. You can select the Text-to-Embedding or Retrieval Augmented Generation use case on the left to filter the appropriate models. You can also find NVIDIA’s text embedding models in the NVIDIA documentation. NVIDIA’s retrieval APIs include text embedding and reranking models. When choosing a model, make sure it explicitly supports text embedding inference. Text embedding models typically include an API description, such as "Creates an embedding vector from the input text."

A good example of an embedding model is the nvidia/nemotron-3-embed-1b model. You can access the deployment page for this model, which allows you to deploy this model on-premises.

Once you have selected a suitable model, open its API reference page, where you’ll find the parameters required to create an Elasticsearch inference endpoint.

Two parameters are relevant:

• model_id(required): Specifies the NVIDIA model to be used for embedding inference. This parameter is named model on the NVIDIA side.

• url(optional): The endpoint URL used to send requests to the NVIDIA model (either deployed on-premises or in a serverless environment). It must be accessible from your Elasticsearch instance.

For most text embedding models, the URL is static and NVIDIA differentiates models solely via the model parameter. If the url parameter isn’t provided during endpoint creation, the default text embedding task specific value https://integrate.api.nvidia.com/v1/embeddings will be used.

To generate text embeddings, set up an endpoint configured with the required NVIDIA model values in the service settings map:

Upon sending this request, you should receive a successful 200 OK response. This response confirms that the endpoint is functioning correctly and the settings are specified accurately, and it will detail your newly created Elasticsearch endpoint for the text embedding task type.

You can now use the newly created endpoint to generate embeddings. The request for this operation will be similar to the example shown below:

The text embeddings will be returned, accompanied by a successful HTTP 200 OK status.

This integration allows users to use the NVIDIA models directly within Elasticsearch, making advanced search and RAG applications easier to build. These production-ready models offer a reliable and robust foundation for enterprise-scale deployments.

Creating and using a completion inference endpoint

To create a completion inference endpoint, you first select an appropriate model.

NVIDIA lists its models in the NVIDIA Build model catalog. You can search for the model there, but you can also find NVIDIA’s completion models in the left-hand navigation of this large language model (LLM) API documentation.Each entry in the list links to a general description of the model. From there, you can navigate to a nested link that opens the API reference specific to the selected model. A good example of a completion model is the nvidia/nemotron-3-super-120b-a12b model. You can access the deployment page for this model, which allows you to deploy this model on-premises.

Once you have selected a suitable model, open its API reference page, where you’ll find the parameters required to successfully create an Elasticsearch inference endpoint.

Two parameters are relevant:

• model_id(required): Specifies the NVIDIA model to be used for completion inference. This parameter is named model on the NVIDIA side.

• url(optional): The endpoint URL used to send requests to the NVIDIA model (either deployed on-premises or in a serverless environment). It must be accessible from your Elasticsearch instance.

For most completion models, the URL is static, and NVIDIA differentiates between models using only the model parameter. If the url parameter isn’t specified during endpoint creation, the default value of https://integrate.api.nvidia.com/v1/chat/completions will be used.

To use a generative model for the Elasticsearch inference completion task, you configure an endpoint that supports completion operations. The service settings map must include the required configuration for the selected NVIDIA model.

Upon success, you’ll receive a 200 OK response. This response provides the details of your new Elasticsearch endpoint, which is configured for completion tasks.

The created endpoint allows you to generate both streaming and non-streaming completions. These refer to how the endpoint delivers its output. Non-streaming completions wait until the entire response is generated before sending it back in a single block, resulting in a single, slower response time. In contrast, streaming completions send the generated text back in small, continuous chunks as they’re produced, which allows you to start reading the response immediately. This continuous delivery creates the perception of faster interaction and is essential for real-time conversational interfaces.

Generating non-streaming completions

To generate non-streaming completions, you call the newly created endpoint with a request similar to the following:

You'll receive a successful 200 OK response, with the completion result:

Generating streaming completions

To use the streaming functionality for the completion task type, you need to send the identical request used for non-streaming completions, but with _stream included in the URL path:

This command will initiate a continuous flow of events, delivering a sequence of outputs similar to the example provided below:

This capability empowers users to easily integrate NVIDIA generative models directly into their Elastic applications, supporting both single-response and engaging streaming experiences for dynamic content generation.

Creating and using a chat completion inference endpoint

To enable more dynamic and flexible interactions than those supported by the standard completion inference endpoint, you configure a chat completion inference endpoint, specifically designed to handle chat-based completion tasks.

To identify the parameters required to construct the service settings map, refer to the completion inference endpoint section of this blog post. The same configuration principles apply to the chat completion inference endpoint.

The service settings map must include the required configuration settings for the selected NVIDIA model.

Upon success, you’ll receive a 200 OK response, which includes the details of your new Elasticsearch endpoint specifically for the chat completion task type.

You can now use the new endpoint to stream generated completions. Your request should resemble the following example:

The chat completion results will be delivered to you as a continuous stream of events, formatted as follows:

The chat completion capability, distinct from the simpler completion API, allows users to build stateful, multi-turn conversational AI applications directly within the Elastic Stack, using the full flexibility of NVIDIA models for dynamic user interactions following Elasticsearch inference chat completion API.

Creating and using a rerank inference endpoint

Reranking is the process of reordering the results from an initial search query to improve their relevance to your intent. Reranking is a second-stage relevance step that reorders the results returned by an initial retriever. In many cases, it uses a different model than the retriever itself, typically a cross-encoder model, which evaluates the query and each candidate document together to produce a more accurate relevance score. The output is a list of results ranked based on their relevancy, thereby drastically improving the quality and contextual accuracy of the search results.

To create a rerank inference endpoint, you first select an appropriate model that can perform reranking operations. NVIDIA lists its models in the NVIDIA Build model catalog. You can use the reranking label to select the appropriate models. You can also find NVIDIA’s reranking models in the left-hand navigation of this retrieval APIs documentation. NVIDIA includes rerankingand text embedding models in the Retrieval APIs section in its API documentation. When selecting a model, ensure that it explicitly supports rerank inference requests. Rerank models typically include an API description, such as “Ranks passages by their relation to a query.” This wording indicates that the model supports the rerank task type.

A good example of a reranking model is the nvidia/llama-nemotron-rerank-vl-1b-v2. You can access the deployment page for this model, which allows you to deploy this model on-premises.

Once you have selected a suitable model, open its API reference page, where you’ll find the parameters required to create an Elasticsearch inference endpoint. Two parameters are relevant:

• model_id(required): Specifies the NVIDIA model to be used for rerank inference.

• url(optional): The endpoint URL used by the inference endpoint to send requests to the NVIDIA service.

For most models, the URL is static and NVIDIA differentiates between models using only the model parameter. If the url parameter isn’t specified during endpoint creation, the default value

https://ai.api.nvidia.com/v1/retrieval/nvidia/reranking will be used. The nvidia/llama-nemotron-rerank-vl-1b-v2 model requires a custom URL to be specified, and it will be used in the example below.

To perform reranking tasks, you configure an inference endpoint that executes reranking operations. The service settings map must include the required configuration for the selected NVIDIA model.

The successful creation of your new Elasticsearch endpoint for the rerank task type will be confirmed by a 200 OK response, which will also provide the specific details of the endpoint.

You can then start using the new endpoint to perform a ranking operation with a request like the one shown below:

A successful HTTP 200 OK status will be returned, and the ranked entries will be included in the response. Since models are not deterministic, the results you receive may vary and may be ordered differently across calls, as the same outcome isn’t guaranteed each time.

Integrating the rerank capability with Elasticsearch and NVIDIA elevates search applications to deliver the most accurate, contextually relevant results. By using the NVIDIA reranking models within the search infrastructure of Elasticsearch, the system moves beyond simple keyword matching. This capability prioritizes the most relevant documents after the initial search, drastically improving the user experience and the utility of the data.

NVIDIA and Elasticsearch: What's next

The integration of Elasticsearch's inference API with NVIDIA marks a considerable step forward for users. By providing a standardized, simpler path to access NVIDIA's high-performance, optimized models, this integration significantly expands Elastic's capabilities. Users can now work with these models for key AI tasks, including generating text embeddings for vector search, generating and streaming content with completion models, building stateful conversational AI applications with chat completion, and drastically improving search result accuracy through reranking. This simplification streamlines the development of sophisticated AI-powered applications, from advanced RAG systems to dynamic conversational interfaces, making powerful AI more accessible for Elastic users.

Ready to get started?

• Explore the Elasticsearch API reference documentation to dive deeper into setup.

• Browse the NVIDIA model catalog to see all available models.

• Check out the NVIDIA API Documentation hub for integration guides and API references.

• Start your journey by creating an NVIDIA account and API key to begin integrating the models today.

Frequently asked questions

How do I connect Elasticsearch to NVIDIA hosted models?

Create an NVIDIA API key at build.nvidia.com, and then create an Elasticsearch inference endpoint using the nvidia service with your API key and a model_id. Elasticsearch's inference API supports four NVIDIA task types: text embedding, completion, chat completion, and reranking.

What's the default endpoint URL for NVIDIA text embedding models in Elasticsearch?

If no url is specified when creating the endpoint, Elasticsearch defaults to https://integrate.api.nvidia.com/v1/embeddings for text embedding tasks. Completion and chat completion tasks default to https://integrate.api.nvidia.com/v1/chat/completions instead.

Can I use NVIDIA NIM models deployed on my own infrastructure with Elasticsearch?

Yes. NVIDIA NIM supports on-premises deployment, and Elasticsearch's inference endpoint accepts a custom url parameter pointing to your self-hosted NIM endpoint instead of NVIDIA's serverless API.

How do I stream chat completion responses from NVIDIA models in Elasticsearch?

Append _stream to the chat completion endpoint's URL path (POST _inference/chat_completion/{id}/_stream). Elasticsearch returns results as a continuous event stream instead of a single blocking response, ending with a [DONE] event.

What's the difference between the completion and rerank task types in Elasticsearch's NVIDIA integration?

Completion and chat completion generate new text from a prompt. Reranking takes an existing list of documents and a query and then reorders them by relevance score using a cross-encoder model; it doesn't generate text, it rescores what you already retrieved.

How do NVIDIA's reranking models improve Elasticsearch search results?

NVIDIA's reranking models evaluate the query and each candidate document together, producing a relevance score used to reorder results beyond keyword matching. Elasticsearch's rerank endpoint returns each document's index and relevance score, so the highest-scoring passages surface first.

Do I need a paid NVIDIA account to use hosted models with Elasticsearch?

You need an NVIDIA account and an API key generated at build.nvidia.com; NVIDIA's build platform offers both free evaluation access and paid production tiers, depending on usage. Elasticsearch itself doesn't add separate licensing for the NVIDIA service beyond your existing NVIDIA account terms.

Prerequisites

• Elastic Cloud deployment running Elasticsearch 9.3+ or start a free trial

• Workflows enabled (Advanced Settings)

• Python 3.9+

• A Mistral API key

How LLM query routing works in this system

We're going to build a two-stage system: a router that decides how to answer, and an answering model that produces the response.

The router looks at the query and the metadata of the top search hits, like scores, categories, and complexity labels. From that, it picks one of two strategies: Answer directly from the top FAQ article, or synthesize across multiple articles with citations. That decision can be made from structured signals alone, so a small, fast model handles it.

The answering step varies. A single-article answer is bounded work that a small model does well and returns quickly. A multisource synthesis with citations benefits from a more capable model, and the extra time is worth it. Matching each query to the model that fits keeps simple answers fast and complex answers good.

Why use a small model for routing instead of the large model?

Because the router runs on every query, including the simple ones. A slow router makes every answer slow, even the ones a small model could have produced in a fraction of the time.

The key design choice is that the routing step only sees metadata, not full documents. A query like "my OTG isn't heating evenly" only needs to know that the top hits are in the "Product Troubleshooting - Appliances" category with issue_complexity: medium, not the full conversation transcripts. This keeps the classification prompt tiny (a few hundred tokens) and cheap. The full article content is only loaded in the response step once.

Set up AI connectors

We use two AI connectors for the workflow:

Connector

Model

Type

Role

Mistral Small

mistral-small-latest

Custom (OpenAI-compatible)

Classify query complexity from metadata, answer simple FAQ-style questions

Anthropic Claude Sonnet 4.6

Claude Sonnet

Elastic Managed LLM

Synthesize complex answers from multiple articles, with citations

Both connectors are billed per million tokens, with the smaller model costing significantly less. Routing simple queries to it saves money on top of the latency win. To learn more about Elastic Managed LLM, see this documentation.

The Claude Sonnet connector is already available as an Elastic Managed large language model (LLM). We only need to create a custom connector for Mistral using the .gen-ai connector type, which supports any OpenAI-compatible API. You can also create it through the Kibana UI.

All the setup code in this article is available in the companion notebook. You can run each section there as you follow along.

The connector ID is auto-generated by Kibana. We let the platform handle this instead of trying to set it manually.

Once created, the connector appears in the Connectors UI:

Load and index the dataset

We use the e-commerce-customer-support-qa dataset from Hugging Face. It contains 1,000 real customer support interactions from an ecommerce platform (BrownBox) with customer questions, agent solutions, issue categories, complexity levels, and customer sentiment.

The index mapping uses semantic_text with the .jina-embeddings-v5-text-small model from Elastic Inference Service. This field handles semantic search end-to-end: embedding generation, chunking, and querying. We use copy_to to aggregate the conversation and QA summary into a single searchable field:

Defining the query routing workflow in Elastic Workflows YAML

The routing workflow has four steps: semantic search, metadata-only classification, conditional branching, and a model-appropriate response step.

We use Elastic Workflows to encapsulate this routing logic. Workflows let us:

1. Expose the triaging as a tool in Elastic Agent Builder, so a conversational agent can call it.

2. Trigger it directly via manual execution, schedules, or alerts.

This flexibility means the same logic serves both programmatic and conversational interfaces without duplicating code.

Workflows are defined in YAML and configured directly in the Workflow UI (Elasticsearch > Workflows > Create a New Workflow). Each step can query Elasticsearch, call Kibana APIs, or prompt an LLM.

Here’s the complete workflow definition:

The workflow has four key parts:

1. search_es uses elasticsearch.search with a semantic query to find the five most relevant articles.

2. classify_querysends the customer query plus only metadata from the search results to Mistral Small. The prompt includes scores, categories, complexity labels, and product categories. This keeps the classification step cheap, preventing the use of large amounts of tokens.

3. route_by_complexity uses an if step to branch based on the classifier's output.

4. The response step depends on the route. For simple queries, Mistral Small gets the top FAQ article and rephrases it. For complex queries, Claude Sonnet gets all five articles and synthesizes a detailed response with citations. This is the only step where full document content is loaded.

Using the workflow as a tool in Agent Builder

Beyond the default triggers (manual, schedule, alerts), workflows can also be exposed as tools in Agent Builder. This adds a conversational layer where users interact through a chat interface, and the agent decides when to call the workflow.

We use the Agent Builder APIs to create the tool and the agent. After creating the workflow in the Kibana UI, copy its ID and use it to register the workflow as a tool:

Then create an agent that uses the tool:

The agent is now available in the Agent Builder UI in Kibana. You can also create the agent and its tools directly through the Agent Builder UI.

Once created, the agent appears in the Agent Builder UI with the workflow tool assigned:

Testing simple vs. complex query routing

Simple query

The workflow searches the knowledge base, finds a direct match in the FAQ articles about order tracking, classifies it as simple, and routes to Mistral Small. The response is concise and drawn directly from the matched article: instructions for using the "My Orders" section or the tracking number from the confirmation email.

Complex query

This query involves three distinct issues (damaged product, duplicate charge, address update) across different support categories. The workflow classifies it as complex and routes to Claude Sonnet, which synthesizes information from multiple knowledge base articles, addresses each issue separately, cites the relevant articles, and provides clear resolution steps for each.

Conclusion

Routing LLM queries by complexity in Elasticsearch reduces latency and cost for simple queries without sacrificing quality on complex ones. The small model answers FAQ-style queries in a fraction of the time the larger model would take, and the larger model is reserved for the queries that actually benefit from its capabilities. Cost savings come along for the ride: Simple queries routed to the smaller model are cheaper, too.

The pattern that makes this work is searching the knowledge base before routing. Without that context, the router is guessing based on surface-level cues. With it, the structure of the search results, like scores, categories, and complexity labels, tells the router whether the answer lives in a single article or needs synthesis across several. That's the actual signal for how to handle the query.

Elastic Workflows makes this possible without writing orchestration code. The entire routing logic lives in YAML inside Kibana, using native steps for search, LLM prompts, and conditional branching. Combined with Agent Builder, the same workflow serves programmatic triggers and conversational interfaces.

Next steps

• Try the notebook with the complete implementation.

• Add LLM monitoring with OpenRouter to track cost per routing tier.

• Explore Elastic Workflows for other automation patterns.

• Learn more about Agent Builder and how to expose workflows as conversational tools.

• Read about building AI agentic workflows with Elastic Agent Builder.

The complete, ready-to-clone code is available at github.com/elastic/terraform-ad-example.

Prerequisites

• An Elastic Cloud account with an organization-level API key. See Elastic Cloud regions, deployment templates, and instances for available regions and templates.

• Terraform installed (>= 1.0.0). The Elastic Stack provider version 0.14.0 or later is required for anomaly detection (AD) job and datafeed resource support. See Install Terraform.

• A terminal opened in a directory containing the clone of the git repo github.com/elastic/terraform-ad-example.

• A valid index suitable for AD jobs (that is, with a timestamp field) should exist in the Elasticsearch cluster once it’s deployed. In this example, the index is filebeat-nginx-elasticco-full.

Terraform project structure for anomaly detection

A modular layout means that each resource has its own module, so job configs, datafeeds and state controllers can be shared and reused across teams independently.

Outputs are key: They allow modules to be chained so that the datafeed automatically receives the job_id from the job module, and Terraform derives the correct creation and destruction order from this dependency graph.

Why separate state from config in Terraform ML jobs?

Note how the state modules are separate from the configuration modules. This reflects a real operational pattern in machine learning (ML): You’ll frequently need to stop a datafeed (for example, to reindex data) or close a job (for example, to reset a model after a pipeline incident) without changing the job's configuration at all. Keeping them separate means operational actions don't create noisy diffs in your config resources.

Configuring the Elastic Cloud deployment in Terraform

Providers and deployment

We use two providers: elastic/ec to provision the Elastic Cloud deployment; and elastic/elasticstack to manage the ML resources within it. The elasticstack provider's connection details are derived directly from the ec_deployment resource, so credentials are never hard-coded:

The deployment itself provisions Elasticsearch (with a dedicated ML node) and Kibana:

The ml block within elasticsearch is essential; it provisions a dedicated ML node. Without it, ML jobs cannot be opened. Here we allocate 1 GB of memory in a single availability zone, which is sufficient for this example. Depending on the characteristics of your AD job and your data, you may need to size your ML node differently.

Because the elasticstack provider references ec_deployment.demo directly, Terraform understands the dependency: It will provision the deployment first and then use the resulting credentials and endpoints automatically.

Wiring the modules together

The output references (module.job.job_id, module.datafeed.datafeed_id) create an implicit dependency graph: Terraform will always create the job before the datafeed, and the datafeed before its state resource. On destroy, the order is automatically reversed.

In the diagram below, solid arrows represent the implicit dependency graph created when one module’s outputs feed into another’s inputs. In contrast, the dotted arrow between job_state and datafeed_state denotes the explicit depends_on defined in the root main.tf.

The explicit depends_on on module.datafeed_state deserves explanation: There's no data flow between the datafeed state and job state modules, but the Elasticsearch API requires the job to be open before a datafeed can start. Without this dependency, Terraform would attempt both in parallel, which would fail.

We start with the job "closed" and the datafeed "stopped". We'll open and start them in later steps to demonstrate lifecycle management.

Module: Anomaly detection job

The job module defines the AD job configuration. Here's the resource itself (modules/job/main.tf):

There are a few things worth noting regarding module reuse:

• The Elasticsearch job API stores arbitrary metadata in a JSON object called custom_settings. In this module, that object is whatever you pass in as the Terraform variable custom_settings: The resource sets custom_settings = jsonencode(var.custom_settings), so the cluster receives the JSON encoding of that map. The default value that is defined in variables.tf is therefore exactly the default metadata (created_by = "terraform" and department = "ITOps) unless a caller overrides custom_settings when invoking the module (for example, to record ownership when importing a legacy job that was created outside Terraform).

• The same pattern applies to the other tunables: analysis_limits, model_snapshot_retention_days, and daily_model_snapshot_retention_after_days are variables with defaults so the module works out of the box, while teams can override them at the call site (for instance, raising model_memory_limit for a higher-cardinality job).

Variable

Default

Purpose

custom_settings

created_by = "terraform"

Arbitrary job metadata; override to record ownership

analysis_limits.model_memory_limit

(see variables.tf)

Tune up for higher-cardinality jobs

model_snapshot_retention_days

(see variables.tf)

Retention period for model snapshots

The full variable definitions and outputs are in the GitHub repo.

Module: Datafeed

The datafeed module connects an index pattern to an AD job and is the primary parameterization point for service teams. (modules/datafeed/main.tf):

The indices variable is the key parameterization point; each service team passes its own index pattern when calling the module.

Modules: Job state and datafeed state

Job state and datafeed state are managed by separate modules, so operational actions (stopping a datafeed, closing a job) don't require a config plan to execute.

How to run and apply the anomaly detection Terraform config

Set your API key and initialize

The repo also includes an elastic-env.sh helper for managing secrets. See the README for details.

Plan and create the resources

The plan shows all six resources that will be created:

• The Elastic Cloud deployment.

• The AD job.

• The datafeed.

• The two state resources.

• A scoped API key for bulk ingestion.

Review the output carefully; this is one of Terraform's greatest strengths. Here's the key section:

Once satisfied, apply:

Notice the creation order:

• The deployment provisions first (~2 minutes).

• Then the API key.

• Then the AD job.

• Then the datafeed and job state in parallel.

• And finally the datafeed state.

Terraform derived this order automatically from the dependency graph:

At this point, you can confirm the job exists in Kibana's ML UI; the nginx job will be visible in the closed state.

Load sample data

The initial terraform apply also creates a scoped Elasticsearch API key for bulk ingestion. We can use it to load some test data. The repo includes a file (sample_data.ndjson) with a few sample documents matching the job's expected fields (@timestamp, nginx.access.body_sent.bytes, and the influencer fields: nginx.access.geoip.city_name and nginx.access.user_agent.build). It can be loaded into the deployment using the Elasticsearch _bulk API:

In practice, you'd want many more documents (hundreds to thousands across weeks/months) for the anomaly detection model to learn meaningful baselines; this sample is enough to verify that the pipeline works end to end.

Open the job

Change the state parameter in the job_state module call (defined in the top level main.tf file):

Terraform updates only the job state resource; the job configuration and datafeed are untouched:

Start the datafeed

Similarly, update the datafeed state:

The datafeed is now running. Since we haven't specified start or end times, it will process all available data in its indices and will continue polling for new data in real time.

Cleaning up

When you're done, a single command tears everything down in the correct reverse order:

• Datafeed state first.

• Then job state.

• Then datafeed.

• Then job.

• Then API key.

• And then the deployment:

How do you promote anomaly detection jobs from dev to production with Terraform?

With this modular structure, promoting a job from dev to production becomes a variable change rather than a manual migration. The platform team validates the job against a dev cluster and then updates a single variable:

The same Terraform configuration, the same modules, the same reviewed workflow, just different parameters.

In practice, you'd use separate Terraform workspaces or .tfvars files per environment, feeding into a continuous integration and continuous deployment (CI/CD) pipeline.

How do I import existing anomaly detection jobs into Terraform?

If you already have AD jobs running that were created through the UI or API, the provider supports importing them into Terraform state:

This lets you gradually shift legacy jobs under Terraform management without recreating them.

What's next

Future releases of the Elasticsearch Terraform provider will add support for ML calendars and filters resources. In the meantime, this modular pattern can be extended to manage other Elasticsearch resources alongside your AD jobs.

To experience the full benefits, upgrade to 9.3 (or later) or start your Elastic Security free trial. If you're also managing detection rules, see Managing Elastic Security Detection Rules with Terraform.

Resources

• Full example code on GitHub

• Anomaly Detection in Terraform documentation

• Elastic Stack Terraform Provider documentation

• Elastic Cloud Terraform Provider documentation

• Elastic Cloud regions, deployment templates, and instances

Frequently Asked Questions

How do I manage Elastic anomaly detection jobs with Terraform?

Use the Elastic Stack Terraform provider (v0.14.0 or later), which includes native resources for anomaly detection jobs (elasticstack_elasticsearch_ml_anomaly_detection_job), datafeeds and their operational state. A single terraform apply provisions the job, datafeed and all state resources in the correct dependency order.

Why should I separate anomaly detection job state from job configuration in Terraform?

Job state (open/closed) and datafeed state (started/stopped) change frequently during normal operations (reindexing, model resets, pipeline incidents) without any change to the underlying configuration. Keeping them in separate Terraform modules means operational actions don't produce diffs in your config resources and don't require a full config plan to execute.

Can I import existing Elastic anomaly detection jobs into Terraform without recreating them?

Yes. The Elastic Stack Terraform provider supports terraform import for existing AD jobs. Run terraform import module.job.elasticstack_elasticsearch_ml_anomaly_detection_job.nginx <deployment_id>/nginx to bring a job created through the Kibana UI or Elasticsearch API under Terraform management without deleting and recreating it.

How do I promote an anomaly detection job from a dev to a production cluster with Terraform?

With a modular Terraform layout, environment promotion is a variable change. Update ec_region and indices in your .tfvars file or workspace variable, then run terraform apply against the production cluster. The same reviewed configuration runs in both environments: no manual migration, no UI steps.

What size ML node do I need for Terraform-managed anomaly detection on Elastic Cloud?

The example allocates 1 GB of memory in a single availability zone, which is sufficient for low-cardinality AD jobs. Higher-cardinality jobs or larger datasets require a larger size value in the ml block of the Elasticsearch resource and the model_memory_limit variable in the job module is the primary tuning point.

Why does the Elasticsearch Terraform provider require an explicit depends_on between datafeed state and job state?

The Elasticsearch API requires a job to be open before its datafeed can start. Because there is no data flow between the two state modules, Terraform would otherwise attempt to start both in parallel and fail. The explicit depends_on = [module.job_state] in the root main.tf enforces the required sequencing.

What is the difference between the Elastic Cloud Terraform provider and the Elastic Stack Terraform provider?

The elastic/ec provider provisions Elastic Cloud infrastructure (deployments, node topology, regions). The elastic/elasticstack provider manages resources within a running Elasticsearch cluster (ML jobs, datafeeds, security API keys, index settings). A typical setup uses both: ec to create the deployment, elasticstack to configure it, with credentials passed automatically between them.

This post explains how the query, discovery, and metadata endpoints build on the earlier ingest and query work to form that API surface. Companion posts go deeper on individual pieces:

• Native PromQL support in ES|QL covers how PromQL queries are translated into ES|QL execution plans.

• Ship Prometheus Metrics to Elasticsearch with Remote Write covers ingestion setup.

• How Prometheus Remote Write Ingestion Works in Elasticsearch covers the remote write internals.

This is still a work in progress. The sections below call out what is supported today and which parts are still evolving.

The API surface

Today, the Prometheus-compatible API surface falls into three groups.

Query endpoints

The query endpoints let Prometheus-compatible clients evaluate PromQL expressions:

• GET /_prometheus/api/v1/query_range evaluates a PromQL expression over a time window (matrix results).

• GET /_prometheus/api/v1/query evaluates at a single point in time (vector results). Currently implemented as a short range query that returns the last sample.

Only GET is supported for query endpoints today. Some clients default to POST, so you may need to configure them to use GET. The Prometheus POST convention uses application/x-www-form-urlencoded bodies, which Elasticsearch's HTTP layer rejects as a CSRF safeguard before the request ever reaches the handler.

For the full PromQL coverage status, see the companion post on PromQL in ES|QL.

Metadata endpoints

The metadata endpoints serve the discovery information that clients need for autocomplete, variable dropdowns, and metric browsing.

The series, labels, and label values endpoints all accept match[] selectors and a time range (start/end). The match[] parameter takes a Prometheus series selector like http_requests_total{job="api"} and restricts the response to time series that match. This keeps responses fast and relevant on clusters with large numbers of metrics. For example:

The first returns all series for http_requests_total where job="api", with their full label sets. The second returns only the label names that exist on http_requests_total series. The third returns only the instance values that appear on matching series.

GET /_prometheus/api/v1/metadata is different: it returns type and unit for each metric, optionally filtered by name via a metric parameter.

It does not accept match[] selectors or a time range. In Prometheus, metadata is collected from active scrape targets (the HELP, TYPE, and UNIT lines they expose), so the response does not involve a data scan. Elasticsearch does not have a dedicated metadata store like that, so the current implementation discovers metric metadata by visiting time series data from the last 24 hours. This keeps the query fast without requiring a full index scan. That 24-hour lookback is fixed today: the Prometheus metadata API does not expose start or end parameters that Elasticsearch could use to make it user-adjustable.

How the metadata endpoints work under the hood, including the TS_INFO and METRICS_INFO commands that power them, is covered below.

Index pre-filtering

All query and metadata endpoints accept an optional {index} path segment after /_prometheus/:

This restricts which Elasticsearch indices the query runs against before any expression evaluation begins. On clusters with many data streams across teams or environments, this avoids scanning unrelated indices and can significantly reduce query latency. You can configure separate data sources per index pattern to give teams scoped access to their own metrics.

A note about Remote Write

For ingestion, Elasticsearch also exposes the standard Prometheus Remote Write endpoint:

• POST /_prometheus/api/v1/write ingests time series via the Prometheus Remote Write v1 protocol. v2 is not yet supported.

Remote Write writes into Elasticsearch's existing time series data streams (TSDS), not a separate Prometheus-specific storage layer. Prometheus labels become TSDS dimensions, and metric names become fields in the index mapping. The remote write architecture post covers the full mapping in detail, including how metric types are inferred and how labels are stored with a labels. prefix.

How it works

Under the hood, all endpoints work the same way: parse the incoming HTTP parameters, build an ES|QL query plan, execute it against time series data streams, and convert the columnar result back into the JSON format Prometheus clients expect.

TS_INFO and METRICS_INFO

The metadata endpoints need to answer questions like "what labels exist?" or "what metric types are defined?" across potentially millions of time series, without scanning every data point.

Internally, the Prometheus metadata endpoints answer those questions by building ES|QL plans around two new processing commands: METRICS_INFO and TS_INFO. You do not need to use these commands directly to use the Prometheus API, but they are the core execution primitives behind the metadata responses. Both work by visiting only one document per time series to extract its metadata, rather than scanning all samples. This means their cost scales with the number of distinct time series, not the number of data points.

METRICS_INFO returns one row per distinct metric with its name, type, unit, and associated dimension fields. TS_INFO is more granular: one row per (metric, time series) combination, including the actual dimension values as a JSON object.

A dedicated blog post on TS_INFO and METRICS_INFO is coming soon, covering the two-phase execution model, how they scale, and how to use them directly in ES|QL queries beyond the Prometheus API.

How the metadata endpoints use them

Each metadata endpoint constructs an ES|QL plan with one of these commands at its core.

/api/v1/labels and /api/v1/series use TS_INFO, since they need per-time-series detail (which labels exist, which dimension values identify each series). /api/v1/metadata and /api/v1/label/__name__/values use METRICS_INFO, since they only need per-metric information (metric names, types, units).

/api/v1/label/{name}/values for regular labels (anything other than __name__) does not use either command. Regular labels like job or instance are actual dimension fields in the index, so the endpoint can query them directly with a group-by aggregation. When match[] selectors are provided, they are translated into a WHERE clause that filters the time series before the aggregation runs.

The __name__ label needs a different strategy because it is not always present as a dimension field. Prometheus Remote Write does store labels.__name__, but metrics ingested through other paths (OpenTelemetry, the bulk API) do not have it. The metric name is encoded in the field name itself (e.g., metrics.http_requests_total). You could look at the index mappings to enumerate field names, but mappings alone do not tell you which metric has which dimensions, and they cannot be filtered by label values from a match[] selector. METRICS_INFO can do both: it enumerates metric names across indices while respecting upstream WHERE filters.

In all cases, the API layer handles the translation back to Prometheus conventions: stripping the labels. and metrics. storage prefixes and synthesizing __name__ for non-Prometheus metrics that lack it.

In conclusion

The result: any Prometheus-compatible client can query and explore Elasticsearch metrics through endpoints it already understands. Remote Write metrics, OpenTelemetry metrics, and metrics indexed through other paths all show up through the same API, backed by the same TSDS indices.

All the Prometheus APIs mentioned here are available as tech preview in Elasticsearch Serverless today. For self-managed clusters and Elastic Cloud Hosted deployments, available as tech preview in Elasticsearch 9.4, with the exception of GET /_prometheus/api/v1/metadata. To experiment locally, use start-local.

In this article, we’ll explore the benefits of building a custom Elasticsearch MCP server and show how to create one in TypeScript that connects Elasticsearch to LLM-powered applications.

Why build a custom Elasticsearch MCP server?

Elastic provides some alternatives for MCP servers:

• Elastic Agent Builder MCP server for Elasticsearch 9.2+

• Elasticsearch MCP server for older versions (Python)

If you need more control over how your MCP server interacts with Elasticsearch, building your own custom server gives you the flexibility to tailor it exactly to your needs. For example, Agent Builder's MCP endpoint is limited to Elasticsearch Query Language (ES|QL) queries, while a custom server allows you to use the full Query DSL. You also gain control over how results are formatted before being passed to the LLM and can integrate additional processing steps, like the OpenAI-powered summarization we'll implement in this tutorial.

By the end of this article, you’ll have an MCP server in TypeScript that searches for information stored in an Elasticsearch index, summarizes it, and provides citations. We'll use Elasticsearch for retrieval, OpenAI's gpt-4o-mini model to summarize and generate citations, and Claude Desktop as the MCP client and UI to take in user queries and give responses. The end result is an internal knowledge assistant that helps engineers discover and synthesize best practices across their organization’s technical docs.

Prerequisites:

• Node.js 20 +

• Elasticsearch

• OpenAI API key

• Claude Desktop

What is MCP?

MCP is an open standard, created by Anthropic, that provides secure, bidirectional connections between LLMs and external systems, like Elasticsearch. You can read more about the current state of MCP in this article.

The MCP landscape is evolving every day, with servers available for a wide range of use cases. On top of that, it’s easy to build your own custom MCP server, as we’ll show in this article.

MCP clients

There’s a long list of available MCP clients, each with its own characteristics and limitations. For simplicity and popularity, we’ll use Claude Desktop as our MCP client. It will serve as the chat interface where users can ask questions in natural language, and it will automatically invoke the tools exposed by our MCP server to search documents and generate summaries.

Creating an Elasticsearch MCP server

Using the TypeScript SDK, we can easily create a server that understands how to query our Elasticsearch data based on a user query input.

Here are the steps in this article to integrate the Elasticsearch MCP server with the Claude Desktop client:

1. Configure MCP server for Elasticsearch.

2. Load the MCP server into Claude Desktop.

3. Test it out.

Configure MCP server for Elasticsearch

To begin, let's initialize a node application:

This will create a package.json file, and with it, we can start installing the necessary dependencies for this application.

• @elastic/elasticsearch will give us access to the Elasticsearch Node.js library.

• @modelcontextprotocol/sdk provides the core tools to create and manage an MCP server, register tools, and handle communication with MCP clients.

• openai allows interaction with OpenAI models to generate summaries or natural language responses.

• zodhelps define and validate structured schemas for input and output data in each tool.

ts-node, @types/node, and typescript will be used during development to type the code and compile the scripts.

Set up the dataset

To provide the data that Claude Desktop can query using our MCP server, we’ll use a mock internal knowledge base dataset. Here’s what a document from this dataset will look like:

To ingest the data, we prepared a script that creates an index in Elasticsearch and loads the dataset into it. You can find it here.

MCP server

Create a file named index.ts and add the following code to import the dependencies and handle environment variables:

Also, let’s initialize the clients to handle the Elasticsearch and OpenAI calls:

To make our implementation more robust and ensure structured input and output, we'll define schemas using zod. This allows us to validate data at runtime, catch errors early, and make the tool responses easier to process programmatically:

Learn more about structured outputs here.

Now let’s initialize the MCP server:

Defining the MCP tools

With everything configured, we can start writing the tools that will be exposed by our MCP server. This server exposes two tools:

• search_docs: Searches for documents in Elasticsearch using full-text search.

• summarize_and_cite: Summarizes and synthesizes information from previously retrieved documents to answer a user question. This tool also adds citations referencing the source documents.

Together, these tools form a simple “retrieve-then-summarize” workflow, where one tool fetches relevant documents and the other uses those documents to generate a summarized, cited response.

Tool response format

Each tool can accept arbitrary input parameters, but it must respond with the following structure:

• Content: This is the response of the tool in an unstructured format. This field is usually used to return text, images, audio, links, or embeddings. For this application, it will be used to return formatted text with the information generated by the tools.

• structuredContent: This is an optional return used to provide the results of each tool in a structured format. This is useful for programmatic purposes. Although it isn't used in this MCP server, it can be useful if you want to develop other tools or process the results programmatically.

With that structure in mind, let’s dive into each tool in detail.

Search_docs tool

This tool performs a full-text search in the Elasticsearch index to retrieve the most relevant documents based on the user query. It highlights key matches and provides a quick overview with relevance scores.

We configure fuzziness: “AUTO” to have a variable typo tolerance based on the length of the token that’s being analyzed. We also set title^2 to increase the score of the documents where the match happens on the title field.

summarize_and_cite tool

This tool generates a summary based on documents retrieved in the previous search. It uses OpenAI’s gpt-4o-mini model to synthesize the most relevant information to answer the user’s question, providing responses derived directly from the search results. In addition to the summary, it also returns citation metadata for the source documents used.

Finally, we need to start the server using stdio. This means the MCP client will communicate with our server by reading and writing to its standard input and output streams. stdio is the simplest transport option and works well for local MCP servers launched as subprocesses by the client. Add the following code at the end of the file:

Now compile the project using the following command:

This will create a dist folder, and inside it, an index.js file.

Load the MCP server into Claude Desktop

Follow this guide to configure the MCP server with Claude Desktop. In the Claude configuration file, we need to set the following values:

The args value should point to the compiled file in the dist folder. You also need to set the environment variables in the configuration file with the exact same names defined in the code.

Test it out

Before executing each tool, click on Search and Tools to make sure that the tools are enabled. Here you can also enable or disable each one:

Finally, let’s test the MCP server from the Claude Desktop chat and start asking questions:

For the question “Search for documents about authentication methods and role-based access control”, the search_docs tool is executed and returns the following results:

The response is, “Great! I found 5 relevant documents about authentication methods and role-based access control. Here's what was found:”

The tool call returns the source documents as part of its response payload, which are later used to generate citations.

It’s also possible to chain multiple tools in a single interaction. In this case, Claude Desktop analyzes the user's question and determines that it needs to first call search_docs to retrieve relevant documents and to then pass those results to summarize_and_cite to generate the final answer, all without requiring separate prompts from the user:

In this case, for the query “What are the main recommendations to improve authentication and access control across our systems? Include references.”, we obtained the following results:

As in the previous step, we can see the response from each tool for this question:

Note: If a submenu appears asking whether you approve the use of each tool, select Always allow or Allow once.

Conclusion

MCP servers represent a significant step toward standardizing LLM tools for both local and remote applications. Though full compatibility is still in the works, we’re moving fast in that direction.

In this article, we learned how to build a custom MCP server in TypeScript that connects Elasticsearch to LLM-powered applications. Our server exposes two tools: search_docs for retrieving relevant documents using Query DSL; and summarize_and_cite for generating summaries with citations via OpenAI models and Claude Desktop as client UI.

The future of compatibility between different client and server providers looks promising. Next steps include adding more functionalities and flexibility to your agent. There’s a practical article on how you can parameterize your queries using search templates to gain precision and flexibility.

Prerequisites

• Elasticsearch 9.3 or Elastic Cloud Serverless: You can create a cloud deployment following these instructions, or you can use the start-local quickstart instead.

• Python 3.12: Download Python here.

• Hugging Face access token.

Chat completions using a Hugging Face inference endpoint

First, we’ll build a practical example that connects Elasticsearch to a Hugging Face inference endpoint to generate AI-powered recommendations from a collection of blog posts. For the app knowledge base, we’ll use a dataset of company blog articles, which contains valuable but often hard-to-navigate information.

With this endpoint, semantic search retrieves the most relevant articles for a given query, and a Hugging Face LLM generates short, contextual recommendations based on those results.

Let’s take a look at a high-level overview of the information flow we’re going to build:

In this article, we’ll test SmolLM3-3B capacity tocombine its compact size with strong multilingual reasoning and tool-calling capabilities. Based on a search query, we’ll send all the matching content (in English and Spanish) to the LLM to generate a list of recommended articles with a custom-made description based on the search query and results.

Here’s what the UI of an article site with an AI recommendations generation system could look like.

You can find the full implementation of this application in the linked notebook.

Configuring Elasticsearch inference endpoints

To use the Elasticsearch Hugging Face inference endpoint, we need two important elements: a Hugging Face API key and a running Hugging Face endpoint URL. It should look like this:

The Hugging Face inference endpoint in Elasticsearch supports different task types: text_embedding, completion, chat_completion, and rerank. In this blog post, we use chat_completion because we need the model to generate conversational recommendations based on the search results and a system prompt.This endpoint allows us to perform chat completions directly from Elasticsearch in a simple way using the Elasticsearch API:

This will serve as the core of the application, receiving the prompt and the search results that will pass through the model. With the theory covered, let’s start implementing the application.

Setting up ​​inference endpoint on Hugging Face

To deploy the Hugging Face model, we’re going to use Hugging Face one-click deployments, an easy and fast service for deploying model endpoints. Keep in mind that this is a paid service, and using it may incur additional costs. This step will create the model instance that will be used to generate the recommendations of the articles.

You can pick a model from the one-click catalog:

Let’s pick the SmolLM3-3B model:

From here, grab the Hugging Face endpoint URL:

As mentioned in the Elasticsearch Hugging Face inference endpoints documentation, text generation requires a model that’s compatible with the OpenAI API. For that reason, we need to append the /v1/chat/completions subpath to the Hugging Face endpoint URL. The final result will look like this:

With this in place, we can start coding in a Python notebook.

Generating Hugging Face API key

Create a Hugging Face account, and obtain an API token by following these instructions. You can choose between three token types: fine-grained (recommended for production, as it provides access only to specific resources); read (for read-only access); or write (for read and write access). For this tutorial, a read token is sufficient, since we only need to call the inference endpoint. Save this key for the next step.

Setting up Elasticsearch inference endpoint

First, let’s declare an Elasticsearch Python client:

Next, let’s create an Elasticsearch inference endpoint that uses the Hugging Face model. This endpoint will allow us to generate responses based on the blog posts and the prompt passed to the model.

Dataset

The dataset contains the blog posts that will be queried, representing a multilingual content set used throughout the workflow:

Elasticsearch mappings

With the dataset defined, we need to create a data schema that properly fits the blog post structure. The following index mappings will be used to store the data in Elasticsearch:

Here, we can see more clearly how the data is structured. We’ll use semantic search to retrieve results based on natural language, along with the copy_to property to copy the field contents into the semantic_text field. Additionally, the title field contains two subfields: the original subfield stores the title in either English or Spanish, depending on the original language of the article; and the translated_title subfield is present only for Spanish articles and contains the English translation of the original title.

Ingesting data

The following code snippet ingests the blog posts dataset into Elasticsearch using the bulk API:

Now that we have the articles ingested into Elasticsearch, we need to create a function capable of searching against the semantic_text field:

We also need a function that calls the inference endpoint. In this case, we’ll call the endpoint using the chat_completiontask type to get streaming responses:

Now we can write a function that calls the semantic search function, along with the chat_completions inference endpoint and the recommendations endpoint, to generate the data that will be allocated in the cards:

Finally, we need to extract the information and format it to be printed:

Let’s test this by asking a question about the security blog posts:

Here we can see the cards in the console generated by the workflow:

You can see the full results, including all hits and the LLM response, in this file.

We’re asking for articles related to: “Security and vulnerabilities.” This question is used as the search query against the documents stored in Elasticsearch. The retrieved results are then passed to the model, which generates recommendations based on their content. As we can see, the model did a great job generating engaging short text that can motivate the reader to click on it.

Conclusion

This example shows how Elasticsearch and Hugging Face can be combined to create a fast and efficient centralized system for AI applications. This approach reduces manual effort and provides flexibility, thanks to Hugging Face’s extensive model catalog. Using SmolLM3-3B, in particular, shows how compact, multilingual models can still deliver meaningful reasoning and content generation when paired with semantic search. Together, these tools offer a scalable and effective foundation for building intelligent content analysis and multilingual applications.

In addition to Python and JavaScript, the LangChain ecosystem also has a community-driven Java project called LangChain4j, which will be the focus of this article, showing how powerful hybrid search can be by writing a complete application using LangChain4j, Elasticsearch, and Ollama.

Setting up the environment

Running a local Elasticsearch instance

Before running the examples, you'll need Elasticsearch running locally. The easiest way is using the start-local script:

After starting, you'll have:

• Elasticsearch at http://localhost:9200.

• Kibana at http://localhost:5601.

Your API key is stored in the .env file (under the elastic-start-local folder) as ES_LOCAL_API_KEY.

> Note: This script is for local testing only. Do not use it in production. For production installations, refer to the official documentation for Elasticsearch.

Running a local Ollama instance

You’ll also need to connect your application to an embedding model. Although you can choose between any provider supported by LangChain4j (check the complete list), for this example we’ll be using Ollama, which can be easily set up locally following the quickstart.

Let’s start coding

The idea for the application is simple: Given a dataset of movies (taken from an IMDb dataset on Kaggle), we want to be able to find movies whose descriptions are relevant to our queries. This demo uses a subset of the data, which has been cleaned. You can download the dataset used for this article from our GitHub repo, along with the full code for this demo.

Step 1: Dependencies and environment

Open your favorite integrated development environment (IDE), create a new blank project, preferably with a modern Java version (we’re using Java24) and a gradle/maven version to match (in our case, Gradle 9.0).

We only need three dependencies:

The first one is needed to ingest the data that we’ll embed and query; the other two are the necessary LangChain4j dependencies to connect and manage our Elasticsearch vector store and Ollama embedding model.

The best way to connect to the external services is to set up environment variables and set them at the start of our main function:

Step 2: Ingesting the dataset

Since the dataset is a CSV, we’ll be using Jackson dataformat’s jackson-dataformat-csv to easily read the data and map it to a Java class, defined as:

Now we can create an instance of CsvSchema mapping the CSV structure and read the file into an iterator:

Each row needs to be embedded first, and then both the embedded content and the text representation will be ingested by Elasticsearch.

Let’s start by creating an instance of the Ollama embedding model class:

And then the Elasticsearch vector store, which needs an instance of the Elasticsearch Java RestClient:

For the ingestion loop, the LangChain4j library requires the data to be split in two lists for ingestion, one for the vector representation and one for the original text, so we’ll set up two lists which will be filled by the loop:

Where Embedding and TextSegment are both library specific classes.

We’ll iterate on the movie dataset iterator, use the embedding model to retrieve the vector representation for each movie information (a text representation of all the fields merged), and add the name separately as metadata so that the result will be easier to read.

Finally, the vector list and text list are passed to the vector store method addAll(), which will handle asynchronously sending the data to the vector store:

Step 3: Querying

Our goal is to find movies with time loops in the plot, so our prompt will be:

Let’s try a simple vector search first, by creating a content retriever with a k-nearest neighbor (kNN) query default configuration and then running the query and printing the results:

This outputs:

Now let’s see how hybrid search performs:

Why these results?

This query (“time loop / reliving the same day”) is a great case where hybrid search tends to shine because the dataset contains literal phrases that BM25 can match and vectors can still capture meaning.

• Vector-only (kNN) embeds the query and tries to find semantically similar plots. Using a broad sci‑fi dataset, this can drift into “trapped / altered reality / memory loss / high-stakes sci‑fi” even when there’s no time-loop concept. That’s why results like “The Witch: Part 1 – The Subversion” (amnesia) and “The Maze Runner” (trapped / escape) can appear.

• Hybrid (BM25 + kNN + reciprocal rank fusion [RRF]) rewards documents that match keywords and meaning. Movies whose descriptions explicitly mention “time loop” or “relive the same day” get a strong lexical boost, so titles like “Edge of Tomorrow” (relive the same day over and over again…) and “Boss Level” (trapped in a time loop that constantly repeats the day…) rise to the top.

Hybrid search doesn’t guarantee that every result is perfect; it balances lexical and semantic signals, so you may still see some non-time-loop sci‑fi in the tail of the top‑k.

The main takeaway is that hybrid search helps anchor semantic retrieval with exact textual evidence when the dataset contains those keywords. Check the previous article for more information on how hybrid search works.

Full code example

You can find the full demo code on GitHub.

Conclusion

In this article, we demonstrated how to use hybrid search in LangChain4j through its Elasticsearch integrations, with a complete Java example. This article is an extension of a previous article, which presents the LangChain integrations for Python and JavaScript and introduces and explains hybrid search. We’re planning to continue our collaboration with LangChain4j in the future by contributing to the embedding models with our Elasticsearch Inference API.

What real problem does it solve for engineers?

OpenClaw is a self-hosted gateway for AI agents: a single runtime that coordinates execution, treats agents as isolated processes, and uses skills (structured instructions in markdown files) as the unit of integration. Conceptually, this isn’t entirely different from what we already do with command line interfaces (CLIs) and scripts, but it’s now formalized around agent-driven workflows.

This led to a practical exploration within the Elastic Stack:

If we treat OpenClaw as an orchestration runtime, how does it behave when Elasticsearch is the back end? And how straightforward is integration using OpenClaw skills?

Let's build an integration using composable skills.

Solution architecture

In this tutorial, we’ll teach OpenClaw how to access and query Elasticsearch data through a custom read-only skill, and we’ll then demonstrate how it composes multiple skills together; for example, combining Elasticsearch queries with real-time weather data to generate dynamic reports.

Before diving into the hands-on steps, let’s look at what we’re building. The solution is composed of three integrated layers that work together through OpenClaw orchestration.

Layer 1: Storage and search (Elasticsearch)

The data layer runs on Elasticsearch via start-local, a single command that spins up Elasticsearch and Kibana locally with Docker.

Two sample indices demonstrate different use cases:

• fresh_produce: 10 products with semantic search (ecommerce scenario)

• app-logs-synthetic: 30 log entries across four services (observability scenario)

The same read-only skill works with both indices without any reconfiguration; the agent inspects the mapping and adapts its queries accordingly.

Layer 2: Orchestration (OpenClaw Gateway)

The gateway receives natural language requests and loads the Elasticsearch skill, and the large language model (LLM) decides which queries to construct. The skill is a pure SKILL.md with reference docs, meaning that its operations require no custom code.

To understand how the gateway organizes this, two core OpenClaw concepts are worth knowing:

• Agents: Independent AI instances, each with its own configuration, workspace, and set of skills. You can run multiple agents for different purposes.

• Workspace: A folder that defines an agent’s context:AGENTS.md (the agent’s permanent briefing), .env(credentials), and a skills/ directory. Think of it as the agent’s working environment.

Layer 3: Skills (composable capabilities)

Skills are structured instructions in markdown files (SKILL.md) that teach the agent how to use specific tools or APIs. They can be global (available to all agents), workspace-specific, or bundled with OpenClaw. The agent selectively loads only the skills relevant to each request.

This tutorial uses two skills:

• Elasticsearch-openclaw (custom, built for this tutorial): A read-only skill that teaches the agent how to search, filter, aggregate, and explore Elasticsearch indices using curl.

• Weather (community skill, used for composition demo): A skill that fetches current weather conditions from external APIs.

Later in the tutorial, we'll demonstrate how OpenClaw composes both skills in a single request, querying Elasticsearch products based on real-time weather data without any custom integration code.

Read-only by design

The elasticsearch-openclaw skill is read-only by design. It provides patterns for searching, filtering, and aggregating data, but it never writes, updates, or deletes. This minimizes the security footprint when giving AI agents access to your Elasticsearch cluster.

Even if the agent environment is compromised, your data remains safe from modification or deletion. This is enforced through:

• Skill design: No write operation patterns in SKILL.md or reference files.

• API key permissions: The tutorial uses a read-only API key with only read and view_index_metadata privileges.

• Agent instructions: AGENTS.md explicitly states "You can SEARCH, FILTER, and AGGREGATE data, but you can NEVER write, update, or delete."

This security-first approach is why infrastructure setup (index creation, data loading) must be done manually; by design, the agent cannot do it for you.

Prerequisites

To follow this tutorial, you’ll need:

Software and tools:

• Docker Desktop installed and running (Docker Engine with Compose V2).

• Elasticsearch running locally via start-local. (We’ll set this up in the next section.)

• Jina API key (free): https://jina.ai/embeddings.

• OpenClaw installed: https://openclaw.ai.

Setting up the environment

Start by cloning the starter project, which contains the skill, workspace configuration, and Dev Tools scripts:

The repository contains:

Note: The devtools*.md files contain Kibana Dev Tools commands formatted as reference documentation.

Installing OpenClaw

OpenClaw is a self-hosted gateway. This means you maintain full control over execution and data, but you need to prepare your local environment or server.

I installed OpenClaw on a separate machine, which is why I included the disclaimer below.

** Security and responsibility disclaimer **

Since OpenClaw is an early-stage, rapidly evolving open-source project, the community has raised important discussions about potential security vulnerabilities, especially around token handling and third-party script execution.

Deployment recommendations:

• Isolated environments: If you’re not an advanced infrastructure security user, we recommend installing OpenClaw strictly in isolated, controlled environments (such as a dedicated virtual machine [VM], a rootless Docker container, or a test machine).

• Do not use in production: Avoid running the gateway on servers containing sensitive data or with unrestricted access to your corporate network until the project reaches a more stable, audited version.

• Least privilege: We reinforce the need to use Elasticsearch API keys with restricted permissions (read-only) to mitigate risks, in case the environment is compromised.

• Network segmentation: Both Elasticsearch and OpenClaw bind to localhost by default. Keep it that way, unless you have a specific reason to expose them.

• Credential rotation: Rotate API keys periodically. OpenClaw stores credentials locally, so treat the machine’s security as the perimeter.

• Audit logging: Enable Elasticsearch audit logging to track all API calls made by OpenClaw. This creates a full trail of what the agent accessed and when.

• Keep the installation up to date.

For a deeper analysis of the security architecture and deployment options, consult the official OpenClaw documentation.

Runtime installation

OpenClaw manages daemons and skill isolation via CLI. Since it’s a recent project that has undergone naming changes, we recommend strictly following the official documentation to ensure installation compatibility.

Preparing the Elasticsearch back end

Before connecting any agent runtime, we need a working Elasticsearch environment with data to query and a secure, read-only access layer. In the next two sections, we’ll spin up Elasticsearch locally using start-local, create an index with semantic_text and Jina v5 embeddings, load sample data, validate that semantic search works, and generate a read-only API key. Once this foundation is in place, the Elasticsearch side is complete and we can focus entirely on teaching the agent how to use it.

Part 1: Setting up Elasticsearch locally

Start a local Elasticsearch and Kibana instance with a single command:

Once complete: Elasticsearch at http://localhost:9200, Kibana at http://localhost:5601, and credentials in elastic-start-local/.env.

Part 2: Configuring the index in Kibana Dev Tools

Open http://localhost:5601 → Dev Tools and run devtools_fresh_produce.md in order.

• Step 1: Replace YOUR_JINA_API_KEY with your actual Jina API key (free).

• Step 2: Save the encoded field immediately; it cannot be retrieved later.

The key commands in the Dev Tools file are:

Create the Jina inference endpoint:

Create the index with semantic_text:

The semantic_text field type handles embedding generation automatically at index time.

Index sample products using the bulk API (see devtools_fresh_produce.md for the full dataset of 10 products).

Validate semantic search:

The semantic query type handles inference on the query side automatically; no need to specify model IDs or embedding details.

Create a read-only API key:

Save the encoded value from the response. This is your API key for the OpenClaw configuration.

Connecting to OpenClaw

With the Elasticsearch back end ready, we can now wire it into OpenClaw. Several Elasticsearch integrations already exist in the ecosystem, from Elastic’s own Model Context Protocol (MCP) server to community-built MCP servers. However, most of these offer full CRUD access or are designed for different agent runtimes. Given that the technology is still in its early stages and security remains a primary concern, I chose to build a dedicated skill, simple, read-only, and purpose-built for OpenClaw. This approach ensures that the agent can search, filter, and aggregate data but never modify it, keeping the blast radius minimal even if the environment is compromised.

In the next sections, we’ll configure credentials, install the skill, create a dedicated agent, and explore how the workspace ties everything together.

Install the skill and create the agent

Step 1: Configure credentials

From the cloned repository, configure the credentials by copying the environment template and filling in your Elasticsearch URL and the read-only API key:

Edit the .env file with these two values:

Example .env file:

Step 2: Install the skill from ClawHub

ClawHub is OpenClaw's public skill registry. Think of it as npm for AI agent skills. At the time of this writing, ClawHub hosts over 3,200 skills, covering everything from Slack and GitHub integrations to Internet of Things (IoT) device automation. For this tutorial, we created elasticsearch-openclaw, a custom skill focused on read-only queries using semantic_text, aggregations, and observability on Elasticsearch 9.x. It’s published on ClawHub so you can install it directly. As a best practice, only install skills from trusted sources with known provenance; as with any package manager, review the content before granting access to your agent.

The elasticsearch-openclaw skill is published on ClawHub.

Recommended: Open the OpenClaw Web UI (http://127.0.0.1:18789/) and ask:

OpenClaw will:

• Fetch the skill from ClawHub.

• Install it in the appropriate directory.

• Confirm when ready to use.

Step 3: Create the agent

Do this by registering a dedicated agent with its own workspace, and then restart the gateway to load the new configuration:

Understanding the workspace

Now that the agent is running, let’s look at what makes it tick.

AGENTS.md

The AGENTS.md file is the agent’s permanent briefing. It defines who the agent is, what it can do, and how it should behave. For our Elasticsearch agent, this file instructs the agent about the available indices, the read-only constraint, and the preferred query patterns.

Skills: When they make a difference

Without skill

With `elasticsearch-openclaw` skill

Agent has no knowledge of Elasticsearch query syntax.

Agent knows semantic, full-text, filtered, and aggregation patterns.

Agent might attempt write operations.

Agent is instructed to never write, update, or delete.

Agent guesses field names and types.

Agent inspects mappings first and then constructs appropriate queries.

Generic curl commands with trial and error.

Structured query templates with best practices for Elasticsearch 9.x.

Exploring with the agent

With the Elasticsearch back end configured and the OpenClaw agent connected, it’s time to see what the agent can actually do. In the next sections, we’ll test natural language queries, explore observability data, and compose multiple skills together.

Testing in OpenClaw

Open the OpenClaw web UI, and try some natural language queries. The agent will inspect the index mapping, choose the appropriate query type, and return results.

Type:

Result:

Others ideas to explore:

• Index exploration: > “What indices do I have in Elasticsearch? Show me the fields of fresh_produce.”

• Filtered search: > “Show me all products on sale under $15.”

• Aggregations: > “What’s the average price by category?”

Observability

To demonstrate that the skill works beyond a single use case, the repository includes a second index: app-logs-synthetic, with 30 synthetic log entries across four fictional services, created from devtools_app_logs_synthetic.md.

Setting up the log data

Since the skill is read-only, you need to populate the index first. The devtools_app_logs_synthetic.md file contains five commands (three for setup and two for verification):

• Create ingest pipeline: Adds @timestamp to log entries automatically.

• Create index mapping: Defines the app-logs-synthetic structure (classic fields only, no semantic_text).

• Bulk insert logs: Loads 30 synthetic log entries across four services.

• Count query: Verify 30 documents were indexed.

• Sample search: Quick test to confirm that data is queryable.

How to run:

1. Open Kibana Dev Tools: http://localhost:5601 → Dev Tools.

2. Copy each numbered block from the .md file.

3. Paste into the Dev Tools console.

4. Press Ctrl/Cmd+Enter to execute.

5. Wait for a successful response before continuing to the next block.

This creates the app-logs-synthetic index with sample data ready for querying.

Try this query in the OpenClaw web UI:

Result:

Other ideas to explore:

• “How many 500 errors do I have in app-logs-synthetic? Which services are failing?”

• “Which endpoints have the slowest response times?”

• “What happened with the payment-service in the last 24 hours?”

This is the same skill, same agent, same setup, just pointed at different data. The agent inspects the new index mapping, adapts its queries, and returns relevant results without any reconfiguration.

Composing skills in action

This is where composable skills truly shine. Start by asking the agent:

OpenClaw will search for the weather skill, automatically attempt the installation, and guide you through the process. Just follow the on-screen instructions; no new API key is required for the weather skill. Afterward, try this:

In a single request, the agent chains multiple skills: the weather skill to check current conditions, the Elasticsearch skill to run a hybrid search on products that match the context, and its built-in file and browser tools to generate an HTML report and open it. No custom integration code, no glue scripts, just skills composed by the LLM at runtime.

This is what makes OpenClaw different from a traditional automation framework. You don’t preprogram the workflow. You describe the outcome, and the agent figures out the composition.

Conclusion

SearchClaw started as a simple experiment and ended up demonstrating what composable, LLM-driven integration looks like in practice. The key takeaway is not the individual tools (all are familiar) but the approach. Instead of writing a specific application with hardcoded queries, we gave the agent capabilities and let it compose solutions dynamically. This is what makes OpenClaw native: composable, LLM-driven, and local-first.

As with any early-stage project, OpenClaw should be used thoughtfully, especially regarding security and environment isolation. The read-only skill approach demonstrated here is one way to limit risk while still unlocking the value of your Elasticsearch data.

The full code is available in the repository and can serve as a starting point for your own integrations: https://github.com/salgado/elasticsearch-openclaw-start-blog.

We’re also planning to support hybrid search in the community-driven Java integration very soon.

What is hybrid search?

Hybrid search is an information retrieval approach that combines keyword-based full-text search (lexical matching) with semantic search (vector similarity). Practically, it means a query can match documents because they contain the right terms and/or because they express the right meaning (even if the wording differs).In simple terms, you can think of it like this:

• Lexical retrieval: “Do these documents contain the words I typed (or related words)?”

• Semantic retrieval: “Do these documents mean something similar to what I typed?”

These two retrieval methods produce scores on different scales, so hybrid search systems typically use a fusion strategy to merge them into one ranking, for example, using reciprocal rank fusion (RRF).

In the figure above, we show an example: BM25 (keyword search) returns Docs A, B, and C, while semantic search returns Docs X, A, and B. The RRF algorithm then combines these two result lists into the final ranking: Doc A, Doc B, Doc X, and Doc C. With hybrid search, Doc C is included in the results thanks to BM25.

Why hybrid search matters

If you’ve built search or retrieval-augmented generation (RAG) features in production, you’ve probably seen the same failure modes show up again and again:

• Keyword search can be too literal. If the user doesn’t use the exact terms that appear in your documents, relevant content gets buried or missed.

• Semantic search can be too fuzzy. It’s great at meaning, but it can also return results that feel related while missing a critical constraint, like a product name, an error code, or a specific phrase the user actually typed.

Hybrid search exists because real user queries in production environments usually need both.

Next we’ll dive into how you get started with hybrid search in the LangChain integration for Python and JavaScript. If you want to read more about hybrid search, check out What is hybrid search?and When hybrid search truly shines.

Setting up a local Elasticsearch instance

Before running the examples, you'll need Elasticsearch running locally. The easiest way is using the start-local script:

After starting, you'll have:

• Elasticsearch at http://localhost:9200.

• Kibana at http://localhost:5601.

Your API key is stored in the .env file (under the elastic-start-local folder) as ES_LOCAL_API_KEY.

Getting started with hybrid search in LangChain (Python and JavaScript)

The dataset is a CSV with information on 1,000 science fiction movies, taken from an IMDb dataset on Kaggle. This demo uses a subset of the data, which has been cleaned. You can download the dataset used for this article from our GitHub gist, along with the full code for this demo.

Step 1: Install what you need.

First you’ll need the LangChain Elasticsearch integration and Ollama for embeddings. (You can also use some other embedding model if you wish.)

In Python:

In JavaScript:

Step 2: Configure your connection and dataset path.

In Python:

At the top of the script, we set:

• Where Elasticsearch is (ES_LOCAL_URL).

• How to authenticate (ES_LOCAL_API_KEY).

• Which demo index name to use (INDEX_NAME).

• Which CSV file we’ll ingest (scifi_1000.csv).

In JavaScript:

Notes for JavaScript:

• JavaScript uses process.env instead of os.getenv.

• Path resolution requires fileURLToPath and dirname for Elasticsearch modules.

• The class is called ElasticVectorSearch (not ElasticsearchStore as in Python).

We can now also create the client.

In Python:

In JavaScript:

Step 3: Ingest the dataset, and then compare vector-only vs. hybrid.

Step 3a: Read the CSV and build what we index.

We build three lists:

• texts: The actual text that will be embedded + searched.

• metadata: Structured fields stored alongside the document.

• ids: Stable IDs (so Elasticsearch can dedupe if needed).

In Python:

In JavaScript:

What’s important here:

• We don’t embed only the description. We embed a combined text block (title/year + director + genre + description). That makes results easier to print and sometimes improves retrieval.

• The same text is what the lexical side uses, too (in hybrid mode), because it’s indexed as searchable text.

Step 3b: Add texts to Elasticsearch using LangChain.

This is the indexing step. Here we embed texts and write them to Elasticsearch.

For asynchronous applications, please use AsyncElasticsearchStore with the same API.

You can find our reference docs for both the sync and async versions of ElasticsearchStore, along with more parameters for advanced fine-tuning RRF.

In Python:

In JavaScript:

Step 3c: Create another store for hybrid search.

We create another ElasticsearchStore object pointing at the same index but with different retrieval behavior: hybrid=False is vector-only search and hybrid=True is hybrid search (BM25 + kNN, fused with RRF).

In Python:

In JavaScript:

Step 3d: Run the same query both ways, and print results.

As an example, let’s run the query “Find movies where the main character is stuck in a time loop and reliving the same day." and compare the results from hybrid search and vector search.

In Python:

In JavaScript:

Example output

Why these results?

This query (“time loop / reliving the same day”) is a great case where hybrid search tends to shine because the dataset contains literal phrases that BM25 can match and vectors can still capture meaning.

• Vector-only (kNN) embeds the query and tries to find semantically similar plots. Using a broad sci‑fi dataset, this can drift into “trapped / altered reality / memory loss / high-stakes sci‑fi” even when there’s no time-loop concept. That’s why results like “The Witch: Part 1 – The Subversion” (amnesia) and “The Maze Runner” (trapped/escape) can appear.

• Hybrid (BM25 + kNN + RRF) rewards documents that match both keywords and meaning. Movies whose descriptions explicitly mention “time loop” or “relive the same day” get a strong lexical boost, so titles like “Edge of Tomorrow” (relive the same day over and over again…) and “Boss Level” (trapped in a time loop that constantly repeats the day…) rise to the top.

Hybrid search doesn’t guarantee that every result is perfect. It balances lexical and semantic signals so you may still see some non-time-loop sci‑fi in the tail of the top‑k.

The main takeaway is that hybrid search helps anchor semantic retrieval with exact textual evidence when the dataset contains those keywords.

Full code example

You can find our full demo code in Python and JavaScript, as well as the dataset used, hosted on GitHub gist.

Conclusion

Hybrid search provides a pragmatic and powerful retrieval strategy by combining traditional BM25 keyword search with modern vector similarity into a single, unified ranking. Instead of choosing between lexical precision and semantic understanding, you get the best of both worlds, without adding significant complexity to your application.

In real-world datasets, this approach consistently yields results that feel more intuitively correct. Exact term matches help anchor results to the user’s explicit intent, while embeddings ensure robustness against paraphrasing, synonyms, and incomplete queries. This balance is especially valuable for noisy, heterogeneous, or user-generated content, where relying on only one retrieval method often falls short.

In this article, we demonstrated how to use hybrid search in LangChain through its Elasticsearch integrations, with complete examples in both Python and JavaScript. We’re also contributing to other open-source projects, such as LangChain4j, to extend hybrid search support with Elasticsearch.

We believe hybrid search will be a key capability for generative AI (GenAI) and agentic AI applications, and we plan to continue collaborating with libraries, frameworks, and programming languages across the ecosystem to make high-quality retrieval more accessible and robust.