Optical character recognition (OCR) has been a goal of AI for a long time, and in some simple cases (such as straightforward layouts, crisp images, standard fonts, and highly resourced standardized languages), reliability can be very high using inexpensive conventional software. But real data is messy. Scans and photos of documents are often blurry, and images are compressed and quality varies. Plus, page layouts can be complex. And these examples don’t even take into consideration handwriting and global languages.

A vision-language model trained for OCR

jina-ocr-v1 is a vision-language model (VLM) with specialized training. Unlike all-purpose large language models (LLMs), it doesn’t answer questions. It’s trained to do one thing: output the text in an image in a coherent order matching how humans would read the image, preserving what’s important about the content’s visual structure.

This integrates into one model a feature set that otherwise requires a pipeline of multiple models and specialized programs to implement, including:

• Layout-aware document processing that produces Markdown format output, retaining as much of the structure of the original document in its text output as possible.

• Support for over 100 languages, with all major international languages and scripts represented.  

• Handwriting recognition, including block text in a wide array of languages and English cursive.

• Table extraction into basic HTML format, suitable for further processing and importing into spreadsheets or other applications.

• Math recognition, turning printed images of formulas into LaTeX math code that you can insert directly into any document engine or scientific software that supports LaTeX.

jina-ocr-v1 has a total of 3.4 billion parameters, but because it uses a mixture-of-experts architecture, it only uses a part of the model at any one time. Only 570 million parameters, about a sixth of the total, get used when responding to user input, but exactly which 570 million is determined dynamically at inference time. This means jina-ocr-v1 takes up as much memory as any other 3.4 billion parameter model, but runs as quickly as a 570 million parameter one.

jina-ocr-v1 specifications

Total size

3.4 billion (10⁹) parameters

Active parameters

570 million

Backbone

DeepSeek-OCR

Input resolution

1024x1024 but with additional higher resolution areas. Features not visible at 2048x2048 are unlikely to be read.

Output

UTF-8 text with Markdown formatting for sections and lists, HTML for tables, and LaTeX for math.

How the jina-ocr-v1 architecture builds on DeepSeek-OCR

The full jina-ocr-v1 model is a 3.4 billion (3.4×109) parameter mixture-of-experts model with 570 million active parameters. It’s based on the encoder-decoder architecture of DeepSeek-OCR with the addition of FastMTP, which reduces computational load at inference time by predicting multiple output tokens at once. This approach results in more efficient processing, with lower latency and less resource consumption at inference time.

Figure 1: The architecture of jina-ocr-v1.

For more information about the model’s architecture, see the technical report and jina-ocr-v1’s model page on Hugging Face. 

Input formats and resolution limits

Following the DeepSeek-OCR architecture, images are resized to 1024x1024 and processed into 256 input tokens for the text decoder. To incorporate additional detail, the image is also processed into up to nine additional cropped higher-detail image tiles, depending on the resolution and geometry of the image, with each reprocessed into up to 100 additional tokens each. Very large images with features too small to make out at 2048x2048 resolution will probably not be processed correctly.

jina-ocr-v1 uses the PIL image processing library, automatically supporting the vast majority of image formats, including PNG, JPG, and GIF. However, it doesn’t have native support for PDFs. You’ll need to convert your PDF pages into an image format (we recommend PNG with minimal compression) for use with jina-ocr-v1. A variety of open source and commercial utilities can perform this task.

For more details on input processing, see the technical reports for jina-ocr-v1 and for DeepSeek-OCR.

Output formats: Markdown, HTML tables, and LaTeX math

jina-ocr-v1generates text with Markdown formatting, supporting only the basic syntactic elements most compatible with different Markdown implementations. Markdown supports common text structure information, like sections and lists, in a simple, human-readable format. It’s readily converted into HTML and other rich text formats.

The model departs from basic Markdown to encode tables and math formulas. Specifically:

• Tables are encoded in basic HTML format, using <table>, <th>, <tr>, and <td> tags, without CSS or other stylesheet information.

• Formulas are encoded in LaTeX math mode syntax, with the commands available in the amsmath package.

Pushing the boundary of performance and cost

jina-ocr-v1 scores among the top models on olmOCR-bench and is the best model overall with less than 600 million active parameters. It sits on the Pareto frontier for AI OCR models, meaning that all the models that perform better on this benchmark have more active parameters, taking more computing power and resources than jina-ocr-v1.

Model

Total parameters

Active parameters

olmOCR-bench

overall score

Infinity-Parser2-Pro

35 billion

2.95 billion

87.6

Chandra 2

5.3 billion

4.96 billion

85.8

dots.mocr

3 billion

1.54 billion

83.9

jina-ocr-v1

3.4 billion

574 million

83.4

Surya OCR 2

650 million

585 million

83.3

LightOnOCR-2-1B

1 billion

596 million

83.2

Chandra 1

9 billion

7.57 billion

83.1

Infinity-Parser-7B

8 billion

7.07 billion

82.5

PaddleOCR-VL-1.6

0.9 billion

0.9 billion

81.0

Falcon-OCR

300 million

270 million

80.3

Qianfan-OCR

5 billion

4.02 billion

79.8

dots.ocr

3 billion

1.54 billion

79.1

DeepSeek-OCR 2

3 billion

570 million

76.3

Table 1: jina-ocr-v1 among olmOCR-bench top performers.

Figure 2:jina-ocr-v1’s place at the Pareto (performance vs size) frontier on the olmOCR-bench test suite. Only much more compute-intensive models have significantly higher published scores.

OmniDocBench (Table 2) offers a more mixed picture, but jina-ocr-v1 scores highly among specialized OCR models adapted from VLMs. However, a comparison of Table 1 and Table 2 reveals that there are no models of comparable size that beat jina-ocr-v1 on both benchmarks, showing that its high performance is robust across different task types.

Model

Total parameters

Active parameters

OmniDocBench

overall score

PaddleOCR-VL-1.6

0.9 billion

0.9 billion

96.34

HunyuanOCR

1 billion

1 billion

94.74

jina-ocr-v1

3.4 billion

574 million

91.14 

dots.ocr

3 billion

3 billion

90.77

DeepSeek-OCR 2

3 billion

575 million

90.25

Table 2: jina-ocr-v1 score on OmniDocBench compared to selected top specialized VLM models.

Table 3 compares jina-ocr-v1 to vision-enabled, nonspecialized LLMs. It significantly outperforms GPT-5.2 and the largest Qwen3 VLM on OmniDocBench, while coming close to Gemini 3 in overall score.

Model

Total parameters

Active parameters

OmniDocBench

Overall score

TextEdit score*

Read-

OrderEdit score*

Ovis2.6-30B-A3B

30 billion

3 billion

93.7

0.035

0.135

Gemini 3 Pro

Closed model, not disclosed

92.91

0.064

0.165

Gemini 3 Flash

Closed model, not disclosed

92.62

0.066

0.172

jina-ocr-v1

3.4 billion

574 million

91.14

0.046

0.142

Qwen3-VL-235B

235 billion

22 billion

89.78

0.063

0.166

GPT-5.2

Closed model, not disclosed

86.59

0.114

0.193

* TextEdit and Read OrderEdit benchmarks are scored so that a smaller value is better than a higher one.

Table 3: jina-ocr-v1 scores compared to top general-purpose LLMs pressed into OCR tasks. 

As shown in Table 3, on OmniDocBench tests focused on correct reading order and overall character-level accuracy (TextEdit and ReadOrderEdit),  jina-ocr-v1 beats all three frontier LLMs handily. Only Ovis2.6-30B-A3B scores higher in all areas in OmniDocBench, at almost nine times as many total parameters and roughly six times as many active parameters.

To test on your data, you can try it out on the Jina AI website, or read the section Five ways to access the jina-ocr-v1 OCR API to see how to integrate it in your document processing and search pipeline.

Layout-aware end-to-end document parsing

OCR has been around for a long time, long enough that many people have experienced struggling with its shortcomings. It’s not that it’s so hard to recognize letters on the page, especially if clearly imaged and using a modern digital font, it’s that recognizing letters in a left-to-right sequence is only the first step to making sense of what’s printed on a page. 

Consider the image below, extracted from the print edition of the June/July 2024 UK edition of Cosmopolitan magazine:

Figure 3: Image extracted from page 17 of UK Cosmopolitan, June/July 2024, from the Internet Archive.

Simply reading from left to right will produce a chaotic, unreadable, useless text, even if every word and letter is recognized correctly. A good OCR model has to parse the image, recognizing, like a human does, the individual elements in relation to each other, and reading the text as intended, attending to fonts and spatial organization.

It should also recognize elements that don’t belong in its output. A page may contain images that incidentally have text in them, like the book covers in Figure 3. There’s also often header and footer information that isn’t part of content, like page numbers and boilerplate. It’s difficult to filter those elements out after OCR, so recognizing and removing them has to be a part of preprocessing or the OCR process itself. 

Figure 4 shows how jina-ocr-v1 parses the image in Figure 3, including elements to ignore. Each colored block indicates text elements that belong together, containing text and possibly other blocks, while the images of books are marked out because, while they do contain text, that text doesn’t belong in the OCR output. 

Figure 4: The same print page but parsed into elements that an OCR process has to take into account and the ones it should ignore.

These problems mean that OCR for materials with complex layouts has traditionally been a multistage process, using different algorithms to:

1. Break up the page.

2. Identify the role of each part of the image.

3. Recognize the text with the help of a statistical language model.

4. Reassemble the result for output.

Each stage is fragile, and errors accumulate through the processing pipeline. 

Instead, end-to-end document parsing does all that in a robust, single-pass generative language process.

jina-ocr-v1 handles this kind of visually complex document with ease, as shown in Figure 5, not only capturing the words, but ordering them correctly, preserving structural information about titles and sections, and ignoring the text in the pictures of books as well as the non-content header and footer.

Figure 5:jina-ocr-v1 output in Markdown format, from processing the image in Figure 3.

jina-ocr-v1 does more than just identify the letters on a page. It manages complex structural information and supports over 100 languages. It also returns information in a Markdown-style format that’s both human-readable and widely supported out of the box by downstream applications.

Table extraction to HTML

Identifying and extracting tables sounds simple, but it’s very challenging to do well without an AI model. 

As an example, consider this table from a recent Jina AI conference paper:

Figure 6: A rendered table, originally composed in LaTeX, from a formal conference paper. 

OCR software that lacks table awareness would likely produce a result like this:

This is useless as a table without further processing. But, when presented with the same table, jina-ocr-v1 produces clean, minimalist HTML:

Figure 7: HTML generated by jina-ocr-v1 for the table image in Figure 6. Some whitespace has been removed for compact display.

This HTML yields a functionally identical table that’s suitable for display, reading into spreadsheets or other applications that support tables, or other further processing.

Figure 8: The HTML in Figure 7, rendered in a browser. Visible borders were added via a stylesheet for clarity. 

When tables appear inside a page with other text, the HTML tables are inserted into the Markdown output, as in Figure 9:

Figure 9: An example document page containing a table (left), and the rendered Markdown output, including HTML table, with visible borders added via CSS for clarity (right).

Math recognition: Printed formulas to LaTeX

Mathematical formulas are visually complex and nonlinear, full of shapes that look like ordinary printed language, but cannot be processed like ordinary language. jina-ocr-v1 has special training to recognize and transform printed math into LaTeX math code. 

Figure 10 is an image of a block of formula-rich text extracted from a recent Jina AI conference paper:

Figure 10: An extract from a scientific paper (left), and jina-ocr-v1’s raw output from processing that image (right). The parts between “$” characters are LaTeX math.

After pasting the generated output from Figure 10 into a fresh LaTeX document (with \usepackage{amsmath}) and compiling it to printable form, the result is functionally identical, missing only the block-level equation alignment present in the original:

Figure 11:jina-ocr-v1 LaTeX formula output, recompiled as LaTeX. Some block alignment information is lost, but the math formulas are all intact and correct.

Handwriting recognition: Print and cursive

jina-ocr-v1 handles both print and cursive handwriting:

Figure 12: A 1941 handwritten letter to US President Franklin D. Roosevelt from a child, preserved in the FDR Presidential Library (left), and jina-ocr-v1's output for it (right). Note that the cursive archivist’s mark in the upper right-hand corner, written in a completely different hand, proves too hard to manage with so little context.

Cursive handwriting is highly varied, and even human readers may struggle with it. While jina-ocr-v1 performs well on neat cursive English, messy writing is as hard for it to read as for humans. When writing is sloppy, poorly captured by OCR, or otherwise difficult to decipher, jina-ocr-v1 can perform quite poorly.

For example, the handwriting below is from the famous children’s books illustrator Beatrix Potter:

Figure 13: A letter from Beatrix Potter to a child acquaintance (top), and the extracted text (bottom).

jina-ocr-v1 is able to accurately capture the flowing cursive in Figure 13, even when it wraps around Potter’s drawings of mice. However, the place-name at the top is much less cleanly written and lacks disambiguating context, resulting in errors. Even the human reader likely finds it difficult to read.

OCR for slides, reports, labels, and business cards

jina-ocr-v1’s expanded training for complex layouts makes it work well with unconventional materials. The examples below show some of the scope of the model’s abilities.

Presentation slides

Figure 14: A slide from Elastic NV’s most recent quarterly financial presentation (left), and jina-ocr-v1’s output, rendered via HTML (right). Note that the model correctly extracts the hierarchy of titles and headings and associates each text with the proper heading. It also ignores the page number and boilerplate Elastic logo.

Business reports

jina-ocr-v1 processes the visually dense pages of corporate business reports, such as in Figure 15:

Figure 15: Page 2 of the SpaceX quarterly earnings report for Q2 2026 (August 4, 2026) (left), and the rendered jina-ocr-v1 output for it (right). jina-ocr-v1 is aware that “Countries with Starlink Coverage” goes with “167,” even though a simple column-based reading wouldn’t have made that association.

Glossy business reports and presentations often have complex, visually interesting layouts that make them hard for traditional OCR to correctly process. `jina-ocr-v1` excels at these kinds of materials, like the example in Figure 16 below:

Figure 16: Page 2 of the 2025 Integrated Report from the Japanese Meiji Group (top), and an HTML rendering of jina-ocr-v1’s output (bottom). Note how the visual hierarchy of elements in the original is preserved in the Markdown output.

Labels and packaging

jina-ocr-v1 can read print from labels and graphic design materials. This is especially helpful with high-information content materials, like medical packaging:

Figure 17: A drug packaging image from the US government’s DailyMed website (top), and the extracted text rendered from Markdown to HTML (bottom).

It works just as well with more conventional consumer goods labeling and photos, if the image quality is high, as in Figure 18:

Figure 18: A common household product with text on the label (left), and extracted output rendered as HTML (right).

Business cards

jina-ocr-v1 has no difficulty with visually compact documents, like business cards:

Figure 19: A photo of a business card sourced from Wikimedia Commons (left), and the HTML rendered text result (right). Note how the three-level header hierarchy is preserved and how jina-ocr-v1 correctly understands the alphanumeric structure of Canadian postal codes, correctly inserting “1” and “0” (instead of “I” and “O”) into “K0K 1Z0.”

Multilingual OCR across 100+ languages

jina-ocr-v1 has been trained in over 100 global natural languages. It performs high-quality OCR for a diverse range of global media, without additional modules to support different language or writing systems.

Diacritics and modified Latin alphabets

Single-language OCR software (especially for English) often struggles to render diacritical marks on letters. Most languages use them, and failing to accurately render diacritics can make indexing, information retrieval, and other downstream applications fail.

Even in Western European languages, like French and German, modified letters like the c-cedilla (Ç) and Eszett (ß), pose well-known problems for many OCR suites:

Figure 20:jina-ocr-v1 handling a French c-cedilla (Ç) and a German Eszett (ß) together in a single image (left), with the plain text Markdown result (right).

Figures 21 and 22 show jina-ocr-v1 handling Czech and Turkish, which are languages written with heavily modified Latin alphabets that use diacritics pervasively:

Figure 21: COVID-19 information from the Czech government, in the original PDF (left), and jina-ocr-v1’s Markdown output rendered to HTML (right).

Figure 22: A Turkish-language presentation slide from Wikimedia Commons (left), and plain text output (right). Note that jina-ocr-v1 correctly recognizes the distinctly Turkish yumuşak g (ğ) and highly problematic “dotless i” (ı).

Greek and Cyrillic

In addition to Latin alphabet writing, jina-ocr-v1 supports Greek and Cyrillic writing:

Figure 23: Greek press release from the construction company GEK TERNA (ΓΕΚ ΤΕΡΝΑ) (left), and extracted text (right).

Figure 24: Ukrainian presentation slide (top), and Markdown output from jina-ocr-v1 rendered via HTML (bottom).

Asian and Middle Eastern languages

jina-ocr-v1 goes beyond Latin scripts. You can see its Japanese support in Figure 16, but it also handles other major Asian languages. 

Chinese:

Figure 25: Chinese. Public promotional materials for a new food traceability reporting system at Lǎoxiāng Jī Restaurants (老乡鸡) downloaded from Sina.cn (left), and rendered Markdown (including HTML table layout) from jina-ocr-v1 (right).

Korean:

Figure 26: Korean. Page 3 of Hyundai Group’s Q2 2026 earnings announcement in the original PDF (left), and rendered Markdown (right).

Thai:

Figure 27: Thai. A screenshot of the home page of the daily newspaper Thai Rath (ไทยรัฐ) on August 26, 2026 (top), and the text extracted text from it (bottom).

Hindi:

Figure 28: Hindi. A screenshotted headline from the BBC Hindi website (left), and jina-ocr-v1's extracted text (right).

Arabic:

Figure 29: Arabic. US government public information pamphlet about COVID-19 (left), and jina-ocr-v1’s extracted text (right).

Mixed-language documents

jina-ocr-v1 excels at materials with mixtures of languages:

Figure 30: US Department of Agriculture notice in English, Spanish, Vietnamese, Chinese, and Arabic (left), all correctly handled by jina-ocr-v1 (right).

Five ways to access the jina-ocr-v1 OCR API

jina-ocr-v1 belongs at or near the front of your data pipeline, preprocessing rendered text images into clean Unicode text and tractable structured formats. This adds value to every downstream part of your processes. Your data ingestion pipeline makes fewer errors, and your retrieval has greater accuracy. Plus, the data retrieved is more immediately usable, whether to human readers or agentic AI.

There are five ways to access and use jina-ocr-v1:

Access method

Interface

Native PDF support

Licensing

Best for

Elastic inference API / EIS

chat_completions inference endpoint

Not yet; convert pages to images first

Included with Elastic Cloud

Teams already ingesting into Elasticsearch

Jina API

HTTP service, prepaid tokens

Not yet; convert pages to images first

Pay per token

Use outside Elasticsearch, no Elastic account needed

Local installation

Jina On-Prem or download from Hugging Face

Not yet; convert pages to images first

CC BY-NC 4.0 for academic and noncommercial; contact Elastic Sales for commercial

Air-gapped, regulated, or high-volume workloads

Jina AI Reader API

HTTP header X-Respond-With: jina-ocr-v1

Yes; converts PDF and HTML to images automatically

Part of the Reader API service

Fastest route if your inputs are PDFs or web pages

Elastic inference API and Elastic Inference Service

jina-ocr-v1 is available via the Elastic inference API and EIS as an inference endpoint.

Because jina-ocr-v1 returns streaming text, like a chat-style LLM, access to it is via the chat_completions interface. We’re integrating native support for PDFs into the Elastic service and will make it available in the near future. Until then, you’ll need to process PDF documents into images first.

Jina API

For users who want to try out jina-ocr-v1 or don’t want to access it via Elasticsearch’s service infrastructure, access via the Jina API is available using a standard HTTP service. This service uses prepaid tokens, with no fixed subscriptions, and comes with 10 million free tokens to try it out at no charge. 

For more information, visit the model’s page at jina.ai and the jina-ocr-v1 sandbox.

Local installation and on-prem licensing

Jina AI models are available for download and licensed commercial use via Jina On-Prem and from the model’s page at Hugging Face. Jina AI’s latest models are free for academic research and noncommercial use under a CC BY-NC 4.0 license. To commercially license local installations of , please contact Elastic Sales.

Jina AI Reader API

You can configure the  Jina AI Reader API service to use jina-ocr-v1 in conjunction with a variety of additional features, including automatic PDF- and HTML-to-image conversion. This is not a default setup. To configure Jina AI Reader API to use  jina-ocr-v1, add the following to the header parameters in your request:

To use this service, see the Reader API documentation page at jina.ai.

jina-ocr-v1 technical report and model card

For more information about , see the model’s technical report and page on Hugging Face.

Start a free Elastic Cloud Serverless trial or log in to Elastic Cloud to follow along, and install the CLI via npm: 

Designing the Elastic CLI for people and AI agents

A useful side effect of building flexible tools for developers is that they’re more useful to AI agents, too. It’s also the tool that our Agent Skills now use to get things done, closing the loop that we opened in March 2026, when we said that a CLI for agent workflows was coming.

The CLI gives every public API across Elasticsearch, Kibana, and Elastic Cloud one shape: the same flags, input and output conventions, authentication method, and failure mode. Consistency is the ergonomic feature; everything else is built on it.

Agents need the same thing, only stricter. An agent won’t know how to craft a valid CLI command or notice if a tool “feels” wrong; it needs output it can parse and input it can validate before sending, along with failures it can branch on. Agents are now a first-class interface to Elastic, alongside people, whether they live on the platform or in your editor and terminal. Agent Skills, and now the CLI, are how we serve the second kind, so those needs are built into the core of the CLI rather than tacked on.

JSON input and output for every command

Agents love structured text, and almost every Elastic API already speaks JSON, so first-class JSON support was a hard requirement. Developers who are quick with a jq query will be equally satisfied.

• JSON output: Any command, like elastic version and elastic es indices delete ..., supports --json, which prints JSON-parseable output to stdout and nothing else. Failed commands print {"error": {"code": "...", "message": "..."}} to stderr.

• JSON input: Every command that takes input accepts JSON on stdin or via --input-file. Every top-level key in that JSON also works as a CLI argument, and inline arguments take precedence, so you can keep a big request body in a file and tweak a value or two per invocation.

• JSON Schema as --help output: Pass --help --json to any command, and it prints a valid JSON Schema for its input, which also feeds nicely into codegen tools. elastic cli-schema prints the whole command tree.

Exit codes that an AI agent can branch on

Agents loop on exit codes as much as they do over stdout. All failure modes are distinguishable from success, even if stdout and stderr are never read.

Safety rails: Keychain storage, allow lists, and validation

No model uses tools perfectly 100% of the time, so an agent-friendly CLI should provide safety rails wherever possible.

• Contexts and secret storage: Connection details live in named contexts in ~/.elasticrc.yml, kubectl-style; switch with --use-context. API commands never take credentials as flags. elastic config context add writes API keys to your OS keychain (macOS, Linux, Windows) and leaves a $(keychain:...) reference in the YAML; $(env:...), $(cmd:...), and $(file:...) work, too. Creating a Serverless project with --save-as writes its credentials straight to the keychain and never prints them, so nothing leaks into logs or LLM transcripts.

• Allowlists/blocklists: A commands.allowed (or commands.blocked) list in the config file, globally or per context, ensures that only the commands an administrator wants are runnable.

• Validation: Every command has a JSON Schema, so inputs are validated before any request is sent. Add --dry-run to any command that takes input, and it validates and exits without sending anything.

• Confirmation: Destructive commands prompt in a terminal. In a noninteractive session, where agents live, they refuse to run without --yes and say so in a structured error.

• Sanitization: Index, field, and pipeline names have length limits and forbidden characters. elastic sanitize index-name '<value>' (and field-name, pipeline-name, …) prints a version stripped of anything invalid.

Keeping API responses inside an agent's context window

Elastic APIs return a lot of data, and an agent’s context window is finite. Three controls help keep unnecessary text out of the context window:

• Field masks: --output-fields takes a comma-separated list, with dot notation for nested fields.

• String templates: For total control, --output-template takes a mustache-style template.

• Command profiles: --command-profile serverless (or default_profile: serverless in your config) hides Elastic Cloud Hosted commands and the Elasticsearch namespaces that don’t exist on Serverless. That means less to scroll past and less for an agent to guess wrong. It’s the profile we recommend for agents.

Control

What it does

Syntax

When to use

Field mask

Returns only the fields you name, using dot notation for nested fields

--output-fields 'name,version.number'

You want valid JSON back, just less of it. This is the default choice for agents parsing structured output.

String template

Renders the response through a mustache-style template

--output-template 'ES version: {{ version.number }}'

You need one value in a specific shape, for a shell variable, a log line, or a prompt.

Command profile

Hides commands and namespaces that don't apply to your deployment

--command-profile serverlessor default_profile: serverless

You want a smaller command surface so an agent has less to scroll past and less to guess wrong. This is recommended for agents.

Helpers for bulk ingest, scroll search, and msearch

Some of Elasticsearch’s most popular APIs have a learning curve, so elastic es helpers wraps them:

• scroll-search: Stream a large result set as NDJSON with paging handled for you.

• bulk-ingest: Ingest from a file, a directory, or stdin (NDJSON, JSON arrays, or CSV) with streaming, batching, concurrency, and retries.

• msearch: Send multiple searches in one request.

• watch: Print new documents from an index to stdout as they’re indexed. This is great for piping into logging tools.

elastic es and elastic kb are aliases for elastic stack elasticsearch and elastic stack kibana. If we don’t ship a command you need, elastic extension create scaffolds one for you.

Searching Elastic docs from the terminal

If you or your agents don’t know which API to use, elastic docs search (plus docs read and docs ask) searches Elastic’s documentation from the terminal, returning Markdown or --json. These are experimental. You’ll see a warning until you pass --accept-experimental, so explore, but don’t script against them yet.

Shell completion for Bash, Zsh, and Fish

Autocomplete hooks are available for Bash, Zsh, and Fish, and they always respect your commands.allowed or commands.blocked policy.

How Elastic Agent Skills use the CLI

Agent Skills teach an AI coding agent how an Elastic expert approaches a job; for example, which cluster health field is the verdict or how to stage a reindex so it doesn’t fall over. They capture process and judgment but not transport. A skill that embeds curl with an auth header has hard-coded a hostname, key, and runtime, and it breaks when any of those change.

So our skills now use a universal format that runs unchanged in any runtime that can execute the elastic CLI, including Claude Code, Codex, Cursor, and GitHub Copilot. The body refers to operations in HTTP shorthand (GET /_cluster/health, POST /_query), and an operations table at the end binds each to a CLI command. That table is the only place transport appears:

HTTP API (shorthand)

elastic CLI command

GET /{index}/_mapping

elastic es indices get-mapping --index '<index>'

POST /_query

elastic es esql query --format tsv --query "<esql>"

POST cloud:/api/v1/serverless/projects/elasticsearch

elastic cloud serverless projects search create --input-file <json> --wait --save-as <ctx>

Every universal skill also inherits a blunt preamble; that is, use the CLI, don’t guess credentials, don’t call the HTTP API directly, and never ask the user to paste an API key into the chat.

The two halves need each other. The skill supplies the expertise that the model doesn’t have, and the CLI supplies a way to act on it that’s validated, credential-safe, and scoped by your allowlist. Tell your agent to spin up a Serverless project and load products.csv into it, and the provisioning skill creates it with --save-as. The ingest skill dry-runs a mapping and loads with elastic es bulk, and the Elasticsearch Query Language (ES|QL) skill writes a query that parses on the first try. Every step returns JSON, exits non-zero on failure, and can only do what your policy allows.

Skills for Elastic Cloud onboarding and provisioning, Elastic Workflows, and Kubernetes investigation are available today. Skills for Elasticsearch query, ingest, reindex, and index design, plus Kibana dashboards and alerting, are close behind.

What’s in the Elastic CLI technical preview, and what’s next

The preview covers all public Elasticsearch Serverless, Kibana Serverless, and Elastic Cloud APIs. Hosted-only 9.x Elasticsearch API coverage is nearly 100%, and hosted-only 9.x Kibana APIs will be added soon.

We’re actively planning more developer experience work, including broader coverage for all supported stack releases, more helpers for common workflows, more skills in the public catalog, and loading the same skills into agents that run on the Elastic platform itself. What shapes that list is hearing how you and your agents use the CLI. Tell us what’s awkward and what’s missing, along with what you’d automate next.

Install the Elastic CLI and Agent Skills

The Elastic CLI is available now on npm (Node.js 22+). Install it and the skills together:

Then, add a context and check it:

Even without a project, you can start a free Serverless trial in about a minute, with no credit card. If you already have a project, log in and create API keys for Elastic Cloud and your Elasticsearch clusters. Before pointing an agent at anything real, start with a trial project, a read-only key, and a scoped commands.allowed list. Be sure to take five minutes to read the security notes in the skills repo.

Replace all those curl commands in your Bash scripts, and add some usage instructions to your AGENTS.md. Then let your agent’s skills work efficiently and accurately with our APIs. Let us know what you think, and don’t hesitate to open an issue if you find a bug or if your use case isn’t well supported. Your feedback directly shapes what we build next.

Elastic CLI and Agent Skills resources

• Elastic CLI on GitHub and CLI documentation

• Elastic Agent Skills on GitHub

• agentskills.io specification

• Elastic Cloud Serverless documentation

• Report a CLI issue · Report a skills issue · Discuss

Managing context depends on good retrieval. Rather than have agents rediscover the same content for every question, burning tokens by retracing similar steps over and over again, Elastic’s agentic AI capabilities enable us to precompute these details and store them in a structured, searchable form, and they let agents load that context directly. We call this precomputed unit of context a Knowledge Indicator.

The default agentic retrieval augmented generation (RAG) pattern does the opposite. It retrieves whole documents and dumps them into the model's context at query time, paying for that retrieval in tokens and latency on every single question. Precomputing the answer as a KI moves that cost out of the hot path and does it once.

How it works: AI Index, Kibana Workflows, and the query-ki skill

Building context through AI indices has three main parts: the AI Index (a special Elasticsearch index where KIs live), Kibana Workflows to create your KIs, and a query-ki skill to help agents directly query KIs using ES|QL: 

This blog post is similar to Part 1 in that we’re using the same core building blocks. But in this post, we’re demonstrating a very different use case. Instead of precomputing index metadata, we’re distilling specific facts from our indexed documents that may be used to directly answer agents’ questions without subsequent searches. We've also provided a notebook, if you'd like to create the same KIs yourself, end to end, as you go through these examples. 

Prerequisites: Elasticsearch Serverless and an LLM API key

This tutorial assumes you have:

1. An Elasticsearch Serverless project. You can sign up for a trial if you don't have one.

2. An API key to access your Elasticsearch project.

3. An OpenAI-compatible large language model (LLM) API key, to access AI indices via Deep Agents scripts.

Load the BrowseComp-Plus sample corpus into Elasticsearch

First, we’ll need some sources. Sources can be data that already exists in your Elasticsearch indices or external data accessed via connectors or ES|QL data sources.  For this blog, we’ll create an index, browsecomp-plus, to hold our example data, with the following mappings:

and populate it with a small sample of BrowseComp-Plus data via the _bulk API. You can use the supporting notebook to load a sample of this data in your project. 

Create the AI Index that stores your KIs

Just like in Part 1, the first step is to create an AI Index:

This is preconfigured with the same required mappings as we listed out in Part 1. We perform hybrid search here using semantic_text out of the box.

How agents retrieve KIs using ES|QL

A KI is a document in the AI Index. What makes KIs useful is retrieval, or querying the AI Index to find the right content. This query is packaged within a small, portable skill that’s harness-agnostic and can be run in any agent harness. 

Here’s a sample query-ki skill:

Save this asskills/query-ki/SKILL.md.

Here’s what this skill is doing: 

• We’re defining corpus_entry as our KI use case.

• We’re performing a hybrid ES|QL search on our AI indices, filtering by the appropriate type, using reciprocal rank fusion (RRF) as the default method to fuse results.

• The KI results will directly ground the agent’s answer when determining what facts are relevant to the users’ query.

When we say that AI indices and KIs are harness-agnostic, it’s because the skill is just instructions plus a query. It will work in Elastic Agent Builder, a Kibana workflow agent, Claude Code, or any other harness. We’ll be using Deep Agents for examples of how to query it outside the Kibana ecosystem. Since an AI Index is, at its core, an Elasticsearch index, you can also explore your data directly. 

Precompute facts as KIs for agentic RAG

In this example, we extract actual facts so agents can retrieve an answer without consuming a full document. We generate one fact-based KI per selected document, though the actual number and structure of KIs you generate are completely customizable.

We'll use a sample of the BrowseComp-Plus corpus, indexed into a browsecomp-plus index, with docid, url, title, and text fields.

Baseline: Retrieving whole documents with RRF

As a baseline, here's a simple RRF query:

This drops several hundred words of raw body text into the model's context. It may work, but it's expensive, and the cost compounds with every miss.

Build the Kibana workflow

The workflow below reads a batch of documents with a single ES|QL query and writes one fact-level KI per document into the AI Index. Each iteration runs two steps: generate_ki distills a raw document into a structured KI, and sink_ki writes it to the AI Index keyed on docid so reruns are idempotent.

Copy and paste the following YAML into the Elastic Workflows editor:

Here’s what this workflow is doing: 

• query_corpus runs an ES|QL query against the browsecomp-plus index, applying some rules, like dropping documents with empty bodies and trimming each body to 12,000 chars so the agent prompt stays inside the context window.

• Note: In this example, we’re cherry-picking some concrete KI IDs, because generating KIs for every document in the index would take a long time, and we want this exercise to be short for those following along.

• loop_corpus_docs iterates over every returned document, running the following two steps per document: 

• generate_ki reads the document and calls an LLM to emit a strictly grounded, structured KI.

• sink_ki bulk-writes each KI into the AI Index (ai-index-idx-my-corpus) as a KI of type corpus_entry. It forces _id to be the same as the document’s docid so rerunning the workflow is idempotent.

To summarize, this workflow turns each raw corpus document into a compact, searchable metadata record that agents can find and judge without reading the full source into the context window.

This workflow is used for example purposes, and the same foreach caveat as in Part 1 applies. For scale, use workflow.executeAsync or native parallel support. The cheat sheet is useful for optimizing Workflows. There could also be cost and efficiency gains in production by using ai.prompt or by choosing different models with which to create KIs. 

Inspect the KIs in your AI Index

Once the workflow runs, you can query the AI Index to browse what was written:

Here’s an example of what one of the KI documents looks like: 

Query KIs from LangChain Deep Agents

We’ll use LangChain Deep Agents with an OpenAI-compatible key to show that AI indices and KIs will work with any agent harness, inside and outside of Kibana’s Agent Builder ecosystem. 

First, let’s create facts_baseline_agent.py to measure our baseline before applying KIs: 

When I ran this baseline query, What was the actress who played Torvi from Vikings also known for?, it output the following:

(Note: Deep Agents automatically adds the read_file tool to handle paginated tool results, which is why it shows up in the output.) 

Next, let’s create an agent that knows how to use our query-ki skill, facts_ki_agent.py: 

This agent will query the KI indices to generate the answer, rather than pulling every document into context. 

When I ran these results against the same query, here was the output: 

How much can precomputing facts reduce agent token usage?

Both agents had similar conclusions, but they took far different paths to get there: 

The same question and the same grounded answer result in 93% fewer tokens and two tool calls instead of eight, when answering from KIs.

Baseline (No AI Index)

With AI Index

Total tool calls

8

2

read_file calls

2

1

esql_query calls

6, all against the browsecomp-plus index

1, from ai-index-idx-*

Tokens consumed

386,187

27,625

Latency

44.86s

15.22s

Answer

Grounded, correct

Grounded, correct

Exact tool call counts, latency, and answers will vary between runs and using different agents. 

Both agents produced solid, grounded answers. The difference is cost. Querying KIs from the AI Index cut token use by 93% and cut latency by roughly two thirds. Here’s how both paths went, side by side:

That was in-depth, but it shows what AI indices and Workflows do together: the same answer, at a fraction of the tokens.

Build precomputed context in Elasticsearch Serverless

This walkthrough shows how to generate more sophisticated KIs based on documented facts and query them for knowledge retrieval use cases using Elasticsearch primitives. 

Managing context is critical in agentic search systems. And at its core, context is a retrieval problem. AI indices help you manage context within the Elastic Stack. Try it out in Serverless, and let us know what you think in our Discuss forums or the #stack-kibana channel in our Community Slack.

We’d also love to hear from you about what use cases you’d like to solve using AI indices.

NASA Aviation Safety Reporting System (ASRS) reports describe unusual events during flights, such as missed altitudes, confusing clearances, runway issues, or mechanical problems. Each report already has an official category, like altitude deviation, course deviation, or ground encounter. In this article, we ask a different question: What does this report reveal about the pilot who wrote it? The idea is to ask a model to infer a schema grounded on the data to classify the report based on criteria that help us figure out information about the report writers. Then ask a second model to apply the labels.

You can find the full workflow definitions and helper scripts here.

What you need to run this LLM pipeline

• Elastic Stack 9.4+ or Elastic Cloud Serverless. Elastic Workflows has been generally available (GA) since 9.4.

• Elastic Agent Builder enabled in your deployment.

• A Kibana generative AI (GenAI) connector pointing at Claude Sonnet (or an equivalent reasoning model). This is the planner.

• A Mistral API key. We’ll use it to register an Elasticsearch inference endpoint.

• Python 3.10+ with elasticsearch>=9.0 and pandas. Used by the dataset loader.

How two-tier LLM orchestration works

The workflow has two jobs: Decide what labels should exist, and then apply those labels to every report.

The first job is open-ended. A large model reads a varied sample of reports and proposes a small schema of categorical fields. A field is one way to describe the writer, such as attribution_style or procedure_orientation. Each field has a few allowed values, such as self_critical, system_attributing, or balanced.

The second job is repeatable. After a human approves the schema, a smaller model reads each report and chooses one value for each field.

We use Elastic Workflows because the steps are known ahead of time: sample reports, propose labels, wait for approval, classify every document, and store the results. Writing those steps in YAML makes the process reproducible, observable, and cheaper to rerun.

Why split LLM work across two model tiers?

A small model could handle classification, but schema discovery is a different shape of problem. It requires reading a diverse sample, spotting latent patterns, and proposing complex structures. In practice, smaller models over-anchor on surface keywords and produce redundant or nonexclusive fields.

Classification is simpler, the schema exists, the values are enumerated, and the task is to pick one per field. A smaller model handles this reliably and at a fraction of the cost, since it runs once per document across the entire corpus.

Large and small here mean reasoning capability. In this article, Claude Sonnet plays the planner and Mistral Small 3.1 plays the executor.

Classifying NASA pilot reports with a two-tier LLM pipeline

We’ll use the NASA ASRS database, which collects voluntary, anonymous incident reports from pilots, controllers, and mechanics. The dataset is public, and the reports are written as free-text narratives.

What we want to ask is:

What does this report reveal about the pilot who wrote it?

The planner reads a varied sample of reports and decides which distinctions are meaningful based on how the reports are actually written.

Step

Role

Model tier

sample

Pull a diverse subset of reports from the corpus.

(no LLM)

discover

Read the question and the sample, propose a schema of fields with enum values.

Large

approve

Human reviews the proposed schema and approves or edits it.

(Human via waitForInput)

apply

Iterate over the corpus, assign one value per field to each report.

Small

store

Write the schema and the per-document field values to Elasticsearch.

(No LLM)

Registering Mistral and Claude as Elasticsearch inference endpoints

The small model will be registered as an Elasticsearch inference endpoint using the native mistral service integration. 

The alias mistral-small-latest resolves to Mistral Small 3.1. It has a 128k context window and supports JSON-mode output.

The large model will be an AI connector pointing at Claude Sonnet. The Agent Builder UI walks you through creating the connector. Take a note of the connector ID since we’ll reference it from the workflow.

Indexing NASA ASRS incident reports into Elasticsearch

The ASRS dataset is indexed with keyword mappings for aggregation fields and text mappings for the narratives the models will read.

Download the ASRS CSV (the database publishes quarterly extracts at the ASRS Database Online page), and index it. The mappings are:

The mapping types follow how each field is used. flight_phase and anomaly are mapped as keyword because we’ll run terms aggregations on them to build the sample, and aggregations need exact, non-analyzed values. narrative and synopsis are mapped as text because they hold free-form prose that the models will read. The companion notebook has the full loader script that reads the CSV and bulk-indexes the documents.

Building a stratified sample for the planning LLM

The YAML snippets in this and the following sections are steps of the workflow definition that the notebook registers via the Workflows API. The first two steps generate a representative sample: They aggregate by flight phase and by anomaly and pull a few documents per bucket with top_hits.

How the large LLM discovers a classification schema from the data

The prompt needs both the question and the sample. A question alone may produce generic labels disconnected from the corpus, and a sample alone produces descriptive clusters that ignore the angle of the question. 

When both are present and the output is structured, the model produces labels that are grounded in the data and oriented to the task: a schema of categorical fields, each with two to four mutually exclusive value options backed by evidence from the sample.

Here’s the planner step from the workflow:

The structured output schema enforces the shape of the response:

• name: Identifier for the categorical field.

• definition: What this field measures, in one sentence.

• why_useful: How this field serves the question; this also helps the downstream classifier understand the intent.

• values: Two to four mutually exclusive options. Each has a value and a definition.

Here’s an example of the produced schema. We can see how the writer is being classified and the reasons why the model decided to create the category. definitionand why_useful fields are used by the second model to classify the documents.

Human-in-the-loop schema approval with waitForInput

The proposed schema is now passed to a person for approval. Elastic Workflows has a waitForInput step that pauses the workflow with a schema, exposes a form, and resumes when the input is submitted.

waitForInput has no timeout of its own, so if nobody responds, the execution waits indefinitely. To put a limit on that, set a workflow-level settings.timeout; if it elapses before the reviewer submits the form, the execution is canceled.

When the workflow reaches this step, the execution pauses and the Kibana UI shows an "Action is required" badge. Clicking Provide action opens a form where the reviewer can paste or edit the schema JSON. Since waitForInput cannot be prepopulated from a previous step, the code polls the discover step output and prints a paste-ready JSON block that can be copied directly into this form.

The step_output helper (in the notebook) polls the execution via GET /api/workflows/executions/{id} until the discover step completes and then returns its output.

Code JSON output pasted on Kibana:

The reviewer can keep useful fields, rewrite unclear ones, merge overlapping values, and add notes. After approval, the workflow resumes and sends the final schema to the executor step.

For a new corpus, keep this human gate in place. Once the schema is stable, you can auto-approve and only fall back to review when it’s worth it: Route just the low-confidence extractions to a person, or compare a new discovery run against the schema stored in the schemas index and trigger review only when fields or values change beyond a threshold.

Classifying the full corpus with a smaller LLM

By the time the workflow reaches this step, the open-ended part of the job is over. From here, the small model takes over and classifies each report against the approved schema.

Note: The classification step uses ai.agent instead of ai.prompt because ai.agent accepts an inference-id, which lets it call the Elasticsearch _inference endpoint directly, while ai.prompt only accepts a connector-id.

The fetch_corpus step is the third elasticsearch.request in the workflow, so it’s worth saying why we read from the index again. The first two (by_phase and by_anomaly) only pulled a small stratified sample for the planner to reason over, not the data to label. Now that the schema is approved, fetch_corpus pulls the documents we actually want to classify. We cap it at 100 with match_all to keep the demo fast; this is where you would page through the full corpus.

For every field, it returns a value (or null), a confidence, and a short quote. Setting additionalProperties: true in the JSON schema lets the step return one entry per field without the workflow having to know the field names ahead of time. A stored extraction looks like this:

Here, review_required is true because procedure_orientation came back at 0.44 confidence, below our 0.5 threshold, which is the signal a confidence-based quality gate would act on.

The fetch_corpus step pulls the documents to classify. The foreachstep iterates over them sequentially, and iteration-on-failure handles the errors: retry covers transient API errors from the inference endpoint, and, if all attempts fail, the fallback step posts to Slack so the failure doesn’t pass silently. (An email connector works the same way.) continue: true then lets the loop move on to the next document instead of failing the whole run. 

For production-scale corpora, consider using executeAsync, which is the fan-out version of execute.

Writing schemas and extractions back to Elasticsearch

The workflow produces two things: the approved schema and the per-document field values. The store_schema step runs right after the human gate, before the classification step fans out:

Each extraction is written inside the foreach loop, so results are persisted as they’re produced rather than batched at the end.

The schemas index holds one document per discovery run (question, approved fields, reviewer notes). The extractions index holds one document per report per schema version. 

What this two-tier LLM orchestration pattern gives you

We built one Elastic workflow that pulls a stratified sample from an incident report index, sends it with a question to a large reasoning model to generate a classification schema based on the data and a user-defined angle, pauses for human approval, and then iterates over the full corpus with a small Mistral model that assigns one value per field. 

The approved schema and per-document field values are written back to Elasticsearch as structured data.

The point of the exercise is that two different shapes of work, schema discovery, and schema application can use two different model tiers and that a workflow lets you write the routing decision down.

Next steps for your own LLM pipeline

• Try it on a corpus of your own. The pattern doesn’t care whether the input is incident reports, customer feedback, weekly status updates, or property listings.

• Promote the foreach step to workflow.executeAsync once you’re comfortable for parallel fan-out at scale.

• Schedule the rediscovery workflow on a cron trigger so you can discover different schema variations based on the data that comes in.

• Read the Elastic Workflows documentation for the full step catalog.

Related reading

• Build AI agents with Elastic Inference Service (EIS) covers the broader multi-model wiring pattern via EIS, complementary to the Workflows-orchestrated split shown here.

• How to build AI agentic workflows with Elasticsearch is a higher-level survey of how Agent Builder and Workflows fit together.

• LangExtract and Elasticsearch tutorial explores a different extraction pattern using a hand-authored schema; useful for contrast with the discover-then-apply approach above.

Elasticsearch's inference API now connects directly to NVIDIA-hosted models. You get text embedding, completion, chat completion, and reranking, plus access to NVIDIA's catalog of NVIDIA Inference Microservices–optimized (NIM-optimized) retrieval and generative models, without writing any custom integration code.

In practice, that's vector search and retrieval augmented generation (RAG) applications built on NVIDIA-hosted embeddings. It's also multi-turn conversations through the chat completion API and reranking with NVIDIA's cross-encoder models to push relevance past keyword matching. All four task types run natively through the inference API, with support for both streaming and non-streaming responses. How do I get an NVIDIA API key?

NVIDIA offers a broad catalog of models designed for a wide range of use cases, all of which can be explored on the NVIDIA Build model catalog. Throughout this article, we provide specific examples of high-performance models optimized for each inference task type. After identifying the model that best aligns with your application requirements, choose the deployment approach that fits your infrastructure and operational needs. This could mean running it on-premises for greater control or using a serverless option for faster experimentation and simplified scaling.

To get started quickly, you’ll first need access to NVIDIA’s model catalog and APIs. Create an account or log in at https://www.build.nvidia.com/ to explore available models, evaluate their capabilities, and compare which ones best fit your use case before proceeding toward full-scale deployment. This site provides a web-based interface for testing models, which is useful during evaluation and experimentation. For production-level requirements, you can use NVIDIA NIM to deploy endpoints on your own infrastructure.

To access NVIDIA models, you need to generate an API key. This key will serve as the authorization mechanism when making calls to NVIDIA's endpoints. You can create, access, and manage your API keys at API keys. To create a new key, click the Generate API Key link in the top right, and then specify a name and expiration period for the key. After generating the API key, select the appropriate model for your task and set up the corresponding Elasticsearch inference endpoint.

Setting up Elasticsearch inference endpoints

Once you have set up your NVIDIA account and obtained the necessary API keys, you can create an Elasticsearch inference endpoint.

Endpoint setup can be done directly in Kibana using the console, which allows you to input the required steps into Elasticsearch even without using an API. The following sections provide examples and details on how to create and use endpoints for text embeddings, completion, chat completion, and reranking.For more examples and detailed information, please consult the Elasticsearch API reference documentation.

Creating and using a text embeddings inference endpoint

To create a text embedding inference endpoint, you first select an appropriate model that can perform embedding operations. NVIDIA lists its models in the NVIDIA Build model catalog. You can select the Text-to-Embedding or Retrieval Augmented Generation use case on the left to filter the appropriate models. You can also find NVIDIA’s text embedding models in the NVIDIA documentation. NVIDIA’s retrieval APIs include text embedding and reranking models. When choosing a model, make sure it explicitly supports text embedding inference. Text embedding models typically include an API description, such as "Creates an embedding vector from the input text."

A good example of an embedding model is the nvidia/nemotron-3-embed-1b model. You can access the deployment page for this model, which allows you to deploy this model on-premises.

Once you have selected a suitable model, open its API reference page, where you’ll find the parameters required to create an Elasticsearch inference endpoint.

Two parameters are relevant:

• model_id(required): Specifies the NVIDIA model to be used for embedding inference. This parameter is named model on the NVIDIA side.

• url(optional): The endpoint URL used to send requests to the NVIDIA model (either deployed on-premises or in a serverless environment). It must be accessible from your Elasticsearch instance.

For most text embedding models, the URL is static and NVIDIA differentiates models solely via the model parameter. If the url parameter isn’t provided during endpoint creation, the default text embedding task specific value https://integrate.api.nvidia.com/v1/embeddings will be used.

To generate text embeddings, set up an endpoint configured with the required NVIDIA model values in the service settings map:

Upon sending this request, you should receive a successful 200 OK response. This response confirms that the endpoint is functioning correctly and the settings are specified accurately, and it will detail your newly created Elasticsearch endpoint for the text embedding task type.

You can now use the newly created endpoint to generate embeddings. The request for this operation will be similar to the example shown below:

The text embeddings will be returned, accompanied by a successful HTTP 200 OK status.

This integration allows users to use the NVIDIA models directly within Elasticsearch, making advanced search and RAG applications easier to build. These production-ready models offer a reliable and robust foundation for enterprise-scale deployments.

Creating and using a completion inference endpoint

To create a completion inference endpoint, you first select an appropriate model.

NVIDIA lists its models in the NVIDIA Build model catalog. You can search for the model there, but you can also find NVIDIA’s completion models in the left-hand navigation of this large language model (LLM) API documentation.Each entry in the list links to a general description of the model. From there, you can navigate to a nested link that opens the API reference specific to the selected model. A good example of a completion model is the nvidia/nemotron-3-super-120b-a12b model. You can access the deployment page for this model, which allows you to deploy this model on-premises.

Once you have selected a suitable model, open its API reference page, where you’ll find the parameters required to successfully create an Elasticsearch inference endpoint.

Two parameters are relevant:

• model_id(required): Specifies the NVIDIA model to be used for completion inference. This parameter is named model on the NVIDIA side.

• url(optional): The endpoint URL used to send requests to the NVIDIA model (either deployed on-premises or in a serverless environment). It must be accessible from your Elasticsearch instance.

For most completion models, the URL is static, and NVIDIA differentiates between models using only the model parameter. If the url parameter isn’t specified during endpoint creation, the default value of https://integrate.api.nvidia.com/v1/chat/completions will be used.

To use a generative model for the Elasticsearch inference completion task, you configure an endpoint that supports completion operations. The service settings map must include the required configuration for the selected NVIDIA model.

Upon success, you’ll receive a 200 OK response. This response provides the details of your new Elasticsearch endpoint, which is configured for completion tasks.

The created endpoint allows you to generate both streaming and non-streaming completions. These refer to how the endpoint delivers its output. Non-streaming completions wait until the entire response is generated before sending it back in a single block, resulting in a single, slower response time. In contrast, streaming completions send the generated text back in small, continuous chunks as they’re produced, which allows you to start reading the response immediately. This continuous delivery creates the perception of faster interaction and is essential for real-time conversational interfaces.

Generating non-streaming completions

To generate non-streaming completions, you call the newly created endpoint with a request similar to the following:

You'll receive a successful 200 OK response, with the completion result:

Generating streaming completions

To use the streaming functionality for the completion task type, you need to send the identical request used for non-streaming completions, but with _stream included in the URL path:

This command will initiate a continuous flow of events, delivering a sequence of outputs similar to the example provided below:

This capability empowers users to easily integrate NVIDIA generative models directly into their Elastic applications, supporting both single-response and engaging streaming experiences for dynamic content generation.

Creating and using a chat completion inference endpoint

To enable more dynamic and flexible interactions than those supported by the standard completion inference endpoint, you configure a chat completion inference endpoint, specifically designed to handle chat-based completion tasks.

To identify the parameters required to construct the service settings map, refer to the completion inference endpoint section of this blog post. The same configuration principles apply to the chat completion inference endpoint.

The service settings map must include the required configuration settings for the selected NVIDIA model.

Upon success, you’ll receive a 200 OK response, which includes the details of your new Elasticsearch endpoint specifically for the chat completion task type.

You can now use the new endpoint to stream generated completions. Your request should resemble the following example:

The chat completion results will be delivered to you as a continuous stream of events, formatted as follows:

The chat completion capability, distinct from the simpler completion API, allows users to build stateful, multi-turn conversational AI applications directly within the Elastic Stack, using the full flexibility of NVIDIA models for dynamic user interactions following Elasticsearch inference chat completion API.

Creating and using a rerank inference endpoint

Reranking is the process of reordering the results from an initial search query to improve their relevance to your intent. Reranking is a second-stage relevance step that reorders the results returned by an initial retriever. In many cases, it uses a different model than the retriever itself, typically a cross-encoder model, which evaluates the query and each candidate document together to produce a more accurate relevance score. The output is a list of results ranked based on their relevancy, thereby drastically improving the quality and contextual accuracy of the search results.

To create a rerank inference endpoint, you first select an appropriate model that can perform reranking operations. NVIDIA lists its models in the NVIDIA Build model catalog. You can use the reranking label to select the appropriate models. You can also find NVIDIA’s reranking models in the left-hand navigation of this retrieval APIs documentation. NVIDIA includes rerankingand text embedding models in the Retrieval APIs section in its API documentation. When selecting a model, ensure that it explicitly supports rerank inference requests. Rerank models typically include an API description, such as “Ranks passages by their relation to a query.” This wording indicates that the model supports the rerank task type.

A good example of a reranking model is the nvidia/llama-nemotron-rerank-vl-1b-v2. You can access the deployment page for this model, which allows you to deploy this model on-premises.

Once you have selected a suitable model, open its API reference page, where you’ll find the parameters required to create an Elasticsearch inference endpoint. Two parameters are relevant:

• model_id(required): Specifies the NVIDIA model to be used for rerank inference.

• url(optional): The endpoint URL used by the inference endpoint to send requests to the NVIDIA service.

For most models, the URL is static and NVIDIA differentiates between models using only the model parameter. If the url parameter isn’t specified during endpoint creation, the default value

https://ai.api.nvidia.com/v1/retrieval/nvidia/reranking will be used. The nvidia/llama-nemotron-rerank-vl-1b-v2 model requires a custom URL to be specified, and it will be used in the example below.

To perform reranking tasks, you configure an inference endpoint that executes reranking operations. The service settings map must include the required configuration for the selected NVIDIA model.

The successful creation of your new Elasticsearch endpoint for the rerank task type will be confirmed by a 200 OK response, which will also provide the specific details of the endpoint.

You can then start using the new endpoint to perform a ranking operation with a request like the one shown below:

A successful HTTP 200 OK status will be returned, and the ranked entries will be included in the response. Since models are not deterministic, the results you receive may vary and may be ordered differently across calls, as the same outcome isn’t guaranteed each time.

Integrating the rerank capability with Elasticsearch and NVIDIA elevates search applications to deliver the most accurate, contextually relevant results. By using the NVIDIA reranking models within the search infrastructure of Elasticsearch, the system moves beyond simple keyword matching. This capability prioritizes the most relevant documents after the initial search, drastically improving the user experience and the utility of the data.

NVIDIA and Elasticsearch: What's next

The integration of Elasticsearch's inference API with NVIDIA marks a considerable step forward for users. By providing a standardized, simpler path to access NVIDIA's high-performance, optimized models, this integration significantly expands Elastic's capabilities. Users can now work with these models for key AI tasks, including generating text embeddings for vector search, generating and streaming content with completion models, building stateful conversational AI applications with chat completion, and drastically improving search result accuracy through reranking. This simplification streamlines the development of sophisticated AI-powered applications, from advanced RAG systems to dynamic conversational interfaces, making powerful AI more accessible for Elastic users.

Ready to get started?

• Explore the Elasticsearch API reference documentation to dive deeper into setup.

• Browse the NVIDIA model catalog to see all available models.

• Check out the NVIDIA API Documentation hub for integration guides and API references.

• Start your journey by creating an NVIDIA account and API key to begin integrating the models today.

Frequently asked questions

How do I connect Elasticsearch to NVIDIA hosted models?

Create an NVIDIA API key at build.nvidia.com, and then create an Elasticsearch inference endpoint using the nvidia service with your API key and a model_id. Elasticsearch's inference API supports four NVIDIA task types: text embedding, completion, chat completion, and reranking.

What's the default endpoint URL for NVIDIA text embedding models in Elasticsearch?

If no url is specified when creating the endpoint, Elasticsearch defaults to https://integrate.api.nvidia.com/v1/embeddings for text embedding tasks. Completion and chat completion tasks default to https://integrate.api.nvidia.com/v1/chat/completions instead.

Can I use NVIDIA NIM models deployed on my own infrastructure with Elasticsearch?

Yes. NVIDIA NIM supports on-premises deployment, and Elasticsearch's inference endpoint accepts a custom url parameter pointing to your self-hosted NIM endpoint instead of NVIDIA's serverless API.

How do I stream chat completion responses from NVIDIA models in Elasticsearch?

Append _stream to the chat completion endpoint's URL path (POST _inference/chat_completion/{id}/_stream). Elasticsearch returns results as a continuous event stream instead of a single blocking response, ending with a [DONE] event.

What's the difference between the completion and rerank task types in Elasticsearch's NVIDIA integration?

Completion and chat completion generate new text from a prompt. Reranking takes an existing list of documents and a query and then reorders them by relevance score using a cross-encoder model; it doesn't generate text, it rescores what you already retrieved.

How do NVIDIA's reranking models improve Elasticsearch search results?

NVIDIA's reranking models evaluate the query and each candidate document together, producing a relevance score used to reorder results beyond keyword matching. Elasticsearch's rerank endpoint returns each document's index and relevance score, so the highest-scoring passages surface first.

Do I need a paid NVIDIA account to use hosted models with Elasticsearch?

You need an NVIDIA account and an API key generated at build.nvidia.com; NVIDIA's build platform offers both free evaluation access and paid production tiers, depending on usage. Elasticsearch itself doesn't add separate licensing for the NVIDIA service beyond your existing NVIDIA account terms.

Model Context Protocol (MCP) is an open-source standard that enables applications and LLMs to communicate with external systems via MCP tools (programmatic capabilities), and LangGraph (an extension of LangChain) provides the orchestration framework for these agentic workflows.

We’ll implement an application that can search both internal knowledge (Elasticsearch stored data) and external sources (on the internet) to identify potential and known vulnerabilities related to a specific tool. The application will gather the information and generate a detailed summary of the findings.

Requirements

• Elasticsearch 9.2

• Python 3.1x

• OpenAI API Key

• Elasticsearch API Key

• Serper API Key

Elastic Agent Builder

Elastic Agent Builder is a set of AI-powered capabilities for developing and integrating agents that can interact with your Elasticsearch data. It provides a built-in agent that can be used for natural language conversations with your data or instance, and it also supports tool creation, Elastic APIs, A2A, and MCP. In this article, we will focus on using the MCP server for external access to the Elastic Agent Builder tools.

To know more about Agent Builder features, you can read this article.

Agent Builder MCP feature

The MCP server is available in the Agent Builder and can be accessed at:

The Agent Builder offers Built-in tools, and you can also create your custom tools.

Reference architecture

To get a complete overview of the elements used by an agentic application in an end-to-end workflow, let’s look at the following diagram:

Elasticsearch is at the center of this architecture, functioning as a vector store, providing the embeddings generation model, and also serving the MCP server to access the data via tools. To better explain the workflow, let’s look at the ingestion and the Agent Builder layer separately.

Here, the first element is the data that will be stored in Elasticsearch. The data passes through an ingest pipeline, where it is processed by the Elasticsearch ELSER model to generate embeddings and then stored in Elasticsearch.

Elastic Agent Builder layer

On this layer, the Agent Builder plays a central role by exposing the tools needed to interact with the Elasticsearch data. It manages the tools that operate over Elasticsearch indices and makes them available for consumption. Then LangChain handles the orchestration via the MCP client.

This architecture allows Agent Builder to work as one of many MCP servers available to the client so that the Elasticsearch agent builder can combine with other MCPs. This way, the MCP client can ask cross-source questions and then combine the answers.

Use case: Security vulnerability agent

The security vulnerability agent identifies potential risks based on a user’s question by combining three complementary layers:

First, it performs a semantic search with embeddings over an internal knowledge base of past incidents, configurations, and known vulnerabilities to retrieve relevant historical evidence.

Second, it searches the internet for newly published recommendations or threat intelligence that may not yet exist internally.

Finally, an LLM correlates and prioritizes both internal and external findings, evaluates their relevance to the user’s specific environment, and produces a clear explanation along with potential mitigation steps.

Developing the application

The application’s code can be found in the attached notebook.

You can see the setup for the Python application below:

We need to access Agent Builder and create one agent specialized in security queries and one tool to perform semantic search. You need to have the Agent Builder enabled for the next step. Once it’s on, we’ll use the tools API to create a tool that will perform a semantic search.

Configure your tools following the best practices defined by Elastic for developing Tools. Once created, this tool will be ready to use in the Kibana UI.

With the tool created, we can start writing the code for the ingestion workflow:

Ingest pipeline

To define the data structure, we need to have a dataset prepared for ingestion. Below is a sample document for this example:

For this type of document, we will use the following index mappings:

We are creating a semantic_text field to perform semantic search using the information from the fields marked with the copy_to property.

With that mapping definition, we can ingest the data using the bulk API.

LangChain MCP client

Here we’re going to create an MCP client using LangChain to consume the Agent Builder tools and build a workflow with LangGraph to orchestrate the client execution. The first step is to connect to the MCP server:

Next, we create an agent that selects the appropriate tool based on the user input:

We’ll use the GPT-5.2 model, which represents OpenAI’s state-of-the-art for agent management tasks. We configure it with low reasoning effort to achieve faster responses compared to the medium or high settings, while still delivering high-quality results by leveraging the full capabilities of the GPT-5 family. You can read more about the GPT 5.2 here.

Now that the initial setup is done, the next step is to define a workflow capable of making decisions, running tool calls, and summarizing results.

For this, we use LangGraph. We won’t cover LangGraph in depth here; this article provides a detailed overview of its functionality.

The following image shows a high-level view of the LangGraph application.

We need to define the application state:

To better understand how the workflow operates, here is a brief description of each function. For full implementation details, refer to the accompanying notebook.

• call_agent_builder_semantic_search: Queries internal documentation using the Agent Builder MCP server and also stores the retrieved messages in the state.

• decide_internet_search: Analyzes the internal results and determines whether an external search is required.

• perform_internet_search: Runs an external search using the Serper API when needed.

• generate_response: Correlates internal and external findings and produces a final, actionable cybersecurity analysis for the user.

With the workflow defined, we can now send a query:

In this example, we want to evaluate whether this specific version of Express is affected by known vulnerabilities.

Research results

See the complete response in this file.

This response clearly correlates internal and internet findings and provides actionable mitigation steps. It successfully highlights the severity of the vulnerability and offers a structured, security-oriented summary.

Extensions and future enhancements

This architecture is modular and allows us to extend its capabilities by replacing, improving, or adding components to the existing list. We could add another agent, consumed by the same MCP client. We can also use an automated ingestion workflow with tools such as Logstash, Kafka, or Elastic self-managed connectors. Feel free to change the LLM, the MCP client framework, or the embeddings model or add more tools depending on your needs.

Conclusion

This reference architecture shows a practical way to combine Elasticsearch, the Agent Builder, and MCP to build an AI-driven application. Its structure keeps each part independent, which makes the system easy to implement, maintain, and extend.

You can start with a simple setup (like the security use case in this article) and scale it by adding new tools, data sources, or agents as your needs grow. Overall, it provides a straightforward path for building flexible and reliable agentic workflows on top of Elasticsearch.