Recap

First, let’s bring you up to speed. Elasticsearch has established itself as a powerful vector database, offering a rich set of features and strong performance for large-scale similarity search. With capabilities such as scalar quantization, Better Binary Quantization (BBQ), SIMD vector operations, and more disk-efficient algorithms like DiskBBQ, it already provides efficient and flexible options for managing vector workloads.

By integrating NVIDIA cuVS as a callable module for vector search tasks, we aim to deliver significant gains in vector indexing performance and efficiency to better support large-scale vector workloads.

The challenge

One of the toughest challenges in building a high-performance vector database is constructing the vector index - the HNSW graph. Index building quickly becomes dominated by millions or even billions of arithmetic operations as every vector is compared against many others. In addition, index lifecycle operations, such as compaction and merges, can further increase the overall compute overhead of indexing. As data volumes and associated vector embeddings grow exponentially, accelerated computing GPUs, built for massive parallelism and high-throughput math, are ideally positioned to handle these workloads.

Enter the Elasticsearch-GPU Plugin

NVIDIA cuVS is an open-source CUDA-X library for GPU-accelerated vector search and data clustering that enables fast index building and embedding retrieval for AI and recommendation workloads.

Elasticsearch uses cuVS through cuvs-java, an open-source library developed by the community and maintained by NVIDIA. The cuvs-java library is lightweight and builds on the cuVS C API using Panama Foreign Function to expose cuVS features in an idiomatic Java way, while remaining modern and performant.

The cuvs-java library is integrated into a new Elasticsearch plugin; therefore, vector indexing on the GPU can occur on the same Elasticsearch node and process, without the need to provision any external code or hardware. During index building, if the cuVS library is installed and a GPU is present and configured, Elasticsearch will use the GPU to accelerate the vector indexing process. The vectors are given to the GPU, which constructs a CAGRA graph. This graph is then converted to the HNSW format, making it immediately available for vector search on the CPU. The final format of the built graph is the same as what would be built on the CPU; this allows Elasticsearch to leverage GPUs for high-throughput vector indexing when the underlying hardware supports it, while freeing CPU power for other tasks (concurrent search, data processing, etc.).

Index build acceleration

As part of integrating GPU acceleration into Elasticsearch, several enhancements were made to cuvs-java, focusing on efficient data input/output and function invocation. A key enhancement is the use of cuVSMatrix to transparently model vectors, whether they reside on the Java heap, off-heap, or in GPU memory. This enables data to move efficiently between memory and the GPU, avoiding unnecessary copies of potentially billions of vectors.

Thanks to this underlying zero-copy abstraction, both transferring to GPU memory and retrieving the graph can occur directly. During indexing, vectors are first buffered in memory on the Java heap, then sent to the GPU to construct the CAGRA graph. The graph is subsequently retrieved from the GPU, converted into HNSW format, and persisted to disk.

At merge time, the vectors are already stored on disk, bypassing the Java heap entirely. Index files are memory-mapped, and data is transferred directly into GPU memory. The design also easily accommodates different bit-widths, such as float32 or int8, and naturally extends to other quantization schemes.

Drumroll…so, how does it perform?

Before we get into the numbers, a bit of context is helpful. Segment merging in Elasticsearch typically runs automatically in the background during indexing, which makes it difficult to benchmark in isolation. To obtain reproducible results, we used force-merge to explicitly trigger segment merging in a controlled experiment. Since force-merge performs the same underlying merge operations as background merging, its performance serves as a useful indicator of expected improvements, even though the exact gains may differ in real-world indexing workloads.

Now, let’s see the numbers.

Our initial benchmark results are very promising. We ran the benchmark on an AWS g6.4xlarge instance with locally attached NVMe storage. A single node of Elasticsearch was configured to use the default, optimal number of indexing threads (8 - one for each physical core), and to disable merge throttling (which is less applicable with fast NVMe disks).

For the dataset, we used 2.6 million vectors with 1,536 dimensions from the OpenAI Rally vector track, encoded as base64 strings, and indexed as float32 hnsw. In all scenarios, the constructed graphs achieve recall levels of up to 95%. Here’s what we found:

• Indexing Throughput: By moving graph construction to the GPU during in-memory buffer flushes, we increase throughput by ~12x.

• Force-merge: After indexing completes, the GPU continues to accelerate segment merging, speeding up the force-merge phase by ~7x.

• CPU usage: Offloading graph construction to the GPU significantly reduces both average and peak CPU utilization. The graphs below illustrate CPU usage during indexing and merging, highlighting how much lower it is when these operations run on the GPU. Lower CPU utilization during GPU indexing frees up CPU cycles that can be redirected to improve search performance.

• Recall: Accuracy remains effectively the same between CPU and GPU runs, with the GPU-built graph reaching marginally higher recall.

Comparing along another dimension: Price

The earlier comparison intentionally used identical hardware, with the only difference being whether the GPU was used during indexing. That setup is useful for isolating raw compute effects, but we can also look at the comparison from a cost perspective.

At roughly the same hourly price as the GPU-accelerated configuration, one can provision a CPU-only setup with approximately twice the comparable CPU and memory resources: 32 vCPUs (AMD EPYC) and 64 GB of RAM, allowing to double the number of indexing threads to 16.

To keep the comparison fair and consistent, we ran this CPU-only experiment on an AWS g6.8xlarge instance, with the GPU explicitly disabled. This allowed us to hold all other hardware characteristics constant while evaluating the cost–performance trade-off of GPU acceleration versus CPU-only indexing.

The more powerful CPU instance does show improved performance compared to the benchmarks in the above section, as you would expect. However, when we compare this more powerful CPU instance against the original GPU-accelerated results, the GPU still delivers substantial performance gains: ~5x improvement in indexing throughput, and ~6x in force merge, all while building graphs that achieve recall levels of up to 95%.

Conclusion

In end-to-end scenarios, GPU acceleration with NVIDIA cuVS delivers nearly a 12x improvement in indexing throughput and a 7x decrease in force-merge latency, with significantly lower CPU utilization. This shows that vector indexing and merge workloads benefit significantly from GPU acceleration. On a cost-adjusted comparison, GPU acceleration continues to yield substantial performance gains, with approximately 5x higher indexing throughput and 6x faster force-merge operations.

GPU-accelerated vector indexing is currently planned for Tech Preview in Elasticsearch 9.3, which is scheduled to be released early in 2026.

Stay tuned for more.

As part of our mission to leave no efficiencies behind, we are exploring acceleration opportunities with these curious computer chips, which you may have heard of—NVIDIA GPUs! (Seriously, have you not?).

When obsessing over performance, we have several problem spaces to explore—how to index exponentially more data, how to retrieve insights from it, and how to do it when your ML models are involved. You should be able to eke out every last benefit available when you have GPUs.

In this post, we dive into our collaboration with the NVIDIA vector search team as we explore GPU-accelerated vector search in Elasticsearch. This work paves the way for use cases where developers could use a mix of GPUs and CPUs for real-world Elasticsearch-powered apps. Exciting times!

Elasticsearch GPUs

We are excited to share that the Elasticsearch engineering team is helping build the open-source cuVS Java API experience for developers, which exposes bindings for vector search algorithms. This work leverages our previous experience with Panama FFI. Elasticsearch and Apache Lucene use the NVIDIA cuVS API to build the graph during indexing. Okay, we are jumping ahead; let’s rewind a bit.

NVIDIA cuVS, an open-source C++ library, is at the heart of this collaboration. It aims to bring GPU acceleration to vector search by providing higher throughput, lower latency, and faster index build times. But Elasticsearch and Apache Lucene are written in Java; how will this work?

Enter lucene-cuvs and the Elastic-NVIDIA-SearchScale collaboration to bring it into the Lucene ecosystem to explore GPU-accelerated vector search in Elasticsearch. In the recent NVIDIA cuVS 25.02 release, we added a Java API for cuVS. The new API is experimental and will continue to evolve, but it’s currently available for use. The question may arise: aren’t Java to native function calls slow? Not anymore! We’re using the new Panama FFI (Foreign Function Interface) for the bindings, which has minimal overhead for Java to native downcalls.

We’ve been using Panama FFI in Elasticsearch and Lucene for a while now. It’s awesome! But... there is always a “but”, isn’t there? FFI has availability challenges across Java versions. We overcame this by compiling the cuVS API to Java 21 and encapsulating the implementation within a multi-release jar targeting Java 22. This allows the use of cuVS Java directly in Lucene and Elasticsearch.

Ok, now that we have the cuVS Java API, what else would we need?

A tale of two algorithms for CPU

Elasticsearch supports the HNSW algorithm for scalable approximate KNN search. However, to get the most out of the GPU, we use a different algorithm, CAGRA [CUDA ANN GRAph], which has been specifically designed for the high levels of parallelism offered by the GPU.

Before we get into how we look to add support for CAGRA, let’s look at how Elasticsearch and Lucene access index data through a “codec format”. This consists of

1. the on-disk representation,

2. the interfaces for reading and writing data,

3. and the machinery for dealing with Lucene’s segment-based architecture.

We are implementing a new KNN (k-nearest neighbors) vector format that internally uses the cuVS Java API to index and search on the GPU. From here, we “plumb” this codec type through Elasticsearch’s mappings to a field type in the index. As a result, your existing KNN queries continue to work regardless of whether the backing index is using a CAGRA or HNSW graph. Of course, this glosses over many details, which we plan to cover in a future blog. The following is the high-level architecture for a GPU-accelerated Elasticsearch.

This new codec format defaults to CAGRA. However, it also supports converting a CAGRA graph to an HNSW graph for search on the CPU.

Indexing and searching on the GPU: Making some “core” decisions

With the stateless architecture for Elasticsearch Serverless, which separates indexing and search, there is now a clear delineation of responsibilities. We pick the best hardware profile to fulfill each of these independent responsibilities.

We anticipate users to consider two main deployment strategies:

1. Index and search on the GPU: During indexing, build a CAGRA graph and use it during search - ideal when extremely low latency search is required.

2. Index on GPU and search on CPU: During indexing, build a CAGRA graph and convert it to an HNSW graph. The HNSW graph is stored in the index, which can later be used on the CPU for searching.

This flexibility provides different deployment models, offering tradeoffs between cost and performance. For example, an indexing service could use GPU to efficiently build and merge graphs in a timely manner while using a lower-powered CPU for searching.

So here is the plan for GPU-accelerated vector search in Elasticsearch

We are looking forward to bringing performance gains and flexibility with deployment strategies to users, offering various knobs to balance cost and performance. Here is the NVIDIA GTC 2025 session where this work was presented in detail.

We’d like to thank the engineering teams at NVIDIA and SearchScale for their fantastic collaboration. In an upcoming blog, we will explore the implementation details and performance analysis in greater depth. Hold on to your curiosity hats 🎩!

In this blog, we’ll demonstrate how to use DocParse and DocPrep to prepare and load a dataset of complex PDFs into Elasticsearch for a RAG application. We will use ~75 PDF reports from the National Transportation Safety Board (NTSB) about aircraft incidents. An example document from the collection is here.

What is Aryn DocParse and DocPrep

Aryn DocParse segments and labels documents, extracts tables, and images, and does OCR – turning 30+ document types into structured JSON. It runs the open-source Aryn Partitioner and its open-source deep learning DETR AI model trained on 80k+ enterprise documents. This leads to up to 6x more accurate data chunking and 2x improved recall on hybrid search or RAG compared to off-the-shelf systems.

Aryn DocPrep is a tool for creating document ETL pipelines to prepare and load this data into vector databases and hybrid search indexes like Elasticsearch. The first step in a pipeline is using DocParse to process each document. DocPrep creates Python code using Sycamore, an open-source, scalable, LLM-powered document ETL library. Though DocPrep can easily create ETL pipelines using Sycamore code, you will likely need to customize the pipeline using additional Sycamore data transforms, chunking/merging, extraction, and cleaning functions.

As can be seen, these documents are complex, containing tables, images, section headings, and complicated layouts. Let’s begin!

Building high-quality RAG apps with effective data preparation

Launch an Elasticsearch vector database container

We’ll install Elasticsearch locally using a Docker container for the demo RAG application. Follow these instructions to deploy it.

Prepare data for RAG using Aryn DocPrep and DocParse

Aryn DocPrep is a tool for creating document ETL pipelines that prepare and load data into vector databases and hybrid search indexes like Elasticsearch. The first step in a pipeline is using DocParse to process each document.

We will use Aryn DocParse in Aryn Cloud to generate our initial ETL pipeline code. You can sign up for free to use Aryn Cloud and go to the DocPrep UI in the Aryn Cloud console.

You can also write an ETL pipeline and run a version of the Aryn Partitioner (used in DocParse) locally. Visit the Sycamore documentation to learn more.

Create ETL pipeline with Aryn DocPrep

DocPrep creates Python code using Sycamore, an open-source, scalable, LLM-powered document ETL library. While DocPrep can easily create ETL pipelines using Sycamore code, you may need to customize the pipeline further with additional Sycamore data transforms, extraction, and cleaning functions.

DocPrep simplifies the creation of a base ETL pipeline to prepare unstructured data for RAG and semantic search.

First, we provide the document type (PDF) and the source location of our PDFs in Amazon S3 (s3://aryn-public/ntsb/):

Next, we will select MiniLM for our embedding model to create our vector embeddings locally. DocPrep uses DocParse for document segmentation, extraction, and other processing, but we don’t need to change the default configuration.

Finally, we select Elasticsearch as our target database and add the Host URL and Index name. Note that the URL is set to “localhost” because we are running Elasticsearch locally. We will also run DocPrep/Sycamore ETL pipeline locally so it can easily load the cluster.

Now, click “Generate pipeline” to create the ETL pipeline. Click “Download notebook” on the next page to download the code as a Jupyter notebook file.

Install Jupyter and Sycamore

We will run the ETL pipeline locally in a Jupyter notebook and use the Sycamore document ETL library. As a first step, install Jupyter and Sycamore with the Elasticsearch connector and local inference libraries to create vector embeddings.

Run Pipeline

Run Jupyter and open the notebook with the ETL pipeline downloaded in the earlier step.

If you haven’t set your Aryn Cloud API key as an environmental variable called ARYN_API_KEY, you can set it directly in the notebook.

In the second-to-last cell, update the Elasticsearch loading configuration. Replace the es_client_args from setting an Elasticsearch password to the Elasticsearch basic auth config from your container:

If the password isn’t set as an environment variable, you can add it directly here.

Now, run the cells in the notebook. Each of the ~75 PDFs is sent to DocParse for processing, and this step in the pipeline will take a few minutes. One of the cells will output three pages of a document with bounding boxes to show how DocParse segments the data.

The final cell runs a read query to verify if the data has been loaded correctly. Now, you can use the prepared data in the Elasticsearch index with your RAG application.

Add additional data enrichment and transforms

The code generated in DocPrep is great for a basic ETL pipeline, however, you may want to extract metadata and perform data cleaning. The pipeline code is fully customizable, and you can use additional transformations in Sycamore or arbitrary Python code.

Here is an example notebook with additional data transforms, metadata extraction, and data cleaning steps. You can use this metadata in your RAG applications to filter your results.

Conclusion

This blog used Aryn DocParse, DocPrep, and Sycamore to parse, extract, enrich, clean, embed, and load data into vector and keyword indexes in the Elasticsearch vector database. We used DocPrep to create an initial ETL pipeline and then used a notebook with additional Sycamore code to demonstrate additional data enrichment and cleaning.

How your documents are parsed, enriched, and processed significantly impacts the quality of your RAG queries. Use the examples in this blog post to quickly and easily build your own RAG systems with Aryn and Elasticsearch and iterate on the processing and retrieval strategies as you build your GenAI application.

Below are some resources for your next steps:

• Sample notebook with Aryn DataPrep and Elasticsearch

• start-local with Elasticsearch vector database

• Get started with Aryn Cloud DocPrep

• Sycamore documentation

• Elasticsearch vector database ecosystem integrations