Tune approximate kNN search

Elasticsearch supports approximate k-nearest neighbor search for efficiently finding the k nearest vectors to a query vector. Since approximate kNN search works differently from other queries, there are special considerations around its performance.

Many of these recommendations help improve search speed. With approximate kNN, the indexing algorithm runs searches under the hood to create the vector index structures. So these same recommendations also help with indexing speed.

If an index exists primarily for vector search, create it with the vectordb_document index mode, which applies several of the recommendations on this page as index defaults: bfloat16 vectors, vectors excluded from _source, preloaded vector index files, and merges that parallelize internally and run unthrottled. Because index.mode is final, set it at index creation and reindex to adopt it for existing data.

For the setup example, see Vector index mode. For the full list of defaults, see Index modes for vector search.

The default element_type is float. But this can be automatically quantized during index time through quantization. Quantization will reduce the required memory by 4x, 8x, or as much as 32x, but it will also reduce the precision of the vectors and increase disk usage for the field (by up to 25%, 12.5%, or 3.125%, respectively). Increased disk usage is a result of Elasticsearch storing both the quantized and the unquantized vectors. For example, when int8 quantizing 40GB of floating point vectors an extra 10GB of data will be stored for the quantized vectors. The total disk usage amounts to 50GB, but the memory usage for fast search will be reduced to 10GB.

For float vectors with dim greater than or equal to 384, using a quantized index is highly recommended.

The speed of kNN search scales linearly with the number of vector dimensions, because each similarity computation considers each element in the two vectors. Whenever possible, it’s better to use vectors with a lower dimension. Some embedding models come in different "sizes", with both lower and higher dimensional options available. You could also experiment with dimensionality reduction techniques like PCA. When experimenting with different approaches, it’s important to measure the impact on relevance to ensure the search quality is still acceptable.

Elasticsearch stores the original JSON document that was passed at index time in the _source field. By default, each hit in the search results contains the full document _source. When the documents contain high-dimensional dense_vector fields, the _source can be quite large and expensive to load. This could significantly slow down the speed of kNN search.

For vector fields specifically, prefer the index.mapping.exclude_source_vectors index setting over the generic excludes mapping parameter. Vectors excluded through this setting are rehydrated automatically, so reindex, update, and recovery continue to work as expected. To retrieve vector values in a specific search response, use the fields option.

This setting is enabled by default for indices created on 9.2+.

You can also disable storing dense_vector fields in the _source through the excludes mapping parameter. This prevents loading and returning large vectors during search, and also cuts down on the index size. Vectors that have been omitted from _source can still be used in kNN search, since it relies on separate data structures to perform the search. Before using the excludes parameter, make sure to review the downsides of omitting fields from _source.

Another option is to use synthetic _source.

Elasticsearch uses either the Hierarchical Navigable Small World (HNSW) algorithm or the Disk Better Binary Quantization (DiskBBQ) algorithm for approximate kNN search.

HNSW is a graph-based algorithm which only works efficiently when most vector data is held in memory. You should ensure that data nodes have at least enough RAM to hold the vector data and index structures.

DiskBBQ is a clustering algorithm which can scale efficiently often on less memory than HNSW. Where HNSW typically performs poorly without sufficient memory to fit the entire structure in RAM, DiskBBQ scales linearly when using less available memory than the total index size. You can start with enough RAM to hold the vector data and index structures but, in most cases, you should be able to reduce your RAM allocation and still maintain good performance.

A dense_vector field stores more than the values you index. On disk, Elasticsearch keeps the raw vectors (for rescoring and reindex), any quantized copy used for approximate search, and the search structure (an HNSW graph, or DiskBBQ centroids and clusters).

Each structure is a Lucene file (also reported under off_heap.*_size_bytes in index stats). Metadata files (.vem, .vemf, .vemq, .vemb) are small and you do not need to preload them.

Disk usage for a dense_vector field includes the raw vectors, quantized vectors when quantization is enabled, and the search structure.

Raw (unquantized) vectors are always stored on disk, including when quantization is enabled. Size depends on element_type:

When quantization is enabled, Elasticsearch stores both the raw vectors and a quantized copy. That increases total disk usage and reduces off-heap RAM. Quantized vector storage applies to float and bfloat16 only. The extra 14 or 16 bytes per vector are a small correction the quantizer stores so it can rescore accurately.

Overhead depends on the search algorithm. The default for HNSW m is 16. The default for DiskBBQ vectors_per_cluster is 384.

For DiskBBQ, add centroid bytes and cluster bytes. The × 2 on cluster vectors is a conservative upper bound: a vector might be written to a second cluster. Real cluster storage is between 1× and 2×.

Each shard replica holds a full copy. Multiply the per-replica figure by 1 + number of replicas for cluster-wide disk. To check the size of vector data in an existing index, use the Analyze index disk usage API.

Disk and off-heap RAM are two different numbers, and they can differ by a lot. Disk is every structure persisted for the field. Off-heap RAM is the working set that must stay in the operating system's filesystem cache for fast, stable query latency. Vector data is memory-mapped, so it lives in the OS page cache, separate from the Java heap.

Provision at least the off-heap RAM figure per copy, plus headroom. Once the working set no longer fits in cache, queries start reading from disk and latency climbs sharply. For quantized indices the raw vectors stay on disk (read only for optional rescoring), so they count toward disk but not toward the required off-heap RAM.

Data nodes should also leave a buffer for other ways that RAM is needed. For example your index might include text fields and numerics, which also benefit from using filesystem cache. Run benchmarks with your dataset to confirm there is enough memory for good search performance. Nightly examples include the so_vector and dense_vector tracks.

If the machine running Elasticsearch is restarted, the filesystem cache will be empty, so it will take some time before the operating system loads hot regions of the index into memory so that search operations are fast. You can explicitly tell the operating system which files should be loaded into memory eagerly depending on the file extension using the index.store.preload setting.

Preload only the files that must stay in RAM. For quantized HNSW or flat, that is the quantized codes (.veq or .veb) plus the HNSW graph (.vex). For DiskBBQ, preload the centroids (.cenivf). Do not preload raw vectors (.vec). Paging them in can evict those index structures from the cache.

You can gather additional detail about the specific files by using the stats endpoint, which displays information about the index and fields.

For example, for DiskBBQ, the response might look like this:

				GET my_index/_stats?filter_path=indices.my_index.primaries.dense_vector
					{
    "indices": {
        "my_index": {
            "primaries": {
                "dense_vector": {
                    "value_count": 3,
                    "off_heap": {
                        "total_size_bytes": 249,
                        "total_veb_size_bytes": 0,
                        "total_vec_size_bytes": 36,
                        "total_veq_size_bytes": 0,
                        "total_vex_size_bytes": 0,
                        "total_cenivf_size_bytes": 111,
                        "total_clivf_size_bytes": 102,
                        "fielddata": {
                            "my_vector": {
                                "cenivf_size_bytes": 111,
                                "clivf_size_bytes": 102,
                                "vec_size_bytes": 36
                            }
                        }
                    }
                }
            }
        }
    }
}
		

Elasticsearch shards are composed of segments, which are internal storage elements in the index. For approximate kNN search, Elasticsearch stores the vector values of each segment as a separate HNSW graph, so kNN search must check each segment. The recent parallelization of kNN search made it much faster to search across multiple segments, but still kNN search can be up to several times faster if there are fewer segments. By default, Elasticsearch periodically merges smaller segments into larger ones through a background merge process. If this isn’t sufficient, you can take explicit steps to reduce the number of index segments.

Elasticsearch provides many tunable settings for controlling the merge process. One important setting is index.merge.policy.max_merged_segment. This controls the maximum size of the segments that are created during the merge process. By increasing the value, you can reduce the number of segments in the index. The default value is 5GB, but that might be too small for larger dimensional vectors. Consider increasing this value to 10GB or 20GB can help reduce the number of segments.

A common pattern is to first perform an initial bulk upload, then make an index available for searches. Instead of force merging, you can adjust the index settings to encourage Elasticsearch to create larger initial segments:

• Ensure there are no searches during the bulk upload and disable index.refresh_interval by setting it to -1. This prevents refresh operations and avoids creating extra segments.
• Give Elasticsearch a large indexing buffer so it can accept more documents before flushing. By default, the indices.memory.index_buffer_size is set to 10% of the heap size. With a substantial heap size like 32GB, this is often enough. To allow the full indexing buffer to be used, you should also increase the limit index.translog.flush_threshold_size.

For indexing-heavy workloads on large vector datasets, GPU acceleration can significantly speed up HNSW index construction and reduce the cost of merging segments into larger ones. See GPU accelerated vector indexing for supported configurations and setup.

Actively indexing documents can have a negative impact on approximate kNN search performance, since indexing threads steal compute resources from search. When indexing and searching at the same time, Elasticsearch also refreshes frequently, which creates several small segments. This also hurts search performance, since approximate kNN search is slower when there are more segments.

When possible, it’s best to avoid heavy indexing during approximate kNN search. If you need to reindex all the data, perhaps because the vector embedding model changed, then it’s better to reindex the new documents into a separate index rather than update them in-place. This helps avoid the slowdown mentioned above, and prevents expensive merge operations due to frequent document updates.

Search can cause a lot of randomized read I/O. When the underlying block device has a high readahead value, there may be a lot of unnecessary read I/O done, especially when files are accessed using memory mapping (see storage types).

Most Linux distributions use a sensible readahead value of 128KiB for a single plain device, however, when using software raid, LVM or dm-crypt the resulting block device (backing Elasticsearch path.data) may end up having a very large readahead value (in the range of several MiB). This usually results in severe page (filesystem) cache thrashing adversely affecting search (or update) performance.

You can check the current value in KiB using lsblk -o NAME,RA,MOUNTPOINT,TYPE,SIZE. Consult the documentation of your distribution on how to alter this value (for example with a udev rule to persist across reboots, or via blockdev --setra as a transient setting). We recommend a value of 128KiB for readahead.

If you use quantized indices and your nodes don't have enough off-heap RAM to store all vector data in memory, then you might experience high query latencies. Vector data includes the HNSW graph, quantized vectors, and raw float vectors.

In these scenarios, on-disk rescoring can significantly reduce query latency. Enable it by setting the on_disk_rescore: true option on your vector indices. Your data must be re-indexed or force-merged to use the new setting in subsequent searches.