One setting for production vector search: How vectordb_document mode tunes Elasticsearch automatically

We’re introducing one setting for production-ready vector search. The new vectordb_document index mode stores raw vectors as bfloat16 to halve their disk footprint and preloads vector data structures into the filesystem cache. It also lets segment merges run unthrottled and in parallel. In our benchmarks, it delivered up to 2× the queries per second (QPS) at high recall with bbq_hnsw and cut time to a search-ready index by roughly 20% with bbq_disk, with no tuning required. It’s available in Stateful Elasticsearch 9.5 and in Elasticsearch Serverless today.

Elasticsearch supports a diverse range of use cases, including observability metrics and logs and complex geospatial analysis. However, as vector search becomes a core component of modern architectures, the need for specialized optimization has grown. Achieving peak performance for vector-heavy workloads often requires navigating a complex space of configuration knobs. To reduce this operational toil, we wanted to provide opinionated, high-performance defaults through a single setting that simplifies performance tuning for production environments. The vectordb_document index mode is designed specifically for optimal vector search workloads.

How to set up vectordb_document mode

Setup is a single setting. When you’re creating an index, define the following in the index settings:

PUT my-index
{
  "settings" : {
    "index" : {
      "mode" : "vectordb_document"
     }
   }
}

vectordb_document mode is available on all Elasticsearch subscription tiers, including Basic.

Most indices used for vector search also support other operations, such as aggregations or geo-search. They also support hybrid search. Because vector search is typically the most computationally demanding part of these mixed workloads, we recommend using the vectordb_document index mode to prioritize performance for your most intensive operations. An index in vectordb_document mode remains highly capable, supporting almost all operations available in the default standard mode, while optimizing specifically for the resource-heavy demands of vector search.

The “_document” suffix represents our roadmap. We're also developing a “vectordb_columnar” mode as another way to optimize vector search, suited to different data and access patterns.

Elasticsearch vector search benchmarks across four datasets

To validate these defaults, we performed extensive benchmarking across various datasets and two index types: bbq_hnsw and bbq_disk. We ran all benchmarks on a single-node Elasticsearch instance on AWS using a c8gd.2xlarge instance (Graviton 4, ARM64, local NVMe SSD) with a pod limited to 8GB RAM (2GB heap) and 4 CPUs, using a single shard.

Datasets:

bbq_hnsw: HNSW index performance with vectordb_document

Query throughput and recall

On all four datasets, vectordb_document produced a better QPS–recall curve, and the shape of the advantage is itself informative. On dbpedia-openai-1M, msmarco-v2-10M, and arxiv-for-fanns-large, the curves start close together at low recall (essentially identical on arXiv) and separate as recall rises, reaching roughly 1.4×, 2.2×, and 2× at the high-recall end. On laion-img-emb-512-20M, the curves are apart from the start and settle at about 2× from recall 0.80 upward.

The gap widens with recall because higher recall is bought with oversampling, and oversampling is exactly where vectordb_document saves. The two effects compound. bfloat16 storage halves the bytes read per rescored candidate. More importantly for hierarchical navigable small world (HNSW), the oversample factor is applied to the graph search itself;  each segment is searched for k × oversample candidates, so cost scales with oversample times segment count. Unthrottled parallel merging leaves fewer, larger graphs (14–21 segments versus 22–29), and the graph and quantized vector files are preloaded into the filesystem cache, so vectordb_document pays far less for each increment of oversample.

Comparing identical search settings, rather than equal recall, makes the effect explicit. With rescoring switched off, the two modes are within 1%–28% of each other; at oversample 5, vectordb_document is 2–4× faster; at oversample 10, up to 10×. Those high-oversample settings sit off the QPS–recall frontier, which is why the curves above top out nearer 2×, but they isolate where the saving comes from.

Indexing speed and merge behavior 

In our benchmarks, vectordb_document increased upload time by roughly 16%. This is expected: merges now run unthrottled and parallelized across threads, so they compete with indexing for CPU while documents are still being ingested. Measured indexing time rose 9%–26% across the four datasets. HNSW graph construction is CPU-bound, so that contention is felt directly.

The same changes make the post-upload phase much cheaper. Disabling auto-throttling removes merge rate limiting entirely (in default mode, DBpedia spent 57% of its merge time throttled), and bfloat16 halves the raw vector data, cutting total bytes merged by 42%–49%. Post-ingest merging finished 75%–86% sooner on three of the four datasets, which brings total time to a search-ready index to only about 6% above the baseline. DBpedia was the exception in the other direction: Its merge tail dominated, so total time actually fell 25%.

bbq_disk: disk-based vector search performance with vectordb_document

bbq_disk query throughput and recall

For bbq_disk indices, vectordb_document mode's impact on search throughput varied by dataset. On the lower-dimensional datasets, QPS was essentially unchanged at equal recall: laion-img-emb-512-20M-cosine (512 dims) and msmarco-v2-10M-jina-v5-1024 (1024 dims) track each other closely across the recall range, with vectordb_document ahead in the lower recall end and a few percent behind at the high-recall end. On the higher-dimensional datasets, we saw a consistent gain of roughly 20% at equal recall: dbpedia-openai-1M-3072-angular (3072 dims) and arxiv-for-fanns-large (4096 dims).

Our interpretation is that this is mainly a rescoring effect. vectordb_document stores vectors as bfloat16, so each rescored candidate reads 2× dims bytes instead of 4× dims. The sweep supports this directly: The advantage grows with the query-time oversample factor, which is exactly what sets how many candidates get rescored. On DBpedia, the QPS ratio rises from 1.16× at oversample 3 to 4.3× at oversample 8, and on arXiv from 1.20× to 1.56×, while on laion and MS MARCO, it stays flat or drifts just below 1. Rescoring is simply a much larger share of the query on the smaller, higher-dimensional datasets; on the 10 million and 20 million ones, scanning 1-bit posting lists dominates.

Indexing speed and merge behavior 

Across the four datasets, vectordb_document cut total time to a fully merged, search-ready index by about 20%. Every dataset improved, from 6% on msmarco-v2-10M to 50% on dbpedia-openai-1M, where the post-upload merge phase alone fell from 322 seconds to 54 seconds. Upload time on its own is less clear-cut: On the day charted below, it finished 2%–15% sooner, depending on dataset, but on the earlier run set, it was marginally slower, so we read upload as unchanged to modestly faster and treat time-to-searchable as the real result.

The gain is almost entirely in merging. In default mode, Elasticsearch rate-limits how fast merges may write, and that limiter was binding hard: 75% of all merge time on DBpedia and 73% on laion was spent paused by it. vectordb_document disables the limiter, so paused time is zero and merge time drops 63% on DBpedia and 42% on laion. bfloat16 helps for the same reason: The limiter meters bytes written, so halving the raw vector data means less to write under the cap. The gradient across datasets follows the throttling rather than the byte count: arXiv, at 21% of merge time throttled, saw 30% less merge time, while MS MARCO, never throttled, got 55% fewer bytes but only 11% less merge time.

The two index types respond differently on ingest because merging costs something different in each. Merging bbq_hnsw segments means rebuilding HNSW graphs: CPU-bound work that competes directly with the equally CPU-bound graph construction on incoming documents, and on a 4-CPU pod that contention surfaces as a slower upload. bbq_disk merges are dominated by writing bytes rather than by CPU, so lifting the rate limiter spends I/O bandwidth that the local NVMe has to spare, and bfloat16 means that there are fewer bytes to write in the first place. Both index types reach a fully merged index far sooner; the difference is only whether the upload phase pays for it.

What vectordb_document sets under the hood

element_type (dense_vector)

• Value: bfloat16.

• Impact: Stores each dimension of the raw vector as bfloat16 instead of the default float32, halving the storage of raw vectors with negligible impact on recall.

• Benefit: Lower disk footprint (reduced total cost of ownership [TCO]), faster fetching of vectors for rescoring.

Dynamic float array mapping

• Value: Float arrays with 32 or more values are dynamically mapped as dense_vector. (In the default mode, this threshold is 128.)

• Impact: No need to explicitly declare the field as a dense vector field; the system identifies it automatically.

• Benefit: Configuration simplicity.

exclude_source_vectors

• Value: true.

• Impact: Vectors are stored once in the vector index and not duplicated in _source. They’re omitted from _source in responses, but they can still be retrieved on request.

Benefit: Lower disk footprint; faster queries, since large vectors are no longer shipped with every _source fetch.

index.store.preload

• Value: ["vex", "veq", "veb", "cenivf"].

• Impact: Preloads search-time vector data structures into the filesystem cache whenever new segments are opened.

• Benefit: Reduced query latency.

index.merge.intra_merge_parallelism_enabled

• Value: true.

• Impact: Use parallel threads for segment merging to achieve optimal segment sizes.

• Benefit: Faster convergence to fewer, larger segments leads to better recall, lower query latency.

index.merge.scheduler.auto_throttle

• Value: false.

• Impact: Allows merges to proceed at full speed.

• Benefit: Merges reach optimal segment sizes sooner, resulting in better recall, lower query latency.

While these defaults are optimized for the majority of use cases, most can be overridden individually to suit unique hardware constraints or extreme performance requirements, with one exception: exclude_source_vectors: true is locked in and cannot be changed on a vectordb_document index.

Summary

The vectordb_document index mode is a production-ready foundation for vector search: By adopting high-performance defaults, teams can focus on building features rather than on hand-tuning merge or storage settings or on adjusting preload settings. 

Across four datasets, the picture is consistently favorable, though the balance differs by index type. With bbq_hnsw, QPS recall improved on every dataset, from roughly on par at low recall to as much as double at the high-recall end, in exchange for a modest ingest cost of about 16% longer upload and 6% longer time to a search-ready index. With bbq_disk, the trade runs the other way: Total time to a search-ready index dropped by about 20%, while search improved by about 20% at equal recall on the higher-dimensional datasets (3072 and 4096 dims) and was essentially unchanged on the lower-dimensional ones. The split comes down to what merging costs in each: Rebuilding HNSW graphs is CPU work that competes with indexing; whereas bbq_disk merges are write-bound and simply run faster once they’re unthrottled.

In both cases, the new defaults move the system in the direction that most users want, with no per-index tuning required. And, for the settings that genuinely depend on the data itself, such as the degree of quantization, auto-calibration now derives them for you. (See How Elasticsearch auto-tunes vector quantization to hit your recall target.)

Try it out by creating an index with the vectordb_document index mode in Stateful Elasticsearch 9.5 or in Serverless.