Optimize approximate kNN search performance and accuracy

After you have approximate kNN search working, use this page to balance latency, recall, and storage. You can change how much of the index each query explores, how vectors are stored, and whether results are rescored with the original vectors.

This page covers those query-time and encoding trade-offs. For mapping, indexing, and a basic search example, refer to Approximate kNN search. For cluster sizing, memory, segment merging, and filesystem cache guidance, refer to Tune approximate kNN search.

Approximate kNN does not score every vector. Each query explores a candidate set, keeps the best k hits on the shard, then merges shard results into the global top k. How you size that candidate set is the main search-time trade-off between latency and recall.

Which parameter you set depends on the index type:

• For HNSW indices, set num_candidates.
• For DiskBBQ (bbq_disk) indices, set visit_percentage.

If the field is quantized, you can also recover recall after retrieval with rescore_vector. Refer to Oversampling and rescoring for quantized vectors.

For each segment, Elasticsearch explores a num_candidates set of approximate neighbors. It then keeps the top k on the shard and merges those shard results into the global top k.

num_candidates must be greater than or equal to k and cannot exceed 10,000. If you omit it, it defaults to 1.5 * k when k is set, or 1.5 * size when k is omitted, capped at 10,000.

• Increase num_candidates to improve recall, at the cost of higher latency.
• Decrease num_candidates for faster queries, with a potential recall trade-off.

Because Elasticsearch searches each segment's HNSW graph separately, more segments means more candidate exploration. For guidance on reducing segment count, refer to Tune approximate kNN search.

				POST image-index/_search
					{
  "knn": {
    "field": "image-vector",
    "query_vector": [-5, 9, -12],
    "k": 10,
    "num_candidates": 100
  }
}
		

1. The number of nearest neighbors to return.
2. The candidate set explored per segment. Larger values improve recall and increase latency.

For DiskBBQ (bbq_disk) indices, visit_percentage controls the percentage of vectors explored during search. It accepts values from 0 to 100, including decimals such as 0.5 (half a percent). On Elastic Stack, bbq_disk requires an Enterprise subscription.

• If you omit visit_percentage, or set it to 0, Elasticsearch picks the visit budget from num_candidates and segment size.
• If you set a non-zero visit_percentage, DiskBBQ uses that value to size the visit budget and does not use num_candidates for that budget.
• Increase visit_percentage to improve recall, at the cost of higher latency.
• Decrease visit_percentage for faster queries, with a potential recall trade-off.

Create a bbq_disk index, then set visit_percentage at search time:

				PUT diskbbq-image-index
					{
  "mappings": {
    "properties": {
      "image-vector": {
        "type": "dense_vector",
        "dims": 3,
        "index_options": {
          "type": "bbq_disk"
        }
      }
    }
  }
}
		

				POST diskbbq-image-index/_search
					{
  "knn": {
    "field": "image-vector",
    "query_vector": [-5, 9, -12],
    "k": 10,
    "visit_percentage": 3
  }
}
		

1. The number of nearest neighbors to return.
2. The percentage of vectors to explore. 3 visits 3% of the vectors. Larger values improve recall and increase latency.

When a search targets both HNSW and DiskBBQ indices, set both num_candidates and visit_percentage. HNSW uses num_candidates. DiskBBQ uses visit_percentage when that value is non-zero.

How you encode vectors is the main mapping-time trade-off between memory, disk, and recall. Start from how your embeddings are produced:

Do not convert float embeddings to byte yourself to save memory. Use quantization instead.

To index float or bfloat16 vectors with a smaller memory footprint, use quantization. Elasticsearch stores a quantized representation for approximate retrieval and keeps the original vectors on disk for rescoring, reindexing, and later quantization improvements. That extra copy means disk use goes up even as RAM for search goes down.

Unless you set index_options.type, Elasticsearch quantizes float and bfloat16 fields automatically:

byte and bit fields are not quantized. They default to hnsw.

Higher compression uses less RAM and usually costs more recall:

bbq_disk is the BBQ variant that can run with less of the index in RAM. For more information on the stored Lucene files, see Vector files. For the full parameter list, see index_options.

To override the default, set index_options.type:

				PUT quantized-image-index
					{
  "mappings": {
    "properties": {
      "image-vector": {
        "type": "dense_vector",
        "element_type": "float",
        "dims": 2,
        "index_options": {
          "type": "int8_hnsw"
        }
      },
      "title": {
        "type": "text"
      }
    }
  }
}
		

1. Overrides the default quantization.

Index float vectors as usual. Elasticsearch quantizes them for approximate retrieval and keeps the original values on disk:

				POST quantized-image-index/_bulk?refresh=true
					{ "index": { "_id": "1" } }
{ "image-vector": [0.1, -2], "title": "moose family" }
{ "index": { "_id": "2" } }
{ "image-vector": [0.75, -1], "title": "alpine lake" }
{ "index": { "_id": "3" } }
{ "image-vector": [1.2, 0.1], "title": "full moon" }
		

At search time, Elasticsearch quantizes the query vector to match the index:

				POST quantized-image-index/_search
					{
  "knn": {
    "field": "image-vector",
    "query_vector": [0.1, -2],
    "k": 10,
    "num_candidates": 100
  },
  "fields": [ "title" ]
}
		

To recover recall, rescore the top candidates with the original vectors. Refer to Oversampling and rescoring for quantized vectors.

Use element_type: byte when your embeddings are already 8-bit integers. Indexed values and query_vector values must be integers in the range [-128, 127].

				PUT byte-image-index
					{
  "mappings": {
    "properties": {
      "byte-image-vector": {
        "type": "dense_vector",
        "element_type": "byte",
        "dims": 2
      },
      "title": {
        "type": "text"
      }
    }
  }
}
		

1. Required when your embeddings are already 8-bit integers.

				POST byte-image-index/_bulk?refresh=true
					{ "index": { "_id": "1" } }
{ "byte-image-vector": [5, -20], "title": "moose family" }
{ "index": { "_id": "2" } }
{ "byte-image-vector": [8, -15], "title": "alpine lake" }
{ "index": { "_id": "3" } }
{ "byte-image-vector": [11, 23], "title": "full moon" }
		

				POST byte-image-index/_search
					{
  "knn": {
    "field": "byte-image-vector",
    "query_vector": [-5, 9],
    "k": 10,
    "num_candidates": 100
  },
  "fields": [ "title" ]
}
		

1. Query values must also be integers in the range [-128, 127].

You can also pass query_vector as an encoded string.

This request is equivalent to the previous search. [-5, 9] encoded as a hex byte string is fb09:

				POST byte-image-index/_search
					{
  "knn": {
    "field": "byte-image-vector",
    "query_vector": "fb09",
    "k": 10,
    "num_candidates": 100
  },
  "fields": [ "title" ]
}
		

You can also pass a base64-encoded query vector. Base64 supports float, bfloat16, byte, and bit encodings depending on the field type.

Set element_type to bfloat16 to store each dimension as a 2-byte value instead of a 4-byte float. Use this when your indexed vectors are already at bfloat16 precision, or when you want to reduce the disk space required to store raw vector data. Elasticsearch rounds 4-byte float values to bfloat16 when indexing.

Retrieved vectors might differ slightly from the values originally indexed, because bfloat16 stores each dimension in 2 bytes.

Quantization reduces RAM for search and costs some recall. Rescoring recovers recall for the top candidates by scoring them again with the original vectors.

• Oversampling retrieves more candidates per shard than k.
• Rescoring recomputes scores for those candidates using the original (non-quantized) vectors.

Higher compression generally needs more oversampling to recover recall. There is no single oversample value that fits every model and dataset: measure recall and latency on your data.

BBQ (bbq_hnsw and bbq_disk) fields already apply a mapping-level rescore_vector.oversample of 3.0 unless you override it. int8_hnsw does not. Set rescore_vector at query time to change the factor.

Prefer the rescore_vector option for this workflow. For DiskBBQ (bbq_disk) indices, you can also let Elasticsearch pick oversampling per segment with auto-calibration.

The additional rescoring techniques cover rescoring after the merge step, or custom per-shard scoring.

Use rescore_vector to oversample and rerank quantized results. When you specify an oversample value, approximate kNN:

• Retrieves num_candidates candidates per segment. If needed, Elasticsearch raises num_candidates to k * oversample.
• Rescores the top k * oversample candidates per shard using the original vectors.
• Merges those candidates after rescoring and returns the global top k.

oversample must be 1 or greater.

Set oversample to 0 to skip oversampling and rescoring.

rescore_vector applies only to quantized dense_vector fields. Elasticsearch ignores it for non-quantized fields, because those fields already score with the original vectors.

A query-time rescore_vector overrides any mapping-level rescore_vector.oversample on the field.

This example uses rescore_vector with an oversample of 2.0:

				POST quantized-image-index/_search
					{
  "knn": {
    "field": "image-vector",
    "query_vector": [0.1, -2],
    "k": 10,
    "num_candidates": 100,
    "rescore_vector": {
      "oversample": 2.0
    }
  },
  "fields": [ "title" ]
}
		

1. The number of nearest neighbors to return (k).
2. Candidates gathered per segment for approximate retrieval. If this is lower than k * oversample, Elasticsearch raises it.
3. Rescore the top k * oversample (20) candidates per shard using the original vectors, then merge shards and return the top 10 (k).

For bbq_disk fields, set auto_calibrate: true in index_options to let Elasticsearch select quantization, oversampling, and preconditioning for each merged segment from the data in that segment.

A query-time rescore_vector.oversample still wins: it applies to every segment and overrides the calibrated values. If you do not set a query-time oversample, calibrated segments use their per-segment factors.

				PUT my-auto-calibrated-index
					{
  "mappings": {
    "properties": {
      "image-vector": {
        "type": "dense_vector",
        "index_options": {
          "type": "bbq_disk",
          "auto_calibrate": true
        }
      }
    }
  }
}
		

1. Lets Elasticsearch pick quantization, oversampling, and preconditioning per merged segment. A query-time rescore_vector.oversample overrides these calibrated values.

For how calibration runs, which segments it skips, and how it interacts with bits and precondition, refer to Auto-calibration for bbq_disk.

By default, Elasticsearch reads raw vector data into memory to perform rescoring. This can slow search if the vector data is too large to fit in off-heap memory at once. When you set on_disk_rescore: true in the field's index_options, Elasticsearch reads vector data directly from disk during rescoring.

This option applies only to newly indexed vectors. To apply it to all vectors in the index, reindex or force-merge after you change it.

For guidance on when on-disk rescoring helps, refer to Use on-disk rescoring when the vector data does not fit in RAM.

Use these approaches when you need to rescore without rescore_vector: after the merge step across shards, or with a custom score.

The examples in this section use an int4_hnsw field:

				PUT my-index
					{
  "mappings": {
    "properties": {
      "my_int4_vector": {
        "type": "dense_vector",
        "dims": 4,
        "index_options": {
          "type": "int4_hnsw"
        }
      }
    }
  }
}
		

Use this option to rescore the top results from all shards, rather than rescoring on each shard.

This example oversamples with the top-level knn search, then uses the rescore section to rerank the merged results:

				POST /my-index/_search
					{
  "size": 10,
  "knn": {
    "query_vector": [0.04283529, 0.85670587, -0.51402352, 0],
    "field": "my_int4_vector",
    "k": 20,
    "num_candidates": 50
  },
  "rescore": {
    "window_size": 20,
    "query": {
      "rescore_query": {
        "script_score": {
          "query": {
            "match_all": {}
          },
          "script": {
            "source": "(dotProduct(params.queryVector, 'my_int4_vector') + 1.0)",
            "params": {
              "queryVector": [0.04283529, 0.85670587, -0.51402352, 0]
            }
          }
        }
      },
      "query_weight": 0,
      "rescore_query_weight": 1
    }
  }
}
		

1. The number of hits to return. This request returns 10 results and gathers 20 nearest neighbors for rescoring.
2. Neighbors returned by approximate kNN, using quantized scores. Because this is the top-level knn object, Elasticsearch gathers the global top 20 from all shards before rescoring. Combined with rescore, this oversamples by 2x.
3. The number of results to rescore. To rescore all kNN results, set this to the same value as k.
4. The script that rescores with the original float vector.
5. Weight of the original query. 0 discards the quantized score.
6. Weight of the rescore query. This example uses only the rescore query score.

Use this option to rescore on each shard when you need more control than rescore_vector provides.

Combine the knn query with a script_score query. This rescores more candidates per shard, which can increase recall at the cost of compute.

				POST /my-index/_search
					{
  "size": 10,
  "query": {
    "script_score": {
      "query": {
        "knn": {
          "query_vector": [0.04283529, 0.85670587, -0.51402352, 0],
          "field": "my_int4_vector",
          "k": 20,
          "num_candidates": 50
        }
      },
      "script": {
        "source": "(dotProduct(params.queryVector, 'my_int4_vector') + 1.0)",
        "params": {
          "queryVector": [0.04283529, 0.85670587, -0.51402352, 0]
        }
      }
    }
  }
}
		

1. The number of results to return.
2. The knn query that runs the initial search. This runs per shard.
3. Neighbors returned per shard from the quantized search. script_score then rescores these hits with the original vectors. num_candidates is the per-segment candidate set and must be at least k.
4. The script that scores with the original float vector.

• Tune approximate kNN search: Production guidance for vector memory, node sizing, indexing, filesystem cache, and on-disk rescoring.
• dense_vector field type: API reference for vector field mapping, including index, similarity, index_options, and quantization parameters.
• Auto-calibration for bbq_disk: How DiskBBQ selects quantization and oversampling per merged segment.
• knn query: API reference for the knn query, including parameters, query_vector_builder options, and usage with dense_vector and semantic_text fields.
• Profile kNN search: Inspect query timing and vector operation counts to diagnose slow kNN searches.