How BBQ shrinks Jina v5 embeddings by 29x without losing recall in Elasticsearch

Try out vector search for yourself using this self-paced hands-on learning for Search AI. You can start a free cloud trial or try Elastic on your local machine now.

BBQ quantization cuts the memory footprint of Jina embeddings v5 vectors by 29x in Elasticsearch. Recall@10 holds at 0.994 against a full-precision float32 baseline. We tested this on a multilingual news corpus across five languages, using jina-embeddings-v5-text-small to build a raw float32 index and a bbq_hnsw index from the exact same vectors. Then we measured memory, disk usage and retrieval quality on both. Disk usage came out nearly identical between the two indices. In-memory footprint is the number that actually decides whether your cluster fits the corpus, and it dropped from 12.71 MB to 0.44 MB for this test set. Jina v5's quantization-aware training is why the recall held.

Prerequisites

• Elasticsearch 9.x with jina-embeddings-v5-text-small inference endpoint available.
• Python 3.10+,
• Elasticsearch API key,

What is quantization?

An embedding is a list of numbers. By default, each number is a float32, which uses 4 bytes. Quantization stores each number with fewer bits, trading precision for space.

Like a JPEG, a quantized vector is a smaller, lower-fidelity copy of the original that still gets the job done.

What is BBQ?

Better Binary Quantization (BBQ) is Elasticsearch's 1-bit quantization mode for dense vectors. Each dimension of the vector is stored as a single bit, plus a few corrective bytes per vector. Then, a rescoring step is applied at query time. This keeps the final retrieval quality close to a full precision search.

For the math behind each level, see Scalar quantization 101, Optimized Scalar Quantization, and the BBQ deep dive.

How does BBQ preserve search accuracy?

Plain 1-bit quantization leads to too high a search quality degradation on its own. BBQ maintains high retrieval quality through three mechanisms:

1. Asymmetric precision: Stored vectors use 1 bit per dimension.
2. Corrective factors: A few floats per vector record the rounding error and correct distances at scoring time.
3. Oversample and rescore: BBQ scans candidates with the bits and then reranks the top ones with higher precision. Fetching the top 10 means scanning about 30 candidates.

The result is the vectors that are roughly 32x smaller, with retrieval quality close to full precision. In the next section of the article, we’ll measure the memory savings and the recall on a real corpus.

How Jina embeddings v5 works

Jina embeddings v5 is a multilingual embedding model with quantization-aware training, which makes it a natural fit for BBQ in Elasticsearch: The 1024-dimensional vectors from jina-embeddings-v5-text-small sit above the dimensional floor where binary quantization stays accurate, and the model is trained so that 1-bit quantization loses little quality. Its main features are:

• One model for many tasks: v5 uses small Low-Rank Adaptation (LoRA) adapters on top of a single base model, one for each task: retrieval, text-matching, clustering, and classification. Elasticsearch picks the right adapter automatically at index and query time.
• Matryoshka dimensions: v5 is trained so you can truncate the vector (1024, 512 to 256) and minimize search quality reduction. This is another way to shrink vectors, independent of quantization.
• Quantization-aware training: v5 is trained to work with BBQ, so its 1-bit vectors lose little accuracy.

We use jina-embeddings-v5-text-small. This model is available through Elastic Inference Service (EIS) and outputs 1024 dimensions with 32k token context and is multilingual across 93 languages. That puts it above the 384-dimension threshold, below which Elasticsearch no longer defaults to bbq_hnsw.

Full model details are in the Jina v5 article on Search Labs.

Setting up the BBQ vs. float32 comparison

We’ll create two indices: Both share mappings, and what changes is the index_options.type parameter, which tells Elasticsearch how to store the dense vector field (as raw float32 HNSW or as 1-bit BBQ):

We then embed the corpus once with Jina v5, index those same vectors into both, and compare them on disk usage, memory footprint, and recall. You can follow along with the full supporting blog content notebook.

Connect to Elasticsearch

Create the two indices

Note: In production, you can use semantic_text to let Elasticsearch manage the mapping and inference endpoint automatically.

Point at the Jina v5 inference endpoint

We call the model jina-embeddings-v5-text-small directly (no need to create an inference endpoint) to turn text into vectors.

As result of the test, we got:

Load a multilingual news dataset

We stream real news articles from hotchpotch/multilingual_cc_news, a parquet mirror of CC-News. We take about 1,000 articles from five languages (around 3,000 docs total), plus a small held-out set of headlines to use as search queries. Using multiple languages also lets Jina v5 show its multilingual strength.

Generate the embeddings and bulk index

We embed the corpus a single time and feed those exact vectors into both indices.

We force-merge to a single segment so the storage numbers are stable and comparable.

Results: Disk versus memory

The disk usage API reports how many bytes each index spends on vectors (knn_vectors).

Result:

On disk, the two indices are about the same size. A quantized index still keeps the raw float32 vectors (needed for rescoring and requantization during merges) and adds the 1-bit vectors on top, so BBQ ends up slightly larger on disk.

The real savings is in memory. The HNSW scan only needs the 1-bit vectors in RAM, while the raw floats are read from disk to rescore the top candidates. We size that footprint using the documented kNN memory formulas: float uses num_vectors × dims × 4 and bbq uses num_vectors × (dims/8 + 14).

BBQ's extra bytes on disk should match the 1-bit payload we computed for memory. Here, that’s 13.25 - 12.80 = 0.45 MB versus the computed 0.44 MB. They line up.

Results: Recall

To check whether the quantized index returns results similar to the float baseline, we use recall:

recall@k = | BBQ top-k ∩ float32 top-k | / k, averaged over all queries.

We vary the oversampling factor (num_candidates / k) that’s the number of candidates BBQ scans with 1-bit vectors before reranking the top ones against the original floats to find the lowest value that still matches float32.

As a result, we have:

BBQ starts at 0.994 recall@10 at 1x oversampling, holds there up to 3x, and then settles at 0.989 at higher factors, meaning it returns at least 98.9% of the same top-10 documents as float32 across all oversampling values. For more on how recall varies across datasets under quantization, see Fast vs. accurate: Measuring the recall of quantized vector search.

BBQ quantization results summary

The same vectors, two storage formats, and one experiment:

• Disk: Roughly the same (12.80 MB versus 13.25 MB). BBQ keeps the raw floats around for rescoring and merging.
• Memory: 29x smaller (12.71 MB versus 0.44 MB). This is the number that decides whether your cluster fits the corpus.
• Recall@10: 0.994 at 1x oversampling. Quantization-aware training pays off.

When to enable BBQ: If your dimension count is above the 384-dim floor, if your vectors are the dominant memory cost, and if you can afford a few extra candidates to rescore. For Jina v5 specifically, the model is trained for it, so the recall hit on most corpora is small.

Further reading on BBQ and vector quantization

• Run the full notebook from this article in the supporting blog content repo.
• For the math behind BBQ, see Better Binary Quantization in Lucene and Elasticsearch.
• For more on Jina v5's architecture, see Jina embeddings v5 on Search Labs.
• For a broader walkthrough on adopting BBQ, see How to implement BBQ into your use case.
• For the original research behind BBQ, see the paper RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Search.