• Geometry-preserving Embeddings: jina-embeddings-v5-omni sits atop the foundation laid by jina-embeddings-v5-text. That original text embedding space is completely unchanged, with its geometry left wholly intact.

• Locked: Synonymous with "frozen." All of the towers in this architecture have their weights locked. 

• Aligned: Aligning the other modalities with the text model's vector space, allowing for cross-modal comparison.

• TOwers: Modality component that converts one type of raw input into vectors.

The model comes in two variants: small and nano. The former has more parameters (1.57 billion) than the latter (0.95 billion), but functionally their architectures are nearly identical. For the sake of brevity, we mostly focus on jina-embeddings-v5-omni-small in this article.

What are vectors, towers and frozen encoders?

jina-embeddings-v5-omni relies on three core machine learning (ML) concepts: vectors, towers, and frozen weights. Here's what each means; feel free to skip ahead if you're already familiar with them. 

How vectors represent data in embedding models

How does AI understand abstract concepts? Can a machine comprehend what "ice cream" is? Does it understand that "chocolate fudge" and "rocky road" have more in common with each other than "sorbet"? The answer, surprisingly, is yes. The mechanism that makes it possible is the vector.

Take a look at this diagram.

If you tried visualizing a way to organize all ice cream flavors, you may end up with something like this: a simplex (triangle) with three vertices, each representing a base flavor. Each flavor is closer to or farther from each vertex, depending on how much of the corresponding base makes up that particular flavor. Chocolate ice cream is all chocolate, so it hugs the top vertex. Vanilla has a similar affinity for the bottom-left vertex. But cookies and cream is roughly a 50/50 mix, so it's about equidistant from both. Neapolitan blends all three, so it sits at the center of the triangle.

This is functionally how vectors work. These flavors get funneled into an ML model (more specifically known as an embedding model) that will then generate coordinates for each of these flavors along this simplex. By measuring the distance between coordinates, software can parse how related or unrelated two flavors are. It's easy to see that "chocolate" and "chocolate brownie fudge" are closely related flavors because they sit close to one another, but "strawberry" and "cookies and cream" are far away from each other, so we can infer that they aren’t similar.

It won't be quite this simple though. Rather than three labeled points, real embeddings run along hundreds or even thousands of dimensions. Nor will they have nice, human-readable labels; the vertex markers are something that only the model understands. The benefit, though, is that we can graph basically anything like this.

To clarify some jargon: These coordinates = vectors = embeddings. For the rest of this article, we use these terms interchangeably.

Much like how embeddings go by many different names, so too do the models that create them.

What is a tower in multimodal embedding models?

The term tower comes from Contrastive Language-Image Pre-training (CLIP), a model released by OpenAI in 2021 that was one of the first to learn a shared embedding space across text and images. In CLIP, each modality is handled by a completely separate model. On an architecture schematic, these models look like towers standing side by side, each taking one type of input and producing vectors in a shared space. The name stuck, and you'll see it used broadly across multimodal ML. 

In the context of jina-embeddings-v5-omni, the word is used a bit more loosely. Its architecture doesn't have true parallel towers in the CLIP sense. All modalities ultimately funnel into a single central text model, rather than sitting as equals beside it. 

With that caveat in place: A tower (or modality component) is a pipeline that converts one type of raw input into vectors. A text tower vectorizes strings, and a vision tower generates image embeddings. An audio tower does the same for sound.

Why freeze a tower instead of training it?

If we want multimodal capabilities, could we Frankenstein multiple towers that handle each of those inputs together into one model? The issue with this approach is that vectors from different models aren’t intelligible to each other. Images will exist in one vector space and audio in another, for example. This means that we have no ability to compare across different modalities. Think of the vectors outputted from one model as existing in their own language. Let's say our audio tower outputs Spanish and our image tower outputs English. Conceptually, the vectors can be describing the same things, but downstream tools that try to make use of these embeddings are "monolingual," so we're out of luck.

The CLIP-style approach is to take several towers and train them together, letting them all reshape each other until their outputs agree. It's sort of like having Spanish and English speakers try to communicate for long enough that they eventually all start speaking Spanglish.

This approach works, but it has a side effect: Towers that you already had working get remodeled in the process. Any embeddings that they produced before are now incompatible with any that are produced by the older version of the tower. If you had a text tower and generated 100 million embeddings with it, you would now need to re-embed all of those strings.

To combat this issue, you can freeze certain towers. This locks their weights, which are the knobs and dials that influence how they behave. This way, training never changes them. What you train instead is a small projector that translates one tower's output into another tower's language. Many different models train some portion of projectors and towers while leaving some others frozen. What makes jina-embeddings-v5-omniunique is that we froze every encoder and trained only the projectors and delimiter tokens. By the end of this article, you’ll understand exactly how that works.

You can think of these frozen and trainable components as clusters of neurons or isolated regions of the mind, and the whole embedding model (jina-embeddings-v5-omni) as the entire brain. Between these learning-capable components sit fixed math operations, such as merging, squashing, selecting, and rescaling numbers on their way from one tower to the next. They have no weights, so there’s nothing in them to freeze or train. 

With that groundwork laid, here’s the full architecture.

How the jina embeddings multimodal architecture works

The architecture routes all modalities through frozen encoders and small trainable projectors into a single shared text embedding space.

Why the text tower is the backbone

The first thing to notice here is the general flow of data in this diagram. Image, video, and audio inputs all (eventually) end up in the same space as text inputs. Why is the architecture set up like this?

jina-embeddings-v5-omni builds on top of jina-embeddings-v5-text: It retains its text-processing backbone and extends it with pretrained vision and audio components.

This has two main advantages:

1. The text model is already state of the art for its size and what it does, and it stays completely untouched. There’s no need to fix what isn't broken. We also don't need to re-embed anything we already embedded with jina-embeddings-v5-text.

2. A single shared vector space is what makes cross-modal search work at all. Because image, audio, and text all resolve to vectors in the same geometry, you can query an image with text, or audio with text, and compare them directly with cosine similarity (a similarity measure based on the angle between two vectors). If each modality lived in its own separate space, those comparisons would be meaningless.

How vision and audio encoders feed into the text model

The first step in building upon the foundation set by jina-embeddings-v5-text is integrating vision and audio encoders into this architecture. In this case, we use the existing Qwen3.5 vision encoders and the Qwen2.5-Omni audio encoder, which themselves have been adapted from SigLIP2 and Whisper-large-v3, respectively. They’re ultimately what’s responsible for generating raw vectors for all vision- and audio-based data. The emphasis is on raw here, since much transformation still needs to be done afterward.

How the vision encoder processes images

In the case of images, we’re borrowing more from Qwen than just the encoder. Additional plumbing inherited from Qwen is attached to the output of the encoder. Let's walk through what comes out of the encoder and how the inherited downstream components transform that output.

Vision encoder (frozen)

Let's use an image as our primary example, since the visual component of video is basically identical. Take this image of a banana split.

Rather than generating one single, clean vector embedding for this image, the vision encoder breaks the image up into 14-pixel by 14-pixel sections and generates a tiny patch token (basically a mini-vector) for each of these sections.

While still inside of the vision encoder, each token looks at every other token that makes up the image and pulls in information from the ones relevant to it, updating its own vector based on that context. This process allows us to preserve fine-grained details. Now, instead of one single vector for the whole image, we have multiple, smaller vectors that represent particulars of the whole dish.

LayerNorm (frozen)

LayerNorm rescales each patch's numbers so they sit in a consistent range before anything else touches them. It stops some patches from being wildly larger than others and drowning out the rest.

2x2 merge 

We saw in the vision encoder section that we split up the image into small, 14-pixel by 14-pixel squares. However, processing this many patch tokens will become expensive downstream. For this reason, the 2x2 merge operation consolidates four patch tokens into one. The squares are now 28 pixels by 28 pixels. The corresponding patch tokens are similarly consolidated.

fc_vision_1 (frozen) 

fc_vision_1 is the first of two matrix multiplies; it mixes the merged patch numbers into a new set. This layer is inherited from Qwen and left as is.

GELU 

Gaussian Error Linear Unit (GELU) is a gate applied to each number. It lets useful signals through and squashes the rest toward zero. It’s the one nonlinear step and is ultimately what lets the two fc_vision layers together learn shapes that a single flat multiply couldn't.

fc_vision_2 (trainable) 

As mentioned earlier, trying to put Qwen image embeddings directly into the same vector space as Jina text embeddings would be like trying to include a Spanish sentence in an English novel. Outside of its native context, its meaning is totally lost.

That is what the trainable projector fc_vision_2 is here to fix. It learns to translate the image embedding into something that jina-embeddings-v5-text can understand. For this reason, you can think of a trainable projector as a translator.

The emphasis belongs on trainable: This is the first component we’ve encountered in our walkthrough so far that isn’t frozen. Many components of this architecture are frozen, meaning that their weights are locked and never change during training, but fc_vision_2 is one of the few parts that actually gets updated, because it has to learn how to translate Qwen's image dimensions into a form that lands meaningfully inside Jina's text space.

× 4

By now, you may have noticed that fc_vision_2 has a "× 4" marked on the bottom, along with fc_audio, both encoders, and the ″Embedding text″ section within jina-embeddings-v5-text. In this case, it represents four different instances of fc_vision_2, each optimized and tuned to one of four slightly different tasks, outlined below.

Task

What it facilitates

Example user input

Example end result

Note

Retrieval

Finds a similar match for the input (that is, standard Google search)

"melting ice cream" as a string/text

Picture of a fallen ice cream cone on asphalt

Text-matching

Judges how similar inputs are

A text string "melting ice cream" and an image of a fallen ice cream cone on asphalt

Score judging how similar the two inputs are

The name of this task is a bit of a misnomer. It's called text-matching, but it works for any modality, not only text.

Clustering

Groups data into clusters

A large array of ice cream images

Lets the user discover natural groups, like "sundaes" and "popsicles"

Classification

Places data into predefined buckets

Two string labels: "melting" and "intact", along with a large array of ice cream images

Sorts the array of ice cream images into the two provided categories, based on their proximity in vector space to the label embeddings

To clarify, the "Example end result" column is the takeaway after some additional math and processing happens once the output vector is generated. The point is that  jina-embeddings-v5-omni only generates vectors. Those vectors take on a mildly different form to optimize for the selected task type.

These task types are also explicitly outlined in the Low-Rank Adaptation (LoRA) component of the architecture diagram, which we'll cover in a moment.

How the audio encoder processes sound

Before we go any deeper into the trenches of our model architecture, let's back up and see how the audio-oriented path differs and how it stays the same. If you were able to follow along during the vision section, the audio portion will be a breeze. We have no extra inherited plumbing from Qwen this time, only the audio encoder and one trainable projector.

Audio encoder (frozen)

Before audio can enter the encoder, it needs to be converted into a form that the encoder can work with. Raw audio is a one-dimensional wave, which isn't particularly useful to a neural network on its own. Instead, the audio is first transformed into a mel spectrogram: a 2D representation that maps frequency against time, weighted to emphasize the frequency ranges that the human ear is most sensitive to. Think of it as a visual fingerprint of the sound. Below is a mel spectrogram of a person saying, "I scream, you scream, we all scream for ice cream."

That spectrogram is then sliced into fixed-length 40ms chunks, analogous to how the vision encoder breaks an image into 14×14 pixel-tiles. The encoder then produces one token per chunk.

This matters for the same reason it does in the vision pipeline: Fine-grained detail would otherwise be lost. A single embedding for the full phrase "I scream, you scream, we all scream for ice cream" would smear everything into one blurry vector. Slicing it into short chunks keeps each fragment of sound intact, so the model can later distinguish "scream" from "cream" rather than collapsing them into an average. The same applies to music, animal noises, environmental sounds, and more.

fc_audio (trainable)

Once the encoder has produced its tokens, fc_audio performs the same translation role that fc_vision_2 does for images: It projects each audio token from the encoder's 1280-dimensional output space into jina-embeddings-v5-text’s hidden dimension (1024 for small, 768 for nano). Like fc_vision_2, it carries a "× 4" in the architecture diagram, meaning that there are four instances, each optimized for a specific task type (retrieval, text-matching, clustering, and classification).

Delimiters (trainable)

We need to convey to jina-embeddings-v5-text that this embedding represents an unexpected modality. We can do this pretty easily with delimiters. In the world of HTML, this looks like:

The first and last <p> tag conveys that everything in between them is a paragraph element.

In our case, things are a bit more complicated. For audio, the architecture diagram shows the delimiters as <aud_start> and <aud_end>, but that isn't quite accurate. The delimiters themselves are actually vectors rather than hard-coded strings.

Each task type has its own pair of delimiter vectors (hence the "4 × special tokens"). These vectors are identical every time rather than being different for each audio embedding. So a task type of retrieval with an audio input type always gets start delimiter vector X and end delimiter vector Y; text-match with audio input always gets a start delimiter vector A and an end delimiter vector B, and so on.

This is necessary because the Transformer layers component only understands vectors. So, by the time the audio embedding makes its way there, it looks like:

The same pattern applies to images:

Video is where this gets interesting. Up to 32 evenly spaced-out frames are pulled from the video and fed into the vision encoder. 

Rather than producing one delimiter-wrapped segment, each sampled frame gets its own <vis_start> / <vis_end> wrapper, and these per-frame segments are concatenated into one long sequence:

This is what makes multi-frame video work: Rather than averaging frames or treating them separately, the transformer receives the whole video as one token stream, allowing its attention to relate tokens across frames. This means that earlier frames can inform how later ones are interpreted. If the video does have an audio component, it's pulled out and fed into the audio encoder and ultimately prepended to the frame sequence.

The transformer layers component then processes this entire concatenated sequence as a single input.

Jina text transformer layers (frozen) 

We’ve generated patch tokens for our images, videos, and audio files. We’ve also wrapped them inside of vector delimiters, all for the sake of having them understood by these layers. They’ll allow each patch token to examine the other patch tokens and determine whether they need to update themselves based on the surrounding context. I know what you're thinking:

Didn't we already do this inside of the encoder? We split up the image into 14-pixel by 14-pixel sections and generated patch tokens for each section, and then the encoder updated each patch token based on surrounding context within the same image.

And you're right! We did. But there's a key difference now.

Originally, that recontextualization ran on the attention of Qwen's vision encoder. The operation within the Transformer layers runs the frozen Jina text transformer's attention. It’s the same operation with different learned parameters, so it transforms the tokens differently.

Think of it like a move: The projector is the flight and the moving trucks. It physically relocates you from vision land to text land, landing you in the right city and even the right neighborhood. The transformer's attention is what happens after you've unpacked. You're already home; you spend the next few weeks figuring out exactly where you fit, meeting the neighbors, finding your bearings, and adjusting your exact spot based on who's actually around you. You did the macro move already. This is the micro fine-tuning.

Lastly, the <vis_end_vector_delimiter> will absorb all the information from the patch tokens it wraps.

LoRA (frozen) 

LoRA is a way of fine-tuning an existing model without completely retraining it. It’s a small set of extra adjustment knobs bolted onto the transformer that nudges its behavior to optimize for one of the specific tasks (such as retrieval or classification).

Last-token pooling 

Since <vis_end_vector_delimiter> absorbed all the other patch tokens into itself, we don't need to consider anything except it, so we throw the rest away. It acts as a stand-alone embedding that represents a summary of the whole.

L2 normalization 

<vis_end_vector_delimiter> could be any length now, which is no good. This step shrinks or stretches it so its length is exactly 1, without changing the direction it points. This is tidying so that comparing it to other vectors later is a fair, clean angle comparison. It changes only the vector’s scale, not what it means.

Enough about ice cream

As we've journeyed our way through jina-embeddings-v5-omni, we’ve been careful to outline which components are frozen and which ones are trainable. By now, you may have noticed that every single tower has been frozen. In fact, only the small projectors (translators) and delimiter tokens have been trainable. We dubbed this architecture pattern GELATO. This makes the entire training process significantly cheaper.

To be clear, we didn't invent the concept of frozen towers. Prior work on Locked-image Tuning (LiT), VISTA, and Multi-modAl Retrieval model via Visual modulE pLugin (MARVEL) froze one side or the other. What no one had done before GELATO was push the idea to its limit: text, image, video, and audio all in one model with every encoder frozen. The only trained pieces are a single projector layer per modality and a handful of delimiter tokens.

But is it any good?

Benchmark results: jina embeddings vs. other multimodal embedding models

There's no point in building out a model and releasing it if you don't even know if it's any good, especially compared to the competition. That's why we have benchmarks and evaluation frameworks. The benchmarks that jina-embeddings-v5-omni was run against are Massive Image Embedding Benchmark (MIEB), Massive Audio Embedding Benchmark (MAEB), Massive Multimodal Embedding Benchmark–Video (MMEB-Video), and Massive Multilingual Text Embedding Benchmark (MMTEB).

As for the models we compare against, it's important not to make apples and oranges comparisons. For that reason, we’re specifically using open-weight omni-style models with support for the same media types:

• LanguageBind

• Omni-Embed-Nemotron-3B

• LCO-Embedding-Omni-3B

• LCO-Embedding-Omni-7B

Evaluation

The table above is sorted by parameter count. Models with the fewest parameters are clustered at the top, and models with the most parameters hug the bottom. This context is important because performance alone isn't the final variable. If a $100 ice cream with gold flakes and the finest dairy milk tastes the same as (or worse than) the average gallon of ice cream that you can buy from your local grocery store, it would be silly to buy it, because you're paying a massive premium for nothing.

A similar situation is unfolding here. jina-embeddings-v5-omni-small and nano outperform every other model on text, despite ranking low to middle in terms of parameter count.

Audio performance is strong, as well. jina-embeddings-v5-omni-small and nano beat out all other models except those from LCO, which they both trail by about 2 to 3 points.

The gap shrinks when considering image performance, particularly with jina-embeddings-v5-omni-small. It beats both LanguageBind and Omni-Embed-Nemotron-3B. It lags less than a point behind both LCO models, despite the fact that they have 4.70 billion and 8.93 billion, respectively, compared to Jina’s 1.57 billion.

Video is the weakest performer for our models, though even in that case jina-embeddings-v5-omni-small still beats Omni-Embed-Nemotron-3B, which has three times as many parameters. Ultimately, when these scores are averaged out, you get the following rankings:

Model

Number of parameters (B)

Average score

LCO-Embedding-Omni-7B

8.93

54.43

jina-embeddings-v5-omni-small

1.57

54.04

LCO-Embedding-Omni-3B

4.70

53.83

jina-embeddings-v5-omni-nano

0.95

47.49

Omni-Embed-Nemotron-3B

4.70

41.21

LanguageBind

1.14

35.82

LCO-Embedding-Omni-7B has nearly six times the number of parameters as jina-embeddings-v5-omni-small but barely squeaks past it in average performance.

One benchmark deserves a special callout for anyone building search or retrieval augmented generation (RAG): visual document retrieval, measured on the ViDoRe benchmark. 

Here, jina-embeddings-v5-omni-small scores 79.25 using only 0.92 billion active text-and-image parameters, ahead of LCO-Embedding-Omni-3B (78.24) and within striking distance of LCO-Embedding-Omni-7B (80.32), a model nearly 10 times its size on that path. nano matches that exact same 79.25 score with just 0.31 billion active parameters. The larger Omni-Embed-Nemotron-3B does take the top spot at 85.64, but it carries roughly five times the active parameters of jina-embeddings-v5-omni-small, so our models remain the most parameter-efficient of the group. If your workload is retrieving pages of documents by their layout and text, this is the number to weigh.

Limitations of frozen-tower multimodal embeddings

GELATO's frozen-tower design delivers strong results at low training cost, but it comes with trade-offs worth naming plainly. As already mentioned, the most consistent weak spot is video. jina-embeddings-v5-omni-small trails the LCO models on video, and moment retrieval (locating a specific event within a clip) is the weakest subtask of all. This is partly structural, since each frame produces its own token set before everything is concatenated and pooled into a single final embedding. Packing that much information into one embedding means that the early dimensions carry a heavier load, so video embeddings degrade faster than image embeddings when truncated to smaller sizes.

Audio has its own gap. While retrieval and classification scores are competitive, audio clustering is the weakest audio subtask (6.13 for jina-embeddings-v5-omni-small).

Cross-modal audio–text retrieval trails LCO-Omni-7B by 11–15 percentage points, a larger gap than the 6–7 points seen on the image–text (I-T) pair. The fc_audio projector is the natural next target for additional trainable parameters, suggesting the audio–text (A-T) alignment path has more room to grow than the multilayer vision pipeline. 

How multimodal embeddings distribute in vector space

We've already discussed performance via benchmarks, but what about how the actual embeddings are distributed in vector space? How does that tangibly differ from model to model?

In the above illustration, data from video clips is funneled into each model and graphed in vector space. It’s then compressed down to two dimensions via the Uniform Manifold Approximation and Projection (UMAP) method for easy visualization. Each modality corresponds to a different component of the video:

Modality

Component of the video

Image

Frame from the middle of the video

Video

The full video

Audio

Audio track from the video

Text

Description of the video

Immediately, some interesting patterns stand out.

Our models and the LCO models seem to have different modalities all mixed together, while LanguageBind and Omni-Embed-Nemotron-3B seem to lean more toward having their embeddings separated by modality.

Our models and the LCO models exhibit interleaved geometry for these vectors. This means that different modalities aren't clearly separated in vector space, but instead intermingle in similar areas. This is less pronounced with Omni-Embed-Nemotron-3B, since only image and video seem to occupy a similar space.

LanguageBind is fully separated, with different modalities occupying entirely different spaces. This is known as the modality-gap pattern.

So which one is better? In practice, interleaved geometry tends to be the more useful of the two, and it’s worth noticing that the strongest models in our benchmarks (ours and LCO's) all exhibit it. However, there are trade-offs.

Interleaved geometry excels at cross-modal retrieval, since everything is jumbled up together in the vector space and, therefore, much closer. It's easier to find a matching picture for the text "strawberry ice cream" when the text and image embeddings sit so close together in vector space.

When you're trying to do a same-modality task though, the image that was so conveniently within reach is now in the way. However, in practice, this is easily mitigated by metadata filtering on something like a “modality” field. 

No such workaround exists for the issues inherent to models that exhibit the modality-gap pattern. It’s easy to find another video of syrup poured on ice cream, since all the videos are sitting together in isolation. But having modalities confined into clusters like that makes finding an accompanying image much harder.

Why this architecture?

GELATO gives a lot of performance for very little training. You keep all your towers as they are and train only small projectors and delimiter tokens, which is significantly cheaper than the alternatives. To put concrete numbers on "cheaper": for jina-embeddings-v5-omni-small, training just the vision projector updates 4.20 million parameters instead of the 920.6 million a full fine-tune would touch. At the same 15,000-step budget, that projector-only run finishes about 1.8 times faster and peaks at 7.52 GiB of GPU memory instead of 12.96 GiB. The audio path shows an even wider gap, with projector-only training running 3.2 to 3.9 times faster than full training. But how did we conclude this was the way to go? We used a process known as ablation.

Ablation is when you remove or change one piece of a system to see how much it actually mattered. Imagine you've been working on an ice cream recipe. Every time you make a tweak, like doubling the milk, swapping brown sugar for white, using vanilla beans instead of extract, or taking out the chocolate chunks, that's ablation.

Ablation in ML functions much in the same way. It asks whether removing, rearranging, freezing, or unfreezing certain components makes the whole system more, less, or equally as performant. In this case, we’re particularly interested in whether unfreezing certain components, and in what order, may affect performance. We conducted five ablation studies on the Qwen3.5 vision stack. The results are measured in mean nDCG@10 (normalized Discounted Cumulative Gain), a standard score for ranking quality where higher is better.

Overall, nearly every ablation study yielded basically identical results, except for case #3, which performed terribly. Before we explain why, let's look at the equivalent ablation diagram for audio.

In this instance, ablation case #2 performs the worst. Do you see the commonality between the worst performer here and the worst performer among the vision ablations? Across both modalities, the same rule holds: If you unfreeze the encoder before the projector has been trained, you’ll see worse performance.

For both modalities, we ultimately chose ablation case #1 for the final architecture. Both had relatively high scores. In vision's case, the configurations that edged out case #1 did so by margins too small to justify their added training stages and extra per-task artifacts. A similar story unfolds for audio, with case #3 beating out case #1 by a small margin but requiring more per-task artifacts.

Ablation validates the GELATO approach: It's cheaper and nearly identical in quality to train a dedicated translator (rather than the speaker).

Summary: why frozen encoders make multimodal embeddings cheaper

Rather than expensively retraining multiple towers to achieve multimodal capabilities, GELATO allows us to minimize cost by freezing our already functioning towers and training only small projectors to translate embeddings. These embeddings get funneled into jina-embeddings-v5-text, ultimately allowing all the output vectors to exist in the same, interleaved geometry. We can now compare text, audio, images, and video at a fraction of the cost of the competition. 

Both jina-embeddings-v5-omni-small and jina-embeddings-v5-omni-nano are open-weight for personal use and available now. You can download them from the Jina AI collection on Hugging Face and start generating multimodal embeddings today, or read the full technical report for the complete set of benchmarks and ablations.

In our previous blog, we laid out the statistical model behind that calibration: a manifold model for how nearest-neighbor distances scale with index size, a Gaussian error model for quantization noise, and a closed-form way to combine the two into an expected recall@k for a given rerank depth. If you haven't read it, the one thing you need going in is this: given a candidate quantization encoding and a rerank depth, we can predict recall@k without building an index and benchmarking it, by fitting two small models to a sample of the corpus.

In this post we’ll go through how to score candidate configurations cheaply and how to leverage that to make merge-time decisions that are themselves cheap, correct, and consistent across real, constantly-merging indexes. This led to some pretty impressive improvements: we see almost 17% average improvement in QPS across a broad range of datasets all while increasing recall (in one case by a factor of 3). What’s more you get this immediately by adding one line to your index options, "auto_calibrate": true, and our plan is to make this our default once it has had the chance to bake a bit.

Why manual vector quantization tuning is unreliable

bbq_disk exposes several knobs: quantization bits for documents (1, 2, 4 or 7), a separate bit width for queries, an oversampling factor for reranking, and whether to precondition vectors before quantizing. None of these act independently, and their effect on recall depends on the data: a 4-bit/1-bit encoding might be plenty for one embedding model and clearly insufficient for another. A single index is also built out of many segments, merged over time, each with an eventually different vector distribution. Hand-tuning one configuration for an entire index is, at best, a compromise, which is the motivation for automatic calibration: let each segment have its own configuration, re-evaluated every time it is involved in a merge operation.

How Elasticsearch runs auto calibration at merge time

When a number of segments are merged and automatic calibration is enabled, Elasticsearch samples documents and queries from the vectors being merged and:

1. fits the manifold model over a sequence of nested samples of the merged corpus;

2. fits the error model, predicting the quantization error's standard deviation for each candidate (query bits, document bits, precondition) combination;

3. sweeps candidate configurations in ascending cost order; the candidate encodings are (1,1), (4,1), (4,2), (4,4) and (7,7) (query bits, document bits), each tried across oversampling factors of 1.25, 1.5, 1.75, 2.0, 2.5 and 3.0;

4. estimates recall@10 for each candidate using the model described in our first post, and stops at the first (cheapest) configuration predicted to hit the target of 90% recall@10.

The winning configuration (encoding, oversample factor, precondition flag) is stored directly in the segment's metadata, so it travels with the segment and is picked up automatically at query time unless a request explicitly overrides it.

Small segments skip this altogether: below 10,000 merged vectors, there isn't enough data to fit a reliable model, so Elasticsearch just uses the current DiskBBQ defaults (4-bit query / 1-bit document encoding, no preconditioning, 3x oversampling).

How the vector quantization cost model works

Following the principles described in our first post, we started by picking candidates with three nested loops that are essentially how you might imagine hand jamming a lookup table. Start with the quantization scheme as the outer loop, ordered cheapest to most expensive by document bits ((1,1) → (4,1) → (4,2) → (4,4) → (7,7)). Then we set the rerank depth within the middle loop, ordered shallow to deep (1.25× → 3.0×). Finally we set preconditioning within the inner loop (off → on).

That ordering has a cost model baked into it, it's just implicit rather than written down: exhaust every rerank depth at the current bit tier before ever trying more bits. Document bits were effectively the only resource priced as expensive; oversampling was treated as nearly free by comparison, since the sweep would always max out rerank depth on a cheap encoding before considering a pricier one.

The current implementation replaces that with an explicit, continuous cost function:

Query bits still don't factor into cost at all, only document bits (which drive index size) and rerank depth (which drives how many candidates get rescored per query). Preconditioning also stays outside the formula: Elasticsearch runs the whole cost-ordered sweep once with preconditioning off, and only if nothing meets the recall target does it re-run the sweep with preconditioning on, treating it as a fallback lever rather than something priced bit-for-bit against the other two.

With this cost model, rerank depth costs noticeably more per unit than a document bit, so the sweep will often prefer stepping up a bit tier over pushing oversampling deeper.

The main reason for this is that once you're running in a serverless deployment, compute and storage are billed and scaled independently, on very different clocks. An extra document bit is mostly a one-time, indexing-time cost; it makes the segment marginally bigger on object storage, which is cheap and doesn't need to be pre-provisioned against a spike in query traffic. It does carry a smaller recurring cost too, since quantized vectors sitting in page cache or loaded for scoring take proportionally more RAM per document as bit width grows, but that scales linearly and predictably with corpus size, and doesn't spike with query load. Rerank depth is the opposite: it's a recurring, per-query cost.

Every extra unit of oversample factor means fetching and rescoring that many more full-precision candidate vectors from disk, on every search request, for as long as the index is queried. That's compute and DRAM pressure on the search-serving tier, which has to autoscale in close to real time to match query concurrency. It sits on the hot path of the latency-and-cost budget in a way storage capacity, and the RAM footprint of the bits themselves, does not. Weighting rerank depth higher than document bits in the cost formula is what makes the sweep reflect that asymmetry.

Efficiently estimating vector quantization error

The cost model above works with the premise that the recall estimate behind it is trustworthy. The manifold and error models need to be accurate for the recall assessment to be trustworthy. While the manifold model of the k-th to N-th nearest neighbors distance is cheap to compute, the standard deviation of the quantization noise for a given candidate encoding is a bit more expensive in principle.

DiskBBQ uses fixed count clusters to accelerate nearest neighbor queries. Our quantization procedure takes advantage of this by only quantizing the vector residuals from the cluster centroids. This means as the data scales, the magnitude of vectors we quantize relative to the various components of the similarity calculation shrinks. As such, quantization accuracy increases. We need to account for this when converting our sample estimates to the segment as a whole.

Clustering the corpus at several sample sizes and fitting how the error scales with cluster size requires re-clustering a real sample of the corpus at several different sizes and fitting a regression model to see how the error shrinks as the effective cluster size grows. We also add a conservative +3σ margin on top of the fitted estimate to guard against noise in the fit itself. This is accurate and appropriately cautious; however, while benchmarking on common dense retrieval datasets, we found that performing several hierarchical k-means passes per candidate was expensive.

To speed things up, we tried approximating residuals with a synthetic isotropic-Gaussian formula. Instead of clustering increasing-size samples, this approach generated synthetic residuals from the manifold model's local density estimate. It was fast and fit for background merges, with the full repeated clustering approach reserved for force-merges only. However, it turned out to inflate error when embeddings (residuals) are anisotropic (some directions carry a lot more variance than others). As a result, the estimated error could grow significantly on strongly anisotropic data (e.g., Fashion-MNIST-style image embeddings).

So instead we looked for a still fast but more accurate way of calculating residuals. We opted for using a single clustering pass over a smaller sample (2,048 vectors). The clustering runs once per merge and is then warm-started for every candidate encoding evaluated afterward, instead of re-clustering from scratch each time. To get the error's dependence on corpus size, which the baseline learns by re-clustering at multiple sizes, this approach instead reuses the manifold model's invDim as a plug-in for that dependence, extrapolating from the single real measurement rather than fitting the size relationship separately.

We also trimmed the query sample used during calibration from 1,024 to 256 vectors, on the reasoning that a smaller sample is enough once the error is being measured from real data rather than synthesized (and validated by benchmarks). The net effect was comparable wall-clock cost to the synthetic residual formula it replaced, but grounded in real per-cluster residuals, accurate enough that force-merge and background merge could be unified onto one path.

As an example, we take five different benchmark datasets and calculate the quantization error standard deviation (SD) by directly measuring the gap between exact and quantized dot products on a sample of real (or, for the synthetic residual formula, fabricated) residuals, then extrapolating that measurement to the full corpus size. They differ only in how much sampling and regression goes into that extrapolation: the multi-sample scaling fit sweeps fifteen sample sizes and fits how error scales with cluster size, the single-pass real residual measurement takes one larger real residual sample and reuses the manifold's intrinsic dimension to estimate the size dependency, and the synthetic residual formula skips real residuals altogether and samples from a synthetic Gaussian from the manifold's expected rank distance. We treat the multi-sample scaling fit as ground truth in this comparison because it's the most sample rich of the three, not because it's a zero variance measurement of the "true" corpus-wide error (it has its own sampling noise too). The table below summarises the methods and findings.

Method

How it works

Speed

Accuracy

When used

Multi-sample scaling fit

Clusters at 15 sample sizes, fits regression

Slow

Gold standard

Ground truth baseline

Single-pass real residual

One clustering pass + manifold invDim plugin

Fast

Near gold standard

Background + force merge

Synthetic residual formula

Gaussian from manifold density estimate

Fast

Inflated on anisotropic data

Deprecated

In order to exchange methods, we only need to be confident that they agree. This question can be answered independently of the correctness of the actual estimates, which we verified in our previous post for the multi-sample scaling fit. The figures below report the predicted quantization SD and the predicted recall@10, which is influenced by how we estimate the error. We report the analytical recall the manifold model predicts as a function of the quantization parameters, given the estimated error distribution perturbing the true distance ordering. This way, we isolate the quantization error's effect on ranking from any separate recall loss the IVF index itself might introduce, which is a distinct error.

The single-pass real residual measurement's calculated error SD is closer to the multi-sample scaling fit (our gold standard), with respect to the synthetic Gaussian residuals. Consequently, the predicted recall is closer when using the single-pass + manifold plugin method. Indeed, we found the models to be essentially interchangeable regarding the indexing decisions they lead to. Critically, we lower the calibration overhead by an order of magnitude.

Auto calibration overhead on indexing performance

We compared the cost of auto calibration on indexing, when compared with ES defaults, over 18 public benchmarks. We noticed that more than 50% of the datasets report an auto calibration overhead below 2%. Three datasets report 16-27% overhead, while two datasets sit in the 31-35% overhead.

The merge overhead is larger for smaller datasets (Fashion-MNIST, FiQA) that get indexed in a few seconds; that is expected as the size of the vector samples being used for calibration is fixed and therefore more noticeable with tiny datasets. In fact, for larger datasets like DBPedia-Entity and HotpotQA (5M doc vectors) the overhead is sometimes not noticeable and within 11% in the worst case.

What quantization parameters does auto calibration choose?

Looking at the encoding auto-calibration landed on for each of the real datasets:

Query bits were 4 in every dataset. While query bits aren't priced into the cost formula, we still iterate through lower query bits first (e.g., at 1 bit doc vectors, we first evaluate recall for 1 bit query vectors, then for 4 bit query vectors); so it’s possible for some datasets to even choose symmetric 1-bit quantization. The center of mass is a 2-bit document encoding with somewhere between 1.5x and 1.75x oversampling; 4-bit only shows up for two genuinely harder datasets (Fashion-MNIST's image embeddings, GIST-1M), and 1-bit only for a handful of the text-embedding models that are most robust to quantization. In fact, our own models are among those that quantize best: we selected 1 bit documents for all three corpuses we tested with Jina v3.

Recall and QPS improvements from automatic calibration

Auto-calibration is a broad win across the eighteen datasets: QPS improves in 15 of 18 cases (often substantially, double digits on about ten, and over +50% on FiQA GTE, Fashion MNIST, and Glove-200), and recall improves in 15 of 18 cases too, including a dramatic +295.7% rescue on Fashion MNIST. Most datasets see gains on both metrics simultaneously, and even the more modest cases still land solidly positive, recall improvements are commonly in the high single digits to double digits, QPS gains follow a similar pattern. Where either metric does dip, the drops are small and contained: the three QPS regressions all stay under 1.5%, and the three recall regressions all stay under 2%.

How to enable auto-calibrated vector quantization in Elasticsearch

The feature is not enabled by default for now, and opt-in via auto_calibrate on bbq_disk index options:

With this set, you no longer need to guess at bits, oversampling, or preconditioning: each segment picks the cheapest configuration that's predicted to hit 90% recall@10 for its own vector distribution, and re-evaluates that choice every time it's merged.

What's next for automatic vector quantization in Elasticsearch

Our first post showed that recall could be predicted in closed form from a small sample. Turning that into something running inside a real merge path meant a second round of engineering decisions that the model itself doesn't answer: how to order a sweep over candidates so it's cheap in the common case, how to price oversampling against document bits given how each is actually paid for at query time, and how to estimate the error term itself cheaply without quietly wrecking its accuracy.

In the end, we have a feature that allows us to tailor indexing choices to the data characteristics, with less than 11% overhead to index time for large indices. This gives us the ability to accurately control recall while optimizing quantization and oversampling choices for query performance. We got an average increase of 16.7% in QPS when we enabled this feature compared to our previous default settings for DiskBBQ. All while reliably achieving our target recall. Taking away the configuration burden from the user actually allows us to make better choices; it is a win-win.

This is the beginning of a longer journey that we’re working on to bring automatic configuration based on a combination of better understanding of the operating environment and better understanding of the data characteristics. We look forward to sharing more of this work with you in the near future.

What’s a reranker and how does it work?

A reranker is an AI model used near the end of an information retrieval pipeline, after other modules have assembled a short list of candidate matches to a query. It’s trained to order the candidate list from best matching to least. Using a specialized model focused purely on ranking candidate matches can improve result quality dramatically.

Jina AI’s latest rerankers use a technique called late interaction, where queries and documents are encoded separately into lists of token embeddings that reflect each token’s semantics in context and then compared to each other.

This is an AI analog of lexical and grammatical disambiguation.

For example, consider the meaning of the word match in these two sentences:

• She looked for a match to light the candle.

• She looked for a match on Tinder.

The first sentence might be a match for queries about matchboxes; the second for queries about romance.

Transformer-based models do this kind of in-context disambiguation but bring much richer information into the token embeddings they produce. The word match might have a semantic embedding near to words like fire or illumination in the first sentence, while in the second, it might be closer to smartphone or swipe.

Late interaction rerankers generate these context-enriched token embeddings for both the query and the candidate documents and then compare them to produce sortable scores. They’re completely agnostic about how candidate match lists are created. The reranker works exactly the same when combined with lexical search schemes, like BM25, AI-driven semantic embeddings-based retrieval, or hybrid and federated search systems that may retrieve multiple candidate lists from different sources or using different algorithms. Of course, the results always depend on the quality of the candidates, so a reranker can’t fix bad first-stage retrieval, but it almost always improves whatever you’ve got.

jina-reranker-v3.5 is a listwise reranker using the last-but-not-late technique developed for jina-reranker-v3. The query and a list of candidate matches are passed into the model together and processed in one pass, returning a numerical score for each candidate. This enables the model to use context information from the query and the full candidate list to make sense of the entire input, producing better results because of the richer information available to it.

jina-reranker-v3 proved that listwise rerankers with last-but-not-late interaction can compete with the largest models on general reranking benchmarks. Only jina-reranker-v3.5 and models with over four billion parameters beat it on Massive Text Embedding Benchmark (MTEB) reranker tasks. However, this approach places strict limits on candidate list sizes. The query and all candidate matches must fit in the input context window of the model.

What problems does Jina Reranker v3.5 solve?

Despite having frontier-level performance overall, jina-reranker-v3 has some notable performance gaps:

Domain-specific text retrieval

jina-reranker-v3 was trained on general text corpora and, as a result, it underperforms on important use cases, particularly:

• Legal texts, like case law and contract clauses.

• Medical literature, like clinical trials and patient records.

• Financial datasets and other texts full of important numbers.

• Computer programming and IT documentation.

• Product catalogs full of technical terminology and specifications.

Structured data: Tables, JSON, and key-value records

Vast quantities of essential, real-world data is encoded in spreadsheets, tables, key-value lists, and structured records, like JSON data. However, rerankers trained purely for textual comparison, like jina-reranker-v3, perform poorly on this kind of data.

Compute costs for long candidate lists

The self-attention architecture at the core of most text-processing AI models means that memory and compute requirements grow quadratically with the size of its input. This makes jina-reranker-v3, like other AI models, very computationally expensive to run with a full input context window. But, to make the most effective use of the model, we want to put as many match candidates as possible into its input. When it’s at its most useful, it’s also slower and more expensive to run.

We’ve developed jina-reranker-v3.5 specifically to address these issues without reducing its performance on general purpose text retrieval.

What’s new in Jina Reranker v3.5?

jina-reranker-v3.5 contains a modified self-attention mechanism that enhances performance, increases processing speed, and reduces the resources required at inference time to process a full input context window. We’ve also introduced a new three-stage self-distillation training process better suited to the sliding-window architecture of large input context models.

We’ve also curated and used training data focusing on the performance gaps we identified in jina-reranker-v3, including:

• Multilingual legal texts drawn from diverse international sources.

• Medical texts drawn largely from scientific literature and materials used for other AI projects, including a collection of Chinese medical question-answer pairs.

• Financial industry data, including investment-related question-answer pairs, regulations, and tables with numbers and associated texts.

• Structured data, especially from ecommerce sources and public corpora of tables.

• Expanded multilingual and cross-language texts.

For details on the data sources and technical innovations in jina-reranker-v3.5, see our technical report.

How Jina Reranker 3.5 performs on retrieval benchmarks

Parameters

597 million

Input modalities

Text only

Context window size

131,072 tokens

Maximum number of candidate matches

No fixed limit, but all candidates and query must fit in the context window.

Languages

Training in 52 languages

General text reranking performance (BEIR and MIRACL)

jina-reranker-v3.5 improves on jina-reranker-v3’s performance on general text reranking benchmarks. On the English-language Benchmarking Information Retrieval (BEIR) benchmark, the average score has increased enough to surpass the frontier Qwen3-Reranker-4B and 0.6B models and Mixedbread AI’s rerankers.

We’ve also improved jina-reranker-v3’s multilingual reranking performance on the Multilingual Information Retrieval Across a Continuum of Languages (MIRACL) benchmark. Only the four billion parameter Qwen3 reranker regularly beats jina-reranker-v3.5’s score.

Legal, medical, and financial reranking

The Retrieval Embedding Benchmark (RTEB) suite consists of diverse domain-specific retrieval benchmarks. jina-reranker-v3.5 outperforms jina-reranker-v3 on all RTEB tasks related to law, medicine, and finance. Only the large Qwen3 reranker, at almost seven times as many parameters, has better average performance in those three domains.

The new model shows particularly strong improvements for legal data, beating jina-reranker-v3’s score by over 50% on case law retrieval tasks.

Task

Reranker v3

Reranker v3.5

Improvement v3 to v3.5

AILA-Case

20.82

32.55

+11.73 (56%)

AILA-Statute

32.15

46.16

+14.01 (44%)

LegalQuAD

81.84

83.09

+1.25 (1.5%)

LegalSum

69.64

70.99

+1.33 (1.9%)

Structured data reranking (Struct-IR and STaRK benchmarks)

We evaluated jina-reranker-v3.5's structured data reranking on two benchmarks: Struct-IR and STaRK. Both benchmarks contain AI-generated JSON text data covering a variety of applications, including product records, scientific papers, and biomedical knowledge bases.

jina-reranker-v3.5 improves substantially on jina-reranker-v3’s score on the Struct-IR benchmark, once again only exceeded by Qwen3-Reranker-4B. On the STaRK benchmark, jina-reranker-v3.5 beats all the other models we tested, of any size.

Inference speed: Latency benchmarks for short and long documents

The longer the candidate documents get, the more significant the architectural improvements we’ve brought to jina-reranker-v3.5 are. To verify this, we used two retrieval datasets distinguished by large differences in the average document length:

Dataset

Avg. doc length

jina-reranker-v3

jina-reranker-v3.5

Speedup

BEIR Natural Questions

145.5 tokens

371.1 ms

305.3 ms

22%

RTEB AILAcasedocs

1,904.0 tokens

16,064.9 ms

10,290.9 ms

56%

jina-reranker-v3.5 is significantly faster in both cases. On the Natural Questions benchmark, there’s a 22% speedup compared to jina-reranker-v3 with average request latency falling from 371.1 ms to 305.3 ms. Each query from the AILAcasedocs benchmark is much larger (more than 10 times larger on average) so it naturally takes longer to rerank on average: 16,064.9 ms for jina-reranker-v3 and 10,290.9 ms for jina-reranker-v3.5. This represents a 56% speedup for the newer model, representing less latency for applications and lower computer costs.

When should you use Jina Reranker v3.5?

Reranking improves search precision in practically every case, and jina-reranker-v3.5 has applications in a wide variety of information retrieval contexts. However, it has some limitations. The table below summarizes our best-practice advice:

Use case

Recommendation

General text retrieval in common international languages

Use `jina-reranker-v3.5`.

Legal, financial, and medical domain retrieval

Use `jina-reranker-v3.5`.

Semi-structured data, tables, product information texts for ecommerce

Use `jina-reranker-v3.5`.

Non-text or mixed-media data

Use `jina-reranker-m0`, which supports both text and image input.

How to use Jina Reranker 3.5 with the Elastic Inference API

jina-reranker-v3.5 is available via the Jina API with free tokens to try it out. It’s also available via the Elastic Inference API and Elastic Inference Service.

If you’re already using jina-reranker-v3, all you have to do is change the name of the model in the model field of your request to the Jina API or model_id field when configuring an Elastic Inference API endpoint. The two models have completely identical interfaces.

You can install jina-reranker-v3.5 as a Jina On-Prem container to get a completely self-contained server that runs on your own hardware. The model weights are also available to download for testing and research. Follow the instructions on the model’s page at Hugging Face. In both cases, the model is available under a CC BY-NC-4.0 license, so you’re free to try it out for testing, building prototypes, or doing scientific research. For commercial use, please contact Elastic sales.

This is where Elasticsearch earns its place in the stack. It is already where a great deal of operational telemetry lives (logs, metrics, traces, events), and it exposes them through an expressive query and aggregation layer. That makes it a natural tool for an agent debugging a live system: it can slice by attribute, aggregate over time, correlate across signals, and drill from a symptom down to the documents that produced it.

We've been building an agentic layer on top of Elasticsearch that continuously monitors a system and root-causes issues as they arise. A recurring primitive in that workflow is time series event analysis. For example, given an error rate, a p99 latency, a queue depth and a throughput counter, tell me whether something happened, what it was and when. A transient spike in errors, a regime change in latency, and a step up in CPU usage are the signatures of the underlying fault, and they're typically what an agent examines first as it forms and tests hypotheses.

Elasticsearch has shipped a single-change-point aggregation for some time. It answers "did this series change?" with one verdict and the most significant change it found. That's a good fit for a dashboard, but less so for an agent, which often wants to interrogate a long window and enumerate everything of interest in it: the error spike at 02:14, the latency regime shift at 02:30, the throughput dip while the pod was being rescheduled. So we upgraded the capability to detect and report multiple events of multiple kinds in a single series. At the same time, we took the opportunity to further harden it to work reliably against whatever telemetry the agent points it at. This post describes how it works.

Why single change point detection isn't enough for agents

Concretely, we want a single entry point that takes a numeric time series and returns a small list of interesting events, each with a type, a location, a significance, and some key characteristics. We care about three classes of event, because they map onto three different kinds of underlying fault:

Event type

What changes

Detection channel

Example fault

Structural change

Level (step) or slope (trend) shifts to a new sustained regime

Value channel

Config push doubles baseline latency; memory leak turns a flat curve into a ramp

Distribution change

Noise level (variance) shifts while the mean holds steady

Dispersion channel

Service responds erratically at the same average latency

Point anomaly

Isolated spike or dip against a stable background

Value channel (pulse detector)

Single burst of errors; one-minute throughput drop during GC pause

The hard part is not detecting any one of these on clean, well-behaved data. The hard part is doing it on arbitrary telemetry without per-series tuning. The agent does not know in advance whether the series it is examining is near-constant, smoothly drifting, heteroscedastic (quiet in places and noisy in others), sparsely populated, or has a magnitude of . It is not scalable to hand-pick parameters for every series it needs to analyze. Whatever we build has to be robust to all of that while maintaining excellent recall and precision. If it fails to detect important events it runs the risk of missing key corroborating evidence for a working hypothesis. Conversely, an analysis tool that reports an event for every minor fluctuation will pollute the context the agent reasons over.

The design goals, in priority order, are: correct on diverse data out of the box, parsimonious (report only what matters), and cheap enough to run interactively across many series.

How PELT and BIC power change point detection

Change-point detection is a well studied field. The classical offline formulation searches for the segmentation of a series that minimizes a penalized cost: a per-segment goodness-of-fit term plus a penalty for each added break to stop the optimizer from putting a boundary between every pair of points. Solved naively, this is combinatorial, but PELT (Pruned Exact Linear Time, Killick et al.) finds the optimal partition in roughly linear time by using a dynamic program to prune candidate boundaries that can never be optimal. On the labeling side, comparing nested models by an information criterion such as the Bayesian Information Criterion (BIC) gives a principled, scale-aware way to decide whether a candidate break is really important and what sort of change it constitutes.

These are good building blocks, and we use them. However, the textbook recipe assumes more than telemetry gives you. It typically assumes a single change type (a mean shift), a known and stationary noise level, and reasonably benign numerics. Real telemetry violates all three: variance changes matter as much as mean changes, the noise level is unknown, often heavy-tailed and changing, and the data spans extreme magnitudes and degenerate cases, such as perfectly constant segments, that wreck an ill-conditioned polynomial fit or a fixed-variance cost. Most of the engineering I describe below is about closing that gap.

Splitting one time series into three detection channels

Rather than trying to find one detector that does everything, we run three focused detectors and then merge their findings. Two of the three are the same structural detector applied to two different views of the data, or channels.

Two detectors run on the value channel: a structural detector that flags step and trend changes, and a pulse detector that identifies point spikes and dips. The dispersion channel, a windowed measure of spread, is fed to a second copy of the structural detector, where a variance change shows up as a level shift and is relabeled a distribution change. The two channels are complementary by construction: a step is a level shift the value channel flags but only has a single large first difference the dispersion channel ignores, while a variance change is invisible to the value channel yet shows up clearly in the dispersion channel. A thin orchestration layer then merges and de-duplicates the streams.

Keeping the three concerns separate makes each one tractable. A mean-shift detector and a variance-shift detector pull in opposite directions if you try to fuse them; a point-anomaly detector and a regime detector need opposite robustness settings. Separated, each can be tuned to its job.

PELT with a scale-free cost

For the structural channel, we fit each candidate segment with a low-order polynomial (constant or linear) and score it with the profiled-variance Gaussian cost. If a segment of length has residual sum of squares , the cost is

This is the negative log-likelihood of a Gaussian segment after profiling out the variance, i.e., after substituting the maximum-likelihood estimate back into the log-likelihood. The total objective PELT minimizes is the sum of segment costs plus a per-break penalty,

Here, is the BIC complexity term (number of parameters times where is the number of values in the time series) and the scale factor lets us trade sensitivity against parsimony in one place.

Using the profiled cost rather than a cost against a fixed global variance is a deliberate and important choice. The global noise level of telemetry is unreliable: on a smoothly varying series the natural estimate (the spread of first differences) can collapse toward zero, and a fixed-variance cost then treats every wiggle as enormously significant and over-segments. The profiled cost depends only on the ratio , so it is invariant to the absolute scale and immune to that failure. A small floor on keeps the logarithm finite, so a zero-residual segment is not rewarded without bound.

Keeping the fit stable using robust local weighting

Before PELT runs, we robustly down-weight points so that an excursion does not create spurious breaks or drag a segment boundary onto itself. Crucially, these weights enter into the weighted residual moments, so they shape not just the segment fit but its residual variance, and so the segment costs themselves. Each point gets a Cauchy weight, , of its residual from a rolling-median baseline against a robust scale : points near the local median keep full weight and points far from it are progressively discounted. A nice side effect of measuring excursions from the median on a window centered on each point is that clean structural breaks do not get down-weighted at all, because the majority of values land on the same side of the break as the point whose residual is being computed. So a sustained regime keeps full weight, but a lone spike does not.

There is a nice justification for scoring a weighted Gaussian cost when what we really want is robustness to a heavy tail. The Cauchy weight is exactly the iteratively reweighted least squares (IRLS) weight of its loss: , so the weighted normal equations are identical to the Cauchy M-estimator's estimating equations . A weighted-mean (or weighted-line) fit at those weights is therefore a Cauchy M-estimate, not a Gaussian one. The cost we actually evaluate inherits the same properties. Because is concave in , its tangent at the current residual lies above it. This gives a pointwise bound with the weight at the tangent point; summing, the weighted residual sum of squares is a tangent upper bound on the total Cauchy loss, touching it in both value and gradient at the weights' anchor point. Minimizing the weighted RSS is thus one step of a majorize–minimize scheme that provably decreases the true Cauchy objective, and the profiled-variance cost we feed the BIC is that majorizer standing in for the Cauchy deviance. The only approximation is that we anchor the weights once, at the rolling-median baseline, rather than iterating IRLS to its fixed point; this is exact for the inliers that sit near the baseline, and correct for gross outliers, whose vanishing weight removes them from the cost wherever the bound is loosest.

Finally, the trick that makes this work on heteroscedastic data is that the residual is judged primarily against a local robust scale, not a global one: the MAD of residuals in the same sliding window. On a series that is quiet in one stretch and noisy in another, this means a spike in the quiet stretch that is multiple local sigmas is correctly suppressed. The local MAD can collapse on quiet stretches, so we use a backstop that is a fraction of a global composite of robust scales and a floor related to the quantization error for discrete series and numerical precision otherwise.

From candidates to labeled events using BIC verification

PELT gives a globally optimal penalized segmentation, so we take its boundaries as candidates and verify each one. For a candidate at index we look at the window of length spanning to its nearest neighboring candidates and compare a no-change null against step and trend alternatives by BIC,

where counts the fitted parameters (the same parameter count as in the PELT penalty above). We map the BIC gain of an alternative over the null to a significance via , to turn a threshold into a decision boundary). We treat this as a significance score for ranking and thresholding, not as a calibrated tail probability.

At this stage we allow higher-degree models to avoid splitting smoothly varying trends. These are problematic in PELT itself because it considers short segments, which they overfit. We keep the most parsimonious alternative that clears the significance threshold and survives a persistence check. The persistence check re-scores with the weights immediately around the candidate muted, and if the evidence collapses, the "change" was driven by a few extreme points – an excursion, not a regime change – and we reject it. The polynomial order is applied symmetrically to the null and to each side of the split, so the alternative is always the same model class merely split at the candidate, and therefore strictly more flexible. The whole process can be thought of as Bayesian model selection with a preference for the null.

When no candidate survives, we still say something useful: we report the series as "stationary" (best no-change model is a constant) or "non-stationary" (best model has a slope), with the trend direction. For an agent, "this series is cleanly trending up over the window" is itself a finding.

Detecting distribution changes with a dispersion channel

A variance change is indirectly visible to the mean channel – worse, the robust weighting there actively mutes the excursions that signal it. So we detect it on a separate dispersion channel and reuse the exact same structural detector, because on this channel a variance change is just an ordinary level change.

The channel is built from one sample per non-overlapping window. Within a window, we take the inter-quartile range of the first differences, rescaled to a standard-deviation equivalent (), and pass it through :

Then . Three choices matter here. First-differencing cancels level and slope, so a mean step contributes a single large difference rather than inflating the whole window, and a steady ramp produces a flat channel. Non-overlapping windows keep the samples independent; overlapping windows autocorrelates the channel and makes the segmenter over-detect. And the IQR is used rather than the median (which is too robust and will miss a window that is 40% noisy then flatlines) or the raw standard deviation (which is not robust enough since one spike's two large differences inflate the window). Because the dispersion channel is a fraction of the original length, the verifier there is restricted to a lower-order null so a genuine low-high-low variance bump is not absorbed.

The functional form is worth dwelling on, because each part earns its place. The log makes the channel respond to ratios of noise level rather than absolute differences. Variance changes in telemetry are typically multiplicative: a regime is "twice as noisy". On a raw-scale channel, a doubling shows up as an enormous absolute jump at a high baseline and a negligible one at a low baseline, so an additive step-cost detector would find variance changes trivially in loud series and miss them in quiet ones. Under a log, a factor- change in scale is the same offset wherever it occurs, which is exactly the additive-step behavior the structural detector is built for. The " is a soft floor. A bare diverges to as the scale goes to zero, which is precisely what happens on a near constant stretch, and would manufacture a huge spurious step at the first noisy window after it. Conversely, is finite and smooth at zero, behaves linearly () while the noise is small, and recovers the multiplicative behavior once the noise is appreciable. This gives graceful degradation instead of a singularity, and with no tuned epsilon to pick. Note that squaring the scale (using a variance instead) only doubles the dynamic range; it makes no difference to the detector either way, since differs only by a constant the threshold absorbs.

Detecting point anomalies as excursions from a local baseline

Spikes and dips are detected as point excursions from the local rolling-median baseline. Working from the local residual rather than raw values means level structure is removed and smooth curvature is tracked; even for time series that change significantly the detector is sensitive to significant local deviations.

The pipeline is a generous proposer followed by a strict gate:

1. Propose every point whose residual exceeds a threshold number of robust sigmas. The scale is the larger of the global first-difference noise (which stays meaningful on smooth data where most residuals are exactly zero) and a composite of robust scales of the residuals (which inflates once a frequent large-residual population appears).

2. Merge adjacent same-sign candidates into excursions, dropping any that span a full minimum segment: that is a regime, and is owned by the structural channels.

3. Rank and cap the excursions by peak z-score, keeping the top , so a pathological series cannot drown the output.

4. Gate using one shared null: build a Gaussian KDE from the series with all of the retained excursions removed, and keep an excursion only if its peak's Bonferroni-corrected upper-/lower-tail probability under that null clears the threshold.

Removing all the tested excursions from the single null at once is a key trick. The leave-one-out alternative – score each excursion against a null containing the others – lets the largest spike and dip mask everything else. Removing them together means several genuinely distinct excursions are each judged against the remainder and all survive, while a recurring population is still rejected.

The proposer and the gate ask deliberately different questions, and that distinction drives two further choices. The proposer works on residuals from the rolling median since it wants recall, and a residual is what tells you a point stands out from its local neighborhood. The gate is value-based: it asks "is this magnitude one we see at other times in the series?", so a spike to a level that recurs elsewhere — such as periodic batch jobs — is suppressed even though it is a large local residual. Those are the right semantics for an agent, but they expose a heteroscedasticity problem, because telemetry noise is almost always a function of magnitude. Periodic spikes can sit orders of magnitude above the background, and a single KDE bandwidth fitted to the whole value range is then far too narrow up in the high tail. So ordinary large values come back as significant, a steady source of false positives.

The fix is a variance-stabilizing transform. We run the value gate in space, where is a robust measure of the spread of the background. is linear for and logarithmic for , which turns a multiplicative (magnitude-dependent) spread into a roughly constant one, so a single bandwidth is valid across orders of magnitude. It is also odd and finite at zero, so exact zeros and sign changes (dips below a small baseline) need no special handling, unlike a bare log. Crucially, it is monotone and so does not change what is tested (for any monotone function , ) so it only fixes the estimate of that tail probability.

One subtlety closes the loop. The KDE null and the kernel bandwidth are taken from different scales, on purpose. The null is the stabilized background values: so it models any mode in the data, which is what makes a recurring large magnitude unsurprising. However, the bandwidth is taken from the stabilized residuals, not the stabilized values, because a genuine level change makes the value distribution bimodal, and a bandwidth computed from that bimodal spread would balloon, masking a real spike sitting on top of a shifted regime. The residual removes the step, so the bandwidth always reflects within-regime noise and the gate stays sensitive to a deviation that is extreme relative to its own neighborhood if it is also outside the envelope for the series as a whole.

Merging structural, distribution, and point anomaly events

Finally, an orchestration layer merges the structural, distribution, and point-anomaly event streams. Structural and distribution events that mark the same regime boundary are de-duplicated to the more significant one (a boundary that shifts both level and spread is one event, not two). Pulses are a separate stream added after de-duplication, because a spike that lands on a structural boundary is a real, separate finding and must not be suppressed. Everything is then mapped back from the internal value-array index space to source-bucket indices.

Handling extreme magnitudes and edge cases

What makes this usable as an unattended tool is a collection of defensive choices for the cases that break naive implementations:

• Variance computed as loses all precision at large magnitudes: a constant series at can manufacture phantom change points purely from floating-point error. We center every PELT input by a constant offset first; the polynomial RSS is invariant to that shift in exact arithmetic, but the working magnitudes drop from to .

• Using raw indices as the regressor results in poor condition polynomial fits: the largest moment is , which is about for a cubic over a 2000-point window. This trips the SVD singularity guard and silently degrades the fit. Mapping affinely onto leaves the fit identical (RSS is invariant under reparametrization) but every moment becomes .

• Scale-free cost, as described, means the segmentation cost doesn't depend on tuning a noise estimate.

• Down-weighting wants a primarily local scale: suppress whatever is anomalous in its own neighbourhood. It uses the maximum of MAD and a small global floor on the differences from the rolling median. Conversely, spike/dip detection wants a global scale: we care about global outliers. It uses a composite of robust scales of all differences. Using the wrong one in either place produces characteristic failures – irrelevant spikes in a quiet segment, or locally large excursions creating spurious breaks – and maintaining separate channels allows us to pick appropriately.

• Using one p-value threshold, Bonferroni-corrected by the number of candidates, applied consistently across all three detectors, means that "how surprised should I be" is consistent everywhere.

The recurring theme is that the difference between a detector that works in a notebook and one that works on a firehose of production time series is mainly in handling the edge cases gracefully.

Performance: ~1ms per series on a single core

Detection cost is dominated by PELT. Its segment cost is a profiled-variance linear fit, which we evaluate in constant time from prefix-summed weighted moments rather than maintaining a regression per candidate boundary, so a single segment cost is a handful of array reads and a 2×2 solve. Cost grows a little faster than linearly with series length since PELT's pruned candidate set does not stay constant on noisy data. Therefore, to bound the worst case on very long series, we downsample ahead of detection: above a cap (2000 samples), the series is collapsed into macro-buckets, keeping two samples per bucket: the median and the largest local deviation. This is inspired by the M4 downsampling scheme, but because we need only the median and the largest excursion for structural-change and outlier detection, respectively, we can then afford to double the bucket resolution. The downsampled series carries its original bucket indices, so every reported event still maps back to a real source bucket; below the cap it is a no-op. The whole analysis is a single pass over the (possibly downsampled) series with no per-series configuration. This lets the agent call it freely across many signals.

In absolute terms, this means the analysis is comfortably interactive. On a single core, post-warmup, a typical series of 140–350 buckets is analysed in about 1 ms (≈220,000 buckets/s), and a series long enough to hit the downsample cap (say 5,000 buckets, collapsed to 2,000) takes about 40 ms, which is the effective worst case per call. For an agent issuing a handful of these calls per investigation, and parallelising across the many series in a `BY` query, the latency is negligible.

Evaluation on synthetic and production telemetry

Synthetic benchmark

We evaluate first on a synthetic generator with known ground truth, because it lets us measure the things that matter precisely. The generator produces random time series with diverse behaviors, which we group into three families. The positive family injects known events of each type: clean and noisy step changes (up and down, SNR around 10, at several positions), trend onsets and ramps (including a flat–ramp–flat sequence with two boundaries), variance changes with a constant mean (single steps and a low–high–low bump), and isolated spikes and dips. The null family ideally produces no event: stationary noise, perfectly constant series, smooth quadratic drift and clean ramps, and periodic signals. (A variance change with a constant mean is not null: it is a distribution change, an abrupt step on the dispersion channel, and so it belongs in the positive family above. The related null requirement, that such a change does not surface on the value channel as a step, is checked separately.) The adversarial family stresses the robustness machinery: a perfectly flat series at a magnitude of (for which a naive variance arithmetic manufactures phantom breaks here through catastrophic cancellation) and, conversely, a genuine spike or step in a noisy baseline as high as (which must still be found and located, the high baseline notwithstanding); a spike on top of a step; a within-regime spike after a 100 level jump; a recurring train of equal peaks (a population, not individual spikes); wide sustained excursions (a structural change, not a spike or dip); and fuzzed random level-shift series. On these series we track recall per event type, precision and the false-positive rate on the null family, localization error, parsimony (events per series and adherence to the count limit), and invariance under constant offsets, rescaling and extreme magnitudes.

The table below summarizes what the suite tests.

Event family

Representative scenarios

Required outcome

Localization tolerance

Step

clean / noisy, up / down, single and multiple, several positions

detected

≤ 4–8 buckets

Trend

slope change and flat–ramp–flat, clean and in noise

detected

≤ 8–12 buckets

Distribution

variance step (mean constant), low–high–low bump

detected

≤ 1 dispersion window

Spike / dip

isolated, multiple distinct, on heavy-tailed and high-magnitude series, within-regime after a step

detected, capped at max(5, 2% of n)

≤ 2 buckets

Null series

stationary noise, constant, smooth drift / ramp, periodic

no event reported

n/a

Invariance / robustness

constant offset, rescaling, 10^5–10^9 magnitudes, spike-on-step, recurring population, wide excursion

result unchanged / no spurious event

n/a

Running this over 400 series per scenario, just over half a million buckets in total, and scoring with the same two categories we use for the real-data evaluation below (any regime change versus point spikes and dips) gives:

Event type

Recall

Precision

Median localization error

Structural change

0.994

0.664

0

Spike / dip

0.847

0.746

0

Two things stand out. When an event is detected, it is placed essentially exactly: the median localization error is zero buckets for both categories; and the point-wise accuracy, 0.998, is directly comparable to the 0.995 we report on real telemetry below: the overwhelming majority of buckets are correctly left unmarked.

The precision figures are lower than on real data, and understandably so. A sixth of this population is a hostile null family (periodic signals, smooth drift, clean ramps) chosen precisely because they tempt a detector into a spurious break, and every false alarm on them counts against precision. The resulting false-positive rate is 0.5% per bucket, with 15% of null series carrying at least one spurious event. We treat this as the conservative end of the range: on the real-telemetry mix below, where the null series are less adversarial, precision rises to 0.85 (structural) and 0.90 (spikes/dips). Finally, offset and scale invariance holds on all 400 series: the same events, to within a few buckets, whether the series is shifted by a constant or rescaled by up to three orders of magnitude.

That raw precision also misses how the result is consumed. The agent reads events most-significant-first, so a false positive only does harm if it outranks a genuine one, and by and large it does not. The median p-value of a true positive is about , against about for a false positive: the genuine events are typically overwhelmingly more significant. Concretely, if we keep only the top events per series by significance (with the true count) precision rises to 0.94, and a randomly chosen true positive is more significant than a randomly chosen false positive 93% of the time. It is not a perfectly clean separation: a strong periodicity or a sharp curve genuinely can contain a significant-looking break, which is why that figure is 0.94 rather than 1. However, the ranking is reliable enough that an agent reading from the top, or applying a stricter significance cut-off, sees the real events first and the false alarms as a lower-significance tail. This is also why exposing the detector's significance to the agent (see What's next) matters more than squeezing the raw precision higher.

Real cloud telemetry

To evaluate on real data, we scraped around 300 metrics from our production cloud environment. These cover HTTP status-code counts, failed memory allocations, memory usage, network usage, page faults, CPU usage, and throttling metrics, measured both per instance and aggregated across the fleet as a whole. Their values range over more than 12 orders of magnitude, and they display a variety of behaviors including ramps, periodicity, step changes, distribution changes, and trend changes. The figure below shows a sample of series together with the detections we make on them.

To get a sense of the accuracy on this set data, we labeled a subset of 150 most interesting time series, marking the visually clearest features in each. This labeling is not necessarily optimized for our target use case, where false negatives are typically more problematic than false positives: an agent consumes these results as part of a broader investigation and can pull additional information to corroborate them. Even so, we find excellent agreement with human judgment on these series. Since each series comprises between 140 and 350 points, the point-wise accuracy, at 0.995, is extremely high: the great majority of points are correctly identified as neither a change, a spike, nor a dip. But the more telling metrics are recall and precision on the human-labeled points, shown in the table below. The human labels did not attempt to categorize each change, so we break the results down only into "structural changes" and "spikes / dips" — the same two categories, and the same point-wise accuracy and recall/precision metrics, as the synthetic benchmark above.

Event type

Recall

Precision

Structural change

0.94

0.89

Spike / dip

0.97

0.92

It is worth covering exactly why we get disagreements. These largely fall into three categories: spikes and dips in context, isolated breaks, and small-magnitude breaks in stable series. We deliberately do not try to detect spikes and dips that are unusual only in their immediate context; that is, not globally unusual but visually significant relative to an inferred periodicity in the data, for example. Trying to account for these without fully modeling the seasonality in the data hurt precision more than it helped recall, and we have a separate persistent anomaly-detection process that builds more complete models of baseline behavior over time. Isolated breaks are an artifact we decided to live with: PELT's cost function tends to isolate a change point with a few values intermediate between the two regimes, because absorbing it into either neighboring span inflates that span's cost. Humans are good at judging such situations visually and assign a single change point. Finally, small-magnitude changes are simply not visually obvious. We detect them deliberately and regard this as a strength of a quantitative approach, since they are often the early precursors of an incident whose later, larger effects drown them out.

How the agent uses change point results in ES|QL

To the agent, all of this is one tool call: it points it at a collection of time series and gets back a typed, located, ranked list of events. That list is small by construction, which means it drops cleanly into the model's context without crowding out everything else it is reasoning about. Because the result contains multiple events, a single call over a window can hand the agent the whole local story – "error spike at 02:14, latency regime change at 02:30, throughput dip at 02:31" – and let it correlate across signals to a root cause.

Operationally, we expose this through both the ES|QL CHANGE_POINT command and the change_point aggregation. Currently, we have not extended the change_point aggregation to return multiple change points since it breaks backwards compatibility of the output schema. It just returns the most significant event. We don't have the same restriction for ES|QL since it returns change points annotated onto the table rows to which they apply. We do plan to revisit the output schema for both ES|QL and the aggregation in a later version. We'd like to migrate to optionally returning significance in log-space, which doesn't underflow, and including a short verbal description of each change, which we expect to help agents when seeing just the change points themselves.

ES|QL is Elasticsearch's piped query language, and CHANGE_POINT runs the detector as one stage in a pipeline. Its BY clause enables it to analyze many series at once (one per group) so the agent can, in a single query, segment every service's latency or every host's error rate side by side rather than issuing a call per series. The actual leverage, compared to the change_point aggregation, is composability: the events come back as ordinary rows in the pipeline, so the agent then has the entire ES|QL language to manipulate them downstream. It can filter to a window, join change points against deploy markers, count events per service, rank by significance, feed the survivors into a further aggregation, and so on.

For example, suppose the agent wants to find out whether any servers have recently seen a sudden CPU spike or a prolonged step change in CPU usage over the last 12 hours, and whether that might point to a load-balancing issue. It could use the following query:

Here it's using a STATS ... BY host.pod to see how the detected events cluster across other dimensions of the data, such as the Kubernetes pod, and so judge whether they share a common cause.

What's next for ES|QL time series analysis

As far as detecting events of interest in time series, the foundation is in place: a single, robust, parsimonious tool that turns a raw telemetry series into the short list of events that actually matter, which is exactly the kind of reach an agentic SRE needs. Going forward, we plan to explore the best mechanism for feeding the detector's uncertainty to the agent, so that a borderline event can be flagged as "worth a second look" rather than silently included or dropped. Also, this is the first of several analytical tools we plan to build into the ES|QL query language to enable agents to triage and RCA issues more effectively; so stay tuned for further updates.

A vector store that achieves good performance without tuning is more valuable than one that requires expert tuning. In fact, our contention is a data store that can be coaxed to exceptional performance by an expert who spends a week hand-tuning it is less useful than one that beats it consistently out of the box. In other words, easily achieving good performance is a first class property, not a nice to have. We can see this clearly in our telemetry. The great majority of users will never tune the internals of vector search at all, and why should they: it is just an enabler for what they're trying to build.

This is the imperative behind features like auto-calibration. The system as a whole should look at your data and your quality target and choose good parameters for you. Indeed we think this is a win-win, since it has far more nuanced information available to it to make these choices than we expose.

To make "good performance" precise, it helps to name the three attributes that characterize any vector search system, because they trade off against one another and you can't talk about one without fixing the others:

1. Performance: throughput (QPS), latency, and so on.

2. Hardware cost: a fair comparison always holds cost fixed. It's trivial to buy your way to more QPS or better recall by throwing hardware at the problem; the interesting question is what you achieve per dollar.

3. Search quality: recall, nDCG, and related measures of whether you're returning the right results.

The three form a frontier. Push one and, at fixed budget, you pay in another. Any honest comparison of approaches pins two down and measures the third. What we describe in this post is the mechanism we're introducing to pick quantization parameters for a fixed recall budget. It is a step on a longer journey towards a vector store that configures itself well across the board.

Why recall is the right quality metric for vector search

Search quality is tricky, because the "right" results depend on relevance labels you usually don't have at index time. So we lean on recall as a safe proxy. The argument is simple: recall measures how well the approximate index reproduces the results of exact search over the same embeddings. If recall is high, you have not degraded search quality relative to what the underlying model can do; you can be confident you’ve faithfully preserved the baseline. You might still wish for a better embedding model, we've got you covered, but that's a separate concern from the index not damaging what the model already gives you.

This is why controlling recall matters so much, and why you should be wary of any system that doesn't reliably control it. If a vendor can't control recall, they can silently degrade your search experience, achieving impressive QPS numbers while quietly returning worse results, and you'd have no way to know without a labeled evaluation set. The method in this post is about maximizing performance while keeping a firm, predictable grip on quality.

Why vector quantization parameters must be chosen at index time

What makes the problem genuinely hard is that vectors are quantized as they are indexed, so the parameters that govern quality (how many bits, how deep to rerank, whether to precondition) have to be evaluated before we've seen the data laid out in its final form. We can't index everything, measure recall, and iterate; by then the quantization is baked in.

So we need to estimate what we'll need from a small sample, cheaply and in advance. Fortunately the Elasticsearch gives us natural moments to do this: segment merges are exactly such an opportunity. When segments are combined we have to rewrite the data anyway and can assess the data and (re)choose parameters. And as we'll see, models fit to small random samples give excellent estimates of the quantities we actually need to control. They’re typically good enough to set parameters once, with a small margin, and trust them as the index grows.

How vector quantization affects nearest-neighbor recall

With that motivation in place, let's start to dig into the details.

Vector quantization is a critical component for making approximate nearest-neighbor (ANN) search affordable at scale; it's an area we've innovated in the past. Instead of storing and comparing full-precision embeddings, we store a lossy, compressed representation and search over that. The catch is the one above: lossy representations move distances around, so the "nearest" neighbors under quantized distances are not always the true nearest neighbors and recall suffers.

The standard fix is to over-retrieve and rerank. We use the cheap quantized distances to pull back the top candidates, then recompute exact distances for those and keep the best . As long as the true top- are present somewhere in the retrieved top-, reranking recovers them exactly.

Reranking isn’t free, we have to fetch high precision vectors from disk. However, we can precisely characterize the performance of reranking based on hardware characteristics alone. This reframes the whole problem. The question is no longer "how much does quantization distort distances?" in the abstract, but something which relates back to the attributes we care about:

This post derives a model that answers exactly that. The core of it is a single, surprisingly clean idea: if we can characterize the distribution of distances to the -th nearest neighbor, and we have a model of the quantization error distribution, then we can compute expected recall after reranking in closed form (up to a one-dimensional integral). Everything else – bit counts, rerank depth, whether to precondition – becomes a search over a model we can fit cheaply from a small sample, instead of an expensive empirical sweep over full indices built with those parameters.

We build it up to this in three stages: the geometry of nearest-neighbor distances, the scaling law that falls out of it, and then the recall model that ties quantization error to recall given a reranking budget. Be warned, the following gets a little bit involved, but to give you intuition about what is happening see the video below.

Quantization error vs. the nearest-neighbor distance gap

Fix a query and rank the database vectors by their true distance to it: , so is the distance to the -th nearest neighbor. Reranking the top succeeds for the true -th neighbor whenever it is not pushed past rank by quantization noise.

Two competing quantities govern this:

• The quantization error that is essentially fixed for a given scheme and dataset: it depends on the embedding dimension, the vector distribution, and the number of bits, but not on how big the index is.

• The criticality gap , which is the distance between the -th and the -th nearest neighbor. This is the margin we have to absorb error. Crucially, it shrinks as the index grows: pack more vectors into the same region and neighbors crowd together.

There’s a detail here we’ll gloss over for the sake of presentation: for IVF style indices, we’re quantizing the residual from a cluster’s centroid. This does in fact couple the quantization error to the index size, but we can handle it much the same way we handle the distance to the -th nearest neighbor.

For reranking to recover the recall lost to quantization, we need the error to only rarely exceed the gap. If we can write down the distribution of and the distribution of the error, we can make that statement quantitative. The first job is to estimate the distribution of nearest-neighbor distances.

Deriving the nearest-neighbor distance distribution

Real embeddings don't fill their ambient space; they concentrate on a lower-dimensional manifold. Near a query, though, we can make a mild local assumption: in a small neighborhood around the query, the data density is roughly uniform. Here is the intrinsic dimension of the manifold; it is unknown and generally far smaller than the embedding dimension. How to estimate it is the subject of Section 4.

Let be the vectors falling in , modeled as i.i.d. uniform on , and define the distance from to its nearest neighbor:

To get the distribution of we use the standard order-statistics trick: rather than ask where the minimum is, ask for the probability it exceeds some radius . The event is exactly the event that every point lands outside the -ball centered on the query .

A single point lands inside with probability equal to the ratio of the ball's volume to the region's volume

where is the volume of the unit -ball. (We assume is large enough that the relevant is small, so the ball doesn't spill outside and boundary effects are negligible.) Because the points positions are assumed to be independent, the survival function is

What we're really interested in is how R behaves on average. To compute this, we use the identity that the expectation of a non-negative random variable is the integral of its survival function, . Evaluating this with (2) gives the headline result:

(The exact integral carries an extra factor; it's an constant that we can fold into a fitted coefficient later, so we drop it here.)

Glacial scaling: why neighbor distances barely change as your index grows

It is interesting to consider what this formula tells us about how distances change with dataset size: . The exponent is , and in high intrinsic dimensions that is a very small number. This is a property the method leans on, so it's worth plugging in some numbers:

• If then doubling multiplies by , so distances drop by ~30%.

• If then doubling multiplies by , so distances drop by a little over 1%.

In high dimensions, neighbor distances barely move even if you add a lot of data; call it glacial scaling. It's the reason we can choose quantization parameters once from a tiny sample, with a small safety margin, and trust them to remain valid even after the index grows substantially before the next re-quantization.

Expected distance to the k-th neighbor and the criticality gap

We actually care about the whole sequence of order statistics , , not just the minimum. There's a simple way to get them.

Map each radius to the cumulative volume it encloses by defining

By (1), each is exactly the probability of landing within radius , so the are uniform on . The order statistics of uniforms are textbook: the -th smallest of uniforms follows a Beta distribution,

Inverting the volume map, , gives the scaling of the -th neighbor distance:

That's all we need for the expected gap:

The last form is the intuitive one: the gap between the -th and -th neighbors is the distance to the -th neighbor, scaled by . Widening the rerank depth relative to opens the gap; higher intrinsic dimension closes it (the exponent pushes toward 1).

Why the expected gap is sufficient to predict recall

Working with an expectation is only legitimate if the gap doesn't fluctuate wildly around it. It doesn't because concentration of measure saves us. Applying the delta method to and using from the Beta distribution, a little algebra gives

So the coefficient of variation is about . For any reasonable intrinsic dimension this is negligible, which justifies modeling only the expected distances. (If you're worried about the delta method approximation, you can check the results numerically: the delta-method variance and the resulting coefficient of variation match the exact expressions to several significant figures.)

Extending the model to cosine similarity and inner product search

The derivation is for the Euclidean metric, but the other common metrics reduce to it:

• For cosine similarity, the equidistant surface is the intersection of a sphere around the query with the unit sphere. This is called a hyperspherical cap, whose volume scales as for small . Therefore, the analysis carries over unchanged up to constants, with the dimension reduced by one.

• For MIPS (maximum inner product), some extra care is needed, because nearest neighbors aren't confined to a compact region. A distant vector can still win on inner product if its norm is large enough, so the gap is really governed by the tail of the norm distribution. However, there is a clean fix, which is to use the Neyshabur–Srebro transformation. This lifts vectors onto a unit hypersphere in dimensions. After this operation, it's just the cosine case.

Fitting intrinsic dimension and scale from a small sample

Equation (3) has a known functional form but two unknown parameters: the intrinsic dimension and the scale . Both are easy to fit, and it's more convenient to fit them from raw neighbor distances than from gaps directly.

Sample several subsets of database vectors of sizes and a set of query vectors . For each query and each subset, measure , the distance to the -th nearest neighbor of within . Taking logs of the scaling law linearises it:

Specifically, this is linear in and , so ordinary least squares recovers and . Varying the subset size is what makes it possible to estimate : it's precisely the rate at which distances shrink with data volume. With the fitted parameters, the whole-index expected gap is

Figure 1 shows how well this fits in practice (and it’s remarkably good): predicted versus actual average distance to the -th neighbor, across a range of datasets and metrics, have between 0.996 and 0.999.

Modeling vector quantization error as Gaussian

With the nearest-neighbor distance model established, the second component is the quantization error distribution. For every metric we use, the quantized distance estimate differs from the true distance by an error that is a sum of many independent per-dimension contributions. By the Central Limit Theorem that sum tends to Gaussian, so we model the error as normal with a variance we estimate empirically:

where is the quantized distance estimate using -bit vectors and is the total number of (query, neighbor) pairs in our sample set. In other words: sample, quantize, measure the squared distance errors, average.

Figure 2 shows the empirical basis for the Gaussian assumption: measured quantization error densities against best-fit Gaussians across a variety of datasets. The fit is good, which is what lets the rest of the model stay analytic.

We could stop here and take a minimax view: threshold the probability that the -th and -th neighbors swap, using the expected gap (4) against the error scale . But that controls a worst-case event, and what we actually want to control is average recall. The outcome would be overly conservative quantization parameters and we'd pay some performance. The next section estimates expected recall properly.

Predicting expected recall after reranking

Combining the distance model and the error model gives a closed-form estimate of expected recall after reranking. Model the noisy distance of the -th true neighbor as a Gaussian centered on its true distance:

The -th neighbor survives reranking, i.e., lands in the retrieved top , if fewer than other vectors have a smaller noisy distance. Condition on and count the competitors closer than :

Then the probability of recalling neighbor integrates over where its own noisy distance lands:

The terms of are independent Bernoullis but not identically distributed, since every neighbor sits at a different true distance , so each has its own probability of intruding on the top- set:

with the standard normal CDF. This makes a Poisson-binomial variable. Since we sum many of them (because ), the Lyapunov CLT applies and we approximate

with the standard Poisson-binomial moments

The survival probability then has a clean closed form:

This is where the two halves of the post so far finally meet. We don't need to know the individual because the manifold scaling law from Section 3 supplies them: . So the moments become explicit sums over ranks, which we truncate at a safe cutoff (say , since distant neighbors contribute negligibly):

Finally, average recall@ given rerank depth sums the per-neighbor recall over the top :

Here is the standard normal density. Each integral is smooth and one-dimensional, so Gauss–Legendre quadrature evaluates it in microseconds. The entire recall prediction for a set of candidate parameters costs a handful of quadrature evaluations, not index build and benchmark run.

Figure 3 validates the end-to-end model: predicted average recall against measured recall across many parameter settings and multiple datasets has .

How the recall model selects vector quantization parameters

With a fast recall predictor available, parameter selection becomes a cheap ordered search. Given a target recall and a rerank budget (typically expressed as a multiple of ), we can find the minimum document and query bit counts, and other knobs, that clear the target. There are a few things to note that are practically important:

1. Glacial scaling gives us some safety because moves so slowly with for even moderate intrinsic dimension. A small margin in the calculation means the chosen parameters stay valid if a lot of vectors are added before parameters are restimated.

2. Small is the worst case if is a fixed multiple of . The gap \mathbb{E}[R_{(k)}]( is smallest for small so if a parameter choice satisfies the recall target at then it will for larger will too.

3. We can treat quantization as a black box because the error model only needs the empirical error variance. This means we can test any configuration, including preconditioning, the same way and we can simply order candidate parameter tuples by increasing index and query cost, and stop at the first choice that hits the target recall. For tuples of (query bits, doc bits, rerank depth, precondition) a sensible search sequence increases query precision first, then document precision , , , , , , , , and each combined (via an outer product ) with rerank depths like and precondition , exiting as soon as the target is met.

Results: auto-selected quantization parameters and recall across datasets

In this section, we discuss the results of the initial experiments on the end-to-end behavior. We’ve made some further refinements as part of the work to fully integrate with Elasticsearch that we discuss in our other post.

The table below shows auto-selected parameters targeting recall 0.97, measured with brute-force search, so the number reflects loss due to quantization alone (64 query clusters, targeting document clusters of size 384, which matches the settings of DiskBBQ).

Dataset

Query bits

Doc bits

Precondition

Depth

Recall

FiQA E5 small

4

2

false

30

0.97

FiQA arctic

2

2

false

30

0.95

FiQA GTE

2

1

true

30

0.98

MNIST

3

1

true

30

0.99

Fashion MNIST

3

1

true

30

0.99

Quora E5 small

2

2

false

30

0.99

Quora arctic

2

1

false

30

0.97

Quora GTE

1

1

false

30

0.98

Dbpedia E5 small

4

2

false

30

0.99

Dbpedia arctic

2

1

false

30

0.94

Dbpedia GTE

2

1

false

30

0.96

Wiki Cohere

2

2

false

30

0.99

Hotpot E5 small

4

2

false

30

0.97

Hotpot GTE

2

1

false

30

0.96

Glove 100

4

2

false

30

0.87

Glove 200

4

2

false

30

0.89

SIFT128

4

4

false

20

0.99

There are a few things worth highlighting:

• The recall is very sensitive to rerank depth. This is why we nearly always end up choosing the maximum depth available: a step up in rerank depth from 20 to 30 is typically what pushes us to hit the recall target for fewer bits and we prefer fewer bits. In the real system, we tuned this behavior based on a more representative reranking cost.

• Glove underperforms partly we approximate the query distribution with random samples from the corpus, but Glove is also less well characterized by the model than the other datasets. A plausible explanation is that the approximately uniform local density assumption from Section 2 is less reliable for Glove embeddings, which would show up as higher recall variance between queries. However, Glove embeddings are not representative of the actual vectors we need to store.

• The FiQA GTE preconditioning choice is a knife-edge case: preconditioning produced only a tiny expected recall improvement, but the prediction sat right at the recall cutoff and allows us to drop the query from 3 to 2 bits. If we'd rather only keep preconditioning where its benefit is clear-cut, we can enforce a minimum uplift threshold. This sort of fine-tuning of the decision logic leaves all the heavy lifting to estimate recall unaffected.

Key takeaways: auto-tuning vector quantization from first principles

We presented a method to pick optimal quantization parameters to achieve a target recall. It rests on two models that compose cleanly:

1. A geometric model of neighbor distances that follows from a local uniform density assumption. We use this to derive the nearest-neighbor distance, the glacial scaling law of the expected distance, and the expected distance profile . We show that fitting and by a simple log-linear regression to average distances in small random samples from the corpus gives an extremely accurate predictive model.

2. A Gaussian quantization error model that is justified by the CLT. Its only parameter is an empirical variance we estimate by comparing quantized and raw vector similarities for a sample of the corpus.

Finally, we show that it is possible to feed the estimated distance model into a Poisson-binomial count of neighbors that intrude on the top- set. Applying the Lyapunov CLT the expected recall@ after reranking to depth falls out as a one-dimensional integral we evaluate by quadrature.

The outcome is an accurate () predictive model of recall as a function of the quantization parameters. Choosing quantization parameters then becomes an ordered search with a predictive model telling us if we’ve hit the recall constraint. And nicely one that also comes with a built-in argument (glacial scaling) for why the chosen parameters remain safe even when estimated from a relatively small fraction of the data.

We’ve built this entire mechanism into Elasticsearch using segment merges as an opportunity to reassess our quantization choices. Aside from the peace of mind this brings (that you’ll achieve good recall whatever vectors you throw at it), it also allows us to chose near optimal parameters from a performance perspective. This closes the loop on our original objective: near optimal performance out of the box, at least as far as quantization goes. We’re pretty excited about the advantages that model based tuning can bring to vector search and look forward to sharing other work we have in this direction in the near future.

Prerequisites

• Elasticsearch 9.x with jina-embeddings-v5-text-small inference endpoint available.

• Python 3.10+,

• Elasticsearch API key,

What is quantization?

An embedding is a list of numbers. By default, each number is a float32, which uses 4 bytes. Quantization stores each number with fewer bits, trading precision for space.

Like a JPEG, a quantized vector is a smaller, lower-fidelity copy of the original that still gets the job done.

Name

Bytes / dim

1024-d vector

Compression

`Float` (Baseline)

4

4096 B

1x

`int8`

1

1024 B

4x

`int4`

0.5

512 B

8x

`bbq`

~0.14

142 B

~29x

What is BBQ?

Better Binary Quantization (BBQ) is Elasticsearch's 1-bit quantization mode for dense vectors. Each dimension of the vector is stored as a single bit, plus a few corrective bytes per vector. Then, a rescoring step is applied at query time. This keeps the final retrieval quality close to a full precision search.

For the math behind each level, see Scalar quantization 101, Optimized Scalar Quantization, and the BBQ deep dive.

How does BBQ preserve search accuracy?

Plain 1-bit quantization leads to too high a search quality degradation on its own. BBQ maintains high retrieval quality through three mechanisms:

1. Asymmetric precision: Stored vectors use 1 bit per dimension.

2. Corrective factors: A few floats per vector record the rounding error and correct distances at scoring time.

3. Oversample and rescore: BBQ scans candidates with the bits and then reranks the top ones with higher precision. Fetching the top 10 means scanning about 30 candidates.

The result is the vectors that are roughly 32x smaller, with retrieval quality close to full precision. In the next section of the article, we’ll measure the memory savings and the recall on a real corpus.

How Jina embeddings v5 works

Jina embeddings v5 is a multilingual embedding model with quantization-aware training, which makes it a natural fit for BBQ in Elasticsearch: The 1024-dimensional vectors from jina-embeddings-v5-text-small sit above the dimensional floor where binary quantization stays accurate, and the model is trained so that 1-bit quantization loses little quality. Its main features are:

• One model for many tasks: v5 uses small Low-Rank Adaptation (LoRA) adapters on top of a single base model, one for each task: retrieval, text-matching, clustering, and classification. Elasticsearch picks the right adapter automatically at index and query time.

• Matryoshka dimensions: v5 is trained so you can truncate the vector (1024, 512 to 256) and minimize search quality reduction. This is another way to shrink vectors, independent of quantization.

• Quantization-aware training: v5 is trained to work with BBQ, so its 1-bit vectors lose little accuracy.

We use jina-embeddings-v5-text-small. This model is available through Elastic Inference Service (EIS) and outputs 1024 dimensions with 32k token context and is multilingual across 93 languages. That puts it above the 384-dimension threshold, below which Elasticsearch no longer defaults to bbq_hnsw.

Full model details are in the Jina v5 article on Search Labs.

Setting up the BBQ vs. float32 comparison

We’ll create two indices: Both share mappings, and what changes is the index_options.type parameter, which tells Elasticsearch how to store the dense vector field (as raw float32 HNSW or as 1-bit BBQ):

Index

`index_options`

Loaded into memory

`vectors-float32`

`hnsw`

Raw `float32` with no quantization (baseline)

`vectors-bbq`

`bbq_hnsw`

1-bit BBQ quantization + corrective factors

We then embed the corpus once with Jina v5, index those same vectors into both, and compare them on disk usage, memory footprint, and recall. You can follow along with the full supporting blog content notebook.

Connect to Elasticsearch

Create the two indices

Note: In production, you can use semantic_text to let Elasticsearch manage the mapping and inference endpoint automatically.

Point at the Jina v5 inference endpoint

We call the model jina-embeddings-v5-text-small directly (no need to create an inference endpoint) to turn text into vectors.

As result of the test, we got:

Load a multilingual news dataset

We stream real news articles from hotchpotch/multilingual_cc_news, a parquet mirror of CC-News. We take about 1,000 articles from five languages (around 3,000 docs total), plus a small held-out set of headlines to use as search queries. Using multiple languages also lets Jina v5 show its multilingual strength.

Generate the embeddings and bulk index

We embed the corpus a single time and feed those exact vectors into both indices.

We force-merge to a single segment so the storage numbers are stable and comparable.

Results: Disk versus memory

The disk usage API reports how many bytes each index spends on vectors (knn_vectors).

Result:

On disk, the two indices are about the same size. A quantized index still keeps the raw float32 vectors (needed for rescoring and requantization during merges) and adds the 1-bit vectors on top, so BBQ ends up slightly larger on disk.

The real savings is in memory. The HNSW scan only needs the 1-bit vectors in RAM, while the raw floats are read from disk to rescore the top candidates. We size that footprint using the documented kNN memory formulas: float uses num_vectors × dims × 4 and bbq uses num_vectors × (dims/8 + 14).

BBQ's extra bytes on disk should match the 1-bit payload we computed for memory. Here, that’s 13.25 - 12.80 = 0.45 MB versus the computed 0.44 MB. They line up.

Results: Recall

To check whether the quantized index returns results similar to the float baseline, we use recall:

recall@k = | BBQ top-k ∩ float32 top-k | / k, averaged over all queries.

We vary the oversampling factor (num_candidates / k) that’s the number of candidates BBQ scans with 1-bit vectors before reranking the top ones against the original floats to find the lowest value that still matches float32.

As a result, we have:

BBQ starts at 0.994 recall@10 at 1x oversampling, holds there up to 3x, and then settles at 0.989 at higher factors, meaning it returns at least 98.9% of the same top-10 documents as float32 across all oversampling values. For more on how recall varies across datasets under quantization, see Fast vs. accurate: Measuring the recall of quantized vector search.

BBQ quantization results summary

The same vectors, two storage formats, and one experiment:

• Disk: Roughly the same (12.80 MB versus 13.25 MB). BBQ keeps the raw floats around for rescoring and merging.

• Memory: 29x smaller (12.71 MB versus 0.44 MB). This is the number that decides whether your cluster fits the corpus.

• Recall@10: 0.994 at 1x oversampling. Quantization-aware training pays off.

When to enable BBQ: If your dimension count is above the 384-dim floor, if your vectors are the dominant memory cost, and if you can afford a few extra candidates to rescore. For Jina v5 specifically, the model is trained for it, so the recall hit on most corpora is small.

Further reading on BBQ and vector quantization

• Run the full notebook from this article in the supporting blog content repo.

• For the math behind BBQ, see Better Binary Quantization in Lucene and Elasticsearch.

• For more on Jina v5's architecture, see Jina embeddings v5 on Search Labs.

• For a broader walkthrough on adopting BBQ, see How to implement BBQ into your use case.

• For the original research behind BBQ, see the paper RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Search.

Vector search is a critical foundation for large language model (LLM) applications, retrieval augmented generation (RAG), and other AI workloads. In this benchmark, Elasticsearch achieved up to 7x higher throughput than Qdrant at comparable recall on the same storage topology. Elasticsearch as a vector database offers strong vector search performance even when network-attached persistent storage remains on the query path.

The difference reflects how the two systems interact with disk. Elasticsearch DiskBBQ is designed to keep vector search efficient when persistent storage remains on the query path, using a compact quantized representation and limiting costly access to full precision vectors during search. In this setup, Qdrant relies on a graph-based search path with rescoring against original vectors stored on disk. On network-attached persistent storage, that random access cost becomes much more significant, which is why the performance gap widens as recall increases. This benchmark therefore focuses specifically on network-attached persistent storage, a common deployment model in managed cloud and Kubernetes environments.

The key pattern in the latency curve is not only the size of the gap but also its shape. Elasticsearch latency remains comparatively flat as recall increases, suggesting that higher recall doesn’t require a dramatic increase in expensive storage activity. Qdrant’s latency rises sharply as hnsw_ef increases, which is consistent with broader candidate exploration leading to more rescoring work against original vectors on disk.

Full results table

The table below shows the full parameter sweep for both Elasticsearch and Qdrant. Because the two engines expose different tuning controls for vector search, the results are reported using each engine’s full parameter key rather than attempting a one-to-one mapping between settings.

A few notes on the metrics:

• ParamKey: The complete parameter setting used for a given run.

• Recall: Recall@100 against a ground-truth top-100 result set for the benchmark queries. Values range from 0 to 1, and higher is better.

• Latency_Avg: The average end-to-end latency per query measured from the benchmarking client across the full run, in milliseconds. Lower is better.

• Latency_P95: The 95th percentile query latency, in milliseconds, showing the upper range of typical slow queries. Lower is better.

• Throughput: The average number of queries processed per second across the full run. Higher is better.

Engine

ParamKey

Recall

Latency_Avg

Latency_P95

Throughput

qdrant

hnsw_ef=50, oversampling=1, size=100

0.8694

315.7849

503.4754

12.629

elasticsearch

k=100, oversample=1, size=100, visit_percentage=1

0.8789

135.0802

218.494

29.343

elasticsearch

k=100, oversample=1, size=100, visit_percentage=1.5

0.9123

127.8286

195.2318

31.1107

qdrant

hnsw_ef=100, oversampling=1, size=100

0.9287

895.9933

1213.0448

4.4493

elasticsearch

k=100, oversample=1, size=100, visit_percentage=2

0.9317

124.846

183.6314

31.8225

elasticsearch

k=100, oversample=1, size=100, visit_percentage=2.5

0.9444

123.517

180.4831

32.1883

qdrant

hnsw_ef=150, oversampling=1, size=100

0.9518

884.7236

1195.2603

4.5066

elasticsearch

k=100, oversample=1, size=100, visit_percentage=3

0.9532

123.276

183.8379

32.2364

elasticsearch

k=100, oversample=1, size=100, visit_percentage=3.5

0.9599

122.5559

184.2858

32.4469

qdrant

hnsw_ef=200, oversampling=1, size=100

0.964

883.2114

1188.6597

4.5143

elasticsearch

k=100, oversample=1, size=100, visit_percentage=4

0.965

122.7946

184.9058

32.3635

elasticsearch

k=100, oversample=1, size=100, visit_percentage=4.5

0.9689

122.7062

182.9559

32.3976

elasticsearch

k=100, oversample=1, size=100, visit_percentage=5

0.9722

122.5761

187.3536

32.4221

qdrant

hnsw_ef=256, oversampling=1, size=100

0.9722

881.9643

1185.4948

4.5192

elasticsearch

k=100, oversample=1, size=100, visit_percentage=5.5

0.9747

122.5609

184.5128

32.4176

Each row pairs the closest measured Elasticsearch and Qdrant configurations in the sweep by achieved recall.

Matched comparisons at similar recall

To make the comparison fair, speedup is calculated only between configurations that achieve similar recall. This avoids comparing settings that trade off accuracy very differently.

Recall band

Elasticsearch recall

Elasticsearch Latency_Avg

Elasticsearch throughput

Qdrant recall

Qdrant Latency_Avg

Qdrant throughput

Throughput speedup

~0.87

0.8789

135.0802

29.343

0.8694

315.7849

12.629

2.32x

~0.93

0.9317

124.846

31.8225

0.9287

895.9933

4.4493

7.15x

~0.95

0.9532

123.276

32.2364

0.9518

884.7236

4.5066

7.15x

~0.96

0.9599

122.5559

32.4469

0.964

883.2114

4.5143

7.19x

~0.97

0.9722

122.5761

32.4221

0.9722

881.9643

4.5192

7.17x

This matched-recall view is the clearest expression of the underlying systems difference. At similar recall levels, Elasticsearch delivers both lower latency and much higher throughput, and the gap widens as recall rises. The recall-throughput pattern matters because higher recall in this benchmark requires broader search. DiskBBQ absorbs that increase with relatively little additional cost, while Qdrant’s graph plus rescoring path becomes much more constrained by random access to original vectors on persistent storage.

Benchmark methodology

Jingra, the benchmarking tool used for these tests, was originally written in Python and has since been rebuilt as a Java project. For these tests, Jingra runs in a Kubernetes pod within the same cluster as the engine being measured. This helps reduce external network variability and keeps the test environment consistent across runs. For each run, Jingra executed the query set at a fixed client concurrency, recorded end-to-end client-side latency and throughput, and computed recall against a precomputed ground-truth top-100 set.

This benchmark was intentionally run on network-attached persistent storage rather than local NVMe. For the published results, the storage used the baseline performance allocation for a 200 GiB GCP Hyperdisk Balanced volume, with no explicit IOPS or throughput provisioning. We chose this topology on purpose because it’s a relevant cloud deployment model and because it keeps storage efficiency materially on the query path.

Qdrant often performs better on local NVMe, so deployments using local NVMe should expect different results than the ones shown here. This benchmark specifically tests network-attached persistent storage because that’s a common managed-cloud deployment model and because it makes storage-path efficiency visible in end-to-end query performance.

Because Elasticsearch and Qdrant expose different query parameters for controlling vector search behavior, there’s no clean one-to-one mapping between their tuning settings. Instead of comparing equivalent parameter values directly, we use recall as the primary point of comparison. The matched comparisons below therefore pair configurations that achieve similar recall, rather than configurations with superficially similar parameter values.

Recall cannot be known in advance for a given parameter setting, so we sweep across a range of search configurations for each engine and then compare results at similar recall levels. In the published results, oversampling was fixed at 1 for both engines so that recall was primarily tuned via search breadth rather than rescoring expansion.

How does Elasticsearch configure vector search?

• query_vector: The input vector used for similarity search. Elasticsearch compares this vector against the stored vectors in the field.

• k: The number of nearest neighbors to retrieve.

• visit_percentage: Controls how much of the DiskBBQ, Elasticsearch’s disk optimized vector index, is explored during the approximate search phase. Higher values usually improve recall but increase latency.

• oversample: Controls how many extra candidate vectors are passed into rescoring relative to k. Higher values can improve recall, but usually at additional cost.

• size: The number of hits returned in the final response.

• _source: false: Disables returning the document _source field, reducing response size and avoiding extra retrieval overhead during benchmarking.

Example

Params

We keep k = size = 100 so the search request is aligned with the benchmark target: returning the top 100 results. To improve recall, we tune visit_percentage rather than inflating the final result count, while keeping oversample = 1 fixed across runs.

How does Qdrant configure vector search?

• query_vector / vector: The input vector used for similarity search. Qdrant compares this vector against the stored vectors in the collection.

• size / limit: The number of nearest neighbor results returned in the response.

• with_payload: false: Disables returning payload fields, reducing response size and avoiding additional retrieval overhead during benchmarking.

• with_vector: false: Disables returning stored vectors in the response, again reducing response size and keeping the benchmark focused on search performance.

• hnsw_ef: Controls the number of candidates explored during HNSW search. Higher values usually improve recall but increase latency. Like visit_percentage in Elasticsearch, it affects search breadth, but the two controls are engine-specific and not directly equivalent.

• quantization.rescore: true: Enables rescoring of the candidate set using the original vectors after quantized search.

• oversampling: Controls how many extra candidates are considered during rescoring relative to the final result count. Higher values can improve recall, but usually at additional cost.

Example

Params

We keep size = 100 so that each request is aligned with the evaluation target, in this case top 100 retrieval. Recall is then tuned by sweeping hnsw_ef, which controls how many candidates are explored during search. Higher hnsw_ef values generally improve recall but also increase latency and reduce throughput. We keep oversampling = 1 fixed across runs so that the main tuning variable is the search breadth rather than the rescoring expansion.

Cluster setup and DiskBBQ configuration

We ran the benchmark on GCP using three n4-standard-8 nodes, with each pod allocated 7 vCPUs and 26 GB of RAM, and using 200 GiB GCP Hyperdisk Balanced volumes at baseline performance allocation. The corpus contains 21 million vectors, (see dataset section below for more details and download links), which account for about 60.1 GiB of raw float vector data. With 2-bit quantization, the vector payload drops to roughly 3.8 to 4.0 GB. However, the full index footprint is much larger once graph and other index structures are included. That means the workload remains meaningfully sensitive to network-attached storage performance, especially because exact vector values still need to be read from disk during rescoring.

We chose this node size intentionally to keep the benchmark in a regime where network-attached persistent storage remains on the query path rather than allowing the full working set to remain comfortably memory-resident. Each system was therefore configured using the best-performing setup we identified for this workload within the tuning scope described in this post. In Elasticsearch, this meant bbq_disk. In Qdrant, the original vectors were stored on disk, while the 2-bit quantized representation used for approximate search was kept in RAM with always_ram: true. Because the two systems expose different search strategies and tuning controls, we compare them at matched recall rather than trying to map parameters one to one.

Elasticsearch was configured to use DiskBBQ, its disk-optimized approach for approximate nearest neighbor vector search, with 2-bit quantization. DiskBBQ uses aggressive quantization to keep the searchable index compact and then rescores with the original vectors to preserve accuracy. This helps maintain strong recall while keeping disk-based search efficient.

bbq_disk is an Elasticsearch Enterprise feature. We used it here because the goal of this benchmark was to compare the strongest disk-oriented vector search configuration available in each engine for this workload, rather than licensing tiers or default features.

We didn’t include bbq_hnsw in this comparison because the benchmark was specifically designed to evaluate disk-oriented vector search under a disk-sensitive workload.

This storage topology matters because Qdrant’s rescore step reads the original float32 vectors from disk with random access on each query. On local NVMe, those reads are much faster, and Qdrant correspondingly performs better. On network-attached persistent storage, the results are consistent with that random-read rescore path becoming a more important bottleneck. Qdrant latency rises sharply as hnsw_ef increases, while Elasticsearch remains comparatively flat across the same recall progression.

We chose 2-bit quantization because Qdrant couldn’t reach the target recall range with 1-bit binary quantization. Since the two systems expose different disk-oriented vector search strategies, we tuned each one to the strongest configuration available within its current feature set.

Both systems were configured with three shards distributed across the three nodes and with two total copies of each shard in the cluster. In Elasticsearch, number_of_shards: 3 and number_of_replicas: 1 means one primary plus one replica, for two total copies. In Qdrant, shard_number: 3 and replication_factor: 2 also means two total copies, since Qdrant’s replication factor refers to the total number of copies rather than the number of additional replicas. So although the field names differ, the effective replication level was the same in both systems.

Setting

Elasticsearch

Qdrant

Shards

number_of_shards: 3

shard_number: 3

Copies

number_of_replicas: 1 (1 primary + 1 replica = 2 total)

replication_factor: 2 (2 total)

Elasticsearch mapping

Qdrant mapping

Dataset

For this benchmark, we used the kenhktsui/wiki_dpr_e5 dataset from Hugging Face, a large-scale Wikipedia passage retrieval dataset designed for dense vector search. The corpus contains 21 million embedded passages, each represented as a 768-dimensional float32 vector, or 3,072 bytes per vector. That corresponds to about 60.1 GiB of raw vector data, before accounting for additional fields and file format overhead in the source dataset. The downloadable data.parquet file is larger at 85.2 GB for that reason.

We chose this dataset because it reflects a common production pattern in LLM, RAG, and retrieval systems: searching a large corpus of semantically embedded text while balancing recall, latency, and throughput. At 21 million vectors and roughly 60 GiB of raw vector data, it’s large enough to make disk-based vector search a relevant operating mode to evaluate.

Both engines used 2-bit quantization, reducing each vector from 3,072 bytes to 192 bytes, a 16x reduction that brings the quantized vector corpus to around 4 GB. In Qdrant, that quantized representation was kept in RAM for search, while the original vectors remained on disk. Even so, the workload remained meaningfully sensitive to network-attached storage performance because rescoring still required access to the original vectors on disk.

You can download the dataset and query files from the links below:

• data.parquet

• queries.parquet

Jingra and recreating the benchmark

For this benchmark, we used Jingra v0.2.3 with the configurations described es-9.4-vs-qd-1.18-vector-search. Jingra handled data loading, query execution, parameter sweeps, and metric collection for both Elasticsearch and Qdrant, making the benchmark repeatable and easier to compare.

To reproduce the experiment, you need the published dataset, query set, engine configurations, and comparable cluster hardware. With those in place, Jingra can rerun the benchmark and generate similar recall, latency, and throughput measurements shown in this post.

Conclusion

At comparable recall levels, Elasticsearch DiskBBQ consistently delivered faster vector search than Qdrant in this benchmark, with higher throughput and lower latency across the recall range we tested. These results are especially notable because the comparison was made on network-attached persistent storage, where efficient storage-aware vector search becomes critical. Elasticsearch as a vector database allows organizations to achieve high recall with lower latency and higher throughput on slower persistent storage.

Just as importantly, this benchmark highlights the value of comparing engines at matched recall rather than by nominal parameter settings. Elasticsearch and Qdrant expose different controls, so the fairest comparison isn’t parameter to parameter but outcome to outcome. Across the recall range tested here, Elasticsearch maintained a clear advantage in both latency and throughput.

If you want to reproduce the experiment yourself, we’re publishing the dataset and query set used in this benchmark so others can validate the results and build on them.

Further reading:

• Introducing a new vector storage format: DiskBBQ

• Elasticsearch BBQ vs TurboQuant

The datafeed isn’t broken. It’s doing exactly what it was built to do: reading every raw document, across every shard, every run. On a large cluster with cross-cluster search (CCS) and a broad index pattern, like logs-*, that means scanning billions of documents per bucket. There’s no hardware that makes that sustainable. The datafeed will always be chasing live data and never reaching it.

The fix is to switch from the default scroll-based datafeed configuration to an aggregation-based datafeed configuration: Let the data nodes summarize locally, and ship only compact bucket results to the ML node. Same detections, a fraction of the load. The speedup can be dramatic. More than you might expect. The numbers are in the next section. The explanation for why the gap is so large is at the end of the post, for those who want to understand the mechanics.

One catch worth knowing now: Switching requires creating a new job. The old model doesn’t transfer; weeks of learned baseline are lost. The right time to make this switch is before the job has been running for months, not after. That’s the main reason to read this before you deploy.

How much faster? Scroll vs. aggregation datafeeds for ML jobs

We ran the same job two ways on production data: first scroll-based, and then aggregation-based. The job covered 13 months of history, monitoring 836,000 log events per hour in 15-minute buckets across multiple clusters.

Training on historical data with scroll-based configuration: five days of wall-clock time, 7.9 million sequential requests, and 3.5 TB transferred; with aggregations: 2.3 minutes, 23 requests, and 34 MB (a 3,374× speedup). Think of it this way: If you start the scroll backfill at 9 a.m. Monday, it will finish Saturday morning. The aggregation version is done by 9:02 a.m.

On live data, the difference is less dramatic but still meaningful: around 20× fewer requests per tick. That adds up quickly when the datafeed runs every few minutes around the clock.

Before you start

Three things worth knowing before diving into the configuration.

This isn't wizard territory. The standard Kibana job wizards (Single Metric, Multi-Metric, Population) don't expose aggregation configuration. To create an aggregation-based job, you need either the Elasticsearch API or Kibana's Advanced Job Wizard, with JSON edited by hand. The worked example below shows the most practical path: Configure the job in the Multi-Metric Wizard, and then click Convert to advanced job before creating it. That gets you a prefilled JSON starting point instead of a blank editor.

The configuration is unforgiving and mostly silent about it. There's no schema validation that catches a misnamed aggregation key or a fixed_interval that doesn't match bucket_span. The job will run, anomalies will fire, and nothing will indicate that the results are based on the wrong data. This is why the five-step pattern exists and why the Preview tab is worth using every time: Catching a misconfiguration before the job trains is a 30-second check; catching it a week later is a much worse afternoon.

The Single Metric Viewer has a known limitation with aggregated jobs. That viewer reconstructs the "actual" data curve by re-querying the index, but it can't reproduce an arbitrary, user-defined aggregation, so the actual-value line is typically missing or approximate. The Anomaly Explorer is unaffected: Anomaly scores, swim lanes, and influencer attribution all work normally. Just don't rely on the Single Metric Viewer's chart for visual validation of what the model saw.

What we can and can’t aggregate

Almost every ML function works with aggregated datafeeds, but the right aggregation pattern depends on the function.

Function

Pattern

`count`, `mean`, `high_mean`, `low_mean`, `sum`, `max`, `min`

Standard: `date_histogram` → `terms` → metric aggregation

`time_of_day`, `time_of_week`

Minimal: plain `date_histogram`, no `terms` or metric needed

`rare`, `freq_rare`, `info_content`

Composite: top-level composite with `date_histogram` as a source

`categorization`

`terms` on the `.keyword` subfield of the categorization field

`lat_long`, `varp`

Scroll only

lat_long and varp are the genuine exceptions. If you want to use these detectors, you are required to use the scroll-based datafeed configuration.

The five-step pattern in the next section covers the standard case. We’ll walk through the remaining patterns at the end of the post.

The standard five-step pattern: Scroll-based to aggregation datafeed

Converting any scroll-based job to an aggregation-based datafeed follows the same five steps. Once you understand the pattern, applying it to any compatible job takes about 10 minutes.

Step 1: Add summary_count_field_name: "doc_count" to the analysis config. This tells the ML engine that incoming data is pre-summarized. Without it, the engine treats each aggregated bucket as a single raw document and produces wrong anomaly scores.

Step 2: Choose the bucket wrapper topology. For most functions (count, mean, sum, max, min, varp, time_of_day, time_of_week, and categorization) use a date_histogram at the top level whose fixed_interval matches your bucket_span exactly to ensure accurate analysis. For rare, freq_rare, and info_content, use a composite at the top level with a date_histogram as one of its sources. This routes the datafeed to the composite extractor, which paginates through all field-value combinations rather than truncating to a top-N.

Step 3: Add a max aggregation on @timestamp. The ML engine needs this to determine the precise end time of each bucket. In the standard topology (Step 2, date_histogram outer), it goes inside the histogram’s aggregations. In the composite topology, it sits as a sibling of the composite aggregation.

Step 4: Map each analysis field to a terms aggregation, named exactly after the corresponding field in the analysis config. One categorical field → a single nested terms. Two or more categorical fields → a composite aggregation nested inside the date_histogram, with one terms source per field. For categorization jobs, use a terms aggregation on the .keyword subfield of the categorization_field_name. The naming rule is strict: The aggregation key must exactly match the field name in the analysis config; the ML engine uses the aggregation name, not the field parameter, to look up values. A mismatch produces silently wrong results; no error, just a job that appears to run while missing everything meaningful.

Step 5: Map each detector’s metric field to its Elasticsearch aggregation equivalent:

ML function

Elasticsearch aggregation

`mean` / `high_mean` / `low_mean`

`avg`

`sum`

`sum`

`max`

`max`

`min`

`min`

For count, rare, freq_rare, info_content, time_of_day, time_of_week, and categorization jobs, the ML engine works from doc_count alone; no metric aggregation is needed, and this step can be skipped.

Step-by-step example: Building an aggregation-based ML job in Kibana

Let’s build this end to end using Kibana’s sample web logs. If you haven’t loaded them yet, go to the Kibana home page and click Integrations → Sample data → Sample web logs → Add data. This gives us a data view called Kibana Sample Data Logs and an index called kibana_sample_data_logs with fields including @timestamp, bytes (response size), and geo.dest (destination country).

We’ll build a job that detects unusually large response sizes: high_mean of bytes, partitioned by destination country (geo.dest), with a 1-hour bucket span.

Creating the job with the Multi-Metric Wizard

This is how most jobs get created in practice. Navigate to Machine Learning → Anomaly Detection → Manage Jobs → Create job.

Select the “Kibana Sample Data Logs” data view, and set the time range to cover the full sample dataset. On the job type screen, choose Multi-metric.

In the Multi-Metric Wizard, configure the detector:

• High mean of bytes.

• Split data by geo.dest.

• Bucket span: 1h.

Give the job an ID, and leave everything else at its defaults, but don’t click Create yet. On this last configuration step, click on Preview JSON and look at the datafeed section. What you’ll see is a plain scroll-based datafeed with no aggregations, just an index pattern and a match_all query.

This is the default every wizard produces. On a small cluster, it works fine. On a large cluster with CCS and a broad index pattern, this datafeed will scan every raw document on every run and never catch up with live data.

Instead of clicking Create, click Convert to advanced job. This keeps everything you just configured (the detector, the partition field, the bucket span) and drops you directly into the Advanced Wizard, where we can apply the five-step pattern.

Analysis configuration

The conversion prefills the detector, partition field, and bucket span. The only change needed here is Step 1 of the pattern: Open the Edit JSON view, and add summary_count_field_name to tell the ML engine that incoming data will be pre-summarized:

Datafeed configuration

Switch to the Datafeed tab. This is where Steps 2 through 5 of the pattern come together. Remove scroll_size if it’s present, and then enter the aggregations:

A few notes on this config:

• Step 2: The date_histogram uses fixed_interval: "1h", matching bucket_span exactly. A mismatch produces incorrect bucket timing.

• Step 3: The max aggregation on @timestamp must be named @timestamp and placed inside the histogram’s aggregations; without it, the ML node can’t determine the precise end of each bucket.

• Step 4: The terms aggregation for the partition field must be named exactly after the partition field: geo.dest, not geo.dest_grouping or any alias. The ML engine uses the aggregation name, not the field parameter, to identify which partition value each bucket belongs to. A mismatch silently drops the partition field from results entirely.

• Step 5: The metric aggregation key bytes matches field_name in the detector exactly. Any mismatch here produces silently wrong anomaly scores.

Validate with the preview

Before we create the job, let’s use the Preview tab. This runs the aggregation against real data and shows exactly what the ML node will receive, a very useful sanity check before committing.

Three things to verify in the preview output: doc_count should be present on every bucket and greater than 1. The bytes values should look like average response sizes: numbers in the hundreds to hundreds of thousands for web traffic. And each row should correspond to a distinct (timestamp, geo.dest) pair. If anything looks off, fix it in the JSON editor and rerun the preview.

Adding influencer fields

In the example above, geo.dest is the partition field. The ML model learns a separate baseline for each destination country, and anomalies are reported per country. But you might also want machine.os to appear as an influencer in anomaly results: When the detector fires, you want to see “this looks anomalous for geo.dest: CN and machine.os: win is a contributing factor.” Influencers don’t drive anomaly detection; they provide context for the anomalies that are found.

To support an influencer alongside a partition field, the analysis config gains an influencers array:

And now the datafeed needs to aggregate on both fields simultaneously. One terms nested inside another terms won’t work; a nested terms surfaces only the top-N values of the inner field per outer bucket, so you’d silently lose combinations. Instead, use a composite aggregation with one terms source per field, next to the date_histogram:

composite generates one bucket per unique (geo.dest, machine.os) combination. The ML node sees every pair and can correctly attribute which operating system was contributing when a country’s response sizes spiked. Use the preview to confirm distinct pairs appear. If you only see a handful of rows where you’d expect many, the size parameter on the composite may need to be raised.

Categorization

Categorization works with aggregated datafeeds: summary_count_field_name and categorization_field_name can coexist in the same job. The five-step pattern applies directly. Step 2 uses the standard date_histogram topology. Step 4 has one adjustment: Instead of a partition field, we aggregate the text field itself using a terms aggregation on its .keyword subfield, named to match categorization_field_name exactly. Step 5 is skipped. The count detector works from doc_count alone. Analysis config:

Datafeed aggregations:

The datafeed sends one bucket per unique message.keyword value with a doc_count for each. The ML node receives those strings, runs categorization on them, assigning an mlcategory to each, and the count detector tracks how many documents fall into each category per bucket. The naming rule from Step 4 applies: The terms aggregation must be named message, matching categorization_field_name in the analysis config exactly.

One thing to watch: Keyword fields have a default ignore_above: 256 limit. Log messages longer than 256 characters won’t be indexed as .keyword and will be silently excluded from the aggregation. If your log messages are long, check the field mapping before using this approach. You may need to raise the limit in your index template.

The minimal pattern for time_of_day and time_of_week

time_of_day and time_of_week are the easiest functions to aggregate: They only need a timestamp and a document count. The C++ process extracts the time component from the bucket timestamp and builds a cyclical model of normal activity; doc_count tells it how many events fell in each bucket. No terms sources, no metric aggregation, no composite. Analysis config:

Datafeed aggregations:

A plain date_histogram is enough; no composite needed. This makes time_of_day and time_of_week particularly CCS-friendly: one request per time chunk, minimal data over the wire. Use the same structure for time_of_week; only the function name changes.

If you want to add a partition_field_name (for example, to model time-of-day patterns per service), add a terms aggregation inside the histogram’s aggregations following the standard Step 4 pattern.

The composite pattern for rare, freq_rare, and info_content

rare, freq_rare, and info_content all need the composite extractor, the one that paginates through all unique value combinations rather than truncating to top-N. The five-step pattern applies here with a different topology in Step 2: composite goes at the top level (not date_histogram), with date_histogram as a source inside it. Step 3 places the max @timestamp aggregation as a sibling of the composite, and Step 5 is skipped since all three functions work from doc_count alone.

The datafeed structure is the same for all three functions: a composite at the top level, a date_histogram as one of its sources, and one terms source per analysis field. The only thing that varies is which fields you include as terms sources: rare needs one source for by_field_name; freq_rare needs sources for both by_field_name and over_field_name; info_content needs a source for field_name plus any by_field_name or over_field_name fields. None of the three require a metric aggregation.

A few notes:

• The composite aggregation must be the top-level aggregation, not nested inside a date_histogram. This is what routes the datafeed to the composite extractor.

• The date_histogram is a source inside the composite, not the outer wrapper. Its fixed_interval must divide evenly into bucket_span.

• The max aggregation on @timestamp sits as a sibling of the composite (inside aggregations), not nested inside it.

• composite.size controls the page size per round trip. Setting it high (10000) reduces round trips, which matters with CCS latency. With three sources and high-cardinality fields, the total combination count can be large; the extractor paginates automatically.

Why aggregation-based datafeeds outperform scroll at scale

The gap is structural, not incidental. A scroll-based datafeed reads raw documents one page at a time: Every 1,000 documents is one request, and each waits for the previous one to complete before issuing the next. The number of requests is therefore proportional to the total document count in the time range being backfilled. At 836,000 events per hour over 13 months, that's roughly 7.9 billion events, or 7.9 million sequential round trips. Each round trip crosses the CCS boundary, waits for shard responses, and transfers matching documents in full. There’s no parallelism: The datafeed holds a scroll context open on the remote cluster and processes one page at a time.

An aggregation-based datafeed works differently. The data nodes summarize data locally, grouping by time bucket and categorical fields, and ship only the bucket results to the ML node. The number of requests is proportional to field cardinalities, not document count. In our example, two influencer fields with six unique combinations produce six result rows per time bucket; the datafeed pages through those in a handful of requests regardless of how many raw events fall in each bucket. Double the ingestion rate and the scroll request count doubles; the aggregation request count stays the same. This is why the gap widens at scale: The more data you have, the worse scroll looks by comparison, and the better aggregations look.

On live data, the picture is different because each real-time tick covers only one fresh bucket: Scroll issues however many pages fit in that bucket's worth of data, while aggregations issue one request. The 20× figure for live data reflects that ratio at 836,000 events per hour with a 15-minute bucket span. The practical threshold where aggregations stop being optional is when (ingestion rate × bucket span) > scroll_size; once a single bucket contains more than one scroll page of documents, the datafeed can't keep pace with live data regardless of hardware. Below that threshold, scroll is fine and aggregations are a nice-to-have. Above it, aggregations are the only sustainable option.

Scroll-based datafeeds are the right default, and the wizards make the right call for most deployments. At scale (more shards, broader index patterns, CCS across tiers), switching to an aggregation-based datafeed is the natural next step: The data nodes summarize where the data lives, the ML node processes compact results, and the detections stay the same. The one cost to know up front is model state: Switching requires a new job, so the earlier you make the move, the less you give up.

If you hit a case not covered here, an aggregation type that doesn’t map cleanly or a composite that behaves unexpectedly, the Elastic Discuss forums are a good place to continue.

Our goal was to transition to an automated, adaptive approach capable of handling both log parsing (field extraction) and log partitioning (source identification). We hypothesized that Large Language Models (LLMs), with their inherent understanding of code syntax and semantic patterns, could automate these tasks with minimal human intervention.

We are happy to announce that this feature is already available in Streams!

Dataset description

We chose a Loghubcollection of logs for PoC purposes. For our investigation, we selected representative samples from the following key areas:

• Distributed systems: We used the HDFS (Hadoop Distributed File System) and Spark datasets. These contain a mix of info, debug, and error messages typical of big data platforms.

• Server & web applications: Logs from Apache web servers and OpenSSH provided a valuable source of access, error, and security-relevant events. These are critical for monitoring web traffic and detecting potential threats.

• Operating systems: We included logs from Linux and Windows. These datasets represent the common, semi-structured system-level events that operations teams encounter daily.

• Mobile systems: To ensure our model could handle logs from mobile environments, we included the Android dataset. These logs are often verbose and capture a wide range of application and system-level activities on mobile devices.

• Supercomputers: To test performance on high-performance computing (HPC) environments, we incorporated the BGL (Blue Gene/L) dataset, which features highly structured logs with specific domain terminology.

A key advantage of the Loghub collection is that the logs are largely unsanitized and unlabeled, mirroring a noisy live production environment with microservice architecture.

Log examples:

In addition, we created a Kubernetes cluster with a typical web application + database set up to mine extra logs in the most common domain.

Example of common log fields: timestamp, log level (INFO, WARN, ERROR), source, message.

Few-shot log parsing with an LLM

Our first set of experiments focused on a fundamental question: Can an LLM reliably identify key fields and generate consistent parsing rules to extract them?

We asked a model to analyse raw log samples and generate log parsing rules in regular expression (regex) and Grok formats. Our results showed that this approach has a lot of potential, but also significant implementation challenges.

High confidence & context awareness

Initial results were promising. The LLM demonstrated a strong ability to generate parsing rules that matched the provided few-shot examples with high confidence. Besides simple pattern matching, the model showed a capacity for log understanding —it could correctly identify and name the log source (e.g., health tracking app, Nginx web app, Mongo database).

The "Goldilocks" dilemma of input samples

Our experiments quickly surfaced a significant lack of robustness because of extreme sensitivity to the input sample. The model's performance fluctuates wildly based on the specific log examples included in the prompt. We observed a log similarity problem where the log sample needs to include just diverse enough logs:

• Too homogeneous (overfitting): If the input logs are too similar, the LLM tends to overspecify. It treats variable data—such as specific Java class names in a stack trace—as static parts of the template. This results in brittle rules that cover a tiny ratio of logs and extract unusable fields.

• Too heterogeneous (confusion): Conversely, if the sample contains significant formatting variance—or worse, "trash logs" like progress bars, memory tables, or ASCII art—the model struggles to find a common denominator. It often resorts to generating complex, broken regexes or lazily over-generalizing the entire line into a single message blob field.

The context window constraint

We also encountered a context window bottleneck. When input logs were long, heterogeneous, or rich in extractable fields, the model's output often deteriorated, becoming "messy" or too long to fit into the output context window. Naturally, chunking helps in this case. By splitting logs using character-based and entity-based delimiters, we could help the model focus on extracting the main fields without being overwhelmed by noise.

The consistency & standardization gap

Even when the model successfully generated rules, we noted slight inconsistencies:

• Service naming variations: The model proposes different names for the same entity (e.g., labeling the source as "Spark," "Apache Spark," and "Spark Log Analytics" in different runs).

• Field naming variations: Field names lacked standardization (e.g., id vs. service.id vs. device.id). We normalized names using a standardized Elastic field naming.

• Resolution variance: The resolution of the field extraction varied depending on how similar the input logs were to one another.

Log format fingerprint

To address the challenge of log similarity, we introduce a high-performance heuristic: log format fingerprint (LFF).

Instead of feeding raw, noisy logs directly into an LLM, we first apply a deterministic transformation to reveal the underlying structure of each message. This pre-processing step abstracts away variable data, generating a simplified "fingerprint" that allows us to group related logs.

The mapping logic is simple to ensure speed and consistency:

1. Digit abstraction: Any sequence of digits (0-9) is replaced by a single ‘0’.

2. Text abstraction: Any sequence of alphabetical characters with whitespace is replaced by a single ‘a’.

3. Whitespace normalization: All sequences of whitespace (spaces, tabs, newlines) are collapsed into a single space.

4. Symbol preservation: Punctuation and special characters (e.g., :, [, ], /) are preserved, as they are often the strongest indicators of log structure.

We introduce the log mapping approach. The basic mapping patterns include the following:

• Digits 0-9 of any length -> to ‘0.’

• Text (alphabetical characters with spaces) of any length -> to ‘a’.

• White spaces, tabs, and new lines -> to a single space.

Let's look at an example of how this mapping allows us to transform the logs.

As a result, we obtain the following log masks:

Notice the fingerprints of the first two logs. Despite different timestamps, source classes, and message content, their prefixes (0/0/0 0:0:0 a a.a:) are identical. This structural alignment allows us to automatically bucket these logs into the same cluster.

The third log, however, produces a completely divergent fingerprint (0-0-0...). This allows us to algorithmically separate it from the first group before we ever invoke an LLM.

Bonus part: Instant implementation with ES|QL

It’s as easy as passing this query in Discover.

Query breakdown:

FROM loghub: Targets our index containing the raw log data.

EVAL pattern = …: The core mapping logic. We chain REPLACE functions to perform the abstraction (e.g., digits to '0', text to 'a', etc.) and save the result in a “pattern” field.

STATS [column1 =] expression1, … BY SUBSTRING(pattern, 0, 15):

This is a clustering step. We group logs that share the first 15 characters of their pattern and create aggregated fields such as total log count per group, list of log datasources, pattern prefix, 3 log examples

SORT total_count DESC | LIMIT 100 : Surfaces the top 100 most frequent log patterns

The query results on LogHub are displayed below:

As demonstrated in the visualization, this “LLM-free” approach partitions logs with high accuracy. It successfully clustered 10 out of 16 data sources (based on LogHub labels) completely (>90%) and achieved majority clustering in 13 out of 16 sources (>60%) —all without requiring additional cleaning, preprocessing, or fine-tuning.

Log format fingerprint offers a pragmatic, high-impact alternative and addition to sophisticated ML solutions like log pattern analysis. It provides immediate insights into log relationships and effectively manages large log clusters.

• Versatility as a primitive

Thanks to ES|QL implementation, LFF serves both as a standalone tool for fast data diagnostics/visualisations, and as a building block in log analysis pipelines for high-volume use cases.

• Flexibility

LFF is easy to customize and extend to capture specific patterns, i.e. hexadecimal numbers and IP addresses.

• Deterministic stability

Unlike ML-based clustering algorithms, LFF logic is straightforward and deterministic. New incoming logs do not retroactively affect existing log clusters.

• Performance and mMemory

It requires minimal memory, no training or GPU making it ideal for real-time high-throughput environments.

Combining log format fingerprint with an LLM

To validate the proposed hybrid architecture, each experiment contained a random 20% subset of the logs from each data source. This constraint simulates a real-world production environment where logs are processed in batches rather than as a monolithic historical dump.

The objective was to demonstrate that LFF acts as an effective compression layer. We aimed to prove that high-coverage parsing rules could be generated from small, curated samples and successfully generalized to the entire dataset.

Execution pipeline

We implemented a multi-stage pipeline that filters, clusters, and applies stratified sampling to the data before it reaches the LLM.

1. Two-stage hierarchical clustering

• Subclasses (exact match): Logs are aggregated by identical fingerprints. Every log in one subclass shares the exact same format structure.

• Outlier cleaning. We discard any subclasses that represent less than 5% of the total log volume. This ensures the LLM focuses on the dominant signal and won’t be sidetracked by noise or malformed logs.

• Metaclasses (prefix match): Remaining subclasses are grouped into Metaclasses by the first N characters of the format fingerprint match. This grouping strategy effectively splits lexically similar formats under a single umbrella.We chose N=5 for Log parsing and N=15 for Log partitioning when data sources are unknown.

2. Stratified sampling. Once the hierarchical tree is built, we construct the log sample for the LLM. The strategic goal is to maximize variance coverage while minimizing token usage.

• We select representative logs from each valid subclass within the broader metaclass.

• To manage an edge case of too numerous subclasses, we apply random down-sampling to fit the target window size.

3. Rule generation Finally, we prompt the LLM to generate a regex parsing rule that fits all logs in the provided sample for each Metaclass. For our PoC, we used the GPT-4o mini model.

Experimental results & observations

We achieved 94% parsing accuracy and 91% partitioning accuracy on the Loghub dataset.

The confusion matrix above illustrates log partitioning results. The vertical axis represents the actual data sources, and the horizontal axis represents the predicted data sources. The heatmap intensity corresponds to log volume, with lighter tiles indicating a higher count. The diagonal alignment demonstrates the model's high fidelity in source attribution, with minimal scattering.

Our performance benchmarks insights:

• Optimal baseline: a context window of 30–40 log samples per category proved to be the "sweet spot," consistently producing robust parsing with both Regex and Grok patterns.

• Input minimisation: we pushed the input size to 10 logs per category for Regex patterns and observed only 2% drop in parsing performance, confirming that diversity-based sampling is more critical than raw volume.

Filters vs facets

Before we begin, let's define what filters and facets are. Filters are predefined attributes used to restrict a set of results. In a marketplace, for example, filters are available even before a search is performed. The user can select a category, such as "Video games", before searching for "PS5", refining the search to a more specific subset instead of the entire database. This significantly increases the chances of obtaining more relevant results.

Facets work similarly to filters but are only available after the search is performed. In other words, the search returns results, and based on them, a new list of refinement options is generated. For example, when searching for a PS5 console, facets such as storage capacity, shipping cost, and color may be displayed to help users choose the ideal product.

Now that we have defined filters and facets, let's discuss the impact of the classical and Machine Learning (ML)-based approaches on their implementation and usage. Each method has advantages and challenges that influence search efficiency.

Classical approach to filters and facets

In this approach, filters and facets are manually defined based on predefined rules. This means that the attributes available for refining the search are fixed and planned in advance, considering the catalog structure and user needs.

For example, in a marketplace, categories such as "Electronics" or "Fashion" may have specific filters like brand, format and price range. These rules are created statically, ensuring consistency in the search experience but requiring manual adjustments whenever new products or categories emerge.

Although this approach provides predictability and control over the displayed filters and facets, it can be limited when new trends arise that demand dynamic refinement.

Pros:

• Predictability and control: Since filters and facets are manually defined, management becomes easier.

• Low complexity: No need to train models.

• Ease of maintenance: As rules are predefined, adjustments and corrections can be made quickly.

Cons:

• Reindexing required for new filters: Whenever a new attribute needs to be used as a filter, the entire dataset must be reindexed to ensure that documents contain this information.

• Lack of dynamic adaptation: Filters are static and do not automatically adjust to changes in user behavior.

Implementation of filters/facets – Classical approach

In Dev Tools, Kibana, we will create a demonstration of filters/facets using the classical approach.

First, we define the mapping to structure the index:

The brand and storage fields are set as keyword, allowing them to be used directly in aggregations (facets). The price field is of type float, enabling the creation of price ranges.

In the next step, the product data will be indexed:

Now, let's retrieve classic facets by grouping the results by brand, storage, and price range. In the query, size:0 was defined. In this scenario, the goal is to retrieve only the aggregation results without including the documents corresponding to the query.

The response will include counts for Brand, Storage, and Price, helping to create filters and facets.

Machine learning/AI-based approach to filters and facets

In this approach, Machine Learning (ML) models, including Artificial Intelligence (AI) techniques, analyze data attributes to generate relevant filters and facets. Instead of relying on predefined rules, ML/AI leverages indexed data characteristics. This enables the dynamic discovery of new facets and filters.

Pros:

• Automatic updates: New filters and facets are generated automatically, without the need for manual adjustments.

• Discovery of new attributes: It can identify previously unconsidered data characteristics as filters, enriching the search experience.

• Reduced manual effort: The team does not need to constantly define and update filtering rules as AI learns from available data.

Cons:

• Maintenance complexity: The use of models may require pre-validation to ensure the consistency of the generated filters.

• Requires ML and AI expertise: The solution demands qualified professionals to fine-tune and monitor model performance.

• Risk of irrelevant filters: If the model is not well-calibrated, it may generate facets that are not useful for users.

• Cost: The use of ML and AI may require third-party services, increasing operational costs.

It's worth noting that even with a well-calibrated model and a well-crafted prompt, the generated facets should still go through a review step. This validation can be manual or based on moderation rules, ensuring that the content is appropriate and safe. While not necessarily a drawback, it is an important consideration to ensure the quality and suitability of the facets before they are made available to users.

Implementation of filters/facets – AI approach

In this demonstration, we will use an AI model to automatically analyze product characteristics and suggest relevant attributes. With a well-structured prompt, we extract information from the catalog and transform it into filters and facets. Below, we present each step of the process.

Initially, we will use the Inference API to register an endpoint for integration with an ML service. Below is an example of integration with OpenAI's service.

Now, we define the pipeline to execute the prompt and obtain the new filters generated by the model.

Running a simulation of this pipeline for the "PlayStation 5" product, with the following description:

Stunning Gaming: Marvel at stunning graphics and experience the features of the new PS5.

Breathtaking Immersion: Discover a deeper gaming experience with support for haptic feedback, adaptive triggers, and 3D Audio technology.

Slim Design: With the PS5 Digital Edition, gamers get powerful gaming technology in a sleek, compact design.

1TB of Storage: Have your favorite games ready and waiting for you to play with 1TB of built-in SSD storage.

Backward Compatibility and Game Boost: The PS5 console can play over 4,000 PS4 games. With Game Boost, you can even enjoy faster, smoother frame rates in some of the best PS4 console games.

Let's observe the prompt output generated from this simulation.

Now a new field, dynamic_facets, will be added to the new index to store the facets generated by the AI.

Using the Reindex API, we will reindex the videogames index to videogames_1, applying the generate_filter_ai pipeline during the process. This pipeline will automatically generate dynamic facets during indexing.

Now, we will run a search and get the new filters:

Results:

To symbolize the implementation of the facets, below is a simple front-end:

The UI code presented is here.

Conclusion

Both approaches to creating filters and facets have their benefits and points of concern. The classic approach, based on manual rules, offers control and lower costs but requires constant updates and does not dynamically adapt to new products or features.

On the other hand, the AI ​​and Machine Learning-based approach automates facet extraction, making the search more flexible and allowing the discovery of new attributes without manual intervention. However, this approach can be more complex to implement and maintain, requiring calibration to ensure consistent results.

The choice between the classic and AI-based approaches depends on the needs and complexity of the business. For simpler scenarios, where data attributes are stable and predictable, the classic approach can be more efficient and easier to maintain, avoiding unnecessary costs with infrastructure and AI models. On the other hand, the use of ML/AI to extract facets can add significant value, improving the search experience and making filtering more intelligent.

The important thing is to evaluate whether automation justifies the investment or whether a more traditional solution already meets the business needs effectively.

Understanding the BEIR benchmark in search relevance evaluation

To improve any system you need to be able to measure how well it is doing. In the context of search BEIR (or equivalently the Retrieval section of the MTEB leaderboard) is considered the “holy grail” for the information retrieval community and there is no surprise in that. It’s a very well-structured benchmark with varied datasets across different tasks. More specifically, the following areas are covered:

• Argument retrieval (ArguAna, Touche2020)

• Open-domain QA (HotpotQA, Natural Questions, FiQA)

• Passage retrieval (MSMARCO)

• Duplicate question retrieval (Quora, CQADupstack)

• Fact-checking (FEVER, Climate-FEVER, Scifact)

• Biomedical information retrieval (TREC-COVID, NFCorpus, BioASQ)

• Entity retrieval (DBPedia)

• Citation prediction (SCIDOCS)

It provides a single statistic, nDCG@10, related to how well a system matches the most relevant documents for each task example in the top results it returns. For a search system that a human interacts with relevance of top results is critical. However, there are many nuances to evaluating search that a single summary statistic misses.

Structure of a BEIR dataset

Each benchmark has three artefacts:

• the corpus or documents to retrieve

• the queries

• the relevance judgements for the queries (aka qrels).

Relevance judgments are provided as a score which is zero or greater. Non-zero scores indicate that the document is somewhat related to the query.

Dataset

Corpus size

#Queries in the test set

#qrels positively labeled

#qrels equal to zero

#duplicates in the corpus

Arguana

8,674

1,406

1,406

0

96

Climate-FEVER

5,416,593

1,535

4,681

0

0

DBPedia

4,635,922

400

15,286

28,229

0

FEVER

5,416,568

6,666

7,937

0

0

FiQA-2018

57,638

648

1,706

0

0

HotpotQA

5,233,329

7,405

14,810

0

0

Natural Questions

2,681,468

3,452

4,021

0

16,781

NFCorpus

3,633

323

12,334

0

80

Quora

522,931

10,000

15,675

0

1,092

SCIDOCS

25,657

1,000

4,928

25,000

2

Scifact

5,183

300

339

0

0

Touche2020

382,545

49

932

1,982

5,357

TREC-COVID

171,332

50

24,763

41,663

0

MSMARCO

8,841,823

6,980

7,437

0

324

CQADupstack (sum)

457,199

13,145

23,703

0

0

Table 1: Dataset statistics. The numbers were calculated on the test portion of the datasets (dev for MSMARCO).

Table 1 presents some statistics for the datasets that comprise the BEIR benchmark such as the number of documents in the corpus, the number of queries in the test dataset and the number of positive/negative (query, doc) pairs in the qrels file. From a quick a look in the data we can immediately infer the following:

• Most of the datasets do not contain any negative relationships in the qrels file, i.e. zero scores, which would explicitly denote documents as irrelevant to the given query.

• The average number of document relationships per query (#qrels / #queries) varies from 1.0 in the case of ArguAna to 493.5 (TREC-COVID) but with a value <5 for the majority of the cases.

• Some datasets suffer from duplicate documents in the corpus which in some cases may lead to incorrect evaluation i.e. when a document is considered relevant to a query but its duplicate is not. For example, in ArguAna we have identified 96 cases of duplicate doc pairs with only one doc per pair being marked as relevant to a query. By “expanding” the initial qrels list to also include the duplicates we have observed a relative increase of ~1% in the nDCG@10 score on average.

Example of duplicate pairs in ArguAna. In the qrels file only the first appears to be relevant (as counter-argument) to query (“test-economy-epiasghbf-pro02a”)

When comparing models on the MTEB leaderboard it is tempting to focus on average retrieval quality. This is a good proxy to the overall quality of the model, but it doesn't necessarily tell you how it will perform for you. Since results are reported per data set, it is worth understanding how closely the different data sets relate to your search task and rescore models using only the most relevant ones. If you want to dig deeper, you can additionally check for topic overlap with the various data set corpuses. Stratifying quality measures by topic gives a much finer-grained assessment of their specific strengths and weaknesses.

One important note here is that when a document is not marked in the qrels file then by default it is considered irrelevant to the query. We dive a little further into this area and collect some evidence to shed more light on the following question: “How often is an evaluator presented with (query, document) pairs for which there is no ground truth information?". The reason that this is important is that when only shallow markup is available (and thus not every relevant document is labeled as such) one Information Retrieval system can be judged worse than another just because it “chooses” to surface different relevant (but unmarked) documents. This is a common gotcha in creating high quality evaluation sets, particularly for large datasets. To be feasible manual labelling usually focuses on top results returned by the current system, so potentially misses relevant documents in its blind spots. Therefore, it is usually preferable to focus more resources on fuller mark up of fewer queries than broad shallow markup.

Leveraging the BEIR benchmark for search relevance evaluation

To initiate our analysis we implement the following scenario (see the notebook):

1. First, we load the corpus of each dataset into an Elasticsearch index.

2. For each query in the test set we retrieve the top-100 documents with BM25.

3. We rerank, the retrieved documents using a variety of SOTA reranking models.

4. Finally, we report the “judge rate” for the top-10 documents coming from steps 2 (after retrieval) and 3 (after reranking). In other words, we calculate the average percentage of the top-10 documents that have a score in the qrels file.

The list of reranking of models we used is the following:

• Cohere's rerank-english-v2.0 and rerank-english-v3.0

• BGE-base

• mxbai-rerank-xsmall-v1

• MiniLM-L-6-v2

Retrieval

Reranking

Dataset

BM25 (%)

Cohere Rerank v2 (%)

Cohere Rerank v3 (%)

BGE-base (%)

mxbai-rerank-xsmall-v1 (%)

MiniLM-L-6-v2 (%)

Arguana

7.54

4.87

7.87

4.52

4.53

6.84

Climate-FEVER

5.75

6.24

8.15

9.36

7.79

7.58

DBPedia

61.18

60.78

64.15

63.9

63.5

67.62

FEVER

8.89

9.97

10.08

10.19

9.88

9.88

FiQa-2018

7.02

11.02

10.77

8.43

9.1

9.44

HotpotQA

12.59

14.5

14.76

15.1

14.02

14.42

Natural Questions

5.94

8.84

8.71

8.37

8.14

8.34

NFCorpus

31.67

32.9

33.91

30.63

32.77

32.45

Quora

12.2

10.46

13.04

11.26

12.58

12.78

SCIDOCS

8.62

9.41

9.71

8.04

8.79

8.52

Scifact

9.07

9.57

9.77

9.3

9.1

9.17

Touche2020

38.78

30.41

32.24

33.06

37.96

33.67

TREC-COVID

92.4

98.4

98.2

93.8

99.6

97.4

MSMARCO

3.97

6.00

6.03

6.07

5.47

6.11

CQADupstack (avg.)

5.47

6.32

6.87

5.89

6.22

6.16

Table 2: Judge rate per (dataset, reranker) pairs calculated on the top-10 retrieved/reranked documents

From Table 2, with the exception of TREC-COVID (>90% coverage), DBPedia (~65%), Touche2020 and nfcorpus (~35%), we see that the majority of the datasets have a labeling rate between 5% and a little more than 10% after retrieval or reranking. This doesn’t mean that all these unmarked documents are relevant but there might be a subset of them -especially those placed in the top positions- that could be positive.

With the arrival of general purpose instruction tuned language models, we have a new powerful tool which can potentially automate judging relevance. These methods are typically far too computationally expensive to be used online for search, but here we are concerned with offline evaluation. In the following we use them to explore the evidence that some of the BEIR datasets suffer from shallow markup.

In order to further investigate this hypothesis we decided to focus on MSMARCO and select a subset of 100 queries along with the top-5 reranked (with Cohere v2) documents which are currently not marked as relevant. We followed two different paths of evaluation: First, we used a carefully tuned prompt (more on this in a later post) to prime the recently released Phi-3-mini-4k model to predict the relevance (or not) of a document to the query. In parallel, these cases were also manually labeled in order to also assess the agreement rate between the LLM output and human judgment. Overall, we can draw the following two conclusions:

• The agreement rate between the LLM responses and human judgments was close to 80% which seems good enough as a starting point in that direction.

• In 57.6% of the cases (based on human judgment) the returned documents were found to be actually relevant to the query. To state this in a different way: For 100 queries we have 107 documents judged to be relevant, but at least 0.576 x 5 x 100 = 288 extra documents which are actually relevant!

Here, some examples drawn from the MSMARCO/dev dataset which contain the query, the annotated positive document (from qrels) and a false negative document due to incomplete markup:

Example 1:

Example 2:

Manually evaluating specific queries like this is a generally useful technique for understanding search quality that complements quantitive measures like nDCG@10. If you have a representative set of queries you always run when you make changes to search, it gives you important qualitative information about how performance changes, which is invisible in the statistics. For example, it gives you much more insight into the false results your search returns: it can help you spot obvious howlers in retrieved results, classes of related mistakes, such as misinterpreting domain-specific terminology, and so on.

Our result is in agreement with relevant research around MSMARCO evaluation. For example, Arabzadeh et al. follow a similar procedure where they employ crowdsourced workers to make preference judgments: among other things, they show that in many cases the documents returned by the reranking modules are preferred compared to the documents in the MSMARCO qrels file. Another piece of evidence comes from the authors of the RocketQA reranker who report that more than 70% of the reranked documents were found relevant after manual inspection.

Update - September 9th: After a careful re-evaluation of the dataset we identified 15 more cases of relevant documents, increasing their total number from 273 to 288

Main takeaways & next steps

• The pursuit for better ground truth is never-ending as it is very crucial for benchmarking and model comparison. LLMs can assist in some evaluation areas if used with caution and tuned with proper instructions

• More generally, given that benchmarks will never be perfect, it might be preferable to switch from a pure score comparison to more robust techniques capturing statistically significant differences. The work of Arabzadeh et al. provides a nice of example of this where based on their findings they build 95% confidence intervals indicating significant (or not) differences between the various runs. In the accompanying notebook we provide an implementation of confidence intervals using bootstrapping.

• From the end-user perspective it’s useful to think about task alignment when reading benchmark results. For example, for an AI engineer who builds a RAG pipeline and knows that the most typical use case involves assembling multiple pieces of information from different sources, then it would be more meaningful to assess the performance of their retrieval model on multi-hop QA datasets like HotpotQA instead of the global average across the whole BEIR benchmark

In the next blog post we will dive deeper into the use of Phi-3 as LLM judge and the journey of tuning it to predict relevance.