Benjamin Trent
Principal Developer II
Benjamin Trent is a software engineer at Elastic, where he works on improving Elasticsearch. He is a Lucene committer and member of the project management committee at The Apache Software Foundation.
Elasticsearch DiskBBQ: 40% faster vector scoring with native SIMD Blocks
A deep dive into how DiskBBQ's block layout, doc ID compression modes and native SIMD kernels combine to deliver 40% improved vector scoring throughput for DiskBBQ in 9.4.
Cutting Elasticsearch DiskBBQ query quantization time by 5x
See how asymmetric quantization cuts DiskBBQ query quantization overhead from about 20% to 4% with little recall impact.
Up to 3x faster stored-vector queries in Elasticsearch
Elasticsearch 9.4 provides a simpler way to search with vectors stored in an Elasticsearch index, with up to 3x lower latency.
Elasticsearch Vector DiskBBQ filter search is now 3–5x faster
Learn how Elasticsearch 9.4 makes restrictive filtered DiskBBQ vector search 3–5x faster and more stable by avoiding wasted centroid and postings-list work when selectivity is high.
Speed up vector ingestion using Base64-encoded strings
Introducing Base64-encoded strings to speed up vector ingestion in Elasticsearch.
Introducing a new vector storage format: DiskBBQ
Introducing DiskBBQ, an alternative to HNSW, and exploring when and why to use it.
Elasticsearch sorting just got up to 900x faster
Discover how we sped up Elasticsearch sorting with faster float/half_float sorting and latency improvements in integer sorting
Scaling late interaction models in Elasticsearch - part 2
This article explores techniques for making late interaction vectors ready for large-scale production workloads, such as reducing disk space usage and improving computation efficiency.
Searching complex documents with ColPali - part 1
The article introduces the ColPali model, a late-interaction model that simplifies the process of searching complex documents with images and tables, and discusses its implementation in Elasticsearch.
Filtered HNSW search, fast mode
Explore the improvements we have made for HNSW vector search in Apache Lucene through our ACORN-1 algorithm implementation.
Concurrency bugs in Lucene: How to fix optimistic concurrency failures
Thanks to Fray, a deterministic concurrency testing framework from CMU’s PASTA Lab, we tracked down a tricky Lucene bug and squashed it
Optimized Scalar Quantization: Improving Better Binary Quantization (BBQ)
Here we explain optimized scalar quantization in Elasticsearch and how we used it to improve Better Binary Quantization (BBQ).
Lucene bug adventures: Fixing a corrupted index exception
Sometimes, a single line of code takes days to write. Here, we get a glimpse of an engineer's pain and debugging over multiple days to fix a potential Apache Lucene index corruption.
Better Binary Quantization (BBQ) vs. Product Quantization
Why we chose to spend time working on Better Binary Quantization (BBQ) instead of product quantization in Lucene and Elasticsearch.
Better Binary Quantization (BBQ) in Lucene and Elasticsearch
How Better Binary Quantization (BBQ) works in Lucene and Elasticsearch.
Looking back: Elastic's vector search improvements in Elasticsearch & Lucene
Looking back at Elastic's vector search innovations in Elasticsearch and Lucene.
Bit vectors in Elasticsearch
Discover what are bit vectors, their practical implications and how to use them in Elasticsearch.
Making Elasticsearch and Lucene the best vector database: up to 8x faster and 32x efficient
Discover the recent enhancements and optimizations that notably improve vector search performance in Elasticsearch & Lucene vector database.
Scalar quantization optimized for vector databases
Optimizing scalar quantization for the vector database use case allows us to achieve significantly better performance for the same retrieval quality at high compression ratios.
Understanding Int4 scalar quantization in Lucene
This blog explains how int4 quantization works in Lucene, how it lines up, and the benefits of using int4 quantization.
Introducing kNN Query: An expert way to do kNN search
Explore how the kNN query in Elasticsearch can be used and how it differs from top-level kNN search, including examples.
Understanding scalar quantization in Lucene
Explore how Elastic introduced scalar quantization into Lucene, including automatic byte quantization, quantization per segment & performance insights.
Scalar quantization 101
Understand what scalar quantization is, how it works and its benefits. This guide also covers the math behind quantization and examples.
Bringing maximum-inner-product into Lucene
Explore how we brought maximum-inner-product into Lucene and the investigations undertaken to ensure its support.
Adding passage vector search to Lucene
Here's how to add passage vectors to Lucene, the benefits of doing so and how existing Lucene structures can be used to create an efficient retrieval experience.
Save space with byte-sized vectors
Elasticsearch is introducing a new type of vector that has 8-bit integer dimensions. This is 4x smaller than the current vector with 32-bit float dimensions, which can result in substantial space savings.
