56% faster, up to 50% better retrieval performance: What's inside Jina's new 600 million parameter listwise reranker reranker

Get hands-on with Elasticsearch: Dive into our sample notebooks in the Elasticsearch Labs repo, start a free cloud trial, or try Elastic on your local machine now.

jina-reranker-v3.5 is a 600 million parameter reranker that delivers major gains over its predecessor, jina-reranker-v3, on legal retrieval, and closes most of the gap to models seven times its size on legal, medical, and financial reranking. On long documents, it runs up to 56% faster than jina-reranker-v3 and scores over 50% higher on case law retrieval. It also beats Qwen3-Reranker-4B on the STaRK structured data benchmark."It’s a drop-in replacement for v3 users and requires no changes to the code that accesses the model.

What’s a reranker and how does it work?

A reranker is an AI model used near the end of an information retrieval pipeline, after other modules have assembled a short list of candidate matches to a query. It’s trained to order the candidate list from best matching to least. Using a specialized model focused purely on ranking candidate matches can improve result quality dramatically.

Jina AI’s latest rerankers use a technique called late interaction, where queries and documents are encoded separately into lists of token embeddings that reflect each token’s semantics in context and then compared to each other.

This is an AI analog of lexical and grammatical disambiguation.

For example, consider the meaning of the word match in these two sentences:

• She looked for a match to light the candle.
• She looked for a match on Tinder.

The first sentence might be a match for queries about matchboxes; the second for queries about romance.

Transformer-based models do this kind of in-context disambiguation but bring much richer information into the token embeddings they produce. The word match might have a semantic embedding near to words like fire or illumination in the first sentence, while in the second, it might be closer to smartphone or swipe.

Late interaction rerankers generate these context-enriched token embeddings for both the query and the candidate documents and then compare them to produce sortable scores. They’re completely agnostic about how candidate match lists are created. The reranker works exactly the same when combined with lexical search schemes, like BM25, AI-driven semantic embeddings-based retrieval, or hybrid and federated search systems that may retrieve multiple candidate lists from different sources or using different algorithms. Of course, the results always depend on the quality of the candidates, so a reranker can’t fix bad first-stage retrieval, but it almost always improves whatever you’ve got.

jina-reranker-v3.5 is a listwise reranker using the last-but-not-late technique developed for jina-reranker-v3. The query and a list of candidate matches are passed into the model together and processed in one pass, returning a numerical score for each candidate. This enables the model to use context information from the query and the full candidate list to make sense of the entire input, producing better results because of the richer information available to it.

jina-reranker-v3 proved that listwise rerankers with last-but-not-late interaction can compete with the largest models on general reranking benchmarks. Only jina-reranker-v3.5 and models with over four billion parameters beat it on Massive Text Embedding Benchmark (MTEB) reranker tasks. However, this approach places strict limits on candidate list sizes. The query and all candidate matches must fit in the input context window of the model.

What problems does Jina Reranker v3.5 solve?

Despite having frontier-level performance overall, jina-reranker-v3 has some notable performance gaps:

Domain-specific text retrieval

jina-reranker-v3 was trained on general text corpora and, as a result, it underperforms on important use cases, particularly:

• Legal texts, like case law and contract clauses.
• Medical literature, like clinical trials and patient records.
• Financial datasets and other texts full of important numbers.
• Computer programming and IT documentation.
• Product catalogs full of technical terminology and specifications.

Structured data: Tables, JSON, and key-value records

Vast quantities of essential, real-world data is encoded in spreadsheets, tables, key-value lists, and structured records, like JSON data. However, rerankers trained purely for textual comparison, like jina-reranker-v3, perform poorly on this kind of data.

Compute costs for long candidate lists

The self-attention architecture at the core of most text-processing AI models means that memory and compute requirements grow quadratically with the size of its input. This makes jina-reranker-v3, like other AI models, very computationally expensive to run with a full input context window. But, to make the most effective use of the model, we want to put as many match candidates as possible into its input. When it’s at its most useful, it’s also slower and more expensive to run.

We’ve developed jina-reranker-v3.5 specifically to address these issues without reducing its performance on general purpose text retrieval.

What’s new in Jina Reranker v3.5?

jina-reranker-v3.5 contains a modified self-attention mechanism that enhances performance, increases processing speed, and reduces the resources required at inference time to process a full input context window. We’ve also introduced a new three-stage self-distillation training process better suited to the sliding-window architecture of large input context models.

We’ve also curated and used training data focusing on the performance gaps we identified in jina-reranker-v3, including:

• Multilingual legal texts drawn from diverse international sources.
• Medical texts drawn largely from scientific literature and materials used for other AI projects, including a collection of Chinese medical question-answer pairs.
• Financial industry data, including investment-related question-answer pairs, regulations, and tables with numbers and associated texts.
• Structured data, especially from ecommerce sources and public corpora of tables.
• Expanded multilingual and cross-language texts.

For details on the data sources and technical innovations in jina-reranker-v3.5, see our technical report.

How Jina Reranker 3.5 performs on retrieval benchmarks

General text reranking performance (BEIR and MIRACL)

jina-reranker-v3.5 improves on jina-reranker-v3’s performance on general text reranking benchmarks. On the English-language Benchmarking Information Retrieval (BEIR) benchmark, the average score has increased enough to surpass the frontier Qwen3-Reranker-4B and 0.6B models and Mixedbread AI’s rerankers.

We’ve also improved jina-reranker-v3’s multilingual reranking performance on the Multilingual Information Retrieval Across a Continuum of Languages (MIRACL) benchmark. Only the four billion parameter Qwen3 reranker regularly beats jina-reranker-v3.5’s score.

Legal, medical, and financial reranking

The Retrieval Embedding Benchmark (RTEB) suite consists of diverse domain-specific retrieval benchmarks. jina-reranker-v3.5 outperforms jina-reranker-v3 on all RTEB tasks related to law, medicine, and finance. Only the large Qwen3 reranker, at almost seven times as many parameters, has better average performance in those three domains.

The new model shows particularly strong improvements for legal data, beating jina-reranker-v3’s score by over 50% on case law retrieval tasks.

Structured data reranking (Struct-IR and STaRK benchmarks)

We evaluated jina-reranker-v3.5's structured data reranking on two benchmarks: Struct-IR and STaRK. Both benchmarks contain AI-generated JSON text data covering a variety of applications, including product records, scientific papers, and biomedical knowledge bases.

jina-reranker-v3.5 improves substantially on jina-reranker-v3’s score on the Struct-IR benchmark, once again only exceeded by Qwen3-Reranker-4B. On the STaRK benchmark, jina-reranker-v3.5 beats all the other models we tested, of any size.

Inference speed: Latency benchmarks for short and long documents

The longer the candidate documents get, the more significant the architectural improvements we’ve brought to jina-reranker-v3.5 are. To verify this, we used two retrieval datasets distinguished by large differences in the average document length:

jina-reranker-v3.5 is significantly faster in both cases. On the Natural Questions benchmark, there’s a 22% speedup compared to jina-reranker-v3 with average request latency falling from 371.1 ms to 305.3 ms. Each query from the AILAcasedocs benchmark is much larger (more than 10 times larger on average) so it naturally takes longer to rerank on average: 16,064.9 ms for jina-reranker-v3 and 10,290.9 ms for jina-reranker-v3.5. This represents a 56% speedup for the newer model, representing less latency for applications and lower computer costs.

When should you use Jina Reranker v3.5?

Reranking improves search precision in practically every case, and jina-reranker-v3.5 has applications in a wide variety of information retrieval contexts. However, it has some limitations. The table below summarizes our best-practice advice:

How to use Jina Reranker 3.5 with the Elastic Inference API

jina-reranker-v3.5 is available via the Jina API with free tokens to try it out. It’s also available via the Elastic Inference API and Elastic Inference Service.

If you’re already using jina-reranker-v3, all you have to do is change the name of the model in the model field of your request to the Jina API or model_id field when configuring an Elastic Inference API endpoint. The two models have completely identical interfaces.

You can install jina-reranker-v3.5 as a Jina On-Prem container to get a completely self-contained server that runs on your own hardware. The model weights are also available to download for testing and research. Follow the instructions on the model’s page at Hugging Face. In both cases, the model is available under a CC BY-NC-4.0 license, so you’re free to try it out for testing, building prototypes, or doing scientific research. For commercial use, please contact Elastic sales.