What’s a reranker and how does it work?

A reranker is an AI model used near the end of an information retrieval pipeline, after other modules have assembled a short list of candidate matches to a query. It’s trained to order the candidate list from best matching to least. Using a specialized model focused purely on ranking candidate matches can improve result quality dramatically.

Jina AI’s latest rerankers use a technique called late interaction, where queries and documents are encoded separately into lists of token embeddings that reflect each token’s semantics in context and then compared to each other.

This is an AI analog of lexical and grammatical disambiguation.

For example, consider the meaning of the word match in these two sentences:

• She looked for a match to light the candle.

• She looked for a match on Tinder.

The first sentence might be a match for queries about matchboxes; the second for queries about romance.

Transformer-based models do this kind of in-context disambiguation but bring much richer information into the token embeddings they produce. The word match might have a semantic embedding near to words like fire or illumination in the first sentence, while in the second, it might be closer to smartphone or swipe.

Late interaction rerankers generate these context-enriched token embeddings for both the query and the candidate documents and then compare them to produce sortable scores. They’re completely agnostic about how candidate match lists are created. The reranker works exactly the same when combined with lexical search schemes, like BM25, AI-driven semantic embeddings-based retrieval, or hybrid and federated search systems that may retrieve multiple candidate lists from different sources or using different algorithms. Of course, the results always depend on the quality of the candidates, so a reranker can’t fix bad first-stage retrieval, but it almost always improves whatever you’ve got.

jina-reranker-v3.5 is a listwise reranker using the last-but-not-late technique developed for jina-reranker-v3. The query and a list of candidate matches are passed into the model together and processed in one pass, returning a numerical score for each candidate. This enables the model to use context information from the query and the full candidate list to make sense of the entire input, producing better results because of the richer information available to it.

jina-reranker-v3 proved that listwise rerankers with last-but-not-late interaction can compete with the largest models on general reranking benchmarks. Only jina-reranker-v3.5 and models with over four billion parameters beat it on Massive Text Embedding Benchmark (MTEB) reranker tasks. However, this approach places strict limits on candidate list sizes. The query and all candidate matches must fit in the input context window of the model.

What problems does Jina Reranker v3.5 solve?

Despite having frontier-level performance overall, jina-reranker-v3 has some notable performance gaps:

Domain-specific text retrieval

jina-reranker-v3 was trained on general text corpora and, as a result, it underperforms on important use cases, particularly:

• Legal texts, like case law and contract clauses.

• Medical literature, like clinical trials and patient records.

• Financial datasets and other texts full of important numbers.

• Computer programming and IT documentation.

• Product catalogs full of technical terminology and specifications.

Structured data: Tables, JSON, and key-value records

Vast quantities of essential, real-world data is encoded in spreadsheets, tables, key-value lists, and structured records, like JSON data. However, rerankers trained purely for textual comparison, like jina-reranker-v3, perform poorly on this kind of data.

Compute costs for long candidate lists

The self-attention architecture at the core of most text-processing AI models means that memory and compute requirements grow quadratically with the size of its input. This makes jina-reranker-v3, like other AI models, very computationally expensive to run with a full input context window. But, to make the most effective use of the model, we want to put as many match candidates as possible into its input. When it’s at its most useful, it’s also slower and more expensive to run.

We’ve developed jina-reranker-v3.5 specifically to address these issues without reducing its performance on general purpose text retrieval.

What’s new in Jina Reranker v3.5?

jina-reranker-v3.5 contains a modified self-attention mechanism that enhances performance, increases processing speed, and reduces the resources required at inference time to process a full input context window. We’ve also introduced a new three-stage self-distillation training process better suited to the sliding-window architecture of large input context models.

We’ve also curated and used training data focusing on the performance gaps we identified in jina-reranker-v3, including:

• Multilingual legal texts drawn from diverse international sources.

• Medical texts drawn largely from scientific literature and materials used for other AI projects, including a collection of Chinese medical question-answer pairs.

• Financial industry data, including investment-related question-answer pairs, regulations, and tables with numbers and associated texts.

• Structured data, especially from ecommerce sources and public corpora of tables.

• Expanded multilingual and cross-language texts.

For details on the data sources and technical innovations in jina-reranker-v3.5, see our technical report.

How Jina Reranker 3.5 performs on retrieval benchmarks

Parameters

597 million

Input modalities

Text only

Context window size

131,072 tokens

Maximum number of candidate matches

No fixed limit, but all candidates and query must fit in the context window.

Languages

Training in 52 languages

General text reranking performance (BEIR and MIRACL)

jina-reranker-v3.5 improves on jina-reranker-v3’s performance on general text reranking benchmarks. On the English-language Benchmarking Information Retrieval (BEIR) benchmark, the average score has increased enough to surpass the frontier Qwen3-Reranker-4B and 0.6B models and Mixedbread AI’s rerankers.

We’ve also improved jina-reranker-v3’s multilingual reranking performance on the Multilingual Information Retrieval Across a Continuum of Languages (MIRACL) benchmark. Only the four billion parameter Qwen3 reranker regularly beats jina-reranker-v3.5’s score.

Legal, medical, and financial reranking

The Retrieval Embedding Benchmark (RTEB) suite consists of diverse domain-specific retrieval benchmarks. jina-reranker-v3.5 outperforms jina-reranker-v3 on all RTEB tasks related to law, medicine, and finance. Only the large Qwen3 reranker, at almost seven times as many parameters, has better average performance in those three domains.

The new model shows particularly strong improvements for legal data, beating jina-reranker-v3’s score by over 50% on case law retrieval tasks.

Task

Reranker v3

Reranker v3.5

Improvement v3 to v3.5

AILA-Case

20.82

32.55

+11.73 (56%)

AILA-Statute

32.15

46.16

+14.01 (44%)

LegalQuAD

81.84

83.09

+1.25 (1.5%)

LegalSum

69.64

70.99

+1.33 (1.9%)

Structured data reranking (Struct-IR and STaRK benchmarks)

We evaluated jina-reranker-v3.5's structured data reranking on two benchmarks: Struct-IR and STaRK. Both benchmarks contain AI-generated JSON text data covering a variety of applications, including product records, scientific papers, and biomedical knowledge bases.

jina-reranker-v3.5 improves substantially on jina-reranker-v3’s score on the Struct-IR benchmark, once again only exceeded by Qwen3-Reranker-4B. On the STaRK benchmark, jina-reranker-v3.5 beats all the other models we tested, of any size.

Inference speed: Latency benchmarks for short and long documents

The longer the candidate documents get, the more significant the architectural improvements we’ve brought to jina-reranker-v3.5 are. To verify this, we used two retrieval datasets distinguished by large differences in the average document length:

Dataset

Avg. doc length

jina-reranker-v3

jina-reranker-v3.5

Speedup

BEIR Natural Questions

145.5 tokens

371.1 ms

305.3 ms

22%

RTEB AILAcasedocs

1,904.0 tokens

16,064.9 ms

10,290.9 ms

56%

jina-reranker-v3.5 is significantly faster in both cases. On the Natural Questions benchmark, there’s a 22% speedup compared to jina-reranker-v3 with average request latency falling from 371.1 ms to 305.3 ms. Each query from the AILAcasedocs benchmark is much larger (more than 10 times larger on average) so it naturally takes longer to rerank on average: 16,064.9 ms for jina-reranker-v3 and 10,290.9 ms for jina-reranker-v3.5. This represents a 56% speedup for the newer model, representing less latency for applications and lower computer costs.

When should you use Jina Reranker v3.5?

Reranking improves search precision in practically every case, and jina-reranker-v3.5 has applications in a wide variety of information retrieval contexts. However, it has some limitations. The table below summarizes our best-practice advice:

Use case

Recommendation

General text retrieval in common international languages

Use `jina-reranker-v3.5`.

Legal, financial, and medical domain retrieval

Use `jina-reranker-v3.5`.

Semi-structured data, tables, product information texts for ecommerce

Use `jina-reranker-v3.5`.

Non-text or mixed-media data

Use `jina-reranker-m0`, which supports both text and image input.

How to use Jina Reranker 3.5 with the Elastic Inference API

jina-reranker-v3.5 is available via the Jina API with free tokens to try it out. It’s also available via the Elastic Inference API and Elastic Inference Service.

If you’re already using jina-reranker-v3, all you have to do is change the name of the model in the model field of your request to the Jina API or model_id field when configuring an Elastic Inference API endpoint. The two models have completely identical interfaces.

You can install jina-reranker-v3.5 as a Jina On-Prem container to get a completely self-contained server that runs on your own hardware. The model weights are also available to download for testing and research. Follow the instructions on the model’s page at Hugging Face. In both cases, the model is available under a CC BY-NC-4.0 license, so you’re free to try it out for testing, building prototypes, or doing scientific research. For commercial use, please contact Elastic sales.

The most powerful AI models run on remote cloud installations with access via a web API, meaning that you have to trust your AI service provider for security, service availability, and stable prices. You can’t easily align reasonable demands for reliability, privacy, manageable costs, and good data governance with increasingly powerful, sophisticated, and resource-intensive AI usage.

Government regulation, court rulings, and business considerations made in someone else’s interest have all recently resulted in restricting access to specific services. And even if you can switch to other services, AI models aren’t components that can just be swapped out whenever you want. Applications that use semantic embeddings depend on having access to the same models at query time as at data ingestion time. To lose access to your embedding model means your search system comes to a halt.

AI pricing models compound that risk. Recent financial disclosures from major AI vendors give customers good reason to be concerned about potential price hikes. Reliance on products with unpredictable costs adds more risk to capital-intensive AI investments that may not produce clear returns.

Jina On-Prem is Elastic’s answer to these challenges.

Who needs on-premises AI?

Local hosting and direct control over your AI models support a variety of technical demands, industry requirements, and business interests.

Local installation reduces what you pay your AI service providers, but it puts the cost of hardware and reliable access on your organization. Depending on your volume of use, it may simply be cheaper. But there are additional pressing reasons to consider running your own AI. If any of the issues described below concern your enterprise, consider a local AI solution like Jina On-Prem. This list is not exhaustive.

Use case

Why on-prem

Example

Air-gapped / high-security

No outbound data transmission; complete network isolation

Defence, intelligence, classified research

Regulatory compliance

Data sovereignty; no cross-border transmission or third-party exposure

Healthcare (Health Insurance Portability and Accountability Act [HIPAA]), finance, EU enterprises (General Data Protection Regulation [GDPR])

Latency-critical

Zero network dependency; no tolerance for connection failures

Robotics, edge computing, vehicles, ships

Cost predictability

Fixed infrastructure cost vs. per-token pricing with uncertain future rates

High-volume continuous inference workloads

Liability reduction

No third-party data exposure; maintains legal privilege and duty of care

Law firms, government agencies

Why air-gapped and firewalled systems need on-prem AI

Air-gapped and firewalled systems cannot use external AI APIs. Jina On-Prem runs entirely within your infrastructure with no outbound connections.

For organizations managing especially sensitive data, security and privacy considerations are paramount. It does little good to invest in protecting your sensitive data if you promptly turn it over to some remote third party that may have insufficient security in place or might be subject to the demands of a foreign government.

Employees in organizations that handle sensitive data often receive some training in secure data handling, but this isn’t very effective when they all have web browsers that may be open to any page on the internet while they handle that data. Isolation is the most effective security measure available, either through air-gapping or very restrictive firewalls, but that makes it difficult to use external services of any kind.

On-prem AI for latency-sensitive and high-availability systems

Software as a service and cloud computing represent a compromise between the cost of offering highly accessible, reliable services on your own computers and outsourcing the problem to someone else. But they come with variable latency, outages, and a complete loss of control when things go wrong. AI services aren’t the exception. If your search system goes offline when you can’t access your embedding model, it may no longer look like a good compromise.

Furthermore, relying on external AI will always involve risks that you can’t easily foresee or manage. Internet access and network latency can degrade without notice, as a result of political events, bad weather, or ships dragging their anchors over underwater fiber-optic cables. Governments can, and recently have, used export bans to suddenly block access to AI models. AI service providers sometimes withdraw models to induce you to switch to newer ones. The flexibility and managed costs of external services have to be balanced against the risks of dependency.

On-prem AI for GDPR, HIPAA, and data sovereignty compliance

Organizations that collect personal data are subject to increasingly stringent regulations which often differ between jurisdictions and may have contradictory requirements. Notably, HIPAA rules place very strict data protections on American healthcare providers, and strong general data protection laws in Canada, the European Union, and many Asian jurisdictions require all enterprises that handle personal information to do so securely and to limit the transmission of that data to other parties or other jurisdictions. These rules can even impose obligations on foreign entities if they have any customers in those jurisdictions. Financial institutions are frequently subject to even stricter rules and bear the same direct liability for information security that they have to protect against other forms of criminal activity.

Regulatory compliance can be incompatible with third-party AI services, especially if using them involves cross-border data transmission.

Furthermore, recent events show that rules restricting the physical location of data stores may not be a reliable source of protection when international cloud operators are subject to pressure from foreign governments. Local laws may conflict between jurisdictions, requiring local data storage and processing and making third-party services impossible to use. In some cases, the only solution is to take all the parts of your processes in house, including your AI systems.

AI liability risks from third-party data transmission

Data protection laws and recognized duties of care toward sensitive data routinely have liability implications, sometimes very severe ones. You can be liable for third-party service providers’ handling of your data. While courts and legal procedures might provide some retrospective protections from insecure service providers, those remedies are not available nor generally effective against national security actors, law enforcement, or criminal hackers.

For governments, there have already been instances of cross-border cloud service providers releasing sensitive state information to foreign actors.

But even if you don’t worry about foreign governments or hackers, and if your external AI service providers are themselves secure, just the fact that they’re external can create liabilities.

For example, in most jurisdictions, lawyers’ communications with their clients enjoy special legal protections, and law offices have strict liabilities when recording or storing this information. In the United States, this “attorney-client privilege” is so famous, it’s central to movie and TV plots. But one of the ways that privilege can be lost is by communicating information with someone who is not privileged, and recent developments suggest that external AI service providers might qualify.

It’s possible, at least in the United States, that just using third-party AI services over an internet API, like embedding models that provide indexing services, might violate critical confidentiality rules. A law firm might be sued, disciplined, or disbarred just for using externally hosted software, even if no security breach occurs.

On-prem AI for offline, edge, and physically isolated systems

Computer systems aren’t just isolated for security reasons. For example, moving vehicles cannot rely on internet access for any essential functions. Ships and aircraft have very extensive onboard computer systems that have to function without internet connections and therefore cannot use external AI services. Offshore platforms, remote facilities in wilderness areas, computer services in the Arctic, Antarctic and on small islands without adequate physical connections to global networks are all examples of installations that benefit from locally hosting all the services they need. As AI’s role in enterprise computing grows, these limitations become more important to address.

Emerging applications of AI to physical systems (robotics and other spatially confined or external-world–focused use cases, like logistics management systems or even supermarket checkouts) may be connected to the global internet, but they have no tolerance for connection failures or spikes in latency. If they rely on an AI system to operate, that AI system needs to be as local and reliable as possible.

Who doesn’t need on-premises AI?

Remote software services and off-site AI do have benefits. Running AI models can require expensive, power-hungry processors with notoriously short lifespans. Access to high-quality hardware is particularly difficult right now due to market factors and external economic shocks. Under the circumstances, it may make sense to pay by the token to use an external API instead of supporting the steep capital costs of local AI.

External APIs make the most sense for intermittent users. If you use AI models primarily to batch process data for analysis, rather than running a search system that has to be online all the time, it makes little sense to invest in capital-intensive hardware and local installations.

Furthermore, when your data processing is already cloud-based, for example, an ecommerce website hosted in the cloud for reliability and accessibility reasons, using AI services located in the same cloud infrastructure may provide a better value for money than introducing your own licensed AI model deployment. You’re already dependent on your cloud service provider, so being dependent on its AI services doesn’t add much risk.

If your use case sounds like it fits that description, Jina AI models are available on EIS, AWS Marketplace, and the Google Cloud Platform specifically to meet your needs.

The table below summarizes the key factors. Your answer depends on your data, infrastructure and usage pattern.

Factor

On-prem favored

Cloud API favored

Usage pattern

Continuous or high-volume inference

Intermittent or batch processing

Data sensitivity

Regulated, sovereign, or classified

No cross-border or third-party restrictions

Network environment

Air-gapped, firewalled, or unreliable

Stable, always-on internet

Existing infrastructure

Own or can procure GPU hardware

Already cloud-hosted with colocated AI

Cost model

Fixed hardware + license; predictable at scale

Per-token; lower up-front, variable long-term

Latency tolerance

None (robotics, edge, real-time)

Network variability is acceptable

Operational responsibility

Your team manages hardware and availability

Provider manages hardware and updates; you manage integration

You have to consider the costs and benefits in light of your particular circumstances and use cases, taking into account the issues highlighted in the previous section that apply to you. The cost-benefit analysis will doubtless change over time. We can’t predict the future of the AI industry or hardware prices even in the short term.

Introducing Jina On-Prem

For users who can benefit from local AI services, we’re introducing Jina On-Prem, a fully self-contained installation suite for Jina AI’s high-performance models.

Jina AI’s models match the accuracy of embedding models many times their size, reducing compute costs, memory footprints, and hardware requirements. This makes them an ideal choice for users who want or need to keep their AI on-premises. Commercial licenses are available with scalable, proportionately priced solutions for use cases of all sizes.

What API schemas does Jina On-Prem support?

• Available as a complete collection of dependencies for local installation or as a Docker container that you can install and run in minutes.

• Jina On-Prem installations do not call out to outside systems.

  • No call to Hugging Face Hub or any model registry (HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1 are baked in).

  • There’s no license server.

  • There are no telemetry or logging endpoints.

• Supports both CPU and GPU hardware, with GPU autodetection.

• All 28 Jina AI models available, including the latest jina-embeddings-v5-omni multimodal embedding models and jina-reranker-v3.

• Access via standard AI API schemas: Jina API, OpenAI, Cohere, Voyage AI, and Gemini. Jina On-Prem is a drop-in solution for applications built on those schemas.

• Drop-in replacement for models served by the EIS. Jina On-Prem integrates directly with air-gapped Elastic deployments.

Hardware requirements for Jina AI on-prem models

The hardware requirements vary for different Jina models. The table below shows the recommendations for the most recent models using GPU settings. You don’t need anything more powerful than an NVIDIA L4 GPU, although an A100 is recommended for the v5 embedding models. Our latest embedding model currently requires a minimum of 8 GB of VRAM.

Model

Minimum VRAM

Recommended GPU

jina-embeddings-v5-text-nano

2 GB

T4 / L4

jina-embeddings-v5-text-small

3 GB

L4 / A10G

jina-embeddings-v5-omni-small

8 GB

L4 / A10G / A100

jina-reranker-v3

3 GB

L4

jina-clip-v2

4 GB

L4

jina-code-embeddings-1.5b

4 GB

L4

ReaderLM-v2

4 GB

L4

If you use more than one model at a time, the VRAM requirements will increase. Please see the Sizing and Hardware page for more information.

How to install Jina On-Prem with Docker

The quickest way to get started is to install Docker (if you haven’t already) and follow the instructions on the Jina On-Prem Quick Start page.

There are pre-composed Docker containers for all 28 Jina models. Download one and transfer it to your installation target, and you can have Jina AI models running in under five minutes.

For multimodal or custom builds, or to download the complete dependency set for installation outside of a container, follow the steps outlined in the bundling guide.

Your Jina On-Prem installation supports all Jina API and EIS functionality and embedding generation via OpenAI, Cohere, Voyage AI, and Gemini APIs, so it can integrate into preexisting applications using standard interfaces. See the API documentation for more information.

Jina models, including models installed with Jina On-Prem, are available on various licensing terms, with the latest models free for noncommercial use under a CC BY-NC 4.0 license. To license Jina On-Prem for commercial use, please contact Elastic Sales.

jina-vlm is a 2.4B-parameter vision-language model that pairs a SigLIP2 vision encoder with a Qwen3 language decoder, using attention pooling over image tiles for token-efficient handling of arbitrary-resolution inputs. Beyond the model itself, the paper's main contribution is its “leave-one-out” ablative data-mixture: By removing one task, domain, modality, or language category at a time during training, you can figure out which slices of data are significant or redundant, and whether learning in one domain transfers to others. The result is a compact model that, despite its size, achieves state-of-the-art multilingual VQA performance.

Rio delivered everything you'd hope for: warm, sunny beach weather, the easy walk between Copacabana and Ipanema, the view from Christ the Redeemer, the colors of Escadaria Selarón. A welcome contrast to a still-chilly European spring.

What was trending at ICLR 2026: RLVR, test-time compute and retrieval

Conferences like ICLR give everyone a chance to take the field’s pulse and find out what’s hot, what’s not, and what’s coming up. After a few days of walking the aisles at the poster sessions and dropping in on oral sessions, you start to get a sense for things. You start to see the same words on poster after poster, and you notice which sessions are the most crowded.

Here are a few things we picked up on:

Reinforcement Learning with Verifiable Rewards (RLVR) is now the dominant paradigm for post-training refinement. Almost every reasoning-focused poster we stopped at was using some form of Group Relative Policy Optimization (GRPO) for math correctness, code execution, and formal-logic checks, rather than Reinforcement Learning from Human Feedback (RLHF). Direct Preference Optimization (DPO) fine-tuning, which felt like the default a year ago, was conspicuously rare. It makes sense: If you can use code to check for correctness, you no longer need to get annotated data and the training loop goes much faster.

Test-time compute has stopped being a curiosity and become a design problem. Test-time compute – the time a system spends generating a response – is an increasingly important study variable. Papers now measure it as part of their experimental setup and developers try to optimize for it. Models are now built with the expectation that inference will be expensive and clever, not just a single forward pass through a neural network.

Vision-Language Models (VLMs) are everywhere, and Vision-Language-Action models (VLAs) are not far behind. A big chunk of the conference was about how to make multimodal AI work better, like better tokenization for images, better positional encodings for non-text media, and more efficient ways to compress visual information before it overwhelms your model. Vision-Language-Action models that extend multimodal AI recipes to robotics and embodied agents are no longer niche research. They brought in the crowds at their presentations and hosted vibrant debates.

Reports of the death of State-Space Models (SSMs) have been greatly exaggerated. Attention models still dominate AI, but Mamba, SSM variants and recurrent neural networks still draw attention and research, both as full replacements for Transformers and as components inside hybrid attention-based stacks. Whether they'll ever genuinely displace Transformers is an open question, but the line of research is alive and well.

Agentic AI safety is taken very seriously. A lot of papers and presentations discussed problems like machine unlearning and jailbreaking, and some of the most interesting work was on prompt injection through agentic tool use, like when a model dutifully follows instructions hidden in a webpage or an API response it just fetched. A repeated, slightly unsettling observation: models that follow instructions better tend to be more vulnerable to this kind of attack, not less. This capability-vulnerability tension is going to define a lot of the next few years of safety research.

Hallucination and factuality are increasingly framed as retrieval problems. Several talks made that point explicitly: A generative model that has to invent facts will inevitably hallucinate them, while a model that retrieves information can ground its responses in verifiable ways. That framing is, of course, exactly the bet that search AI engineers have been making all along.

ICLR 2026 invited talks: hidden universe imaging and open AI development

Two of the invited talks stood out to us, albeit for very different reasons:

Images of the Hidden Universe

Katie Bouman presented a tour of how physics, prior knowledge, and machine learning combine to reconstruct information that the universe never gives us directly, like the silhouettes of supermassive black holes and the invisible dark matter structures. She walked us through the Event Horizon Telescope's imaging of M87 and Sagittarius A, building images up from indirect and incomplete radio measurements, and then extended the same machinery to mapping dark matter through gravitational lensing.

This talk was a useful reminder of why machine learning matters outside the LLM bubble. The more you already know, the more you can learn from a little bit more information. This principle generalizes beyond astronomy to knowledge in general, and to machine learning in particular. Any decision system that uses sparse, noisy observations is confronted with it.

_____________________________________________________________________________________

Marin: Open Development of Frontier AI

Percy Liang opened his presentation with a blunt observation: As AI capabilities skyrocket, openness plummets. His response is Marin, a platform for community-driven AI research where every experiment is open, every suggestion or discussion is on public fora, and anyone can review or rerun a result.

What makes Marin interesting isn't just creating open weight models - plenty of projects do that - but creating an open process for making models. Project pre-registration, peer review, and reproducibility have long been part of the natural sciences, and Marin attempts to maintain that tradition for AI. Model training is treated as a matter of public scientific record.

The talk presented concrete scientific results from this approach (optimizer findings and scaling-law results), suggesting that community-scale science isn't just an aspiration but a workable methodology.

_____________________________________________________________________________________

Bouman and Liang made a pleasingly complementary pair: one a reminder of how much ML has to offer the world outside ML, the other a challenge to how the field organizes itself.

ICLR 2026 research highlights: embedding models, retrievers and sparse representations

We attended many oral presentations and poster sessions. The papers below stood out because of their potential to impact how we make and use embedding models.

Rethinking pretraining for representations

Decoder-only models have dominated the LLM leaderboards for years, but one paper makes a case for encoder models.

Seq vs Seq: An Open Suite of Paired Encoders and Decoders does a repeatable, open-data, architecture-controlled comparison of encoder-only and decoder-only models trained identically. They used the same data, same architecture, same training recipe, and differed only in their training paradigms: Bidirectional Masked Language Modeling (MLM), typically associated with encoders, vs. Causal Language Modeling (CLM), usually used in decoders. Their results confirm prior findings that encoders excel at classification and retrieval while decoders excel at generation. A key finding is that cross-objective continuous pretraining does not close the performance gap between the encoders and decoders. A 400M parameter encoder beats a 1B parameter decoder in classification and retrieval, and vice versa for generative tasks. All artifacts including data, checkpoints, and code are open-sourced.

Their study delivers a definitive empirical finding for the AI community: Encoder-only pretraining is substantially more efficient for classification and retrieval tasks than adapting decoders to act like encoders, even with post-training on high-quality data. This challenges the recent trend of adapting large decoder LLMs (like LLM2Vec) for embedding tasks. Dedicated encoder pretraining from scratch remains the most reliable path to strong retrieval performance. Additionally, the public release of 200+ checkpoints with batch-ordered training data makes their work an invaluable resource for studying how retrieval-relevant representations emerge during training and how they scale with parameter count and tokens.

New paradigms for training retrievers and embedders

Revela: Dense Retriever Learning via Language Modeling reframes dense retriever training as a language modeling problem. Rather than using supervised training with query-document pairs, it trains a retriever model jointly with a language model by conditioning next-token prediction on all the other documents in the batch. This innovative in-batch attention mechanism modifies the model’s Transformer blocks by injecting the similarity scores of documents in each batch into the cross-document attention weights. Training is done on raw text, without query-document pairs, hard negatives, or synthetic data generation. The resulting 3B parameter model outperforms E5-Mistral-7B-Instruct (with 7B parameters) as well as proprietary closed-weight embedding models like OpenAI, Cohere, and Voyage. On retrieval benchmarks, it matches E5 despite using roughly 1000 times less training data and approximately 10 times less compute.

This demonstrates that next-token prediction can still serve as an effective training objective for high-quality dense retrieval AI. This is important because plain text data – what you need for next-token prediction – is widespread and inexpensive and this paper shows that it’s all you need to train competitive embedding models.

Let LLMs Speak Embedding Languages: Generative Text Embeddings via Iterative Contrastive Refinementadvances the proposition that LLMs should learn to "speak an embedding language," i.e., generate sequences of “soft tokens” optimized for semantic representation rather than human readability. They outline innovative loss functions and objectives in support of this goal, and show that the resulting models have very competitive performance, while generating only a handful of additional tokens.They also show that generating more tokens at inference time steadily improves embedding quality in a way analogous to chain-of-thought scaling in reasoning LLMs. KV-caching reduces the computational overhead of the generation process to within 1.1 times that of standard single-pass embedding models. This approach represents a new paradigm for representation learning, complementary to encoder-only and single-pass approaches.

Towards Improved Sentence Representations using Token Graphs frames the problem of generating embeddings for sentences from token-level representations as a relational learning problem rather than a compression problem. Instead of pooling tokens, it uses a supplementary neural network that processes a dynamically constructed graph made from output token similarities. This added network is compact, with very few trainable parameters, and can be implemented without doing any additional training on the main language model. The result is competitive with current frontier models.

This approach can be dropped into any language model at a very reasonable additional training cost, giving it immediate practical significance. Furthermore, the resulting models hold up well in the presence of noise, a known problem, especially for long-context models.

Sparse and ultra-efficient embeddings

LightRetriever: A LLM-based Text Retrieval Architecture with Extremely Faster Query Inference introduces an asymmetric dual-encoder architecture for embeddings-based retrieval in which the query encoder is much smaller and faster than the document one. The key insight is that while document embeddings benefit from the modeling power of a large language model, query embeddings are much less demanding. During training, they propose to learn per-token query embeddings, then, at query time, those embeddings are retrieved and averaged to produce a full query embedding. Documents must still be encoded at storage time using a potentially large encoder, but there is no need to invoke an embedding model at query time at all. The result retains approximately 95% of the performance of the query encoder it replaced. This has immediate implications for computational constrained, time-sensitive, or resource-efficient text information retrieval systems.

CSRv2: Unlocking Ultra-Sparse Embeddings addresses the computational cost of embeddings-based retrieval using dense, high-dimensional vectors. It tackles that cost with Contrastive Sparse Representation (CSR), which maps dense vectors into a much higher-dimensional space where only a few vector entries are non-zero, so that search can use highly efficient sparse-vector search techniques like inverted-indexes.

CSR approaches tend to break down when the number of dimensions with non-zero values becomes very low. This paper addresses this problem with an innovative training approach that makes ultra-sparse representations viable, opening up the possibility of much faster, less computationally demanding retrieval without loss of accuracy.

Multi-step and multimodal retrieval

Q-RAG: Long-Context Multi-Step Retrieval via Value-Based Embedder Training frames the problem of multi-step retrieval-augmented generation (RAG) in terms of optimizing the embeddings used in RAG search. RAG systems are typically based on a single retrieval step: Input to an LLM becomes a query to a vector store, and a selection of the results are presented to the LLM as a basis for composing a response. However, agentic approaches that involve multi-step interactions between the LLM and vector store can improve RAG performance significantly, especially for large input contexts that might contain millions of tokens. This paper seeks to optimize the embedding model used for retrieval to better support this usage scenario with Reinforcement Learning with Verifiable Rewards (RLVR).

This paper is one of the more elegant intersections of two of the conference's biggest themes — RLVR and retrieval — and it gives a glimpse of what retrieval looks like when it has to operate inside an agentic loop, not just before one.

Foundations and evaluation

HUME: Measuring the Human-Model Performance Gap in Text Embedding Tasks undertakes the unusual task of systematically measuring human performance on the Massive Text Embedding Benchmark (MTEB), the most widely used benchmark for embeddings-based information retrieval. Using 16 datasets in 5 languages, they find that average human retrieval accuracy is 77.6%, while the best embedding models currently score over 80%. However, this performance gap is uneven. Models may outperform humans on standard tasks but fall apart when faced with low-resource languages, where human intuition still holds a significant lead.

This paper also shows that "superhuman" scores on low-agreement tasks are mostly artifacts of fitting noise, not genuine capability. This underlines the problem of our current suite of embedding benchmarks: New models are not improving benchmark performance very much. To make progress, we need new, harder challenges and a total rethink of how we evaluate models.

Training dynamics for foundation models

How Learning Rate Decay Wastes Your Best Data in Curriculum-Based LLM Pretraining identifies a significant but underexplored problem in AI model training. Large training sets can create a problem with models forgetting things that they’ve learned as they’re presented with more data. Curriculum-based pretraining — sorting data from low to high quality — should help, but in practice the results have been disappointing. The reason, the authors argue, is that the model encounters the highest-quality data late in the training schedule when the learning rate is at its lowest. Its gradient contribution is therefore greatly reduced. They confirm that hypothesis empirically by showing that curriculum training significantly beats random shuffling if training uses a constant learning rate.

They propose two simple strategies to fix this: Let the learning rate decay more slowly, or replace learning rate decay with weight-averaging over the multiple final checkpoints. Combining the two yields a 1.64% average benchmark improvement over standard practices with no additional data refinement. The broader message - that data composition and optimization schedule need to be co-designed - applies well beyond pretraining, and is a useful frame for embedding training too.

What ICLR 2026 means for retrieval and embedding research

Science has always been conducted through print and publication, but in-person conferences are still the only way to put people together in a room. Over five days, we met a steady stream of researchers from very different backgrounds — academia and industry, large labs and small startups, half a dozen countries — and conversations ranged from research trends to philosophical questions that have haunted AI from the beginning. Are LLMs really reasoning, or are they doing something more like very high-dimensional memorization with interpolation? Where exactly is the line, and does it matter for what we can build on top of them?

These conversations rarely produce answers, but they sharpen the questions, which is most of what good research is.

For the information retrieval work we do at Jina by Elastic, the future looks bright. Retrieval, long relegated to merely applied research, is increasingly recognized as the engine for keeping language models grounded. Better encoders, better embedding training paradigms, sparser representations, and retrieval that operates at the core of reasoning loops – these things matter to us all. What we saw and heard at ICLR 2026 convinces us that this is where a meaningful share of the next round of progress will come from.

We're already looking forward to seeing where the field is next year.