What is Elasticsearch Relevance Engine?

Elasticsearch Relevance Engine is a set of tools that help developers build AI search applications and includes: Industry leading advanced relevance ranking features, including traditional keyword search with BM25, a foundation of relevant, hybrid search for all domains. Full vector database capabilities – including the ability to create embeddings, in addition to storage and retrieval of vectors. Elastic Learned Sparse Encoder – our new machine learning model for semantic searchacross a range of domains. Hybrid ranking (RRF) for pairing vector and textual search capabilities for optimal search relevance across a variety of domains. Support to integrate 3rd- party transformer models such as OpenAI GPT-3 and 4 via APIs. A full suite of data ingestion tools such as database connectors, 3rd- party data integrations, web crawler, and APIs to create custom connectors. Developer tools to build search applications across all types of data:text, images, time-series, geo, multimedia, and more.

What can I build with Elasticsearch Relevance Engine?

Elasticsearch is a leading search technology for websites (like ecommerce product and discovery) and internal information (such as customer success knowledge bases and enterprise search). With ESRE, we're providing a toolkit to build AI powered search experiences. Let users express their queries in natural language, in the form of a question or a description of the kind of information they seek. Combine this natural language capability with Generative AI to further enhance these models' abilities with context from your own, private or proprietary data.

How are Elasticsearch and Elasticsearch Relevance Engine different?

Capabilities included with Elasticsearch Relevance Engine are designed to use Elasticsearch as an underlying storage and search technology. Developers can use the Elastic API or familiar tools, such as Kibana, to interact with capabilities that make up Elasticsearch Relevance Engine.

What is Elastic Learned Sparse Encoder?

Elastic Learned Sparse Encoder is a model built by Elastic for high relevance semantic search across a variety of domains. Currently, an English-only machine learning model, it captures the relationships between meanings and words for information retrieval. Interested in benchmark tests with our new retrieval model? Watch this video to learn more.

What is a transformer, and is Elastic Learned Sparse Encoder a transformer model?

A transformer is a deep neural network architecture which serves as the basis for LLMs. Transformers consist of various components and can be composed of encoders, decoders, and many "deep"neural network layers with many millions (or even billions) of parameters. They are typically trained on very large corpus of text like data on the Internet, and can be fine-tuned to perform a variety of NLP tasks. Our new retrieval model uses a transformer architecture but consists only of an encoder designed specifically for semantic search across a wide variety of domains.

How do I get started with Elasticsearch Relevance Engine? Do I need to purchase Elasticsearch Relevance Engine separately?

All of Elasticsearch Relevance Engine’s capabilities come with Elastic Enterprise Search Platinum and Enterprise plans, as part of the 8.8 release. You can easily get started with embeddings and vector search, and try out the retrieval model model. Check out a demo of Elastic Learned Sparse Encoder’s capabilities. If you have an Elasticsearch license, Elasticsearch Relevance Engine is included as part of your purchase.

Search in the time of generative AI

Use Elasticsearch with large language models (LLMs) to create powerful, new applications for your customers and employees. Tailor generative AI experiences to your business using real-time, proprietary data. Build cost-effective and secure AI apps that are accurate and relevant using Elastic’s vector database, out of the box semantic search, and transformer model flexibility. The future is possible today with Elastic.

Check out this primer on different ways to build AI Search and integrate with ChatGPT or other LLMs.

See blog: Demystifying ChatGPT

Find out how the Elastic AI Assistant employs generative AI to help keep your environment secure.

Read announcement: Elastic introduces Elastic AI Assistant

Learn about how Elastic is accelerating AI in this video featuring Chief Product Officer, Ken Exner.

Watch video: The role of open source in AI acceleration

ALL EYES ON AI

Subscribe for AI updates

Stay informed on all things related to AI. Sign up to receive exclusive news, Elastic® product updates, AI trends, hands-on demos, and more.

Straight from the source

LLMs, NLP, ChatGPT, RRF, BM25, ESRE, vectors, transformers, embeddings — there’s a lot to keep up with in the world of AI. Hear from Elastic product and engineering leadership and development teams about this fast-moving AI landscape so you can build better AI-powered search applications.

Sign up

MarketoFEForm

WHAT IS GENERATIVE AI USED FOR?

A world of AI possibilities

Build generative AI applications using Elastic to give end-users customized, prescriptive responses with sophisticated question-answering that references your organization's real-time data.

Learn about gen AI use cases

Retail & Ecommerce
Product recommendations, real-time insights and automation, fraud detection
Financial Services
Risk assessment, fraud detection, personalized experiences, customer insights
Human Resources
Policy requirements, regional Q&A
Government
Personalized public services, streamlined investigations, and intelligence
Legal
Compliance reporting and discovery
Manufacturing
Smart monitoring, predictive maintenance, and reporting
Medical & Pharma
Patient analysis, accelerated drug innovation, and discovery
Security
Risk assessment and analysis, incident response, improved threat detection, alert prioritization

PRIVACY & FLEXIBILITY FIRST

Link private data to generative AI with Elastic

Create a generative AI experience that's tailored to your own business and end-user needs. Elastic connects your datastore — whether it's a database, knowledge base, or case history — with large language models like OpenAI ChatGPT, Google Bard, and Hugging Face. Have your own transformer model? Bring it and manage it within Elastic. Using Langchain to build your app? We can integrate with your preferred open source frameworks too.

See sample apps

POWERING GENERATIVE AI

Your data creates the best responses

How can your organization take advantage of generative AI's massive promise and differentiate itself? Put your proprietary data to work using Elasticsearch. Pull real-time data for better efficiency and automation to create innovative customer experiences.

What is gen AI?

Customer information
Past purchases or interactions, location, contract details
Product details
Product catalog, specifications, inventory levels, promotions, documentation, support resources
Process documentation
Service level agreements, who to contact for what, how to self-serve

GEN AI FUNDAMENTALS

See through the context window

Deliver generative AI experiences with better context for customers and employees. Elastic provides generative AI models with relevant search results from your data using retrieval augmented generation (RAG).

When users query your application, Elastic provides relevant search results pulled from the data you have stored in Elasticsearch. These secure results, which contain proprietary context from your organization, get passed to the generative AI model to create more accurate responses for end-users.

ELASTIC'S CAPABILITIES

A new generation of tools

Build relevant, enterprise search experiences and AI apps with the Elasticsearch Relevance Engine™, a suite of powerful development tools that make use of a vector database, semantic search, and transformer models. It's production-ready, highly scalable, and trusted by developers worldwide.

See documentation

Vector database
Get the foundation for a full vector search experience and generative AI integration. Use a single platform to create, store, and search embeddings for dense retrieval and capture your unstructured data’s meaning and context — across text, images, videos, audio, geo-location, or other data.
Elasticsearch goes further than other vector databases with a full suite of search capabilities: filters and faceting, document level security, on-prem or cloud deployment, and more.
Learn about Elastic's vector database
Elastic's ML model for semantic search
Get relevant semantic search out of the box across domains with the Elastic Learned Sparse Encoder model. Implement it easily with a single click when setting up your new search application. Query expansions with related keywords and relevance scores make the model easily understood and ready for prime time on any dataset — no fine-tuning required.
Read Learned Sparse Encoder blog
Large language models
Incorporate your proprietary, business-specific information with LLMs so that generative AI applications don’t have to simply rely on publicly trained data. Elasticsearch is your data source for highly relevant search results that enhances the quality of LLM output via context window. Integrate with generative AI or your preferred LLM using Elasticsearch’s APIs and plugins.
Learn how OpenAI meets private data

"Elastic is a feature-rich environment for creating a variety of search solutions. The ability to combine semantic search aspects into solutions is key for search evolution and Elastic is well-positioned for it."

Prem Malhotra, Director, ML/AI & Search at Cisco

Elasticsearch Advantage

Enterprise ready

Elastic's natural language understanding and real-time data insights help you provide relevant results with generative AI, at speed and scale.

Learn about Elasticsearch Relevance Engine

Secure
Secures data and user access and removes private information with document-level control.
Relevant & cost effective
Reduce compute, storage, and costs. Deliver relevant content to gen AI, and return the best responses to your users.
Compliance baked-in
Full support for a comprehensive set of widely recognized compliance standards
Proven at scale
Trusted by 50% of the Fortune 500 for mission critical use cases at petabyte scale.
AI app platform
An end-to-end platform for building AI search applications that goes beyond gen AI.
Deploy anywhere
Run Elasticsearch where you are: on-prem, on a public cloud provider of your choice, hybrid environments, or serverless (coming soon).

Elasticsearch Platform

ELK Stack

Elastic Cloud

Observability

Security

Search

By industry

By solution

Customer spotlight

Developers

Connect

Learn

Help

See what's happening at Elastic

Elasticsearch Platform

ELK Stack

Elastic Cloud

Observability

Security

Search

By industry

By solution

Customer spotlight

Developers

Connect

Learn

Help

See what's happening at Elastic

Search in the time of generative AI

ALL EYES ON AI

Subscribe for AI updates

Straight from the source

Sign up

WHAT IS GENERATIVE AI USED FOR?

A world of AI possibilities

Retail & Ecommerce

Financial Services

Human Resources

Government

Legal

Manufacturing

Medical & Pharma

Security

PRIVACY & FLEXIBILITY FIRST

Link private data to generative AI with Elastic

POWERING GENERATIVE AI

Your data creates the best responses

Customer information

Product details

Process documentation

GEN AI FUNDAMENTALS

See through the context window

ELASTIC'S CAPABILITIES

A new generation of tools

Vector database

Elastic's ML model for semantic search

Large language models

Prem Malhotra, Director, ML/AI & Search at Cisco

Elasticsearch Advantage

Enterprise ready

Secure

Relevant & cost effective

Compliance baked-in

Proven at scale

AI app platform

Deploy anywhere

Learn how to use ChatGPT with Elasticsearch

Follow us

About us

Join us

Press

Partners

Trust & Security

Investor relations

EXCELLENCE AWARDS