At its core, this solution combines the power of Elasticsearch's search capabilities with Amazon's serverless AI infrastructure. Here's how it works:

• Your natural language questions are processed through the MCP server that is deployed on Amazon Bedrock AgentCore Runtime

• The MCP server translates these questions into precise Elasticsearch queries

• Results are returned in human-readable format, making your data instantly accessible

• All of this happens in a secure, scalable environment that's production-ready, deployed on Amazon Bedrock AgentCore Runtime

In this blog post, we'll explore how to:

• Deploy Elastic's MCP server on Amazon Bedrock AgentCore Runtime

• Transform local MCP prototypes into production-ready solutions

• How to Build Conversational Interfaces on top of Elasticsearch

• Implement secure, scalable AI agent architectures

Background

Model Context Protocol (MCP)

MCP is an open protocol that revolutionizes how businesses interact with their data through AI. Unlike traditional Retrieval-Augmented Generation (RAG) systems that simply retrieve documents, MCP enables AI agents to dynamically construct and execute complex tasks in real-time, mirroring the flexibility of human problem-solving

In practice, this means a business analyst can ask a series of increasingly specific questions about market trends, and the MCP-powered system will intelligently select and combine the appropriate data sources and analytical tools to provide comprehensive answers, while still maintaining context, allowing for follow-up questions without repetition.

For instance, when analyzing a product launch, the AI might integrate data from sales reports, customer feedback, and social media sentiment, orchestrating multiple tools simultaneously to provide a holistic view, thus enabling businesses to uncover deeper insights and make informed decisions, all through natural language interactions with their data systems.

Agents

Agents are AI-powered software applications that can think, plan, and act to achieve specific goals with minimal human supervision. They use foundation models (advanced AI models) to understand and complete complex tasks.

There are two types of AI agents.

Knowledge AI agents

These agents focus on enterprise knowledge. They gather context from company data — documents, logs, dashboards, communications, and customer records — and use that information to complete business tasks.

Example: A knowledge AI agent that can search across contracts, policies, and past tickets to help a customer support representative instantly resolve an issue.

General AI agents

These agents go further. They can understand goals and autonomously execute tasks on behalf of a user in broader, cross-domain workflows.

Example: A general AI agent that books travel, manages schedules, and negotiates with other systems to complete a user’s request end-to-end.

Elastic’s role in agentic AI

For knowledge AI agents: Elastic enables secure access to enterprise data, retrieves relevant context, and grounds responses in facts.

For general AI agents: Elastic serves as the knowledge store and context engine, providing trusted information so agents can perform more complex, goal-driven tasks.

In short, Elastic isn’t just storing data; it’s making enterprise knowledge usable, actionable, and AI-ready. This is the foundation for building intelligent agents that both understand and act.

AWS partnership and MCP server

Elastic has earned the AWS Generative AI Competency status. This recognition is awarded to AWS partners who deliver cutting-edge generative AI solutions that drive measurable gains in business efficiency, creativity, and productivity

Elastic also integrates with the Model Context Protocol (MCP), providing a seamless way for AI agents and applications to interact with Elasticsearch data through natural language conversations.

With the MCP server, you can connect to Elasticsearch directly from any MCP client — such as Claude Desktop, MCP Inspector, or an agentic application. The Elasticsearch MCP server is free to use (though infrastructure and Elasticsearch cluster costs may apply).

And with Amazon Bedrock models (such as Anthropic’s Claude) supporting MCP clients, organizations can now deploy intelligent, data-aware agents more easily and powerfully than ever before.

Amazon Bedrock AgentCore

Amazon Bedrock AgentCore is an enterprise-grade orchestration platform designed for scalable AI agent deployment and management. 

• The platform provides serverless runtime environments with session isolation capabilities, enabling concurrent agent operations across multiple frameworks.

• It implements memory management systems for both session-state and persistent storage, facilitating context-aware model interactions and learning capabilities.

• The architecture includes observability features with granular logging, metrics collection, and advanced debugging capabilities for agent trajectory analysis.

• The platform's robust identity and access management layer enables secure service-to-service authentication and fine-grained authorization controls for AWS and third-party service integrations.

It features a protocol-agnostic gateway for API transformations and tool discovery, supporting MCP-compliant interfaces. The infrastructure also includes containerized browser instances for web automation workflows and isolated compute environments for secure code execution. This end-to-end solution eliminates the need for building custom infrastructure components while maintaining enterprise security and compliance standards.

Solution overview

High-level architecture

The architecture consists of four main components:

1. Python client: Handles user interactions and AWS authentication

2. Amazon Bedrock AgentCore Runtime: Provides serverless hosting and session management

3. Elastic MCP server: Processes MCP protocol requests and queries Elasticsearch

4. Elasticsearch cluster: Stores and indexes the searchable data

Step-by-step workflow walkthrough

1. User authenticates using an authentication mechanism such as OAuth.

2. User access secured Client application running in a Customer AWS account using authenticated credentials.

3. The client application invokes a Supervisor Agent that further invokes and orchestrates other Agents.

4. All the agents are deployed on Amazon AgentCore Runtime and their tools are made available for the Agents, including Elastic’s MCP server and its tools.

5. Foundation Models are available for the agentic AI application through Amazon Bedrock.

6. Elastic Cloud is deployed on AWS and its endpoints are accessed by the Elastic MCP Server. Elastic MCP server automatically crafts the required queries, runs the queries against the Elastic data and fetches the response back to the Supervisor Agent.

7. Supervisor Agent responds back to the end user via the Client Application.

Implementation guide

Please refer to this GitHub repo to get a hands-on experience of how this solution can be implemented. Pay close attention to the prerequisites before getting started.

Step 1: Deploy Elastic MCP server to ECR

The automated deployment script handles the entire container build and upload process:

Here is what the script does when you execute:

1. Downloads the official Elastic MCP server repository

2. Builds Docker container using Dockerfile-8000

3. Creates ECR repository with image scanning enabled

4. Uploads container image to ECR with proper tagging

Step 2: Create AgentCore Runtime host

Navigate to the AWS Console and configure your AgentCore Runtime:

1. Access AgentCore: Go to Amazon Bedrock AgentCore > Build and Deploy > Agent Runtime > Host Agent

2. Basic configuration: Click on “Host Agent” and give it a meaningful name if you prefer. Point to the Container Image you have uploaded to Amazon ECR.

3. Service role: Choose "Create and use a new service role"

4. Protocol settings: Choose MCP, and for the Inbound Identity, select Use IAM username

5. Environment variables: Finally, configure your Elasticsearch endpoints and pass them as environment variables to your Docker container.

6. After creating the host agent, go ahead and copy the Agent Runtime ARN from the "View invocation code" section. Here is an example:

Step 3: Configure Python client

Install dependencies:

Go ahead and initialize a virtual environment and install the Python libraries.

Update Agent ARN in my_mcp_client_remote.py:

Next, update the Python MCP client with the agent ARN you obtained in the previous steps.

Here are the key client components in this Python file.

AWS authentication class:

MCP request formation / payload:

Step 4: Run the client

Execute the Python client to test the integration. This Python program implements an asynchronous client for interacting with Amazon Bedrock AgentCore, specifically designed to query event information. The code utilizes AWS SigV4 authentication and consists of two main functions: test_mcp_endpoint() and chat_with_agentcore(). The first function demonstrates basic API interaction by listing available tools and performing a search query, while the second function implements a more sophisticated search functionality specifically for events in Paris.

The program uses the httpx library for async HTTP requests and handles Server-Sent Events (SSE) responses, parsing and displaying event details including names, venues, dates, and descriptions. The authentication is managed through a custom AWSAuth class that handles AWS SigV4 signing of requests. The code includes comprehensive error handling and formatted output display, making it suitable for both testing and production use cases.

Use case demonstrations

Use case 1: Data discovery

Scenario: Finding events in a specific city using natural language.

Query: "Events in Paris"

MCP request: Here is the payload you supply to the Amazon Bedrock Agentcore Runtime.

Response: And here is the response you get after Elastic’s MCP server runs a query in Elastic Search AI Platform and returns the result.

Use case 2: Elastic MCP tool discovery

Scenario: Discovering the available MCP tools that Elastic’s MCP server offers.

MCP request:

This returns a list of available tools that the MCP server provides, enabling dynamic tool discovery.

Use case 3: Complex search queries

Scenario: Advanced filtering with multiple criteria. You can run more advanced Elastic Search Query Language based queries, like one shown below.

Query: Events with specific price ranges, dates, and categories.

MCP request:

How it works (technical deep dive)

MCP implementation

The Model Context Protocol uses JSON-RPC 2.0 format for all communications:

Key protocol features:

• Stateless operation: Each request is independent with session isolation

• Tool discovery: Dynamic discovery of available capabilities

• Structured responses: Consistent response format across all tools

• Error handling: Standardized error reporting and recovery

AWS authentication flow:

Session management

AgentCore automatically adds Mcp-Session-Id headers for session isolation:

• Each client session gets a unique identifier

• Stateless servers can maintain conversation context

• Automatic cleanup of inactive sessions

Response processing pipeline

1. Server-Sent Events (SSE): Responses come as data: {...} in JSON format

2. JSON parsing: Extract JSON from SSE wrapper

3. Content extraction: Parse the MCP result structure

4. Data formatting: Convert Elasticsearch results to user-friendly format

Cleanup

After you have played around with this setup, if you would like to clean up the environment, please follow the steps outlined below.

Delete AgentCore Runtime:

1. Navigate to Amazon Bedrock AgentCore in AWS Console

2. Select your agent runtime

3. Click "Delete" and confirm

Remove ECR repository:

Clean local environment:

Conclusion

By deploying Elastic's MCP server on Amazon Bedrock AgentCore Runtime, we've created a powerful, scalable, production-ready solution for natural language interaction with Elasticsearch data. This implementation opens up new possibilities for data exploration and analysis, making complex queries accessible through simple conversations.

Key takeaways include:

• Seamless integration: MCP protocol enables natural language querying of complex data

• Production scalability: AgentCore provides enterprise-grade hosting with minimal configuration

• Developer productivity: Transform local prototypes to production with minimal code changes

• Security first: Built-in AWS security and authentication mechanisms

Potential applications can be in any of the following areas of implementation:

• Customer support: Natural language querying of support ticket databases

• Business intelligence: Conversational analytics for business metrics

• Content discovery: Intelligent search across document repositories

• IoT data analysis: Natural language queries for sensor and telemetry data

Additional resources:

• Amazon Bedrock integration documentation

• Elastic MCP server documentation

To the public, generative AI has seemingly appeared from nowhere. But if you dig deeper, you’ll note that the ideas underlying generative AI solutions trace their lineage back to inventions such as the Mark I perceptron in 1958 and neural networks in the late twentieth century.

Advancements in statistical techniques, the vast growth of publicly available data and advancements in Machine Learning (specifically the invention of the transformer-based neural network architecture) have led to the rise of models that contain billions of parameters or variables. To give a sense for the change in scale, the largest pre-trained model in 2019 was 330M parameters. Cohere's Command XL model, one of the leading models in Stanford’s Holistic Evaluation of Language Models (HELM) benchmark is trained on 52.4 billion parameters - ~1580x increase in size in just a few years.

Foundation Models (FMs) are ML models trained on massive quantities of structured and unstructured data, which can be fine-tuned or adapted for more specific tasks.

Large Language Models (LLMs) are a subset of FMs focused on understanding and generating human-like text. These models are ideal for needs such as translation, answering questions, summarizing information, and creating or identifying images.

LLMs can perform a wide range of tasks that span multiple domains, like writing blog posts, solving math problems, engaging in dialog, and answering questions based on a document. The size and general-purpose nature of FMs make them different from traditional ML models, which typically perform specific tasks, like analyzing text for sentiment, classifying images, and forecasting trends.

The primary goal of LLMs is to enable meaningful and engaging conversation between humans and machines and have become an immensely effective resource in countless industries, helping business to improve the customer experience.

Limitations of LLMs

LLMs have certain limitations. One notable constraint is that they are trained on general domain corpora, making them less effective on domain-specific tasks. There are scenarios when you want models to generate text based on specific data rather than generic data. For example, a health service provider company may want their chatbot to answer questions using the latest information stored in an enterprise document repository, so that the answers are specific to the health service provider’s business.

Also, these LLMs are trained offline until a knowledge cutoff date. It will be agnostic of any developments that have happened after the knowledge cutoff date. This may lead to inaccurate interpretations. For example, in 2021 San Francisco was the most expensive Bay Area City for renters. Today, it is Mountain view - source.

Approaches to enhance LLMs

There are two popular ways to reference contextual data in LLMs.

The first option is to fine-tune the base LLMs with contextual data. But, using this approach generating the correctly formatted information is time consuming. Also, it is costly to fine-tune a model. In addition to that, if the domain specific data is changing frequently, it would require frequent fine-tunings and retraining to provide accurate responses. This impacts time to market and also increases overall cost of the solution. In addition to that, not all LLMs provide an option to fine-tune.

To overcome these constraints, we can use a technique called Retrieval Augmented Generation (RAG). RAG is a process in which the model retrieves contextual documents from an external data source like Elasticsearch as part of its execution. These contextual documents are used in conjunction with the original input to produce an output.

Below are a few examples of how RAG can be used in various applications to improve the quality and relevance of generated content.

• Chatbot responses: In a chatbot system, RAG refers to combining a retrieval-based approach with a generative model. The retrieval component obtains relevant responses from a pre-defined database or knowledge base, while the generative model can add additional context or generate more fluent and diverse responses. This combination helps the chatbot provide more accurate and contextually appropriate answers to user queries.

• Content generation: RAG can be used in content generation tasks such as summarization or paraphrasing. The retrieval component can retrieve relevant sentences or paragraphs from existing documents or articles, and the generative model can then augment or rephrase the retrieved content to create new and original summaries or paraphrases.

• Recommendation systems: RAG can also be applied in recommendation systems. The retrieval component can retrieve a set of candidate items or products based on user preferences or history, and the generative model can then generate personalized recommendations or provide additional information about the recommended items to enhance the user’s decision-making process.

RAG using Elasticsearch and Cohere Command model through Amazon Bedrock

Why Cohere?

Cohere is the leading AI platform for enterprise. The company builds world-class LLMs that allow computers to search, understand meaning, and converse in text. Cohere's models are uniquely suited to the needs of business, providing ease of use and strong security and privacy controls across multiple deployment options. Companies can use the models out-of-the-box or tailor them to their particular needs using their own custom data.

Command is Cohere’s flagship text generation model. It is trained to follow user commands and to be instantly useful in practical business applications, such as text generation, summarization, RAG, and chat. Command ranks as one of the leading language models according to the Stanford’s HELM website an evaluation leaderboard comparing large language models on a wide number of tasks from Stanford University (March 2023 results). Customers can use Cohere's Command LLM through Amazon SageMaker Jumpstart and Amazon Bedrock.

Embed is Cohere’s representative model which translates text into numerical vectors that models can understand. Cohere provides industry-leading English and multilingual models (100+ languages) for a range of use cases, including semantic search, text classification, and semantic engine for RAG.

Why Elasticsearch?

To make the most of generative AI, it is essential to have a unified data platform where the organization's data is stored, making it easily (and safely) accessible and searchable in one centralized location.

Elasticsearch is a distributed, open source search and analytics engine for all types of data, including textual, numerical, geospatial, structured, and unstructured. Raw data from Enterprises flows into Elasticsearch from a variety of sources, including logs, system metrics, and web applications. Elasticsearch is built on top of Lucene and it excels at full-text search. Elasticsearch is fast and excels at delivering the most relevant responses to users.

In addition to full-text search, Elasticsearch also supports vector-search.

Vector search leverages ML to capture the meaning and context of unstructured data, including text and images, transforming it into a numeric representation. Frequently used for semantic search, vector search finds similar data using approximate nearest neighbor (ANN) algorithms. Compared to traditional keyword search, vector search yields more relevant results and executes faster. Users can enhance the search experience by combining vector search with filtering and aggregations to optimize relevance by implementing a hybrid search and combining it with traditional scoring.

Elasticsearch provides an easy-to-use and performant API to enable integration with other services. These features make Elasticsearch a preferred choice for enterprises to store business data and improve search experience.

Elasticsearch allows for the seamless integration of domain-specific context from the organization's data, thereby enhancing the performance and value of generative AI for achieving desired business objectives.

Why Amazon Bedrock?

Amazon Bedrock is a fully managed service that offers a choice of high-performing FMs from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon with a single API, along with a broad set of capabilities you need to build generative AI applications, simplifying development while maintaining privacy and security. With the comprehensive capabilities of Amazon Bedrock, you can easily experiment with a variety of top FMs, privately customize them with your data using techniques such as fine-tuning and RAG, and create managed agents that execute complex business tasks—from booking travel and processing insurance claims to creating ad campaigns and managing inventory—all without writing any code. Since Amazon Bedrock is serverless, you don't have to manage any infrastructure, and you can securely integrate and deploy generative AI capabilities into your applications using the AWS services you are already familiar with.

Amazon Bedrock offers several capabilities to support security and privacy requirements and has achieved HIPAA eligibility and GDPR compliance. With Amazon Bedrock, content is not used to improve the base models and is not shared with third-party model providers. Data in Amazon Bedrock is always encrypted in transit and at rest, and can encrypt the data using your own keys. AWS PrivateLink can be used with Amazon Bedrock to establish private connectivity between FMs and your Amazon Virtual Private Cloud (Amazon VPC) without exposing your traffic to the Internet.

Solution overview

Here's how to use RAG to enable Generative AI capabilities on domain-specific business data using Elasticsearch and Cohere Generate Model -Command through Amazon Bedrock.

The below architecture diagram explains how to get domain-specific responses from Cohere Command Model through Amazon Bedrock using enterprise data hosted in Elastic Enterprise Search using a technique called RAG.

Figure 1. RAG Architecture using Elasticsearch and Amazon Bedrock

Step by step explanation

Offline Data Ingestion:

i. The documents are ingested using web crawler or any other ingestion mechanism.

ii. The Elastic Learned Sparse EncodeR (ELSER) model will embed the text and store the resulting tokens in the Elasticsearch Index

Real-time flow on user query:

1. The User provides a question via the RAG web application

2. The RAG application generates a retrieval request initialized from the vector store (Elasticsearch index). At query time, the text will be embedded using the ELSER model and the resulting tokens will be used to perform a text expansion query.

3. The Retriever component of RAG application fetches the relevant documents from Elasticsearch vector store.

4. The RAG application passes the retrieved documents (context) along with user question (prompt) to the Cohere Command Model through Amazon Bedrock

5. The Cohere Command Model through Amazon Bedrock generates a textual response and sends it back to RAG application

6. The RAG application performs any required post processing tasks. For example, it adds source to the response generated from Cohere Command Model. The User views the response in the web application.

We used the following AWS and third-party services:

1. Amazon Bedrock for interacting with LLMs from Cohere.

2. Cohere Command Model for Text Generation.

3. Elasticsearch for storing embeddings of the enterprise knowledge corpus and doing similarity search with user questions.

4. Python, LangChain and Streamlit for building the RAG application

5. Amazon EC2 for hosting the Streamlit application

6. AWS Identity and Access Management roles and policies for access management.

Prerequisites:

1. Sign up for a free trial of Elasticsearch cluster with Elastic Cloud

2. Create a new deployment on AWS following the steps

3. Reset and download the elastic user password following these steps

4. Copy the Cloud ID from the My Deployment page listed under Deployments

5. Deploy ELSER Model: In Kibana, navigate to Machine Learning> Trained models. ELSER can be found in the list of trained models. Click the Download model button under Actions. After the download is finished, start the deployment by clicking the Start deployment button. Go to the Elastic ELSER page to find more details.

1. Install packages and import modules: Firstly, we need to install modules. Make sure python is installed with min version 3.8.1. Then we need to import modules.

1. Initialize Amazon Bedrock client using the following code

1. Connect to Elasticsearch using Elastic Cloud Id, Elastic username and Elastic password. Use ElasticsearchStore to connect to our elastic cloud deployment. As we’re using ELSER we use “SparseVectorRetrievalStrategy”. This strategy uses Elasticsearch’s sparse vector retrieval to retrieve the top-k results.

1. Download the dataset, deserialize the document and split the document into passages. We’ll chunk the documents into passages in order to improve the retrieval specificity and to ensure that we can provide multiple passages within the context window of the final question answering prompt. Here we are chunking into 800 tokens with an overlap of 400 tokens. Here, we are using a simple splitter but LangChain offers more advanced splitters to reduce the chance of context being lost.

1. Index data to Elasticsearch using ElasticsearchStore.from_documents.

1. Initialize the Amazon Bedrock LLM. In the Amazon Bedrock instance, will pass bedrock_client and specific model_id. In this case model_id = cohere.command-text-v14.

1. Asking a question: Now that we have the passages stored in Elasticsearch and LLM is initialized, we can now ask a question to get the relevant passages.

1. (Optional) You can also add a reranking step to the search pipeline which can further improve the ranking of the results returned in the search step. See the Deploying with Amazon SageMaker guide on using Rerank.

Conclusion

In this post, we showed how to create a RAG web application using a combination of Elasticsearch, Amazon Bedrock, Cohere Command Model and open source python packages like LangChain and Streamlit.

We encourage you to learn more by exploring Amazon SageMaker Jumpstart, Amazon Bedrock, Cohere, and Elasticsearch and building a solution using the sample implementation provided in this post and a dataset relevant to your business. If you have questions or suggestions, leave a comment.

Elastic and Amazon Web Services (AWS) understand this pressing need and have taken the lead in offering cutting-edge solutions to meet these demands. Using Amazon SageMaker JumpStart combined with Elasticsearch’s capabilities, businesses can now confidently explore and adopt the most suitable AI models for their specific use cases while maintaining cost-effectiveness, security, and privacy.

Elasticsearch’s integration with advanced AI models further enhances its capabilities. By leveraging Elasticsearch’s retrieval prowess, LLMs can access the most relevant documents to provide accurate and up-to-date responses. This synergy between Elasticsearch and LLMs ensures that users receive contextually relevant and factual answers to their queries, setting a new standard for information retrieval and AI-powered assistance.

Elasticsearch is a scalable data store and vector database that offers a range of features to ensure exceptional search performance. It supports traditional keyword and text-based search using the BM25 algorithm, as well as AI-ready vector search with exact match and approximate kNN (k-Nearest Neighbor) search capabilities. These advanced features allow Elasticsearch to retrieve highly relevant results for queries expressed in natural language. By combining traditional, vector, or hybrid search approaches, Elasticsearch delivers precise results, making it effortless for users to find the information customers need.

The Elasticsearch platform seamlessly incorporates robust machine learning and artificial intelligence capabilities directly into its solutions, empowering you to create highly sought-after applications and accomplish tasks with remarkable efficiency. By leveraging these advanced technologies, you can harness the full potential of Elasticsearch to deliver exceptional user experiences and expedite your workflow.

Implementing RAG using Elasticsearch and open source LLM available in Amazon SageMaker JumpStart

The solution below explains how to use Retrieval Augmented Generation (RAG) to enable GAI capabilities on domain-specific business data using Elasticsearch, Amazon SageMaker JumpStart, and your choice of open source LLMs.

Solution overview

We will start by reviewing the architecture diagram below. It explains how to get domain-specific responses from an LLM hosted in Amazon SageMaker JumpStart using enterprise data hosted in Elasticsearch using RAG.

We used the following AWS and third-party services:

1. Amazon SageMaker and Amazon SageMaker JumpStart for hosting the open source LLMs from Hugging Face

2. Falcon 40B Instruct and Flan-T5 XL LLM from Hugging Face

3. Elasticsearch for storing embeddings of the enterprise knowledge corpus and doing similarity search with user questions

4. Python, LangChain, and Streamlit for building the RAG application

5. Amazon EC2 for hosting the Streamlit application

6. AWS Identity and Access Management roles and policies for access management

Step-by-step explanation

Offline data ingestion:

We ingest data from an enterprise knowledge corpus – for example this could be internal web pages, documents describing a company’s process, or corporate financial data.

1. The documents are ingested using a web crawler or any other ingestion mechanism.

2. The textual content is converted into vectors and stored in a dense_vector field by a sentence transformer type ML model.

Real-time flow on user query:

1. The user provides a question via the Retrieval Augmented Generation (RAG) web application.

2. The RAG application generates a hybrid search request for Elasticsearch based on the user's question and sends it to Elasticsearch. The hybrid search request does a BM25 match on the text field and kNN search on the dense_vector field.

3. Elasticsearch returns the document body and source URL (if applicable) to the RAG application. The RAG application accepts only the top scored document.

4. The RAG application passes the top scored document body (context) along with user question (prompt) to the LLM hosted as Amazon SageMaker endpoint.

5. The Amazon SageMaker endpoint generates a textual response and sends it back to the RAG application.

6. The RAG application performs any required post processing tasks. For example, it adds a source url to the response generated from the LLM. The user views the response in the web application.

Let’s now look at a few setup steps and a few implementation steps to create a working search solution:

Setup steps:

1. Sign up for a free trial of an Elasticsearch cluster with Elastic Cloud.

2. Create a new deployment on AWS following these steps.

3. Add a new machine learning node following the steps below. This will enable you to run machine learning models in your deployment.

4. Click on Edit under Deployment Name in the left navigation bar.

5. Scroll down to the Machine Learning instances box.

6. Click +Add Capacity.

7. Under Size per zone, click and select 2GB RAM.

8. Click on Save and then Confirm.

9. Reset and download the elastic user password following these steps.

10. Copy the deployment ID from the Overview page under Deployment name.

11. Load an embedding model into Elasticsearch. Here, we have usedall-distilroberta-v1 model hosted in the Hugging Face model hub. You can choose other sentence transformer types based on your case. Import this Python notebook here in Amazon SageMaker and run it. Provide the Cloud Id , Elasticsearch username , and Elasticsearch password when prompted. This will download the model from Hugging Face, chunk it up, load it into Elasticsearch, and deploy the model onto the machine learning node of the Elasticsearch cluster.

12. Create an Elasticsearch index by opening Kibana from the Elastic Cloud console and navigating to Enterprise Search - Overview. Click on Create an Elasticsearch Index. Choose Web Crawler as the Ingestion method. Enter a suitable Index name and click Create Index.

13. Add an Inference Pipeline by clicking on Pipelines tab > Copy and customizing it in the Ingest Pipeline Box. Click Add Inference Pipeline in the Machine Learning Inference Pipelines box. Enter the Name for the new pipeline. Select the trained Model loaded in step 6 and Select title as source field. Click Continue in two subsequent screens and click Create Pipeline at the Review stage.

1. Update the mapping for dense vector by clicking on Dev Tools and running the following code. This will enable you to run kNN search on the title field vectors. From Elasticsearch version 8.8+, this step will be handled automatically.

1. Configure web crawler to crawl Elastic Docs (you can replace this with your Enterprise Domain corpus). Click on the relevant index under Available indices. Click on the Manage Domains tab. Click Add domain. Enter https://www.elastic.co/guide/en and click Validate Domain. Click Add domain and then Add Crawl rules. Add the following rules. Click Crawl and then Crawl all domains on this index. This will start Elasticsearch’s web crawler and it will crawl the targeted documents, generate vectors for the title field, and index the document and vector.

The implementation steps for instantiating the solution presented in this post are as follows:

1. Choose your LLM. Amazon SageMaker JumpStart offers a wide selection of proprietary and publicly available foundation models from various model providers. Log in to Amazon SageMaker Studio, open Amazon SageMaker JumpStart, and search for your preferred Foundation model. Please find the list of models available for each task here.

1. Deploy your LLM. Amazon SageMaker JumpStart studio also provides a no-code interface to deploy the model. You can easily deploy a model with few clicks. After the deployment is successful, copy the Endpoint Name.

1. Download and set up the RAG Application. Launch an EC2 instance and clone the code from this GitHub link. Set up a virtual environment following these steps. Install the required Python libraries by running the command pip install -r requirements.txt. Update the config.sh file with the following:

2. ES_CLOUD_ID: Elastic Cloud Deployment ID

3. ES_USERNAME: Elasticsearch Cluster User

4. ES_PASSWORD: Elasticsearch User password

5. FLAN_T5_ENDPOINT: Amazon SageMaker Endpoint Name pointing to Flan T5

6. FALCON_40B_ENDPOINT: Amazon SageMaker Endpoint Name pointing to Falcon 40B

7. AWS_REGION: AWS Region

8. Run the application using the command streamlit run rag_elastic_aws.py. This will start a web browser and the url will be printed to the command line.

9. Response of LLM without context.

1. Response of LLM with context derived from Elasticsearch.

Conclusion

In this post, we showed you how to create a Retrieval Augmented Generation-based search application using a combination of Elasticsearch, Amazon SageMaker JumpStart, open-source LLMs from Hugging Face, and open source Python packages like LangChain and Streamlit.

Learn more by exploring JumpStart, Amazon Titan models, Amazon Bedrock, and Elastic to build a solution using the sample implementation provided in this post and a data set relevant to your business.

Or, start your own 7-day free trial by signing up via AWS Marketplace and quickly spin up a deployment in minutes on any of the Elastic Cloud regions on AWS around the world. Your AWS Marketplace purchase of Elastic will be included in your monthly consolidated billing statement and will draw against your committed spend with AWS.

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.

In this blog post, we may have used or referred to third party generative AI tools, which are owned and operated by their respective owners. Elastic does not have any control over the third party tools and we have no responsibility or liability for their content, operation or use, nor for any loss or damage that may arise from your use of such tools. Please exercise caution when using AI tools with personal, sensitive or confidential information. Any data you submit may be used for AI training or other purposes. There is no guarantee that information you provide will be kept secure or confidential. You should familiarize yourself with the privacy practices and terms of use of any generative AI tools prior to use.

Elastic, Elasticsearch, ESRE, Elasticsearch Relevance Engine and associated marks are trademarks, logos or registered trademarks of Elasticsearch N.V. in the United States and other countries. All other company and product names are trademarks, logos or registered trademarks of their respective owners.