Elasticsearch open inference API for Anthropic’s Claude

Elasticsearch has native integrations with the industry-leading Gen AI tools and providers. Check out our webinars on going Beyond RAG Basics, or building prod-ready apps with the Elastic vector database.

To build the best search solutions for your use case, start a free cloud trial or try Elastic on your local machine now.

We are excited to announce our latest addition to the Elasticsearch Open Inference API: the integration of Anthropic's Claude. This work enables Elastic users to connect directly with the Anthropic platform, and use large language models like Claude 3.5 Sonnet to build GenAI applications with use cases such as question answering. Previously customers could access this capability from providers like Amazon Bedrock, but now can utilize their Anthropic account for these purposes.

Using Anthropic’s messages to answer questions

In this blog, we’ll use the Claude Messages API to answer questions during ingestion to have answers ready ahead of searching. Before we start interacting with Elasticsearch, make sure you have an Anthropic API key by creating an evaluation account first and generating a key. We’ll use Kibana's Console to execute these next steps in Elasticsearch without setting up an IDE.

First, we configure an inference endpoint, which will interact with Anthropic’s messages API:

We’ll get back a response similar to the following with status code 200 OK on successful inference endpoint creation:

We can now call the configured endpoint to perform completion on any text input. Let’s ask the model for a short description of GenAI:

We should get a response back with a status code 200 OK providing a short description of GenAI:

Now we can set up a catalog of questions which we want to be answered during ingestion. We’ll use the Elasticsearch Bulk API to index these questions about Elastic products:

A response similar to the one below should be returned upon successful indexing:

We’ll now create our question and answering ingest pipeline using the script, inference, and remove processors:

The pipeline prefixes the question field with the text: “Please answer the following question: “ in a temporary field called prompt. The content of the temporary prompt field is sent to the Anthropic service via the inference API. Using an ingest pipeline provides extensive flexibility as you can set the pre-prompt to fit your needs. This approach can be used to summarize documents as well.

Next, we’ll send our documents containing the questions through the question and answering pipeline by calling the reindex API.

We should get back a response similar to the following:

In a production setup, you’ll likely use another ingestion mechanism to ingest your documents in an automated manner. Check out our Adding data to Elasticsearch guide to learn more about the various options offered by Elastic to ingest data into Elasticsearch. We’re also committed to showcasing ingest mechanisms and providing guidance on bringing data into Elasticsearch using 3rd party tools. For example, take a look at Ingest Data from Snowflake to Elasticsearch using Meltano: A developer’s journey to see how to use Meltano for ingesting data.

We can now search for our pre-generated answers using the Search API:

The response will contain the pre-generated answers:

Pre-generating answers for frequently asked questions is particularly effective in reducing operational costs. By minimizing the need for on-the-fly response generation, you can significantly cut down on the amount of computational resources required. Additionally, this method ensures that every user receives the same precise information. Consistency is critical, especially in fields requiring high reliability and accuracy such as medical, legal, or technical support.