This new functionality further solidifies our commitment to not only working with Microsoft and the Azure platform, but also toward our commitment to offering our customers more flexibility with their AI solutions.

Ongoing Investment in AI at Elastic

This is the latest in a series of additional features and integrations on AI enablement for Elasticsearch following on from:

• Elasticsearch open inference API adds Azure AI Studio support

• Elasticsearch open inference API adds support for Azure OpenAI chat completions

• Elasticsearch open inference API adds support for OpenAI chat completions

• Elasticsearch open inference API adds support for Cohere Embeddings

• Introducing Elasticsearch vector database to Azure OpenAI Service On Your Data (preview)

The new inference embeddings service provider for Azure OpenAI is already available in our stateless offering on Elastic Cloud, and will be soon available to everyone in an upcoming Elastic release.

Using Azure OpenAI Embeddings with the Elasticsearch Inference API

Deploying an Azure OpenAI Embeddings Model

To get started, you will need a Microsoft Azure Subscription as well as access to Azure OpenAI service. Once you have registered and have access, you will need to create a resource in your Azure Portal, and then deploy an embedding model to Azure OpenAI Studio. To do this, if you do not already have an Azure OpenAI resource in your Azure Portal, create a new one from the “Azure OpenAI” type which can be found in the Azure Marketplace, and take note of your resource name as you will need this later. When you create your resource, the region you choose may impact what models you have access to. See the Azure OpenAI deployment model availability table for additional details.

Once you have your resource, you will also need one of your API keys which can be found in the “Keys and Endpoint” information from the Azure Portal's left side navigation:

Now, to deploy your Azure OpenAI Embedding model, go into your Azure OpenAI Studio's console and create your deployment using an OpenAI Embeddings model such as text-embedding-ada-002. Once your deployment is created, you should see the deployment overview. Also take note of the deployment name, in the example below it is “example-embeddings-model”.

Using your deployed Azure OpenAI embeddings model with the Elasticsearch Inference API

With an Azure OpenAI embeddings model deployed, we can now configure your Elasticsearch deployment's _inference API and create a pipeline to index embeddings vectors in your documents. Please refer to the Elastic Search Labs GitHub repository for more in-depth guides and interactive notebooks.

To perform these tasks, you can use the Kibana Dev Console, or any REST console of your choice.

First, configure your inference endpoint using the create inference model endpoint - we'll call this “example_model”:

For your inference endpoint, you will need your API key, your resource name, and the deployment id that you created above. For the “api_version”, you will want to use an available API version from the Azure OpenAI embeddings documentation - we suggest always using the latest version which is “2024-02-01” as of this writing. You can also optionally add a username in the task setting's “user” field which should be a unique identifier representing your end-user to help Azure OpenAI to monitor and detect abuse. If you do not want to do this, omit the entire “task_settings” object.

After running this command you should receive a 200 OK status indicating that the model is properly set up.

Using the perform inference endpoint, we can see an example of your inference endpoint at work:

The output from the above command should provide the embeddings vector for the input text:

Now that we know our inference endpoint works, we can create a pipeline that uses it:

This will create an ingestion pipeline named “azureopenai_embeddings” that will read the contents of the “name” field upon ingestion and apply the embeddings inference from our model to the “name_embedding” output field. You can then use this ingestion pipeline when documents are ingested (e.g. via the _bulk ingest endpoint), or when reindexing an index that is already populated.

This is currently available through the open inference API in our stateless offering on Elastic Cloud. It'll also be soon available to everyone in an upcoming versioned Elasticsearch release, with additional semantic text capabilites that will make this step even simpler to integrate into your existing workflows.

For an additional use case, you can walk through the semantic search with inference tutorial for how to perform ingestion and semantic search on a larger scale with Azure OpenAI and other services such as reranking or chat completions.

Plenty more on the horizon

This new extensibility is only one of many new features we are bringing to the AI table from Elastic. Bookmark Elastic Search Labs now to stay up to date! Ready to build RAG into your apps? Want to try different LLMs with a vector database? Check out our sample notebooks for LangChain, Cohere and more on Github, and join the Elasticsearch Engineer training starting soon!

Developers can use the capabilities of the world's most downloaded vector database to store and utilize embeddings generated from OpenAI models from Azure AI studio or access the wide array of chat completion model deployments for quick access to conversational models like mistral-small.

Just recently we've added support for Azure OpenAI text embeddings and completion, and now we've added support for utilizing Azure AI Studio. Microsoft Azure developers have complete access to Azure OpenAI & Microsoft Azure AI Studio service capabilities and can bring their Elasticsearch data to revolutionize conversational search.

Let's walk you through just how easily you can use these capabilities with Elasticsearch.

Deploying a model in Azure AI Studio

To get started, you'll need a Microsoft Azure subscription as well as access to Azure AI Studio. Once you are set up, you'll need to deploy either a text embedding model or a chat completion model from the Azure AI Studio model catalog. Once your model is deployed, on the deployment overview page take note of the target URL and your deployment's API key - you'll need these later to create your inference endpoint in Elasticsearch.

Furthermore, when you deploy your model, Azure offers two different types of deployment options - a “pay as you go” model (where you pay by the token), and a “realtime” deployment which is a dedicated VM that is billed by the hour. Not all models will have both deployment types available, so be sure to take note as well as which deployment type is used.

Creating an Inference API Endpoint in Elasticsearch

Once your model is deployed, we can now create an endpoint for your inference task in Elasticsearch. For the examples below we are using the Cohere Command R model to perform chat completion.

In Elasticsearch, create your endpoint by providing the service as “azureaistudio”, and the service settings including your API key and target from your deployed model. You'll also need to provide the model provider, as well as the endpoint type from before (either “token” or “realtime”). In our example, we've deployed a Cohere model with a token type endpoint.

When you send Elasticsearch the command, it should return back the created model to confirm that it was successful. Note that the API key will never be returned and is stored in Elasticsearch's secure settings.

Adding a model for using text embeddings is just as easy. For reference, if we had deployed the Cohere-embed-v3-english model, we can create our inference model in Elasticsearch with the “text_embeddings” task type by providing the appropriate API key and target URL from that deployment's overview page:

Let's perform some inference

That's all there is to setting up your model. Now that that's out of the way, we can use the model. First, let's test the model out by asking it to provide some text given a simple prompt. To do this, we'll call the _inference API with our input text:

And we should see Elasticsearch provide a response. Behind the scenes, Elasticsearch is calling out to Azure AI Studio with the input text and processes the results from the inference. In this case, we received the response:

We've tried to make it easy for the end user to not have to deal with all the technical details behind the scenes, but we can also control our inference a bit more by providing additional parameters to control the processing such as sampling temperature and requesting the maximum number of tokens to be generated:

That was easy. What else can we do?

This becomes even more powerful when we are able to use our new model in other ways such as adding additional text to a document when it's used in an Elasticsearch ingestion pipeline. For example, the following pipeline definition will use our model and anytime a document using this pipeline is ingested, any text in the field “question_field” will be sent through the inference API and the response will be written to the “completed_text_answer” field in the document. This allows large batches of documents to be augmented.

Limitless possibilities

By harnessing the power of Azure AI Studio deployed models in your Elasticsearch inference pipelines, you can enhance your search experience's natural language processing and predictive analytics capabilities.

In upcoming versions of Elasticsearch, users can take advantage of new field mapping types that simplify the process even further where designing an ingest pipeline would no longer be necessary. Also, as alluded to in our accelerated roadmap for semantic search the future will provide dramatically simplified support for inference tasks with Elasticsearch retrievers at query time.

These capabilities are available through the open inference API in our stateless offering on Elastic Cloud. It'll also be soon available to everyone in an upcoming versioned Elasticsearch release.