Elastic Search 8.11: Announcing GA of Elastic’s Learned Sparse EncodeR model and a new set of enriched connectors

Elastic Learned Sparse EncodeR — or ELSER — is the Elastic retrieval model trained by Elastic^® that enables you to perform semantic search to retrieve more relevant search results. This search type provides you search results based on contextual meaning and user intent, rather than exact keyword matches.

• Universal Version, which can run on any hardware
• Optimized Version for x86 family of architectures

Elasticsearch can now be used as a vector store when using LlamaIndex. Read the docs for more details on how to go about it. 

Elasticsearch as a vector store keeps getting more love from the LangChain community. Elasticsearch is the most upvoted vector store integration on LangChain.

Elastic Search Labs continues to add great content to its list. Some of the notable recent ones are listed below.

• Use Amazon Bedrock with Elasticsearch and Langchain
  Learn to split fictional workplace documents into passages, transform these passages into embeddings in Elasticsearch, and integrate Amazon Bedrock LLM.
• How to create customized connectors for Elasticsearch
  Learn how to create customized connectors for Elasticsearch to simplify your data ingestion process.
• Less merging and faster ingestion in Elasticsearch 8.11
  Elasticsearch 8.11 improves how it manages its indexing buffer, resulting in less segment merging.

Elastic Search 8.11 comes with dense vector improvements that improve performance and make for a more intuitive experience across broader use cases. 

Below are some improvements made to the dense_vector mapping settings.

• The dense_vector field type supports indexing, which is now enabled by default, so users don’t have to explicitly set index:true in mappings.
• The number of dimensions can now be dynamically set: rather than define the number of dimensions in advance, if left undefined it will be set when the first document with a value for that field is ingested. 
• The similarity field can be left undefined, where it will default to cosine.
• Float arrays > 128 (and < 4096) are automatically mapped as a dense_vector.

With the above improvements, dense vector mappings can be defined just in one line.

Boost your unified knowledge search across your organization by using content from  private data sources such as Outlook, Teams, Zoom, and Box. 

Customers utilizing our Google Drive, OneDrive, and GitHub connector clients can now offload the cost of self managed infrastructure by switching to the newly promoted native connectors available in Elastic Cloud.

Our extended connector catalog provides customers (who were previously evaluating Enterprise Search’s Workplace Search content sources) with a full path to sync those sources of data and use it fully with ESRE capabilities. 

With the release of 8.11:

• Google Drive, OneDrive, and GitHub connectors are now available as native connectors on Elastic Cloud.
• Dropbox, ServiceNow, and GitHub are now generally available.
• OneDrive and Network Drive are enhanced with document level security capabilities to ensure that data shared across these integrations is exposed to the right privileged audience.

Binary content can be extracted from small files only via Ingest Pipelines. If you want to extract binary content from larger files (GBs in size), you can now use connector clients together with Extraction Service.

Most of our connectors support binary content extraction on edge for large files via our Extraction Service. Learn more about Extraction Service.

Read about these capabilities and more in the Elastic Search release notes.

Existing Elastic Cloud customers can access many of these features directly from the Elastic Cloud console. Not leveraging Elastic on cloud? Start a free trial.

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.