Elasticsearch .NET client evolution: From NEST to Elastic.Clients.Elasticsearch

Learn about the evolution of the Elasticsearch .NET client and the transition from NEST to Elastic.Clients.Elasticsearch.

Introduction to .NET client and NEST

In the .NET world, integration with Elasticsearch has long been facilitated by the NEST library, which serves as a robust interface for developers to interact with Elasticsearch's powerful search and analytics capabilities. NEST, born out of a need for a native .NET client for Elasticsearch, quickly gained popularity among developers for its rich feature set and seamless integration capabilities.

For nearly 14 years and only 8 months after Elasticsearch's first commit NEST has been faithfully tracking Elasticsearch releases.

Transitioning from NEST to Elastic.Clients.Elasticsearch

As Elasticsearch evolved, maintaining NEST's complex codebase became increasingly difficult. We recognized the need for a more sustainable approach to client development and went on a journey to redesign the .NET client from the ground up. It took us almost a year to release a first beta version and another year to get close to supporting every single server endpoint. One of the most difficult decisions was to reduce the scope of the library in order to prioritize maintainability instead.

Given the size of the Elasticsearch API surface today, it is no longer practical to maintain over 450 endpoints and nearly 3000 types (requests, responses, queries, aggregations, etc.) by hand. To ensure consistent, accurate, and timely alignment between language clients and Elasticsearch, the 8.x clients, and many of the associated types are now automatically code-generated from a shared specification. This is a common solution to maintaining alignment between client and server among SDKs and libraries, such as those for Azure, AWS and the Google Cloud Platform.

The Elasticsearch specification was created over 8 years ago by exporting the type mappings from NEST and through the hard work of the clients team we can now use the same specification to create a new .NET client (and clients for multiple other languages like Java, Go, etc.).

With the release of version 8.13, the deprecation of NEST was officially announced. As Elasticsearch transitions to Elastic.Clients.Elasticsearch, NEST will gradually phase out, reaching its end-of-life at the close of the year. Developers are strongly encouraged to commence migration efforts early to ensure a smooth transition and mitigate any potential disruptions. Embracing Elastic.Clients.Elasticsearch not only ensures compatibility with the latest server features but also future-proofs applications against deprecated functionality.

Elastic.Clients.Elasticsearch: features and changes overview

Switching to the v8 client Elastic.Clients.Elasticsearch enables access to all the new features of Elasticsearch 8 and also brings numerous modernizations to the library itself but also implies a reduction in convenience features compared to its predecessor. Some of the new core features include the query language ES|QL, modern machine learning (ML) capabilities and improved diagnostics in the form of OpenTelemetry-compatible activities. Starting with version 8.13, Elastic.Clients.Elasticsearch supports almost all server features of Elasticsearch 8.

An important breaking change, for example, is related to aggregations. In NEST, the fluent API usage looks like this:

s => s
.Aggregations(aggs => aggs
    .Children<CommitActivity>("name_of_child_agg", child => child
        .Aggregations(childAggs => childAggs
            .Average("average_per_child", avg => avg.Field(p => p.ConfidenceFactor))
            .Max("max_per_child", max => max.Field(p => p.ConfidenceFactor))
            .Min("min_per_child", min => min.Field(p => p.ConfidenceFactor))
        )
    )
)

while the v8 client requires the following syntax:

s => s
.Aggregations(aggs => aggs
	.Add("name_of_child_agg", agg => agg
		.Children(_ => {})
		.Aggregations(childAggs => childAggs
			.Add("average_per_child", agg => agg.Avg(avg => avg.Field(p => p.ConfidenceFactor)))
			.Add("max_per_child", agg => agg.Max(max => max.Field(p => p.ConfidenceFactor)))
			.Add("min_per_child", agg => agg.Min(min => min.Field(p => p.ConfidenceFactor)))
		)
	)
)

Migrating from NEST v7 to .NET client v8

A comprehensive migration guide is available here: Migration guide: From NEST v7 to .NET Client v8.

Additional resources

Ready to try this out on your own? Start a free trial.

Elasticsearch has integrations for tools from LangChain, Cohere and more. Join our advanced semantic search webinar to build your next GenAI app!

Related content

LangChain4j with Elasticsearch as the embedding store

October 8, 2024

LangChain4j with Elasticsearch as the embedding store

LangChain4j (LangChain for Java) has Elasticsearch as an embedding store. Discover how to use it to build your RAG application in plain Java.

Using Eland on Elasticsearch Serverless

October 4, 2024

Using Eland on Elasticsearch Serverless

Learn how to use Eland on Elasticsearch Serverless

Testing your Java code with mocks and real Elasticsearch

October 3, 2024

Testing your Java code with mocks and real Elasticsearch

Learn how to write your automated tests for Elasticsearch, using mocks and Testcontainers

How to ingest data from AWS S3 into Elastic Cloud - Part 1 : Elastic Serverless Forwarder

October 2, 2024

How to ingest data from AWS S3 into Elastic Cloud - Part 1 : Elastic Serverless Forwarder

Learn about different ways you can ingest data from AWS S3 into Elastic Cloud

Automating traditional search with LLMs

September 30, 2024

Automating traditional search with LLMs

Learn how to use LLMs to write Elastic Query DSL and query structured data with filters

Ready to build state of the art search experiences?

Sufficiently advanced search isn’t achieved with the efforts of one. Elasticsearch is powered by data scientists, ML ops, engineers, and many more who are just as passionate about search as your are. Let’s connect and work together to build the magical search experience that will get you the results you want.

Try it yourself