NEST and Elasticsearch.Net: upgrading your Elasticsearch server and codebase

Picture this: you've spent the last few months working on your company's application, using Elasticsearch and the NEST client quite happily, only to discover that the feature you really need has shipped in Elasticsearch server version 7.3, and you happen to be using server and client version 6.5.

The system administrators are comfortable in upgrading the Elasticsearch server, but you'd really like to continue using the 6.5 cluster and the new 7.3 cluster (with the migrated data) at the same time, in the same code base.

A note on client-server compatibility

Looking at the compatibility matrix for the NEST client you quickly discover that using a 6.x .NET client against a 7.3 Elasticsearch cluster is not recommended, and the same problem exists in attempting to use a 7.x client against a 6.5 server.

There are no compatibility assurances across using different major versions of the client against different major versions of the server. Differences can exist across major versions of the server, particularly around request and response object formats. Given that NEST is a high level client with defined types for all API requests and responses, these differences are also reflected in the client.

So, how do you solve this problem, given that you can only reasonably reference just one version of a .NET assembly at a time?

The solution

Add our package source

We offer namespaced versions of the NEST and Elasticsearch.Net clients through our continuous integration (CI) server. You can add our CI package source as a feed and then reference different client assemblies that are tied to a specific major version.

The package source address is: https://f.feedz.io/elastic/all/nuget/index.json

You will need to add our CI package source, referring to your IDE documentation:

• JetBrains Rider "Managing package sources" documentation
• Microsoft Visual Studio "Package sources" documentation

...or by creating a nuget.config file in the project or solution directory, and adding it there:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <add key="Elastic Feedz.io" value="https://f.feedz.io/elastic/all/nuget/index.json" />
  </packageSources>
</configuration>

You can pull in the latest Nest.v6 package from our CI package source, in addition to the NEST package version 7.3.0 from the official Nuget package source:

Upgrade your codebase

1. Remove the NEST and Elasticsearch.Net 6.x official Nuget packages from your project, and replace with the NEST.v6 and Elasticsearch.Net.v6 packages from our CI package source.
2. Prefix existing uses of Nest and Elasticsearch.Net types with Nest6. and Elasticsearch.Net6, respectively, to fully qualify the type names. You have now tied your code base to the 6.x client version from our CI package source.
3. Add the official nuget package of NEST and Elasticsearch.Net to the latest 7.x, and start working with this later major version.

You should then see the project file references the two different assemblies (note the CI version identifier):

<Project Sdk="Microsoft.NET.Sdk">
    <PropertyGroup>
        <OutputType>Exe</OutputType>
        <TargetFramework>netcoreapp2.2</TargetFramework>
    </PropertyGroup>
    <ItemGroup>
      <PackageReference Include="NEST" Version="7.3.0" />
      <PackageReference Include="NEST.v6" Version="6.9.0-ci20190826T071619" />
    </ItemGroup>
</Project>

Usage

The two versioned clients can then be used in the same .NET project side-by-side:

public static void Main()
{
    var uri6 = new Uri("http://v6.domain.com:9200");
    var uri7 = new Uri("http://v7.domain.com:9200");
    
    // 6.x client from Nest.v6 package
    var settings6x = new Nest6.ConnectionSettings(new Elasticsearch.Net6.SingleNodeConnectionPool(uri6)).DefaultIndex("posts");
    var client6x = new Nest6.ElasticClient(settings6x);

    // 7.x client from Nest 7.0.0 package
    var settings = new ConnectionSettings(new SingleNodeConnectionPool(uri7)).DefaultIndex("posts");
    var client = new ElasticClient(settings);
}

You can continue to use the latest 7.x release from the official nuget package source and then reference the 6.x client from our CI package source. This allows you to use two packages, and therefore clients, within the same project and helps with migrating your code base from one major version to the next.

Please note, this approach should be considered a short term solution to aid with upgrading; ideally you should remove the older client version over time. As they are CI packages, they are not held to the same standard of support as officially released packages.

Behind the scenes

As part of our CI process, we often need to be able to benchmark two different versions of the client, to identify where performance improvements have been made and to reduce the chances of introducing a performance regression.

To aid in this endeavor, a successful CI build of the client rewrites the NEST and Elasticsearch.Net assemblies using Mono.Cecil, conveniently packaged in the AssemblyRewriter project, and referenced as a global .NET CLI tool. AssemblyRewriter rewrites dependent assemblies first, then the target assembly. This ordering of rewriting ensures the rewritten target assembly references the rewritten dependencies.

Considering we already generated these packages as part of our CI process and found them to be useful, we think others might find them useful too.

In closing

The intention with providing the namespaced clients is to help customers with upgrading the client to the next major version, at the same time as upgrading the Elasticsearch server.

Looking for a seamless way to upgrade your cluster? Consider migrating to Elasticsearch Service on Elastic Cloud, where upgrading your cluster is as simple as clicking a button. Spin up a 14-day free trial of the Elasticsearch Service on Elastic Cloud — now also available on Microsoft Azure.