After a very successful beta1 period, we are pleased to announce the availability of the release candidate for the Elasticsearch .NET clients.

So what does release candidate mean for you exactly? Just like the beta, the RC is being released on the nuget unstable channel. However, we’ve now committed our public API as released in this RC. No breaking changes will be introduced between this RC release and the final 1.0 release forthcoming on the nuget stable feed.

We’re pushing for a 1.0 stable release 10 days after this release candidate. We’ll spend these 10 days ramping up the documentation for NEST and Elasticsearch.NET and responding to bugs discovered in the RC.

As with all of our .NET releases, we’ve tested this RC against every version of Elasticsearch 1.0 and up, including the latest 1.2.2 release. In addition to our automated testing, we’ve also spent time upgrading some of our customers’ existing applications to the new NEST 1.0 release candidate. The firsthand experience and feedback on the state of the new client from these real world scenarios has been incredibly helpful in improving from beta1.

Wait, wait – 2 clients? Quick recap on our story so far…

Starting with 1.0, we made a conscious decision to split out the bare metal low level moving parts from NEST into Elasticsearch.NET. The low level client does not come with design choices around types, so you can inject your own routines for serialization, connection handling and connection pooling handling. This change makes Elasticsearch.NET a great choice for interfacing to small libraries such as logging adapters, as the low level client is completely dependency free.

If you are writing more involved applications, we recommend NEST since it is strongly typed around 90% of the Elasticsearch universe. If you are using NEST and need to drop down to the low client, you can always do by simply calling into client.Raw property. NEST uses Elasticsearch.NET internally, allowing you to inject your own moving parts as you see fit.

Breaking changes

This list is by no means exhaustive, but in our work converting existing applications using the beta1 release we’ve found these to be the main breaking changes you might run into when upgrading to the release candidate:

• Enums are no longer postfixed with Option
• BaseFilter has been renamed to FilterContainer
• BaseQuery has been renamed to QueryContainer
• Enums casing now follow C# naming guidelines so i.e not_analyzed is now NotAnalyzed
• OnField() on some queries is now called DefaultField() to match the Elasticsearch DSL more closely.
• SourceInclude() is now called _SourceInclude() to match the Elasticsearch DSL. In most cases you want the more expressive .Source() overload.
• DeleteMapping() is now typed again DeleteMapping<T>()
• When using bulk index/create/update.Object() has been renamed .Document() to more closely resemble the Elasticsearch DSL.
• Removed EsRegexFlags enum, places that took one now take a string.

If you find others, please let us know!

Whats new

As always, we’d like to thank the community first and foremost. The amount of feedback coming in from GitHub issues and Stack Overflow has been absolutely stellar.

All of this continuous feedback means we’ve been able to address many quirks and bugs that the beta1 release introduced, plus add new functionality for the deepest places where the Elasticsearch and .NET universes intersect.

New hire

I’m very pleased to be able announce the .NET team at Elasticsearch Inc has doubled in size with the hiring of Greg Marzouka. Greg is based in Jersey City, New Jersey, so the team now spans two continents!

Ok, so technically Greg is not a new feature or bug fix of the release candidate, but he’s been instrumental in getting the RC out the door. Welcome Greg!

Object Initializer Syntax

The big new feature of the RC is the object initializer syntax. When I first started to write NEST, I wrote it for me and the Elasticsearch projects I was doing at that time. I really like the fluent syntax, but not all NEST users feel the same way. With this release, we hope to get the fluent syntax haters back on board!

So what does this mean in practice? Lets consider an example:

CreateIndex() fluent syntax

var response = this._client.CreateIndex(c => c
    .Index("new-index-name")
    .Settings(s => s
        .Add("index.settings", "value")
    )
    .AddMapping<object>(m => m
        .Type("my_root_object")
        .Properties(p => p
            .String(sm => sm.Name("my_field").Analyzer("default"))
        )
    )
);

CreatIndex() object initializer syntax

var request = new CreateIndexRequest("new-index-name")
{
    IndexSettings = new IndexSettings
    {
        Settings = new Dictionary<string, object>
        {
            {"index.settings", "value"}
        },
        Mappings = new List<RootObjectMapping>
        {
            { new RootObjectMapping
            {
                Name = "my_root_object",
                Properties = new Dictionary<PropertyNameMarker, IElasticType>
                {
                    {"my_field", new StringMapping() { Analyzer = "default" } }
                }
            }}
        }
    }
};
var response = this._client.CreateIndex(request);

In most cases, the fluent syntax is more terse, but the object initializer syntax has the following benefits:

• If lambdas are not your deal, you can still use NEST and fully mapped requests without resorting to the low level client and anonymous C# objects
• The lambda syntax relies on Func<R, R> as method argument which is really hard to mock
• The object initializer syntax pushes required arguments to the constructor

The choice is now yours! We’ve enabled this on all the API endpoints mapped by NEST.

Object initlizer query syntax

A special mention should go to show that the new syntax extends also in to Query DSL

csharp
QueryContainer query = new TermQuery()
{
    Field = Property.Path<ElasticsearchProject>(p=>p.Name),
    Value = "value"
} && new PrefixQuery()
{
    Field = "prefix_field", 
    Value = "prefi", 
    Rewrite = RewriteMultiTerm.ConstantScoreBoolean
};

var request = new CountRequest()
{
    AllowNoIndices = true,
    ExpandWildcards = ExpandWildcards.Closed,
    MinScore = 0.6,
    Query = query
};

Deserialize query / dsl visitor

Since we’ve separated the internal state of the descriptors to specialized interfaces, both syntaxes are now assertable.

It also opens interesting use cases such as iterating over your query structure with specialized visitors.

It also means that you can deserialize strings into NEST query objects as is shown by this suite of unit tests.

Fields()

One of the biggest problems people in the .NET world face when upgrading to Elasticsearch 1.0, is that when specifying fields they are always returned as arrays. While the beta fully supported the new situation, the NEST API became very verbose. We’ve simplified the API, making it more readable.

The beta also had a bug in the .Documents view into your .Hits that would return an array of nulls if you specified .Fields(). Thanks to Chris McKee for reporting it! #606

We now offer a specialized view into your field selections using .FieldSelections and improved the syntax around getting you field values. See pull request 619 for the updated syntax.

Metrics support

With the new connection pooling / cluster failover support in the client, a single call might be doing multiple calls in the background. Starting with this RC, every response in Elasticsearch.Net and NEST will return a .Metrics object. This object lists the total time all the requests took, (de)serialization time and an individual metric record for each request that the client performed.

This change gives you insights into what nodes were pinged, sniffed, and on which nodes the call was actually done.

Metrics is enabled by default under DEBUG builds, but you can turn it on explicitely by calling .EnableMetrics() on ConnectionSettings.

Connection pooling / cluster failover

The beta1 was the first release that included support for connection pooling and cluster failover. Sadly, that version orphaned exceptions into MaxRetryException making it hard to see what the actual exception was even if you did not even use a connection pool! Thanks to Andrew Ochsner for identifying and fixing the issue.

A MaxRetryException is something you should now only see if you are using connectionpooling and all the attempts resulted in an actual network exception or an HTTP 503.

Elasticsearch server exceptions

By default, NEST never throws an exception if it gets a response from Elasticsearch. While we’ve always exposed the status code and raw response, we now also support a .ServerError property on all the responses.

This property gives more information about the Elasticsearch server exception that occurred, e.g. IndexMissingException, SearchParseException, etc.

If you wish to throw on these kinds of exceptions, you can instruct the client by specifying .ThrowOnElasticsearchServerExceptions() on ConnectionSettings. In this case a special ElasticsearchServerException is thrown.

The throw/not throw behaviour is quite similar to .NET WebRequest vs HttpClient. The older WebRequest always throws on anything not in the 200 range, where as the new HttpClient does not.

Continuous builds

We’ve also set up continuous bleeding edge builds of the client on the myget nuget feed. If a commit on our develop branch passes all our unit tests, myget will build and graciously host a bleeding edge version of the client. Even if we do plan to release often after the big 1.0 release, if you need to use a fix that went in to develop now rather then 2 weeks later, it’s all yours!

Changelog

As always the level of community feedback never ceases to amaze us. The Changelog only lists those that resulted in fixes or new features but the influx of general questions from GitHub and Stack Overflow questions tagged with ‘nest’ is wonderful to see. Many thanks to all of you for your contributions – you’ll see many familiar names thanked there!

Moving forward

As stated, we will take a 10 day period to solidify the release-candidate, after which we will push 1.0 into the nuget stable channel. As always we are actively looking for feedback so please don’t hesitate to let us know what you think! Stay tuned for our announcement in a few days time that Elasticsearch .Net & NEST are in GA!