Tech Topics

NEST 1.5 Released

Today we are happy to release NEST and Elasticsearch.Net 1.5 which brings us to feature parity with the new features introduced in Elasticsearch 1.5. A couple weeks later than our usual feature parity release cadance, but the .NET team was on the road in the states, which we'll report on later in this blog post.

Put plainly: NEST 1.4.3 was an Elasticsearch 1.5 compatibilty release, NEST 1.5 is a complete feature parity release. 


The Bad!

Before we get the good news show rolling, let's start with the bad that has been fixed in this 1.5 release.

We broke .NET 4.0 support in our Nuget package meta-data causing folks on .NET 4.0 not see any update past NEST 1.3.3.  We are very happy that this is now fixed and folks still on .NET 4.0 can now upgrade to 1.5!

If you had an alternative id configured on your POCO either through [ElasticType(IdProperty="Name")] or .MapIdPropertyFor on ConnectionSettings, the method we constructed to get the id out of your POCO was not cached, causing it to not take advantage of cached reflection.  

We are happy this is now fixed but upset we had to include a bad section in this blog post to begin with!


Inner Hits

This feature allows you to get the matching parents, childs or nested objects back separately but in a single search request.

Imagine for a second that we're indexing royal families.  Instead of indexing the complete graphs, we'll index Kings, Princes, Dukes, Earls and Barons separately. The mapping for which might look something like this in NEST:

var create = this.Client.CreateIndex("royalty", c => c
    .NumberOfReplicas(0)
    .NumberOfShards(1)
    .AddMapping<King>(m => m.MapFromAttributes())
    .AddMapping<Prince>(m => m.MapFromAttributes().SetParent<King>())
    .AddMapping<Duke>(m => m.MapFromAttributes().SetParent<Prince>())
    .AddMapping<Earl>(m => m.MapFromAttributes().SetParent<Duke>())
    .AddMapping<Baron>(m => m.MapFromAttributes().SetParent<Earl>())
);


When we are searching for princes we'd also like to get back the king he's serving, inner hits allows you to do this!

var results = this.Client.Search<Prince>(s => s
    .Index("royalty")
    .Query(q => q
        .HasParent<King>(hp => hp
            .Query(qq => qq.MatchAll())
            .InnerHits()
        )
    )
);


To get back each hit's king (remember a document can only have one parent) you can use the following C# code:

foreach(var hit in results.Hits)
{
    var king = hit.InnerHits["king"].Documents<King>().FirstOrDefault();
}


Here we specified inner_hits as part of the query DSL, but you can also specify more deeply nested inner hits structures as part of the _search request. NEST fully supports this. Be sure to read the Elasticsearch documentation on inner hits for more information.


IDocument

In order to implement inner hits, we had to support lazy document serialization. Because the inner hits on the response is basically an IDictionary<string, object> where object can be of any unrelated type, we had to come up with a way to defer deserialization until you, the programmer, can get a hold of the dictionary and know what types you want to get out of it.

To solve this, inner hits are mapped to IDocument, a special construct that you can call .As<T>() on later to deserialize into the type of your pleasing. The .Documents<King>() earlier is a convenience method on the inner hits meta data object to do so for all the inner hits.

A cool side effect is that you can now also use this on .Search<T>() using IDocument as T and defer determening what type to deserialize to in your application code.


Variables in suffix expressions

Best explained through an example.

var sortSuffix = "raw";
var path = Property.Path<ElasticsearchProject>(p => p.Name.Suffix(sortSuffix));
var resolved = client.Infer.PropertyPath(path);

resolved will be now be name.raw


and so much more...

Please see our release notes for all the fixes and new features that went into this release, and as always a shout out to everyone who submitted a PR or reported an issue. We can't thank you enough!


.NET team US tour

The .NET team within Elasticsearch ( @Mpdreamz & @gregmarzouka) spent 8 jam-packed days together doing meetups, trainings and hashing out our roadmap for the rest of the year.


.NET Fringe

First stop: Portland, Oregon, Where we attended the amazing .NET Fringe conference, a grassroots .NET OSS conference. We are still a bit buzzed by the good vibes we felt at this conference!

In addition to attending, we also did a 4 hour training on .NET and Elasticsearch, where we reworked our new getting started tutorial project into training format.

We are pleased to open up this tutorial for folks to play with right now and provide us with feedback. 

Please note its still a tad unpolished and we plan to release and blog about it more properly as part of our 2.0 release!

While in Portland we also had the pleasure of doing a virtualized meetup at the Mozilla offices where we were being video beemed to the internet and the Mountain View office where some of the meetup group had also assembled. 

You can watch this presentation here.


New York, New York!

After 3 very short days, we flew back to Greg's hometown where we had a couple of days to hash out our plans for the future. In addition to this planning, we did the same talk as we did at Mozilla at the Microsoft Times Square office for the Elasticsearch-NY meetup group. Thank you again Maria Naggaga for hosting us!

Roadmap

So what did we hash out in the Big Apple?

NEST 2.0

NEST 2.0 will move on to C# 6, we will also develop primarely in Visual Studio 2015 and move on from sln/csproj based architecture to DNX's global.json/project.json.

With that we will move from Paket, which is an amazing package manager, to dnu. What Paket gave us was Nuget retargeting inside of csproj files and being able to call msbuild targetframework= on the command line. Since this is now handled as well by dnu pack, we can move over.

With the jump to DNX we will also start building for the .NET Core Framwork, which will be Microsoft's foray into a cross-platform CLR.

If none of these terms make any sense to you yet, fear not! As a consumer you will still be able to use our DLLs on desktop .NET 4.0 and up as well!

We are going to remove the thrift (now deprecated in Elasticsearch itself) and http client connection nuget packages, as they offer no performance benefits and with the move to .NET Core FX we'll most likely have to move from WebRequest to HttpClient by default. 

Another huuuge focus of our 2.0 release is a reworking our testing and documentation, merging them into one project where we can leverage the Roslyn compiler to do literate programming inside our unit tests.  This will ensure our documentation will never go stale, and the act of writing unit/integration tests adds to the documentation in the same vein. 


Expect good things here!

But also expect some breaking changes as we move to 2.0, some of which will come from Elasticsearch itself and others from the client. We'll try to keep it at a minimum and document them extensively!