This powerful capability allows the .NET APM agent to instrument and trace code that would otherwise be unfeasible.

blog-thumbnail-apm-waterfall.png

How auto instrumentation with the next iteration of the Elastic APM .NET Agent works

<p>We're happy to announce an initial alpha release of a new Rust client for Elasticsearch! You can find it on <a href="https://crates.io/crates/elasticsearch" target="_blank">crates.io with the crate name elasticsearch</a> and dive into the <a href="https://docs.rs/elasticsearch" target="_blank">documentation to get started</a>.
</p><h2 id="why-a-rust-client">Why a Rust client?
</h2><p>Rust has been voted <a href="https://insights.stackoverflow.com/survey/2019#most-loved-dreaded-and-wanted" target="_blank">the most loved programming language for the last four years on StackOverflow</a>, gaining adoption at many companies such as <a href="https://msrc-blog.microsoft.com/2019/11/07/using-rust-in-windows/">Microsoft</a> and <a href="https://blogs.dropbox.com/tech/2016/06/lossless-compression-with-brotli/" target="_blank">Dropbox</a> as a fast and memory-safe low-level language alternative for C and C++ systems programming. Indeed, there's a growing fondness of Rust within Elastic, with a Rust Guild and Slack channel to share knowledge, ideas, and projects with other like-minded folks.
</p><p>The question of having an official Rust client has <a href="https://github.com/elastic/elasticsearch/issues/35538" target="_blank">been raised in the past</a> and there are <a href="https://www.elastic.co/guide/en/elasticsearch/client/community/current/index.html#rust" target="_blank">a couple of popular community clients</a> for working with Elasticsearch. It's fair to say that Rust has been moving fast however, and with the release of <a href="https://blog.rust-lang.org/2019/11/07/Async-await-stable.html" target="_blank">async-await to stable Rust in 1.39.0</a>, there seemed like no better time than now to see what an official client could look like.
</p><p><a href="https://www.elastic.co/about/our-source-code#space-time" target="_blank">A spacetime project</a> was started to explore how much effort it would be to build a minimum viable Rust client, as well as become more acquainted with the Rust language and ecosystem. As someone familiar with what it takes to build a client in a statically typed language, I am aware of the challenges in taking on such an effort. Before going into some of the design, however, let's look at a simple example.
</p><h2>Getting started
</h2><p>To demonstrate getting started with the client, we'll run against an <a href="https://www.elastic.co/cloud/" target="_blank">Elasticsearch cluster hosted on Elastic Cloud</a>. First, add the <code>elasticsearch</code> crate as a dependency to the Cargo.toml of your project, as well as <code>serde</code> and <code>serde_json</code>, which we'll use for serialization
</p><pre class="prettyprint">[dependencies]
elasticsearch = "7.5.1-alpha.1"
serde = "~1"
serde_json = "~1"
</pre><p>After going through the steps in the Elastic Cloud web console and deploying a cluster, get the authentication credentials and the <a href="https://www.elastic.co/guide/en/cloud/current/ec-cloud-id.html" target="_blank">cloud id</a> from the web console and use them to create an instance of the client:
</p><pre class="prettyprint">use elasticsearch::{
    auth::Credentials, http::transport::Transport, 
    params::Refresh, Elasticsearch, IndexParts,
    SearchParts,
};
use serde_json::{json, Value};
// change to cloud_id retrieved from cloud web console
let cloud_id = "cluster_name:Y2xvdWQtZW5kcG9pbnQuZXhhbXBsZSQzZGFkZjgyM2YwNTM4ODQ5N2VhNjg0MjM2ZDkxOGExYQ==";
// change to username and password retrieved from cloud web console
let credentials = Credentials::Basic("&lt;username&gt;".into(), "&lt;password&gt;".into());
let transport = Transport::cloud(cloud_id, credentials)?;
let client = Elasticsearch::new(transport);
</pre><p>With the client in place, index some documents with the index API:
</p><pre class="prettyprint">let index_response = client
    .index(IndexParts::IndexId("tweets", "1"))
    .body(json!({
        "user": "kimchy",
        "post_date": "2009-11-15T00:00:00Z",
        "message": "Trying out Elasticsearch, so far so good?"
    }))
    .refresh(Refresh::WaitFor)
    .send()
    .await?;
if !index_response.status_code().is_success() {
    panic!("indexing document failed")
}
let index_response = client
    .index(IndexParts::IndexId("tweets", "2"))
    .body(json!({
        "user": "forloop",
        "post_date": "2020-01-08T00:00:00Z",
        "message": "Indexing with the rust client, yeah!"
    }))
    .refresh(Refresh::WaitFor)
    .send()
    .await?;
if !index_response.status_code().is_success() {
    panic!("indexing document failed")
}
</pre><p>serde_json's <a href="https://docs.rs/serde_json/1.0.44/serde_json/macro.json.html" target="_blank"><code>json!</code></a> macro passed to the <code>body()</code> function expands JSON into a <a href="https://docs.rs/serde_json/1.0.44/serde_json/enum.Value.html" target="_blank"><code>serde_json::Value</code></a>, which is useful in representing a document to index into Elasticsearch. The body function can accept any type that implements <a href="https://docs.rs/serde/1.0.104/serde/ser/trait.Serialize.html" target="_blank"><code>serde::Serialize</code></a> trait, however, allowing your own structs to represent documents to be used.
</p><p>Now, search the tweets index:
</p><pre class="prettyprint">let response = client
    .search(SearchParts::Index(&["tweets"]))
    .body(json!({
        "query": {
            "match": {
                "message": "Elasticsearch rust client"
            }
        }
    }))
    .send()
    .await?;
</pre><p>And read values from the response body:
</p><pre class="prettyprint">let response_body = response.read_body::&lt;Value&gt;().await?;
for hit in response_body["hits"]["hits"].as_array().unwrap() {
    println!(
        "id: {}, message: '{}', score: {}",
        hit["_id"].as_str().unwrap(),
        hit["_source"]["message"].as_str().unwrap(),
        hit["_score"].as_f64().unwrap()
    );
}
</pre><p>The following will be printed to standard output:
</p><pre class="prettyprint">id: 2, message: 'Indexing with the rust client, yeah!', score: 1.4313364
id: 1, message: 'Trying out Elasticsearch, so far so good?', score: 0.6720003
</pre><p>Take a look at the <a href="https://github.com/russcam/elasticsearch_rust_example" target="_blank">elasticsearch_rust_example GitHub repository</a> for the example in its entirety.
</p><h2 id="design">Design
</h2><p>The client is largely generated from the <a href="https://github.com/elastic/elasticsearch/tree/master/rest-api-spec" target="_blank">Elasticsearch</a> and <a href="https://github.com/elastic/elasticsearch/tree/master/x-pack/plugin/src/test/resources/rest-api-spec/api" target="_blank">X-Pack</a> REST API specs within the Elasticsearch repository, first by reading the JSON specs into structs that model the specs, then by generating streams of Rust syntax tokens from this representation, using the syn and quote crates, and finally, writing the resulting Rust code to source files. There are several advantages in taking such an approach:
</p><ul>
	<li>With such a large number of APIs, generating as much of the client as possible from an intermediate spec makes development more manageable, reducing the burden of maintenance and potential for mistakes.</li>
	<li>It is easy to reason with the resulting code at both development and run time, than, for example, defining well-behaved macros in source code to expand an intermediate representation to executable code.</li>
</ul><h3 id="heaps-of-apis">Heaps of APIs
</h3><p>Elasticsearch has some 280 APIs at present, and counting! An effective client should be structured to make it easy to discover and call them. The client exposes each API as an associated function, either on the root client, <code>Elasticsearch</code>, or on one of the namespaced clients, such as <code>Cat</code>, <code>Indices</code>, <code>Slm</code>, etc. The namespaced clients are based on the grouping of APIs within the REST API specs. All API functions are async only, and can be awaited, taking advantage of the new language features.
</p><p>Many Elasticsearch APIs have several URL variants. Take the search API as an example; it has the following URL variants, where each <code>{token}</code> URL part value can be supplied by the consumer:
</p><ul>
	<li><code>/_search</code>
	</li>
	<li><code>/{index}/_search</code>
	</li>
	<li><code>/{index}/{type}/_search</code>
	</li>
</ul><p>Putting aside the fact that the last variant is deprecated and will be removed in the future since <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.0/removal-of-types.html" target="_blank">types are going/gone</a>, the client models API URL variants with enums. For example, the above search API variants are represented with the SearchParts enum:
</p><pre class="prettyprint">#[derive(Debug, Clone, PartialEq)]
#[doc = "API parts for the Search API"]
pub enum SearchParts&lt;'b&gt; {
    #[doc = "No parts"]
    None,
    #[doc = "Index"]
    Index(&'b [&'b str]),
    #[doc = "Index and Type"]
    IndexType(&'b [&'b str], &'b [&'b str]),
}
</pre><p>Each <code>*Parts</code> enum has an associated function to build a relative URL path from the enum variant. Using such types allows users to be explicit about which variant they want to call, and removes the burden on the consumer of constructing the API path.
</p><h3 id="builder-patterns">Builder patterns
</h3><p>Each API is modeled with a struct, using a <a href="https://doc.rust-lang.org/1.0.0/style/ownership/builders.html#consuming-builders:" target="_blank">consuming builder pattern</a> to set values for the API call. All query string parameters accepted by an API are exposed as associated functions, as is a function for the request body, if the API accepts one. A <code>send()</code> async associated function acts a terminal function for the builder pattern, consuming <code>self</code> and creating a <code>Future</code> that can be awaited.
</p><p>Most Elasticsearch APIs accept a request body in the form of JSON, but there are a few that accept a request body in the form of <a href="http://ndjson.org/" target="_blank">newline-delimited JSON</a> (NDJSON). A <a href="https://docs.rs/elasticsearch/7.5.1-alpha.1/elasticsearch/http/request/trait.Body.html" target="_blank">Body trait</a> models the body of an API call, with implementations to handle both JSON and NDJSON. For APIs that expect JSON, the <code>body()</code> associated function of the API constrains the input to a type that implements <code>serde::Serialize</code> trait, and for APIs that expect NDJSON, the body associated function constrains the input to a vector of types that implement the <code>Body</code> trait. These function signatures should serve to guide consumers to the correct body form, while being open to extension in the future. The <code>Body</code> trait abstraction also allows experimental or beta Elasticsearch APIs, which aren't modeled as structs in the client, to be called by the client. For example, the <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.5/put-transform.html" target="_blank">Create transform API</a>, a beta API in Elasticsearch 7.5.1, can be called with:
</p><pre class="prettyprint">let response = client
    .send(Method::Put,
        "_transform/ecommerce_transform",
        HeaderMap::new(),
        Option::&lt;&Value&gt;::None,
        Some(JsonBody::new(json!({
            "source": {
            "index": "kibana_sample_data_ecommerce",
            "query": {
                "term": {
                "geoip.continent_name": {
                    "value": "Asia"
                }
                }
            }
            },
            "pivot": {
            "group_by": {
                "customer_id": {
                "terms": {
                    "field": "customer_id"
                }
                }
            },
            "aggregations": {
                "max_price": {
                "max": {
                    "field": "taxful_total_price"
                }
                }
            }
            },
            "description": "Maximum priced ecommerce data by customer_id in Asia",
            "dest": {
            "index": "kibana_sample_data_ecommerce_transform",
            "pipeline": "add_timestamp_pipeline"
            },
            "frequency": "5m",
            "sync": {
            "time": {
                "field": "order_date",
                "delay": "60s"
            }
            }
        })))
    )
    .await?;
let status_code = response.status_code();
let response_body = response.read_body::&lt;Value&gt;().await?;
let successful = status_code.is_success() &&
    response_body["acknowledged"].as_bool().unwrap();
</pre><h2 id="future-roadmap">Future roadmap
</h2><p>The immediate next steps on the roadmap are towards a general availability (GA) release, ensuring that the API is easy to use, idiomatic, and importantly, fits the needs of the Rust community wanting to interact with Elasticsearch. These steps include ensuring that the behavior of the client is functionally aligned with the other official clients, for example, controlling when failed API calls can be retried, how nodes are selected to perform API calls against, etc.
</p><p>One important item on the future roadmap for a statically typed language client like Rust is to expose requests and responses in a strongly typed fashion. This is something that we are looking into, to try to find an approach that reconciles maintaining such a large API with some of the more dynamic structural and behavioral elements of those APIs.
</p><h2 id="thank-you">Thank you
</h2><p>A special thank you to <a href="https://twitter.com/KodrAus" target="_blank">Ashley Mannix</a> and <a href="https://twitter.com/mwilliammyers" target="_blank">William Myers</a> for taking the time to chat about their experiences working with the community Rust clients and Elasticsearch. In particular, to Ashley, for the many conversations and inspiration!
</p><h2 id="feedback">Feedback
</h2><p>This initial alpha release is just the beginning. Please try the client out; we would love to hear your thoughts and feedback on it, <a href="https://github.com/elastic/elasticsearch-rs/issues" target="_blank">on a GitHub issue</a> and beyond!
</p>

rust-ferris-blog-banner.png

rust-ferris-blog-thumb.png

A Rust client for Elasticsearch (alpha)

2000-scaffold.jpg

720-scaffold.jpg

NEST and Elasticsearch.Net: upgrading your Elasticsearch server and codebase

The NEST and Elasticsearch.Net 7.0 clients are now GA. Learn more about performance improvements, a simplified codebase, and parity with Elasticsearch 7.0+.

blog-banner-release-elasticsearch.png

blog-thumb-release-elasticsearch.png

NEST and Elasticsearch.Net 7.0 now GA

<p>We’re pleased to announce that Elastic Stack 7.2.0 is available on the <a href="https://azuremarketplace.microsoft.com/marketplace/apps/elastic.elasticsearch">Azure Marketplace</a> through our <a href="https://github.com/elastic/azure-marketplace">Azure Resource Manager (ARM) solution template</a>. With this release, users running the Elastic Stack in Azure will be able to take advantage of all the many features that <a href="/blog/elastic-stack-7-2-0-released">Elastic Stack 7.2.0</a> has to offer, including <a href="/solutions/siem">Elastic SIEM</a>.
</p><p><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltb802e9d4f0ae7edc/5d1614acc807c769b0b1b548/ESS-Azure-Elastic-SIEM.jpg" data-sys-asset-uid="bltb802e9d4f0ae7edc" alt="View of the Elastic SIEM UI" "="" style="display: block; margin: auto;">
</p><h2>Leveraging Azure features</h2><p>Through our close partnership with Microsoft, our ARM template has continued to benefit from the evolving capabilities of the Azure platform. We’d like to take this opportunity to highlight some of the new features recently introduced into the ARM template.
</p><h3>8-, 16-, and 32TiB managed disks</h3><p>With the <a href="https://azure.microsoft.com/blog/larger-more-powerful-managed-disks-for-azure-virtual-machines/">general availability of Standard HDD, Standard SSD, and Premium SSD disks up to 32TiB</a>, it is now possible to deploy Elasticsearch clusters with large amounts of storage space on managed disks of varying capability, to suit a variety of use cases.
</p><p>For example, deploying a three master-eligible node cluster of the <a href="https://docs.microsoft.com/azure/virtual-machines/linux/sizes-memory">memory optimized Standard_E16s_v3 virtual machines (VMs)</a>, where each VM has 32 managed disks attached (each 32TiB in size), results in total space for data of 3072TiB (or roughly 3 pebibytes), configured as a RAID 0 array on each VM to maximize IOPS throughput:
</p><pre class="prettyprint">template_version=7.2.0
template_base_uri="https://raw.githubusercontent.com/elastic/azure-marketplace/$template_version/src/"
resource_group="&lt;resource group&gt;"
location="&lt;location&gt;"
az group create --name "$resource_group" --location "$location"
az group deployment create \
    --resource-group "$resource_group" \
    --template-uri ${template_base_uri}mainTemplate.json \
    --parameters _artifactsLocation=$template_base_uri \
        esClusterName=elasticsearch \
        adminUsername=russ \
        authenticationType=password \
        adminPassword=Password1234 \
        dataNodesAreMasterEligible=Yes \
        vmSizeDataNodes=Standard_E16s_v3 \
        vmDataNodeCount=3 \
        vmDataDiskCount=32 \
        vmDataDiskSize=32TiB \
        securityBootstrapPassword=BootstrapPassword123 \
        securityAdminPassword=AdminPassword123 \
        securityKibanaPassword=KibanaPassword123 \
        securityLogstashPassword=LogstashPassword123 \
        securityBeatsPassword=BeatsPassword123 \
        securityApmPassword=ApmPassword123 \
        securityRemoteMonitoringPassword=RemoteMonitoringPassword123
</pre><h3>Accelerated networking</h3><p><a href="https://azure.microsoft.com/blog/maximize-your-vm-s-performance-with-accelerated-networking-now-generally-available-for-both-windows-and-linux/">Accelerated networking</a> has been generally available for some time for certain VM sizes. This moves much of Azure’s software-defined networking stack off the CPUs and into FPGA-based SmartNICs, reclaiming compute cycles and reducing the load on VMs, which improves network latency consistency in the process. For those VMs that support accelerated networking, it’s enabled by default within the ARM template.
</p><h3>New VM SKUs</h3><p>The template continues to be updated to support the new virtual machine sizes on Azure, such as the <a href="https://azure.microsoft.com/blog/announcing-the-general-availability-of-lsv2-series-azure-virtual-machines/">Lsv2-series</a>.
</p><p>The Lsv2-series VMs have directly mapped local NVMe storage of 1.92TB per 8 vCPUs, making them an attractive choice for Elasticsearch nodes. The ARM template has been able to utilize local temp storage mounted on VMs for some time, as a cost-effective way of running an Elasticsearch cluster with good IOPS characteristics, with the caveats that come with the ephemeral nature of such storage. Expect an update to the ARM template in the future to be able to use local NVMe storage over local temp storage, where present.
</p><h3>Shard allocation awareness with update and fault domains</h3><p>When data, master, and coordinating nodes are provisioned, they are added to separate availability sets and assigned an <a href="https://docs.microsoft.com/azure/virtual-machines/windows/manage-availability#configure-multiple-virtual-machines-in-an-availability-set-for-redundancy">update and fault domain</a>. The ARM template specifies the maximum number of 20 update domains for each availability set, as well as the maximum number of <a href="https://docs.microsoft.com/azure/virtual-machines/windows/manage-availability#number-of-fault-domains-per-region">fault domains available within the target Azure region</a>.
</p><p>The fault and update domain values are added as node attributes to the elasticsearch.yml configuration file of each Elasticsearch node, and the values are used to configure <a href="/guide/en/elasticsearch/reference/7.2/allocation-awareness.html">shard allocation awareness</a> on the node. This allows Elasticsearch to take the fault and update domain values into account for shard allocation.
</p><h2>Azure Marketplace Elastic Stack features</h2><p>In addition to leveraging new features available in Azure, the ARM template has continued to evolve its own feature set, from <a href="/blog/saml-based-single-sign-on-with-elasticsearch-and-azure-active-directory">SAML-based Single Sign-On with Azure Active Directory</a>, to Transport Layer Security (TLS) configuration, to Application Gateway for load balancing. Some of the more recent features include multiple Logstash instances, shared storage for snapshots, and configuration of all built-in users.
</p><h3>Multiple Logstash instances</h3><p>The Azure Marketplace and ARM template now has the ability to deploy multiple instances of Logstash alongside Elasticsearch and Kibana. The VMs running Logstash are placed into their own availability set, ensuring the VMs run across multiple physical servers, compute racks, storage units, and network switches.
</p><h3>Shared storage for snapshots</h3><p>In addition to being able to specify an existing storage account to configure for snapshotting cluster data with the <a href="/guide/en/elasticsearch/plugins/7.2/repository-azure.html">Azure Repository plugin</a>, the shared storage account resource deployed with the template can be configured as the account to use for snapshots, keeping them together with the rest of the Stack resources within the resource group.
</p><h3>Configured with built-in users</h3><p>Simple things should be simple! When the <a href="/blog/security-for-elasticsearch-is-now-free">Elastic Security features</a> are enabled, the cluster configures all of the built-in users needed to start using it to receive data from Beats, APM Server, and Logstash, making it simple to get started with the Elastic Stack in Azure for a plethora of use cases.
</p><p>
	<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt09542825a58c66ab/5d1614c5e12f075e3a9339ff/New-users-and-groups.jpg" style="background-color: initial; font-family: proximaNovaRegular, Arial, Helvetica, sans-serif; font-size: 14px; color: rgb(116, 133, 144); display: block; margin: auto;" data-sys-asset-uid="blt09542825a58c66ab" alt="Kibana dashboard showing new users and groups">
</p><h2>Coming soon: Elasticsearch Service on Azure</h2><p>We trust you’ll continue to get real value from using our Azure Marketplace and ARM template offering to easily deploy the Elastic Stack to Azure; go <a href="https://azuremarketplace.microsoft.com/marketplace/apps/elastic.elasticsearch">try out the new release now</a>, and check out the <a href="/guide/en/elastic-stack-deploy/7.2/index.html">documentation</a> to get started!
</p><p><a href="/cloud">Elasticsearch Service</a> on Elastic Cloud — that’s hosted Elasticsearch and Kibana — is coming to Microsoft Azure. With this new offering, users will be able to launch a hosted Elasticsearch and Kibana cluster on Microsoft Azure, offering the latest versions of the Elastic Stack as they’re released, one-click upgrades and scaleups, and access to Elastic support. <a href="/partners/microsoft-azure">Sign up for updates about this offering</a> as news develops.
</p>

ES72-Azure-fullbleed.jpg

ES72-Azure-thumbnail2.jpg

Elastic Stack 7.2.0 is now available on the Azure Marketplace

Introducing integration of SAML Single Sign-On with Azure Active Directory and our Elastic ARM template offering, including a walkthrough of the steps involved.

saml-sso-azure-feature-padlocks-fullbleed.jpg

saml-sso-azure-feature-padlocks.jpg

SAML based Single Sign-On with Elasticsearch and Azure Active Directory

<html><head></head><body><p>Today we are pleased to announce the General Availability (GA) release of our .NET clients for Elasticsearch 6.x.
</p><p><em>&quot;But wait, wasn&apos;t Elasticsearch <strong>6.1</strong> just released?&quot;</em>
</p><p>Yes, this is true; whilst the .NET clients have had several pre-releases compatible with Elasticsearch 6.0 and 6.1, we intentionally pushed back on our GA release until we were overwhelmingly happy it was ready. We hope that the reasoning for this delay will become clear as you read through this post.
</p><h2 id="6-0-compatibility">6.0 Compatibility</h2><p>This client has full feature parity with Elasticsearch 6.0 and has been tested against all currently released Elasticsearch 6.x versions. You will be able use this release with versions 6.1 and the forthcoming 6.2 release without issue.
</p><p>For some of the newer features in Elasticsearch 6.1, such as the <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-composite-aggregation.html">composite aggregation</a> and <a href="https://www.elastic.co/blog/elasticsearch-machine-learning-on-demand-forecasting">machine learning forecasting</a>, we&apos;ll push out a newer release of 6.x in the future, to incorporate these.
</p><h2 id="ch-ch-changes">Ch... Ch... Changes</h2><p>The 6.0 GA release comprises well over <strong>650</strong> commits and, in the process, has shrank our Github issues down from four pages to just half of one. We&apos;ll spare you from all the gory details and effort that went into this, and call out only the most impactful changes that you&apos;re likely to come across.
</p><p>As with any major client release, <a href="https://www.elastic.co/guide/en/elasticsearch/client/net-api/6.x/_breaking_changes.html" target="_blank">we publish an exhaustive list of all the binary breaking changes</a>. Please don&apos;t let this long list intimidate you! Most of these you are unlikely to encounter, and we provide them all simply for completeness. So, onto the big changes!
</p><h2 id="serialization-changes">Serialization changes</h2><p>There are two big changes related to how we now perform serialization within the NEST client.
</p><h3 id="user-defined-types">User defined types</h3><p>We now have an explicit path for user defined types in the client serialization pipeline.
</p><p>In NEST 5.x, if you were to use a custom serializer with the client by implementing <code>IElasticsearchSerializer</code> and passing it to the client configuration, the custom serializer would take charge of serializing the <em>entire</em> request and response, including the underlying client types within NEST used to model requests and responses. Now in NEST 6.x, a custom serializer is used <em>only</em> in places where a user defined type is expected, for example,
</p><ul>
	<li>The <code>_source</code> and <code>fields</code> on <code>hits</code> in the response</li>
	<li>The term query <code>value</code></li>
	<li>A document sent as part of the Multi Term Vector API</li>
	<li>A document returned by the Get API</li>
	<li>A document sent through the Bulk API</li>
	<li>An upsert or document part of the Upsert API</li>
</ul><p>This change allows you to use your own serialization for your own user defined types without fear of affecting the rest of the client behaviour.
</p><h3 id="newtonsoft-json">Newtonsoft.Json</h3><p>NEST 6.x no longer depends on the popular <a href="https://www.nuget.org/packages/newtonsoft.json/" target="_blank">Newtonsoft.Json a.k.a. Json.NET library</a> directly as a nuget package reference. We&apos;ve taken the leap and internalized Newtonsoft.Json within the NEST assembly, and <em>shaded</em> the dependency by re-namespacing it to Nest.Json. This required some IL and assembly rewriting using <a href="https://github.com/gluck/il-repack" target="_blank">ILRepack</a> and <a href="http://www.mono-project.com/docs/tools+libraries/libraries/Mono.Cecil/" target="_blank">Mono.Cecil</a> to get it to work for both .NET Framework and .NET Core target platforms.
</p><p>What this means in practice as a consumer, is that your application is now free to use whatever version of Newtonsoft.Json that you desire without conflicting with the version that NEST uses. What this means for us as client authors, is that we are now free to work on improving the client serialization pipeline over the course of NEST 6.x without having to wait until NEST 7.x. There are many exciting developments happening in the .NET ecosystem around performance with the likes of <a href="https://msdn.microsoft.com/en-us/magazine/mt814808.aspx" target="_blank"><code>Span&lt;T&gt;</code></a>, <a href="http://adamsitnik.com/Array-Pool/" target="_blank"><code>ArrayPool&lt;T&gt;</code></a> and active discussion around a more <a href="https://github.com/dotnet/coreclr/issues/7083" target="_blank">compact UTF-8 string representation</a>, and we would like to be able to explore these unhindered.
</p><p><strong>Note</strong> we have been indexing our client benchmarking CI test runs into an Elasticsearch cluster running in <a href="https://www.elastic.co/cloud" data-vivaldi-spatnav-clickable="1">Elastic Cloud</a> for a while, and will be looking to expose these results publicly in the future. We&apos;ll also look at creating an Elasticsearch <a href="https://github.com/dotnet/BenchmarkDotNet" target="_blank">BenchmarkDotNet</a> reporter, to allow anyone running benchmarking tests with BenchmarkDotNet to easily index results into Elasticsearch. For the time being however, our implementation is wired together in our <a href="https://github.com/elastic/elasticsearch-net/blob/master/build/scripts/Benchmarking.fsx" data-vivaldi-spatnav-clickable="1">FAKE build</a>.
</p><h2 id="caution-breaking-changes-ahead">Caution! Breaking changes ahead</h2><h3 id="single-types">Single types</h3><p>The <a href="https://www.elastic.co/guide/en/elasticsearch/reference/master/removal-of-types.html" data-vivaldi-spatnav-clickable="1">staged removal of types</a> means that in Elasticsearch 6.0 onwards, <strong>multiple types in a single index are no longer allowed</strong>. This will impact your NEST code in several ways.
</p><p>As an example, if you are using NEST 5.x to index <code>Company</code> and <code>Employee</code> types in the following way:
</p><pre class="prettyprint">var settings = new ConnectionSettings()
    .DefaultIndex(&quot;my-index&quot;);
var client = new ElasticClient(settings);

client.IndexDocument(new Company { Id = 1 })
client.IndexDocument(new Employee { Id = 1 })
</pre><p>The documents would end up as different types within the same index in versions of Elasticsearch prior to 6.0
</p><pre class="prettyprint">/my-index/company/1
/my-index/employee/1
</pre><p>This will no longer work in Elasticsearch 6.0, and both <code>Company</code> and <code>Employee</code> types will now need to live in their own index. When upgrading to NEST 6.x, you can use the <a href="https://www.elastic.co/guide/en/elasticsearch/client/net-api/6.x/configuration-options.html#_options_on_connectionsettings" target="_blank"><code>DefaultMappingFor&lt;T&gt;()</code></a> method for each type so that documents are indexed into their own index. Rewriting the above example to take this approach would look as follows
</p><pre class="prettyprint">var settings = new ConnectionSettings()
  .DefaultIndex(&quot;my-index&quot;)
  .DefaultMappingFor&lt;Company&gt;(m =&gt; m.IndexName(&quot;company-index&quot;))
  .DefaultMappingFor&lt;Employee&gt;(m =&gt; m.IndexName(&quot;employee-index&quot;));
var client = new ElasticClient(settings);

client.IndexDocument(new Company { Id = 1 })
client.IndexDocument(new Employee { Id = 1 })
</pre><p>and results in documents ending up in their own index:
</p><pre class="prettyprint">/company-index/company/1
/employee-index/employee/1
</pre><p>If you <em>really need</em> multiple .NET types in a single index, you need to ensure that both .NET types are indexed using the same type name, which can be done using <code>DefaultTypeName()</code>
</p><pre class="prettyprint">var settings = new ConnectionSettings()
  .DefaultIndex(&quot;my-index&quot;)
  .DefaultTypeName(&quot;doc&quot;);
var client = new ElasticClient(settings);  

client.IndexDocument(new Company { Id = 1 });
client.IndexDocument(new Employee { Id = 1 });
</pre><p><em>However</em>, in doing this, you will now face another problem! Both documents will end up being indexed with the same id
</p><pre class="prettyprint">/my-index/doc/1
/my-index/doc/1
</pre><p>with the last document indexed overwriting the previous. The solution here is to change your .NET classes to give the documents a unique id when invoking <code>client.IndexDocument&lt;T&gt;()</code>.
</p><p>Another problem you&apos;ll encounter if you decide to index multiple .NET types into a single index is when you need to search. When searching in NEST 5.x using
</p><pre class="prettyprint">client.Search&lt;Company&gt;();
</pre><p>a search request would be made against the <code>company</code> type within the index
</p><pre class="prettyprint">/my-index/company/_search
</pre><p>and result in a collection of <code>Company</code> types being returned from the deserialized _source documents. Now that Elasticsearch 6.0 allows only a single type name within an index <em>and in future, no type name at all</em>, the previous search request will now translate to the following in NEST 6.x
</p><pre class="prettyprint">/my-index/doc/_search
</pre><p>which will now be searching across both <code>Company</code> and <code>Employee</code> types within the one index, since they are both indexed using the <code>doc</code> type name. If you are looking to constrain the search to <em>only</em> <code>Company</code> types, you can use a discriminator field within the document source, and apply a query on this field to the search criteria:
</p><pre class="prettyprint">client.Search&lt;Company&gt;(s =&gt; s
  .Query(q =&gt; +q
    .Term(&quot;my_type_field&quot;, &quot;company&quot;)
  )
);
</pre><h3 id="covariant-searches">Covariant search results</h3><p>NEST 5.x used to support returning covariant search results by inspecting the <code>_type</code> metadata for each hit, mapping each _type back to a .NET type <em>automagically</em>. For example
</p><pre class="prettyprint">client.Search&lt;IInterface&gt;(s =&gt; s.Types(typeof(A), typeof(B), typeof(C)));
</pre><p>would perform a search on <code>a</code>, <code>b</code> and <code>c</code> types within an index. NEST would then use the <code>_type</code> field in the response to deserialize each <code>_source</code> document into the the correct .NET type, and not the common interface <code>IInterface</code>. Since you can no longer have multiple types within the one index, this feature no longer makes sense in NEST 6.x.
</p><p>It is still possible to achieve covariant search results going forward, but not without telling NEST 6.x how to do so. There is now a new nuget package, <a href="https://www.nuget.org/packages/NEST.JsonNetSerializer/" target="_blank"><code>Nest.JsonNetSerializer</code></a>, with a dependency on Newtonsoft.Json, that you can reference to configure Json.NET to use <a href="https://www.newtonsoft.com/json/help/html/SerializeTypeNameHandling.htm" target="_blank"><code>TypeNameHandling.All</code></a>, such that <code>_source</code> documents are deserialized into the .NET types you expect. You can of course also implement your own <code>IElasticsearchSerializer</code> implementation to use a discriminator field with the <code>_source</code> to control deserialization to your own user defined types instead.
</p><h3 id="parent-child-joins">Parent/Child joins</h3><p>If you rely on the parent and child mapping in your usage of Elasticsearch, you still need to, conceptually at least, index multiple types into a single Elasticsearch index. Elasticsearch 6.0 introduced the <a href="https://www.elastic.co/guide/en/elasticsearch/reference/6.0/parent-join.html" target="_blank"><code>join</code> data type</a> as a mechanism to support parent/child relationships.
</p><p>The NEST documentation has a <a href="https://www.elastic.co/guide/en/elasticsearch/client/net-api/6.x/parent-child-relationships.html" data-vivaldi-spatnav-clickable="1">full example</a> of how to use the <code>join</code> data type, illustrating the removal of the <code>_parent</code> mapping and request parameter. Be sure to have a thorough read of the documentation if you&apos;re using this feature.
</p><h3 id="mapping-attributes">Mapping attributes</h3><p>Since the Newtonsoft.Json package dependency has now been removed, NEST provides additional attributes you can use to decorate your user defined types to indicate how properties should be serialized when sent to Elasticsearch. This allows you to include and rename properties without having to reference <code>Nest.JsonNetSerializer</code> and provide your own serializer instance.
</p><p>For example:
</p><pre class="prettyprint">[ElasticsearchType(Name = &quot;employee&quot;)]
public class Employee
{
    [Text(Name = &quot;first_name&quot;)]
    public string FirstName { get; set; }
    [Text(Name = &quot;last_name&quot;)]
    public string LastName { get; set; }
    [Ignore] // Property will not be included in serialization
    public bool IsSmith =&gt; !string.IsNullOrEmpty(LastName) &amp;&amp; LastName.Equals(&quot;smith&quot;, StringComparison.OrdinalIgnoreCase);
    [Number(DocValues = false, IgnoreMalformed = true, Coerce = true)]
    public int Salary { get; set; }
    [Date(Format = &quot;MMddyyyy&quot;)]
    public DateTime Birthday { get; set; }
    [Boolean(NullValue = false, Store = true)]
    public bool IsManager { get; set; }
    [Nested]
    [PropertyName(&quot;empl&quot;)] // serialize this property to a field named &quot;empl&quot; in JSON
    public List&lt;Employee&gt; Employees { get; set; }
}
</pre><h2 id="low-level-client">Low-Level Client</h2><h3 id="code-generation-fixes">Code generation improvements</h3><p>In generating parts of the low-level client, <a href="https://www.nuget.org/packages/Elasticsearch.Net/" target="_blank">Elasticsearch.Net</a>, we take the <a href="https://github.com/elastic/elasticsearch/tree/master/rest-api-spec" data-vivaldi-spatnav-clickable="1">Elasticsearch REST API specifications</a> and use it to generate C# source code. This code generation implementation has been mostly untouched since the days of NEST 1.x. With this release however, we&apos;ve taken the opportunity to fix several outstanding issues, one of which was that request parameters were fluent methods, meaning it was possible to write code such as
</p><pre class="prettyprint">client.Search&lt;dynamic&gt;(r =&gt; r.Parameter())
</pre><p>This was done in the beginning to facilitate the high-level client but now is no longer needed, thus has been removed. Instead of using fluent methods, one can use
</p><pre class="prettyprint">client.Search&lt;DynamicResponse&gt;(new OperationRequestParameters { Parameter = .. })
</pre><h3 id="response-types">Response types</h3><p>In previous versions of Elasticsearch.Net, any operation such as <code>client.Search&lt;T&gt;(..)</code> accepted any of <code>string</code>, <code>byte[]</code>, <code>dynamic</code>, <code>Stream</code>, <code>VoidResponse</code> or an <code>object</code> as the generic type parameter <code>T</code>, to deserialize the response from Elasticsearch into, and the return value of the method call was of type <code>ElasticsearchResponse&lt;T&gt;</code>.
</p><p>In NEST 6.x, the generic type parameter <code>T</code> is now constrained to be of type <code>IElasticsearchResponse</code> and returns <code>T</code> instead of <code>ElasticsearchResponse&lt;T&gt;</code>. To help aid the transition for the common use cases, Elasticsearch.Net provides <code>BytesResponse</code>, <code>StringResponse</code>, <code>DynamicResponse</code>, <code>VoidResponse</code> and <code>ElasticsearchResponse&lt;T&gt;</code> where <code>T</code> can be any object to deserialize the response into.
</p><p>As part of this change, support has been removed for returning the response from Elasticsearch as a <code>Stream</code>. This was a potentially dangerous feature because the onus for disposing this stream lay with the consumer, and the result of not disposing it could result in memory leaks and keeping connections open to Elasticsearch. If you do need to access the response stream directly for some reason, this is best handled by creating a custom <code>IElasticsearchSerializer</code> implementation.
</p><h3 id="exception-handling">Exceptions</h3><p>In previous versions, the low-level client was in charge of deserializing exceptions from the server, and would short-circuit this deserialization if a <em>bad</em> HTTP status code such as a 400 were returned from Elasticsearch. This approach was fine for Elasticsearch 2.x, but starting with Elasticsearch 5.x, several endpoints could also return additional information for <em>bad</em> HTTP status codes, and the client needed to perform a double-read of the response stream in order to surface this additional information to the consumer. How inefficient!
</p><p>Starting with NEST 6.x, the response stream is now fed directly into the configured serializer for both <em>good</em> and <em>bad</em> HTTP status codes, for deserialization. No more double reads, yay! If you want to disable this behaviour, you can set <code>DisableDirectStreaming()</code> or <code>DebugMode()</code> on the instance of <code>ConnectionSettings</code> that is passed to instantiate the client. This will result in the request and response bytes becoming available on the response through the <code>response.DebugInformation</code> property.
</p><p>The built-in <code>BytesResponse</code> and <code>StringResponse</code> types within Elasticsearch.Net both have a <code>TryGetServerError()</code> method to access an instance of <code>ServerError</code> from the exception that occurred on the server.
</p><p>For the high-level client NEST, nothing changes with usage here; the <code>.ServerError</code> property is still available on all responses, but what this does change is that serializer used by NEST is now in charge of deserializing the server error within the response, and not the low-level client.
</p><h2 id="summary">Summary</h2><p>Quite a few changes to digest, but we hope you&apos;ll agree that these continue to make working with the client as smooth as possible. We&apos;d like to give a shout out to the community who contributed to this 6.0 GA release by finding and reporting issues, opening pull requests and generally helping to further the betterment of Elasticsearch with .NET. Your contributions are highly appreciated!
</p><p>Please feel free to take NEST 6.0 and Elasticsearch.Net 6.0 for a spin, and <a href="https://github.com/elastic/elasticsearch-net/issues" target="_blank">let us know if you come across issues</a> &#x1F60A;
</p></body></html>

nest-elasticsearch-6-ga.jpg

nest-elasticsearch-6-ga-thumb.jpg

NEST and Elasticsearch.Net 6.0: Now GA!

A walkthrough of different Elasticsearch and Kibana deployment scenarios on Azure, using Elastic's Azure Resource Manager (ARM) template.

azure-topology-elasticsearch-fullbleed.jpg

azure-topology-elasticsearch-thumbnail.png

Elasticsearch and Kibana Deployments on Azure

blog-docs-fb.jpg

blog-docs-thumb.jpg

The Future of Attachments for Elasticsearch and .NET

<html><head></head><body><p>This marks the first GA release of our 2.0 client with well over a 1000 commits since 1.7.1 (the currently last released NEST version in the 1.x range).
</p><h2>Back to the drawing board</h2><p>We took some time to go back to the drawing board. NEST was originally started in 2010 and there are many choices that have accumulated in the code base that don&apos;t make sense anymore. So we stepped back to properly formalize how we see the lifetime of a call and worked off of that. Armed with the following diagram, we completely rewrote NEST&apos;s internals; The old <a href="https://msdn.microsoft.com/en-us/library/dd460717.aspx">Task Parallel Library (TPL)</a> based code is now replaced with <a href="https://msdn.microsoft.com/en-us/library/hh191443.aspx">async/await</a> with a much saner approach to exceptions and errors in addition to exposing enough information as an audit trail so you don&apos;t ever have to guess what went down during a call.
</p><p><img src="https://api.contentstack.io/v2/assets/575e4d2b3dc542cb38c09d96/download?uid=bltec66d884d4f8e393?uid=bltec66d884d4f8e393" alt="Request pipeline in NEST 2.0" style="max-width:100%;">
</p><p>Our internals now also reflect this:
</p><ul>
	<li><code>IElasticClient</code> exposes all the Elasticsearch API endpoints e.g <code>client.Search</code> this calls into <code>ITransport</code>&apos;s 2 methods <code>Request</code> and <code>RequestAsync</code> </li>
	<li>the default <code>ITransport</code> uses the passed in <code>IRequestPipelineFactory</code> to create an <code>RequestPipeline</code> which implements <code>IPipeline</code>.</li>
</ul><p>This pipeline now handles all of the failover/sniffing/pinging logic and directly reflects the flow diagram.
</p><p>We also simplified <code>IConnection</code> down just 2 methods. This means the outer edges are clean (<code>ITransport</code> and <code>IConnection</code>) and implementing your own should be really really simple. All of these (and also <code>IMemoryStreamProvider</code> and <code>IDateTimeProvider</code>) can be injected on the constructor of the client.
</p><h2>Test Framework</h2><p>Another huge endeavour is the rework of our test framework. NEST 1.x was always well tested but used 5 different test projects and 5 years worth of changing our minds as how best to write tests and assertions, thus becoming a big hodgepodge of <a href="http://www.nunit.org/">NUnit assertions</a>, <a href="http://www.fluentassertions.com/">Fluent assertions</a>, <a href="https://fakeiteasy.github.io/">FakeItEasy</a>, <a href="http://www.moqthis.com/">Moq</a> combined with several different ways to compare json with object graphs and vice-versa. Trying to write a new test quickly became <a href="http://www.hanselman.com/blog/YakShavingDefinedIllGetThatDoneAsSoonAsIShaveThisYak.aspx">an exercise in yak shaving</a> because there was no clear cut way for how best to write said test.
</p><p>So the first thing we did as part of our 2.0 branch was to completely delete all of our tests. This <em>act of insanity</em> gave us carte blanche during our rewrite.
</p><p>As of 2.0, we have one test project, <strong>Tests</strong>, with all tests written in such a way that they can be run in unit test mode and integration test mode. <em>Write once, run differently</em>.
All of the API endpoint tests test all four request variations - two DSL&apos;s (using fluent and object initializer syntax) with both synchronous and asynchronous variations of each. We also test all of the moving parts of the Elasticsearch DSL (Aggregations, Sorting, IndexSettings, etc) in the same way.
</p><p>In addition to the more formal unit and integration tests, we also implemented a thing we dubbed <em>Literate Testing</em> to allow us to write tests in a more story telling form, with multi-line comments serving as the narrative for our asciidoc documentation while using the <a href="https://github.com/dotnet/roslyn/wiki/Roslyn%20Overview">Roslyn compiler</a> to pick out the interesting bits of code. This gives us the benefit of always compiling our documentation in addition to having only one place where we document, test and assert how a piece of code is supposed to work.
</p><p>Another huge component of our testing framework is the <em>Virtual Cluster</em> that allows us to write tests for any situation and how we expect the client to behave.
</p><p>For example:
</p><pre class="prettyprint">/** we set up a 10 node cluster with a global request time out of 20 seconds.
* Each call on a node takes 10 seconds. So we can only try this call on 2 nodes
* before the max request time out kills the client call.
*/
var audit = new Auditor(() =&gt; Framework.Cluster
  .Nodes(10)
  .ClientCalls(r =&gt; r.FailAlways().Takes(TimeSpan.FromSeconds(10)))
  .ClientCalls(r =&gt; r.OnPort(9209).SucceedAlways())
  .StaticConnectionPool()
  .Settings(s =&gt; s.DisablePing().RequestTimeout(TimeSpan.FromSeconds(20)))
);
audit = await audit.TraceCalls(
  new ClientCall {
    { BadResponse, 9200 }, //10 seconds
    { BadResponse, 9201 }, //20 seconds
    { MaxTimeoutReached }
  },
  /**
  * On the second client call we specify a request timeout override to 80 seconds
  * We should now see more nodes being tried.
  */
  new ClientCall(r =&gt; r.RequestTimeout(TimeSpan.FromSeconds(80)))
  {
    { BadResponse, 9203 }, //10 seconds
    { BadResponse, 9204 }, //20 seconds
    { BadResponse, 9205 }, //30 seconds
    { BadResponse, 9206 }, //40 seconds
    { BadResponse, 9207 }, //50 seconds
    { BadResponse, 9208 }, //60 seconds
    { HealthyResponse, 9209 },
  }
);
</pre><p>This showcases the <em>Virtual Cluster</em> tests combined with <em>Literate Tests</em> and the extensive audit trail information available on each response (or exception).
</p><p>I&apos;m pleased to say we are back at a decent coverage rate (60%) and will continue to iterate and improve this.
</p><h2>Exception handling</h2><p>Another big change in NEST 2.0 is how we deal with exceptions.
</p><p>In NEST 1.x, the client threw a multitude of exceptions: <code>MaxRetryException</code>, <code>ElasticsearchAuthException</code>, <code>ElasticsearchServerException</code>, <code>DslException</code>, etc.. This made it challenging for users to handle exceptions and invalid responses, and understand the root cause of errors. On top of that, the types of exceptions thrown depended on what kind of <code>IConnectionPool</code> was injected, in order to maintain maximum backwards compatibility with NEST 0.x.
</p><p>In NEST 2.x, exceptions are much more deterministic.  The former <code>ThrowOnElasticsearchServerExceptions()</code> setting has been replaced with the more succinct <code>ThrowExceptions()</code>, which determines whether the client should ever throw an exception or not (client side and server exceptions). Furthermore, the types of exceptions have been reduced and simplified down to three types of exceptions:
</p><p><strong>ElasticsearchClientException:</strong> These are <strong>known</strong> exceptions, either an exception that occurred in the request pipeline(such as max retries or timeout reached, bad authentication, etc...) or Elasticsearch itself returned an error (could not parse the request, bad query, missing field, etc...). If it is an Elasticsearch error, the <code>ServerError</code> property on the response will contain the the actual error that was returned.  The inner exception will always contain the root causing exception.
</p><p><strong>UnexpectedElasticsearchClientException:</strong>  These are <strong>unknown</strong> exceptions, for instance a response from Elasticsearch not properly deserialized.  These are usually bugs in the client and <a href="https://github.com/elastic/elasticsearch-net/issues">we encourage you to report them</a>.  This exception also inherits from <code>ElasticsearchClientException</code> so an additional catch block isn&apos;t necessary to handle but can be helpful in distinguishing between the two.
</p><p><strong>Runtime exceptions:</strong> These are CLR exceptions like <code>ArgumentException</code>, <code>NullArgumentException</code> etc. that are thrown when an API in the client is misused.
</p><h1>Breaking Changes</h1><p>Even though a lot of work went into the interior, the exterior did not escape unscathed! On top of the many breaking changes that Elasticsearch 2.0 introduces, there are more then a few NEST 2.0 introduces.
We revalidated all the request and response domain objects against Elasticsearch 2.0.
</p><p>A pretty complete list of breaking changes are available:
</p><ul>
	<li><a href="https://github.com/elastic/elasticsearch-net/blob/master/docs/2.0-breaking-changes/elasticsearch-net-breaking-changes.md">Low level client changes, Elasticsearch.NET</a></li>
	<li><a href="https://github.com/elastic/elasticsearch-net/blob/master/docs/2.0-breaking-changes/nest-breaking-changes.md">High level client changes, NEST</a></li>
</ul><h2>Elasticsearch 2.x support</h2><p>NEST 2.0 supports all the new features in Elasticsearch 2.0 including pipeline aggregations. <a href="https://www.elastic.co/blog/elasticsearch-2-2-0-and-2-1-2-and-1-7-5-released">New features from Elasticsearch 2.2</a> have not yet been mapped.
</p><p>Here we&apos;ll just highlight a couple features that are reflected in the NEST changes to whet your appetite!
</p><h2>Filters are <strong>Gone Gone Gone</strong></h2><p>In Elasticsearch 2.0, <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking_20_query_dsl_changes.html#_queries_and_filters_merged">query and filter constructs have merged into one concept called queries</a> and NEST 2.0 reflects this. So if you were previously using the <code>AndFilter</code>, this is now&#xA0;the <code>AndQuery</code>; beware however&#xA0;that some of these filters have been obsoleted and chances are high <a href="https://www.elastic.co/blog/all-about-elasticsearch-filter-bitsets">you were using them wrong!</a>.
</p><h2>Isolated descriptors</h2><p>In NEST 1.x we took pride in being a 1-to-1 mapping with the Elasticsearch API. In some cases however, this <em>purity</em> hid the real depth of parameters. As an example,
</p><p>in NEST 1.x you could add sorts on <code>Search()</code> using:
</p><pre class="prettyprint">client.Search&lt;Project&gt;(s=&gt;s
  .SortAscending(...)
  .SortScript(...)
)
</pre><p>in NEST 2.0 you have to stoop down a level first in order to access the same functionality
</p><pre class="prettyprint">client.Search&lt;Project&gt;(s=&gt;s
  .Sort(ss=&gt;ss
    .Field()
    .Script()
    .Ascending()
    .GeoDistance()
  )
)
</pre><p>This encapsulates all of the sort options properly and adheres stricter to the 1-to-1 mapping. NEST 1.x did also contain this full descriptor however the mix and matching
of the convenience methods of the parent meant that some fluent methods were additive whilst others overwrote what was previously set. With NEST 2.0, this discrepancy is gone!
</p><p>This happens in more places e.g index settings and mappings.
</p><h2>Filtered query deprecation</h2><p>With the removal of filters, NEST has added a special construct in its Query DSL to easily create a <a href="https://www.elastic.co/guide/en/elasticsearch/reference/2.0/query-dsl-bool-query.html">bool query</a> with a filter clause
</p><pre class="prettyprint">.Query(q=&gt; +q.Term(p=&gt;p.Name, &quot;NEST&quot;))
</pre><p>the <code>+</code> will cause&#xA0;the term query to be wrapped inside a bool query&apos;s <code>filter</code> clause
</p><p>You can even combine <code>+</code> with <code>!</code>
</p><pre class="prettyprint">.Query(q=&gt; !+q.Term(p=&gt;p.Name, &quot;NEST&quot;))
</pre><p>This will wrap the term query inside a <code>bool filter</code> and subsequently inside a <code>bool must_not</code>&#xA0;clause.&#xA0;This approach also works with the object initializer syntax (OIS)
</p><pre class="prettyprint">!+new TermQuery {}
</pre><h2>Attribute based mapping</h2><p>The single <code>ElasticPropertyAttribute</code> has been broken up into individual attributes per property type.
</p><p>For instance, the following:
</p><pre class="prettyprint">[ElasticType(Name = &quot;othername&quot;, IdProperty = &quot;MyId&quot;)]
public class Foo
{
  [ElasticProperty(Type = FieldType.String)]
  public Guid MyId { get; set; }
  [ElasticProperty(Type = FieldType.String)]
  public string Name { get; set; }
  [ElasticProperty(Type = FieldType.String, Analyzer = &quot;myanalyzer&quot;, TermVector = TermVectorOption.WithOffsets)]
  public string Description { get; set; }
  [ElasticProperty(Type = FieldType.Date, Format = &quot;mmmddyyyy&quot;)]
  public DateTime Date { get; set; }
  [ElasticProperty(Type = FieldType.Integer, Coerce = true)]
  public int Number { get; set; }
  [ElasticProperty(Type = FieldType.Nested, IncludeInParent = true)]
  public List&lt;Bar&gt; Bars { get; set; }
}
</pre><p>becomes
</p><pre class="prettyprint">[ElasticsearchType(Name = &quot;othername&quot;, IdProperty = &quot;MyId&quot;)]
public class Foo
{
  [String]
  public Guid MyId { get; set; }
  [String]
  public string Name { get; set; }
  [String(Analyzer = &quot;myanalyzer&quot;, TermVector = TermVectorOption.WithOffsets)]
  public string Description { get; set; }
  [Date(Format = &quot;mmddyyyy&quot;)]
  public DateTime Date { get; set; }
  [Number(NumberType.Integer, Coerce = true, DocValues = true)]
  public int Number { get; set; }
  [Nested(IncludeInParent = true)]
  public List&lt;Bar&gt; Bars { get; set; }
}
</pre><p>Aside from a simpler and cleaner API, this allows each attribute to only reflect the options that are available for the particular type instead of exposing options that may not be relevant (as <code>ElasticPropertyAttribute</code> did).
</p><h2>Inferred types</h2><p>Many places that only took a string or primitive type now take a more strongly typed object such as <code>Id</code>, <code>Field</code>, <code>Fields</code>, <code>Index</code>, <code>Indices</code>, <code>Type</code>, <code>Types</code>, <code>DocumentPath&lt;T&gt;</code>, etc. It&apos;s good to know that in most cases you can still pass a string or primitive type and it will be implicitly converted to the type where it makes sense.
</p><p>If you are using <code>C# 6</code> you can also statically import the <code>Infer</code> static class using <code>using static Nest.Infer</code>, allowing to write
</p><ul>
	<li><code>Field&lt;Project&gt;(p =&gt; p.Name)</code> any where that takes a <code>Field</code></li>
	<li><code>Field&lt;Project&gt;(p =&gt; p.Name).And(p=&gt;p.Description).And(&quot;field&quot;)</code> anywhere that takes <code>Fields</code></li>
	<li><code>Index&lt;Project&gt;()</code> for <code>Index</code></li>
</ul><p>so on and so forth. If you are using the fluent API using these infer methods is not required since the fluent API is strongly typed through lambda expressions, but they are another tool at your disposal nonetheless.
</p><h2>C# 6 support</h2><p>NEST&apos;s codebase has been largely rewritten to take advantage of all the cool new c# features making almost all the fluent code one liners
</p><pre class="prettyprint">using static Nest.Infer;
//later..
Field&lt;Project&gt;(p =&gt; p.Name);
Index&lt;Project&gt;();
Indices&lt;Project&gt;().And&lt;Developer&gt;();
</pre><h2>DNX Support</h2><p>The 2.0 release has been released with a <code>dotnet5.1</code> version on nuget that can be used on both the Desktop and Core CLR runtimes of DNX rc1. We are actively tracking <a href="https://github.com/aspnet/Home/wiki/Roadmap">rc2</a>, the <a href="https://github.com/dotnet/cli">.NET Commandline interface (dotnet-cli)</a> and the new <code>netstandard</code> NuGet target framework, with a plan to release compatible packages once these hit the scene.
</p><h2>Feedback</h2><p>We&apos;d like to thank everyone who took the alpha and rc releases out for a spin and provided invaluable input while we incubated the 2.0 release. A special shoutout to <a href="https://github.com/niemyjski">Blake Niemyjski</a> for sharing his screen with us so we could see firsthand where the pain points were during an upgrade from 1.x to 2.0.
</p><p>As always we very much welcome all feedback on <a href="https://github.com/elastic/elasticsearch-net/issues">our github issues page.</a>
</p></body></html>

birds-nest.jpg

GA Release of NEST 2.0, our .NET client for Elasticsearch

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

headshot-russ-cam-300x300.jpg

Principal Software Engineer

Russ Cam

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_self">Terms of Service</a>, and you agree to receive the <a href="/legal/privacy-statement#how-we-use-the-information" target="_self">selected communications</a> at the contact details you provided above. See <a href="/legal/privacy-statement/" target="_self">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p>

Communication Preferences - ONLY for /communication-preferences

<p>Durch das Absenden erklären Sie, dass Sie unsere <a href="/de/legal/elastic-cloud-account-terms" target="_self">Nutzungsbedingungen</a> gelesen haben und mit ihnen einverstanden sind und dass Sie Elastic erlauben, <a href="/de/legal/privacy-statement#how-we-use-the-information" target="_self">ausgewählte Nachrichten und Informationen</a> an die oben von Ihnen bereitgestellten Kontaktangaben zu senden. Weitere Informationen, darunter auch dazu, wie Sie Ihre Einwilligung zurückziehen können, finden Sie in der <a href="/de/legal/privacy-statement/" target="_self">Datenschutzerklärung von Elastic</a>.</p>

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/serverless-preview-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p>

Serverless GDPR Text

<p>Durch das Absenden erklären Sie, dass Sie unsere <a href="/de/legal/serverless-preview-terms" target="_self">Nutzungsbedingungen</a> gelesen haben und mit ihnen einverstanden sind und dass Sie Elastic erlauben, bezüglich unserer zugehörigen Produkte und Dienstleistungen und unter Nutzung der von Ihnen oben bereitgestellten Kontaktangaben <a href="/de/legal/privacy-statement#how-we-use-the-information" target="_self">mit Ihnen in Kontakt zu treten</a>. Weitere Informationen, darunter auch dazu, wie Sie Ihre Einwilligung zurückziehen können, finden Sie in der <a href="/de/legal/privacy-statement/" target="_self">Datenschutzerklärung von Elastic</a>.</p>

<p>By submitting you agree to <a href="/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud Terms of Service</a>. Your personal data will be processed in accordance with <a href="/legal/privacy-statement">Elastic's Privacy Statement</a>.</p>

Cloud Trial GDPR Text

<p>Beim Absenden akzeptieren Sie die <a href="/de/agreements/global/cloud-services-monthly" rel="nofollow">Nutzungsbedingungen von Elastic Cloud</a>. Die Verarbeitung Ihrer personenbezogenen Daten unterliegt der <a href="/de/legal/privacy-statement">Datenschutzerklärung von Elastic</a>.
</p>

<p>By submitting you acknowledge that you've read and agree to our <a href="/legal/terms-of-use" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.</p>

Newsletter GDPR Text

<p>Durch das Absenden erklären Sie, dass Sie unsere <a href="/de/legal/terms-of-use" target="_self">Nutzungsbedingungen</a> gelesen haben und mit ihnen einverstanden sind und dass Sie Elastic erlauben, bezüglich unserer zugehörigen Produkte und Dienstleistungen und unter Nutzung der von Ihnen oben bereitgestellten Kontaktangaben <a href="/de/legal/privacy-statement#how-we-use-the-information" target="_self">mit Ihnen in Kontakt zu treten</a>. Weitere Informationen, darunter auch dazu, wie Sie Ihre Einwilligung zurückziehen können, finden Sie in der <a href="/de/legal/privacy-statement/" target="_self">Datenschutzerklärung von Elastic</a>.</p>

Explore similar demos

Overview

Speakers

Close

See more insights

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

The content on this page is not available in the selected language.

Der Inhalt dieser Seite ist in der ausgewählten Sprache nicht verfügbar. Wir bei Elastic arbeiten daran, die bereitgestellten Inhalte in verschiedenen Sprachen anzubieten. Bis dahin bitten wir Sie um etwas Geduld und hoffen auf Ihr Verständnis!

Author

Watch now (no PT)

Watch now

See all top stories

All (no PT translation)

Contact information

Press Release

Share on Reddit

Articles by

Share this story

Share by Email

Thank you for your interest!

Contact Info

Follow us on Youtube

Load More

Share on LinkedIn

Follow us on LinkedIn

Search Integrations

All Solutions

Video for

Follow us on Twitter

See when this webinar starts in my time zone

Register to Watch

Register now

See All Posts

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

Sie können nicht dabei sein? Registrieren Sie sich, damit wir Ihnen die Aufzeichnung zukommen lassen können. Außerdem erhalten Sie eine E‑Mail mit Begleitmaterial.

Small image for

On-demand webinar

Featured webinar

Register to Attend

Share on Facebook

Reset all

Upcoming webinar

View more posts

Print

Hosted by

Sign In to Attend

View next

Follow us on Facebook

Share

Highlights

Learn More

Read less

You'll also receive an email with related content

Wir schicken Ihnen zudem relevante Informationen, die von Interesse sein könnten.

Thank you for registering. We will send you a confirmation email soon.

Vielen Dank für Ihre Registrierung. Sie erhalten in Kürze eine Bestätigungs-E‑Mail von uns.

Author

Articles by Russ Cam

Principal Software Engineer, Elastic

How auto instrumentation with the next iteration of the Elastic APM .NET Agent works

A Rust client for Elasticsearch (alpha)

NEST and Elasticsearch.Net: upgrading your Elasticsearch server and codebase

NEST and Elasticsearch.Net 7.0 now GA

Elastic Stack 7.2.0 is now available on the Azure Marketplace

SAML based Single Sign-On with Elasticsearch and Azure Active Directory

NEST and Elasticsearch.Net 6.0: Now GA!

Elasticsearch and Kibana Deployments on Azure

The Future of Attachments for Elasticsearch and .NET