<html><head></head><body><table style="background: #FFFFD2;">
<tbody>
<tr>
	<td><strong>DON&apos;T PANIC.&#xA0;</strong>This article contains outdated information. Get the latest from&#xA0;<em><strong><a href="https://www.elastic.co/blog/state-of-the-official-elasticsearch-java-clients">The State of the Official Elasticsearch Java Clients</a>&#xA0;</strong>post,</em>&#xA0;instead.
	</td>
</tr>
</tbody>
</table><p>One of the important aspects of Elasticsearch is that it is programming language independent. All of the APIs for indexing, searching and monitoring can be accessed using HTTP and JSON so it can be integrated in any language that has those capabilities. Nevertheless Java, the language Elasticsearch and Lucene are implemented in, is very dominant. In this post I would like to show you some of the options for integrating Elasticsearch with a Java application.
</p><section id="the-native-client" class="level2">
<h2><a href="#the-native-client">The Native Client</a></h2>
<p>The obvious first choice is to look at the <a href="/guide/en/elasticsearch/client/java-api/current/client.html">client Elasticsearch provides natively</a>. Unlike other solutions there is no separate jar file that just contains the client API but you are integrating the whole application Elasticsearch. Partly this is caused by the way the client connects to Elasticsearch: It doesn&#x2019;t use the REST API but connects to the cluster as a cluster node. This node normally doesn&#x2019;t contain any data but is aware of the state of the cluster.
</p>
<p>
	<img src="https://api.contentstack.io/v2/assets/575e4c8b3dc542cb38c08224/download?uid=bltbd099733456972a2?uid=bltbd099733456972a2" data-sys-asset-uid="blte38f3ba3bf6718cb" alt="The node client integrates with your Elasticsearch cluster" style="max-width:100%;">
</p><figcaption>The node client integrates with your Elasticsearch cluster</figcaption>
<p>On the right side we can see two normal nodes, each containing two shards. Each node of the cluster, including our application&#x2019;s client node, has access to the cluster state as indicated by the cylinder icon. That way, when requesting a document that resides on one of the shards of Node 1 your client node already knows that it has to ask Node 1. This saves a potential hop that would occur when asking Node 2 for the document that would then route your request to Node 1 for you.
</p>
<p>Creating a client node in code is easy. You can use the <code>NodeBuilder</code> to get access to the <code>Client</code> interface. This then has methods for all of the API functionality, e.g. for indexing and searching data.
</p>
<pre class="sourceCode java prettyprint">Client client = NodeBuilder.nodeBuilder()
                                .client(true)
                                .node()
                                .client();
    boolean indexExists = client.admin().indices().prepareExists(INDEX).execute().actionGet().isExists();
    if (indexExists) {
        client.admin().indices().prepareDelete(INDEX).execute().actionGet();
    }
    client.admin().indices().prepareCreate(INDEX).execute().actionGet();
    SearchResponse allHits = client.prepareSearch(Indexer.INDEX)
                                .addFields(&quot;title&quot;, &quot;category&quot;)
                                .setQuery(QueryBuilders.matchAllQuery())
                                .execute().actionGet();
</pre>
<p>You can see that after having the client interface we can issue index and search calls to Elasticsearch. The fluent API makes the code very readable. Note that the final <code>actionGet()</code> call on the operations is caused by the asynchronous nature of Elasticsearch and is not related to the HTTP operation. Each operation returns a <code>Future</code> that provides access to the result once it is available.
</p>
<p>Most of the operations are available using dedicated builders and methods but you can also use the generic <code>jsonBuilder()</code> that can construct arbitrary JSON objects for you.
</p>
<p>An alternative to the node client we have seen above is the <code>TransportClient</code>. It doesn&#x2019;t join but connects to an existing cluster using the transport module (the layer that is also used for inter-node communication). This can be useful if maintaining the cluster state in your application can be problematic, e.g. when you are having tight constraints regarding your memory consumption or you are restarting your application a lot.
</p>
<p>The <code>TransportClient</code> can be created by passing in one or more urls to nodes of your cluster:
</p>
<pre class="sourceCode java prettyprint">Client client = new TransportClient()
                        .addTransportAddress(new InetSocketTransportAddress(&quot;localhost&quot;, 9300))
                        .addTransportAddress(new InetSocketTransportAddress(&quot;localhost&quot;, 9301));
</pre>
<p>Using the property <code>client.transport.sniff</code> the <code>TransportClient</code> will also retrieve all the URLs for the other nodes of the cluster for you and use those in a round robin fashion.
</p>
<p>The native Java client is the perfect solution if you need to have all of the features of Elasticsearch available. New functionality is automatically available with every release. You can either use the node client that will save you some hops or the <code>TransportClient</code> that communicates with an existing cluster.
</p>
<p>If you&#x2019;d like to learn more about the two kinds of clients you can have a look at <a href="http://www.devdiscoveries.com/elasticsearch/java-elasticsearch-become-up-and-running/">this article on using Elasticsearch from Java</a> or <a href="/blog/found-elasticsearch-networking">this post on networking on the Found blog</a>.
</p>
<p>Note: Elasticsearch service providers that have built a highly secure platform and service, e.g. implementing security measures such as ACLs and encryption, do not support unmodified clients. For more details regarding Found&#x2019;s requirements, see <a href="/guide/en/found/current/java-transport.html">Found Elasticsearch Transport Module</a>.
</p>
</section><section id="jest" class="level2">
<h2><a href="#jest">Jest</a></h2>
<p>For when you need a lightweight client in your application (regarding jar size or memory consumption) there is a nice alternative. <a href="https://github.com/searchbox-io/Jest">Jest</a> provides an implementation of the Elasticsearch REST API using the <a href="http://hc.apache.org/">Apache HttpComponents project</a>.
</p>
<p>The API of Jest is very similar to the Elasticsearch API. It uses a fluent API with lots of specialized builders. All of the interaction happens using the <code>JestClient</code> that can be created using a factory:
</p>
<pre class="sourceCode java prettyprint">JestClientFactory factory = new JestClientFactory();
        factory.setHttpClientConfig(new HttpClientConfig.Builder(&quot;http://localhost:9200&quot;)
                .multiThreaded(true)
                .build());
        JestClient client = factory.getObject();
</pre>
<p>When it comes to communicating with Elasticsearch you have two options: You can either create strings in the JSON-API of Elasticsearch or you can reuse the builder classes of Elasticsearch. If it&#x2019;s not a problem to have the Elasticsearch dependency on your classpath this can lead to cleaner code. This is how you conditionally create an index using Jest:
</p>
<pre class="sourceCode java prettyprint">boolean indexExists = client.execute(new IndicesExists.Builder(&quot;jug&quot;).build()).isSucceeded();
        if (indexExists) {
            client.execute(new DeleteIndex.Builder(&quot;jug&quot;).build());
        }
        client.execute(new CreateIndex.Builder(&quot;jug&quot;).build());
</pre>
<p>And this is how a search query can be executed.
</p>
<pre class="sourceCode java prettyprint">String query = &quot;{\n&quot;
                + &quot;    \&quot;query\&quot;: {\n&quot;
                + &quot;        \&quot;filtered\&quot; : {\n&quot;
                + &quot;            \&quot;query\&quot; : {\n&quot;
                + &quot;                \&quot;query_string\&quot; : {\n&quot;
                + &quot;                    \&quot;query\&quot; : \&quot;java\&quot;\n&quot;
                + &quot;                }\n&quot;
                + &quot;            }&quot;
                + &quot;        }\n&quot;
                + &quot;    }\n&quot;
                + &quot;}&quot;;
        Search.Builder searchBuilder = new Search.Builder(query).addIndex(&quot;jug&quot;).addType(&quot;talk&quot;);
        SearchResult result = client.execute(searchBuilder.build());
</pre>
<p>You can see that concatenating the query can become complex so if you have the option to use the Elasticsearch builders you should try it.
</p>
<p>The really great thing about Jest is that you can use Java Beans directly for indexing and searching. Suppose we have a bean <code>Talk</code> with several properties we can index instances of those in bulk in the following way:
</p>
<pre class="sourceCode java prettyprint">Builder bulkIndexBuilder = new Bulk.Builder();
        for (Talk talk : talks) {
            bulkIndexBuilder.addAction(new Index.Builder(talk).index(&quot;jug&quot;).type(&quot;talk&quot;).build());
        }
        client.execute(bulkIndexBuilder.build());
</pre>
<p>Given the <code>SearchResult</code> we have seen above we can then also retrieve our talk instances directly from the Elasticsearch results:
</p>
<pre class="sourceCode java prettyprint">List&lt;Hit&lt;Talk, Void&gt;&gt; hits = result.getHits(Talk.class);
        for (Hit&lt;Talk, Void&gt; hit: hits) {
            Talk talk = hit.source;
            log.info(talk.getTitle());
        }
</pre>
<p>Besides the execute method we have used so far there is also an async variant that returns a <code>Future</code>.
</p>
<p>The structure of the JEST API is really nice, you will find your way around it immediately. The possibility to index and retrieve Java Beans in your application makes it a good alternative to the native client. But there is also one thing I absolutely don&#x2019;t like: It throws too many checked Exceptions, e.g. a plain <code>Exception</code> on the central execute method of the <code>JestClient</code>. Also, there might be cases where the Jest client doesn&#x2019;t offer all of the functionality of newer Elasticsearch versions immediately. Nevertheless, it offers a really nice way to access your Elasticsearch instance using the REST API.
</p>
<p>For more information on Jest you can consult the <a href="https://github.com/searchbox-io/Jest/tree/master/jest">project documentation on GitHub</a>. There is also a <a href="http://www.ibm.com/developerworks/java/library/j-javadev2-24/index.html">nice article on Elasticsearch at IBM developerWorks</a> that demonstrates some of the features using Jest.
</p>
</section><section id="spring-data-elasticsearch" class="level2">
<h2><a href="#spring-data-elasticsearch">Spring Data Elasticsearch</a></h2>
<p>The <a href="http://projects.spring.io/spring-data/">Spring Data project</a> is a set of APIs that provide access to multiple data stores using a similar feeling. It doesn&#x2019;t try to use one API for everything, so the characteristics of a certain data store can still be available. The project supports many stores, <a href="http://projects.spring.io/spring-data-jpa/">Spring Data JPA</a> and <a href="http://projects.spring.io/spring-data-mongodb/">Spring Data MongoDB</a> being among the more popular. Starting with the latest GA release the implementation of <a href="https://github.com/spring-projects/spring-data-elasticsearch">Spring Data Elasticsearch</a> is also officially part of the Spring Data release.
</p>
<p>Spring Data Elasticsearch goes even one step further than the Jest client when it comes to indexing Java Beans. With Spring Data Elasticsearch you annotate your data objects with a <code>@Document</code> annotation that you can also use to determine index settings like name, numbers of shards or number of replicas. One of the attributes of the class needs to be an id, either by annotating it with <code>@Id</code> or using one of the automatically found names <code>id</code> or <code>documentId</code>. The other properties of your document can either come with or without annotations: without an annotation it will automatically be mapped by Elasticsearch, using the <code>@Field</code> annotation you can provide a custom mapping. The following class uses the standard mapping for speaker but a custom one for the title.
</p>
<pre class="sourceCode java prettyprint">@Document(indexName=&quot;talks&quot;)
    public class Talk {
        @Id
        private String path;
        @Field(type=FieldType.String, index=FieldIndex.analyzed, indexAnalyzer=&quot;german&quot;, searchAnalyzer=&quot;german&quot;)
        private String title;
        private List&lt;String&gt; speakers;
        @Field(type= FieldType.Date)
        private Date date;
        // getters and setters ommitted
    }
</pre>
<p>There are two ways to use the annotated data objects: Either using a repository or the more flexible template support. The <code>ElasticsearchTemplate</code> uses the Elasticsearch <code>Client</code> and provides a custom layer for manipulating data in Elasticsearch, similar to the popular <code>JDBCTemplate</code> or <code>RESTTemplate</code>. The following code indexes a document and uses a GET request to retrieve it again.
</p>
<pre class="sourceCode java prettyprint">IndexQuery query = new IndexQueryBuilder().withIndexName(&quot;talks&quot;).withId(&quot;/tmp&quot;).withObject(talk).build();
    String id = esTemplate.index(query);
    GetQuery getQuery = new GetQuery();
    getQuery.setId(id);
    Talk queriedObject = esTemplate.queryForObject(getQuery, Talk.class);
</pre>
<p>Note that none of the classes used in this example are part of the Elasticsearch API. Spring Data Elasticsearch implements a completely new abstraction layer on top of the Elasticsearch Java client.
</p>
<p>The second way to use Spring Data Elasticsearch is by using a <code>Repository</code>, an interface you can extend. There is a general interface <code>CrudRepository</code> available for all Spring Data projects that provides methods like <code>findAll()</code>, <code>count()</code>, <code>delete(...)</code> and <code>exists(...)</code>. <code>PagingAndSortingRepository</code> provides additional support for, what a surprise, paging and sorting.
</p>
<p>For adding specialized queries to your application you can extend the <code>ElasticsearchCrudRepository</code> and declare custom methods in it. What might come as a surprise at first: You don&#x2019;t have to implement a concrete instance of this interface, Spring Data automatically creates a proxy for you that contains the implementation. What kind of query is executed is determined by the name of the method, which can be something like <code>findByTitleAndSpeakers(String title, String speaker)</code>. Besides the naming convention you can also annotate the methods with an <code>@Query</code> annotation that contains the native JSON query or you can even implement the method yourself.
</p>
<p>Spring Data Elasticsearch provides lot of functionality and can be a good choice if you are already using Spring or even Spring Data. Some of the functionality of Elasticsearch might not be available at first or is more difficult to use because of the custom abstraction layer.
</p>
</section><section id="more" class="level2">
<h2><a href="#more">More</a></h2>
<p>Besides the clients we have seen in this post there are more available for the JVM. The <a href="/guide/en/elasticsearch/client/groovy-api/current/index.html">Groovy Client</a> wraps the Java API in Groovy, <a href="http://clojureelasticsearch.info/">Elastisch</a> is a client that implements the Elasticsearch API in a Clojure way. Have a look at the <a href="/guide/en/elasticsearch/client/community/current/clients.html">clients supported by the community</a> to see more, e.g. several clients for Scala.
</p>
<p>Some of the clients might be a little behind so if you need to have the newest features it might be best to choose the native client. But of course all of this projects are open source, so if you need a feature why not implement it yourself and contribute it back to the project?
</p>
</section></body></html>

Java Clients for Elasticsearch

<html><head></head><body><p><strong>UPDATE:</strong> <em>This article refers to our hosted Elasticsearch offering by an older name, Found. Please note that Found is now known as Elastic Cloud.</em></p><p><strong>Update May 1, 2017:</strong>&#xA0;<em>At the time of this writing, Sense existed purely as a Chrome plugin. &#xA0;It was later made into a Kibana plugin and has now been integrated directly into Kibana as &quot;Console&quot; in what is&#xA0;known as the &quot;Dev Tools&quot; section. &#xA0;The Dev Tools&#xA0;can be accessed in Kibana without the need to install a&#xA0;browser plugin. &#xA0;See <a href="/guide/en/kibana/5.3/console-kibana.html">the Console docs in Kibana</a> for more information on that functionality. &#xA0;As a result of the migration into Kibana, the Chrome plugin is no longer maintained, and the blog below is retained mostly for legacy purposes, though you may find that many of the usage patterns contained on this page (including autocomplete and the request/response panes) are directly applicable in Kibana&apos;s Dev Tools Console.</em><br></p><p>This article will introduce you to Sense, a Chrome plugin for Elasticsearch. It offers autocompletion, code highlighting and formatting and can help you with exploring Elasticsearch.
</p><section id="introduction" class="level2">
<h2><a href="#introduction">Introduction</a></h2>
<p>HTTP and JSON are the backbone for any work with Elasticsearch. Indexing, searching, administrative operations: Everything is done with HTTP and JSON. Most examples you will see are using <a href="http://curl.haxx.se/">cURL</a>, the http client widely available on different systems. With cURL you pass the HTTP method and the body of a request as command line parameters.
</p>
<p>While because of its availability on different systems cURL is great to be used for examples you might long for something more comfortable for your daily work. Enter <a href="https://github.com/bleskes/sense">Sense</a>, a <a href="http://www.google.com/chrome">Chrome</a> plugin for Elasticsearch.
</p>
</section><section id="basics" class="level2">
<h2><a href="#basics">Basics</a></h2>
<p>After installing Sense, it will allow you to use a simpler query request format, offers code completion and a highlighted response section.
</p>
<p>Like any extension, it can be started from the menu on the top right of Chrome. The UI consists of two larger panes, a white one on the left that is used to enter your requests to Elasticsearch and a black response section on the right. The host to be used can be configured in the top line and defaults to a local installation.
</p>
<p>
	<img src="https://api.contentstack.io/v2/assets/575e4c8a12798d8e381fd5fd/download?uid=bltccab886e4a31cd63?uid=bltccab886e4a31cd63" data-sys-asset-uid="bltde7dc0c68917467d" alt="Sense consists of two panes: One for the request and one for the response" style="max-width:100%;">
</p><figcaption>Sense consists of two panes: One for the request and one for the response</figcaption>
<p>You will see that there already is an example query that you can immediately execute using the green arrow on the top right of the query pane. It queries the current instance for all documents of all types in all indices. If you are starting with a fresh Elasticsearch installation this will return an empty result list but you can immediately start with some examples from the Elasticsearch guide. Let&#x2019;s index some data.
</p>
</section><section id="copy-and-paste" class="level2">
<h2><a href="#copy-and-paste">Copy and Paste</a></h2>
<p>As so many examples are already using cURL Sense offers a nice copy and paste feature that translates cURL requests to the proper syntax to use for Sense. Just copy the following snippet to your clipboard:
</p>
<pre class="prettyprint">curl -XPUT &apos;http://localhost:9200/twitter/tweet/1&apos; -H &apos;Content-Type: application/json&apos; -d &apos;{
    &quot;user&quot; : &quot;kimchy&quot;,
    &quot;post_date&quot; : &quot;2009-11-15T14:12:12&quot;,
    &quot;message&quot; : &quot;trying out Elastic Search&quot;
}&apos;
</pre>
<p>Mark all of the existing query in the left pane and press Ctrl-V to paste the text. You will notice that Sense automatically removes the cURL parameters and transforms the query to its internal syntax.
</p>
<pre class="prettyprint">PUT /twitter/tweet/1
{
    &quot;user&quot; : &quot;kimchy&quot;,
    &quot;post_date&quot; : &quot;2009-11-15T14:12:12&quot;,
    &quot;message&quot; : &quot;trying out Elastic Search&quot;
}
</pre>
<p>Once you click the green arrow to execute the indexing query you will see the familiar response from Elasticsearch that your document has been indexed.
</p>
<p>You can also go the other way around. If you&#x2019;d like to get the cURL equivalent of a query you can mark it in the pane, press the little tool icon and choose Copy as cURL. You can then paste the result to a shell and execute the same command using cURL. The keyboard shortcut for this is Ctrl + Shift + C.
</p>
</section><section id="autocompletion" class="level2">
<h2><a href="#autocompletion">Autocompletion</a></h2>
<p>Instead of copying and pasting we can of course also create the query ourselves. Remove all the existing text from the query pane and start typing GET. Sense will immediately offer a small tooltip with possible completions. Continue typing the / and you will notice that the popup has some internal knowledge of our Elasticsearch instance: It offers our index, twitter, that we just created.
</p>
<p>
	<img src="https://api.contentstack.io/v2/assets/575e4c8a43e9adc538713aa0/download?uid=bltf312c168e5c212f2?uid=bltf312c168e5c212f2" data-sys-asset-uid="bltbe74fa336c406c11" alt="Autocompletion works on indexes" style="max-width:100%;">
</p><figcaption>Autocompletion works on indexes</figcaption>
<p>We can continue to enter our query, e.g. a simple term query on the user:
</p>
<pre class="prettyprint">GET /twitter/tweet/_search
{
    &quot;query&quot;: {
        &quot;term&quot;: {
           &quot;user&quot;: {
              &quot;value&quot;: &quot;kimchy&quot;
           }
        }
    }
}
</pre>
<p>If during any part of typing the query you are unsure what is allowed or wonder about the correct spelling you can always press Ctrl-Space to get a list of possible options. This powerful context sensitive completion can help a lot when experimenting with Elasticsearch.
</p>
</section><section id="history" class="level2">
<h2><a href="#history">History</a></h2>
<p>When working with Sense you might notice that you did something exciting a while ago but can&#x2019;t remember what you did. No need to undo all your edits; just open the history that can be accessed on the top right. It will show you the last 500 edits and you can choose one that you would like to execute again.
</p>
<p>
	<img src="https://api.contentstack.io/v2/assets/575e4c8a9985d589763769d8/download?uid=blt8f4ebfa5f3270d60?uid=blt8f4ebfa5f3270d60" data-sys-asset-uid="bltf46bc349e11f4a2e" alt="The history will show you your recent queries" style="max-width:100%;">
</p><figcaption>The history will show you your recent queries</figcaption>
</section><section id="structuring-json" class="level2">
<h2><a href="#structuring-json">Structuring JSON</a></h2>
<p>Sometimes requests and responses can become rather large with different indentation levels. To structure them you can use two features of Sense: Code indentation and code folding. If you don&#x2019;t insert well indented code at first Sense can do the indentation for you: Just press Ctrl-I to have it formatted. And if your query or response really gets too long you can use the code folding feature. In the query as well as the response pane you can see little arrows at some line numbers. Those can be used to fold or unfold a code block. This can help a lot to make the JSON more approachable.
</p>
</section><section id="conclusion" class="level2">
<h2><a href="#conclusion">Conclusion</a></h2>
<p>Sense is a fantastic tool for everyday work with Elasticsearch as well as for learning the query syntax. It will guide you with the queries, format everything correctly and do syntax highlighting on the query and response for you. If you&#x2019;re not using it yet you should at least give it a try.
</p><p><em>Editor&#x2019;s Note (May 1, 2017): Starting with 6.0, any curl command to Elasticsearch containing content will require a valid content type header. As a result, this post has been updated to reflect this change and to set readers of this post up for success with future versions.</em><br></p>
</section></body></html>

Articles by Florian Hopf

Java Clients for Elasticsearch

Sense - a Cool JSON Aware Interface to Elasticsearch

Products & Solutions

Company

Resources

Solutions

Elastic (ELK) Stack

Register Today

Featured topics

News

Articles by Florian Hopf

Java Clients for Elasticsearch

Sense - a Cool JSON Aware Interface to Elasticsearch