Index Directly into Elasticsearch from Hadoop

The native integration allows you to efficiently push data into Elasticsearch using the existing Hadoop tools you know and love.

Query Elasticsearch from Hadoop

The rich query API of Elasticsearch allows you to ask complex questions and use the real-time results in Hadoop.

Use HDFS as a Long-Term Archive for Elasticsearch

es-hadoop allows Elasticsearch to push backup data to HDFS using the built-in snapshot and restore capability.

How People are Using Elasticsearch and Hadoop

Klout Queries Over 400M Users' Data To Build Marketing Campaigns

Using HDFS to store user data and index it into Elasticsearch, Klout builds real-time targeted marketing campaigns that are generated in seconds rather than minutes.

MutualMind Replaces 15-Minute Batch Process with Real-Time Analysis

With customers like AT&T, Kraft, Nestle, and Starbucks interested in keeping a pulse on their brands, MutualMind uses Elasticsearch to get quick insight and Hadoop for batch-based statistical analysis.

International Financial Services Firm Quickly Analyzes Access Logs

Instead of waiting hours to run MapReduce jobs to analyze access logs, a global financial institution gets value from its data with Elasticsearch in minutes—and even increased the quantity of log data it processed from one hour to a full week.

Works with Any Flavor of Hadoop Distribution

We are official partners with a number of organizations within the Hadoop ecosystem, including Cloudera, MapR, Hortonworks, Databricks, and Concurrent. Whether you're using vanilla Hadoop, or other distributions like CDH, HDP, and MapR, Elasticsearch has got you covered. As an added bonus, we are also certified on Cloudera Enterprise 5 and are Certified Technology Partners with Hortonworks.

Take a Look Under the Hood

Visualize Your Big Data

Elasticsearch works with the visualization tool Kibana to help you explore your big data with in real time. With beautifully designed graphs, charts, and maps, Kibana transforms your data into real-time, customizable dashboards that let you visualize the value of your data.

Leave the Real-Time Analytics to Us

Gone are the days of waiting hours or more for a batch process to run in order to get insight into your Hadoop data. Elasticsearch provides responses in milliseconds, which can significantly reduce a Hadoop job's execution time and the cost associated with it, especially on "rented resources" such as Amazon EMR or EC2.

Ask More Sophisticated Questions

Elasticsearch provides a robust query DSL that lets users to ask sophisticated questions that result in more complete answers, faster.

Prepared for When Things Go Awry

Elasticsearch is designed to tolerate hardware failures. Es-hadoop continues communicating with the cluster, even when failures occur.

Added Efficiency with Our Native Integration

Elasticsearch is natively integrated with Hadoop so there is no gap for the user to bridge. We provide a dedicated Input and Output format for vanilla MapReduce, taps for reading and writing data in Cascading, storages for Pig and Hive, a native Spark Resilient Distributed Dataset (RDD) for both Java and Scala, and support for Storm's bolt and spout abstractions so you can access Elasticsearch just as if the data were in HDFS.

Enhance Your Workflow to Get the Best of Both Worlds

Get maximum flexibility with the es-hadoop connector by leveraging everything that Hadoop has to offer (via MapReduce, Hive, Pig, Cascading, Spark, and Storm) and combining it with a real-time search and analytics capability of Elasticsearch.

Need to Grow? Just Add More Nodes.

Elasticsearch can be scaled in the same way as your Hadoop cluster – add more Elasticsearch nodes and the data will be automatically re-balanced.