Elasticsearch-Hadoop

Best of two worlds for real-time analysis

Connect the massive data storage and deep processing power of Hadoop with the real-time search and analytics of Elasticsearch. The Elasticsearch-Hadoop (ES-Hadoop) connector lets you get quick insight from your big data and makes working in the Hadoop ecosystem even better.

Getting started with Elasticsearch: Store, search, and analyze with the free and open Elastic Stack.
Watch video
Intro to ELK: Get started with logs, metrics, data ingestion and custom vizualizations in Kibana.
Watch video
Getting started with Elastic Cloud: Launch your first deployment.
Learn more

Interactive analytics on your Hadoop data

Hadoop shines as a batch processing system, but serving real-time results can be challenging. For truly interactive data discovery, ES-Hadoop lets you index Hadoop data into the Elastic Stack to take full advantage of the speedy Elasticsearch engine and beautiful Kibana visualizations.

With ES-Hadoop, you can easily build dynamic, embedded search applications to serve your Hadoop data or perform deep, low-latency analytics using full-text, geospatial queries and aggregations. From product recommendations to genomic sequencing, ES-Hadoop opens up a new world of broad applications.

Diagram of Elasticsearch-Hadoop

Seamlessly move data between Elasticsearch and Hadoop

Live decision making only happens with lightning fast data movement. With dynamic extensions to existing Hadoop APIs, ES-Hadoop lets you easily move data bi-directionally between Elasticsearch and Hadoop while exposing HDFS as a repository for long-term archival. Partition awareness, failure handling, type conversions, and co-location are all done transparently.

Natively interface with Spark and friends

ES-Hadoop offers full support for Spark, Spark Streaming, and SparkSQL. Additionally, whether you are using Hive, Pig, Storm, Cascading, or standard MapReduce, ES-Hadoop offers a native interface allowing you to index to and query from Elasticsearch. No matter what you use, the absolute power of Elasticsearch is at your disposal.

Logos for Spark, Hive, Storm, Pig, MapReduce and Cascading

Your data is secure everywhere

ES-Hadoop ships with all the security features you'll need, including HTTP authentication and SSL/TLS support, to securely move your data between your Hadoop and Elasticsearch clusters. It also works with Kerberos-enabled Hadoop deployments.

Works with any flavor of Hadoop

We are official partners with Cloudera, MapR, Hortonworks, and Databricks, so whether you're using vanilla Hadoop or any other distribution, we've got you covered. ES-Hadoop has been certified with CDH, MapR, and HDP.