Our Story

A bit about how we got started and what’s unfolded since.

From the early days of Elasticsearch to how the ELK Stack came to be, a period of awesome (but chaotic) development, the introduction of the Elastic Stack, and a new era of openness — there’s a lot of goodness to unpack in our narrative.

circle_only_path

Early 2000s

Downloads: 0

It Started with a Recipe App

In a London apartment, Shay Banon was looking for a job while his wife attended cooking school at Le Cordon Bleu. In his spare time, he started building a search engine for her growing list of recipes.

His first iteration was called Compass. The second was Elasticsearch (with Apache Lucene under the hood). He open sourced Elasticsearch, created the #elasticsearch IRC channel, and waited for users to appear.

The response was impressive. Users took to it naturally and easily. Adoption went through the roof, a community started to form, and people noticed — namely Steven Schuurman, Uri Boness, and Simon Willnauer. Together, they founded a search company.

founders.jpg

circle_only_path

June 2012

Downloads: <16,000

You Know, for Search Inc.

Around the time Elasticsearch Inc. was founded two other open source projects were taking flight.

Jordan Sissel was working on Logstash, an open source, pluggable ingestion tool for sending logs to the “stash” of the user’s choosing, one of which was Elasticsearch. He was also developing a UI on top of it to visualize log data — and it was shaky at best.

Fortunately, someone else was tinkering with the visualization challenge. Enter Rashid Khan, who was working on an open source UI called Kibana.

Shay, Jordan, and Rashid had known each other and their projects for some time and decided to team up, resulting in the ELK Stack — Elasticsearch, Logstash, and Kibana Stack.

A while later, we released two commercial plugins: Marvel for monitoring and Shield for security.

Heya, Elastic. Welcome, Found.

At Elastic{ON} 2015 in San Francisco, we made two big announcements. The first: we rebranded the company to be called Elastic. The new name better represented our growing product ecosystem and suite of use cases. The second: we joined forces with Found, a company delivering hosted and managed Elasticsearch on AWS. By teaming up, we could provide the simplest, most complete offering on the market.

circle_only_path

July 2015

Downloads: 36,431,145

Emerging from Our Primordial Ooze

In the early days, building and releasing software at Elastic took an every-engineer-for-themselves approach: ship whatever version you want, whenever you want — just make it awesome. Kibana had betas, Logstash had milestones, Elasticsearch had numbers. Plugins happened as they pleased. It was chaotic, but it worked...until it didn't.

As users were doing more with the product, we needed to build a product that did more for the users. We added more capability, submitted more pull requests, built new plugins and extensions. The awesomeness increased, complexity emerged, and things got messy for our technology stack.

For instance, if you were running version 1.7 of Elasticsearch and version 2.3 of some plugin, there wasn’t an automatic way to know if they were compatible or if the plugin was silently failing. This was a bug.

We also started to hear ourselves say things like, “If you want to use Shield, you need Elasticsearch 1.4.2….unless you’re using Watcher. In that case, you’ll need Elasticsearch 1.5.2. And if you’re using Elasticsearch 1.5.2, that’s only compatible with Kibana 4.0.x, Logstash 1.4.x, Shield 1.2.x, and Watcher 1.0.x.”

We had arrived at a special sort of versioning hell — and the support matrix didn’t look any better. It was time for a change.

Pausing to Take a Beat(s)

While the product teams wrestled with version numbers, another product story was unfolding. In 2015, we welcomed Packetbeat, a Berlin-based, husband-and-wife team engineering a lightweight way to send network data to Elasticsearch, to the Elastic family.

That got us thinking: what if we had a family of single-purpose, lightweight data shippers to send network data, logs, metrics, audit data, and more from edge machines to Logstash and Elasticsearch? And so Beats was born.

circle_only_path

October 2015

Downloads: 44,378,846

The Bonanza Begins

October 2015 marked a turning point for addressing our product versioning and compatibility complexities.

Dubbed the “release bonanza,” it was the first time all of our products — Elasticsearch 2.0, Logstash 2.0, Watcher 2.0, Shield 2.0, and Kibana 4.2 — shipped together on the same day. (Beats 1.0 had another month to bake.)

Coordinating this effort wasn’t easy. The engineering teams had to change the way they worked together to build and test the products. It was worthwhile, though. This shift made it easier for users to get started with our products and made our products more reliable to do amazing things with.

Enter, Elastic Cloud

A few months later, the release bonanza would no longer only be a downloadable experience. Elasticsearch and Kibana were now available as a service on AWS through Elastic Cloud, the offering formerly known as Found.

circle_only_path

October 2016

Downloads: 91,183,928

BELK 5.0 Elastic Stack 5.0

Aligning release cadence with Elasticsearch 2.0 was the first step toward a more mature product offering. The 5.0 launch was the second step. It introduced a more integrated, better tested, and easier getting started experience than ever before.

The 5.0 release also bundled all of our commercial plugins (which we called Shield, Marvel, and Watcher at the time) into a single extension called X-Pack. It consisted of features like security, monitoring, and alerting for our core products, and grew to include machine learning when we brought a London-based company called Prelert into the Elastic family.

So Modules, Much Simplified

In version 5.3 (released in March 2017), Filebeat formally introduced the concept of “modules,” or a set of safe configurations to ship, parse, store, analyze, and visualize common log formats (e.g., Apache, Nginx, MySQL, etc.) in the Elastic Stack. Modules simplified the getting started experience of going from dataset to dashboard.

Metricbeat and Packetbeat had their own flavors of modules, and months later, Logstash would introduce modules of its own for ArcSight and NetFlow data.

circle_only_path

May 2017

Downloads: 137,715,884

A New Frontier: Introducing ECE

From the beginning, we had a vision for simplifying how users deploy Elastic in their organizations. We took the technology we use to manage our own Elastic Cloud service and released Elastic Cloud Enterprise (or ECE) to allow businesses big and small to download all the goodness of our hosted offering and run it themselves. ECE made managing one cluster or thousands seamless, streamlining the management and orchestration of Elastic products and solutions in any environment.

circle_only_path

October 2017

Downloads: 192,865,831

Elastic Solutions Precipitate

As modules started to multiply, getting started with the Elastic Stack to address a particular use case like logging or metrics got easier and easier. And momentum continued to build when we joined forces with Opbeat, a Copenhagen-based application performance monitoring (APM) company, and Swiftype, a San Francisco-based site and enterprise search company, a few months later. Both companies became part of Elastic.

By this time, our company had matured to a place where we could offer streamlined ways for solving common problems leading us to formally introduce our solutions. While our solutions range from a DIY experience to something more turnkey, each has real product behind it and can be deployed in just a few minutes.

circle_only_path

February 2018

Downloads: 267,972,265

Opening Our X-Pack Code

From open source to open communication, being open is at the heart of all we do. This is why we made the decision to open the code to our commercial X-Pack features in order to speed up development time and increase community engagement, allowing everyone to contribute to, comment on, and inspect our code.

As a result, getting started with the Elastic Stack became even easier, with all of the X-Pack features now shipping with the default distributions of Elasticsearch, Kibana, Beats, and Logstash. This change didn’t take away any Apache 2.0 code. Instead, we doubled down on being open.

There’s more to our story. Stay tuned for updates as our adventure continues to unfold.

Explore More Things Elastic

Discover what we make, how we live it, and how it’s part of the overall community.

Our Customers

Learn how companies use Elastic every day.

Learn More

Our Culture

Read about what we’re up to.

Learn More

Our Products

See what we build.

Learn More