18 April 2017 News

Hosted Elasticsearch Services Roundup: Elastic Cloud and Amazon Elasticsearch Service

By Haley EshaghTyler Hannan

Our Elastic Cloud product, which runs Elasticsearch and Kibana as a service, has become popular. But there's been some confusion. Elastic Cloud and the Amazon Elasticsearch Service are different offerings, and neither is the same as running vanilla Elasticsearch on AWS.

To be very clear, the Amazon Elasticsearch Service is not related to Elastic. Elastic does not partner with, participate in, or support, the Amazon Elasticsearch Service.

Instead, we offer Elastic Cloud, our hosted Elasticsearch, Kibana, and X-Pack service that runs on AWS (and now available on Google Cloud Platform (GCP)). We also support a variety of customers who self-manage their own clusters on AWS.

And, again, Elastic Cloud is not the same as the Amazon Elasticsearch Service. In fact, we believe Elastic Cloud brings unique value to mission-critical applications, validating the importance of our offering.

We'd like to explain why.

Features That Delight with the Projects You Love, Hosted

When we talk about Elastic Cloud, we often dive straight into the details about our open source products in the Elastic Stack — Elasticsearch, Kibana, Beats, and Logstash. While they're fantastic, we'd like to start from a different angle: X-Pack.

X-Pack is an extension that bundles together security, alerting, monitoring, reporting, and graph exploration capabilities that extend what you can do with the Elastic Stack. Furthermore, Elastic Cloud is the only offering that ships with free features like Search Profiler and the Elastic Maps Service (with zoom levels you can dial up to 18).

Access to these features opens up brave new worlds and possibilities with Elasticsearch and Kibana. And X-Pack features, along with the open source products, get better and better with every release.

This is why we take pride in being able to offer the latest versions of Elasticsearch, Kibana, and X-Pack on Elastic Cloud the day a release drops. When Elasticsearch 5.4 or 6.0 or 10.3 ships, you can count on Elastic Cloud having the option to upgrade to it at the same time. Similarly, if there is a security vulnerability, we patch user clusters as soon as the fix is released.  

Our ability to get you the latest and greatest the moment it's available is critical to your success. We also know there's more to it than that, whether you need to make a change to your elasticsearch.yml (specific settings are whitelisted in Elastic Cloud), click a single button to upgrade your cluster to the most recent version, restore from an automated snapshot that occurs every 30 minutes, scale your cluster by moving a slider, or, recently, file a ticket in the support portal provided all Elastic Cloud customers.

All of this matters because it makes good sense, but also because you've told us it matters. And we will always strive to honor that and listen to you.

There Is No Compression Algorithm for Experience

As an Elastic Cloud user, you are, in a way, benefiting from the wisdom of the relative ancients. Elastic Cloud has been around since 2012 (known as Found in its early life). Amazon's Elasticsearch Service was introduced in 2015.

Operational excellence is not something to take lightly or undervalue. There is a lot of work that goes into ensuring that the lights stay on and customers are getting a high level of service. There is no compression algorithm for experience. (Amazon, of course, has been offering services via AWS since 2006. In the context of managing and supporting hosted Elasticsearch, however, our team has a few years head start.)

Elastic Cloud is a mature platform. It supports customers like IBM, Fandango, Activision Blizzard, Unilever, Shopify, and more. They put their trust in us to keep their mission-critical systems firing on all cylinders.

To make sure that we are providing the uptime our Elastic Cloud premium customers require, we now provide an SLA for Elastic Cloud. And, we do so by measuring cluster-level availability instead of infrastructure-level availability.

We Eat, Sleep, and Breathe the Elastic Stack

The team that builds the Elastic Stack works alongside the team that builds and runs Elastic Cloud. No one, and we mean no one, understands the nature of issues like the Elastic team. We strive for a bug-free, problem-free experience. But it is software and things do, indeed, happen. Support from the source can make the difference between a pleasant day with a resolved interruption and one that is downright unpleasant.

Amazing Performance, Very Little Cost

As of today, Elastic Cloud uses only I-series instances on AWS (currently I2). As such, even the smallest cluster in Elastic Cloud runs on best-in-class AWS instances. Given a production need for I/O, network, and CPU guarantees, T-series instances aren't terribly applicable. And, of course, I3 instances are coming soon.

That said, simple price comparison is a race to the bottom. The Amazon Elasticsearch Service is cheaper in some circumstances. In an apples-to-apples comparison (i2 instances with similar memory to disk ratios, retention periods that match, matching deployment across AZs, and ignoring support and commercial products), the Amazon Elasticsearch Service is actually ~20% more expensive. (We've crunched some numbers as an example at the end of this post **.)

So that means Elastic Cloud is price competitive and you get the benefit of current releases, security patches, and X-Pack.

"But wait," you may say. "Amazon already has my credit card number." Or sometimes, "I have to use my Amazon credentials for payment."

We've got you covered.  Now you can pick between having your bill come through Elastic or AWS via the hosted Elasticsearch service on the AWS Marketplace!

Running Elasticsearch on AWS

We didn't forget. Read this post.

The Details Matter

"I get it," you say, "Elastic knows their products better than anyone else. But what about the details?" They are below and we will keep the table current as things change.

The beauty of modern software is that there is flexibility in deployment and consumption models. If you desire a cloud deployment, make sure you understand what you are getting and from whom. As the creators of Elasticsearch, Kibana, Beats, and Logstash, we want you to have the best possible experience with the Elastic Stack. That's why we created and stand behind only one hosted offering — Elastic Cloud.

Last updated on February 6, 2018

Amazon Elasticsearch Service Elastic Cloud
Elastic Stack - Open Source Features Some All
Current Version 6.0.0 6.2.0
Same Day Elastic Stack Version Release No Yes
(Elastic Commercial Plugins)
No Yes
Security, Alerting, Monitoring, Graph, Reporting
Elastic Technical Support No Yes
Elastic Cloud Subscriptions
One-Click Upgrades No Yes
Default Snapshots 1 time per day 48 times per day
Every 30 minutes
Stored for 48 hours
Instant Rollout of Elasticsearch and Kibana Security Patches No Yes
Custom Plugin Support Not supported Supported
Java Transport Client Not supported Supported
Cross Zone Replication Support for up to 2 availability zones Support for up to 3 availability zones
SLA-Based Support General level support, not specific to AWS ES Yes
Uptime SLA No Yes
99.95% cluster uptime in a given month as long as the cluster is deployed across 2 or more zones
Elastic Maps Service (geo-visualizations in Kibana) Does not work out of the box Yes
Data Transfer Charges Regular AWS data transfer charges apply No additional data transfer charges
Security Only perimeter-level security and standard IAM policies
  • Transport encryption
  • Authentication
  • Role-based access control
  • Field- and document-level security
Alerting Need to build and manage your own system to create alerting functionalities. This depends on Amazon Cloudwatch, which comes with predefined, simple metrics. If you want something more sophisticated, or related to your data, you'll need to build a custom metric and alerts.
  • Allows you to create scheduled queries, conditions, and actions on your data in Elasticsearch.
  • UI to create, manage, and take actions on alerts.
Monitoring Depends on Amazon Cloudwatch, that covers a few metrics including cluster state, node information, etc. Feature-rich and complete monitoring product specifically designed for Elasticsearch and Kibana.
  • Captures a wider range of metrics including search/index rate and latency, garbage collection count and duration, thread pool bulk rejection/queue, Lucene memory breakdown, and more with 10-second data granularity to ensure that clusters are running healthy.
  • Robust tools to diagnose, troubleshoot and keep your cluster healthy including automatic alerts on cluster issues.
Graph No Yes
Reporting No Yes

If you haven't already, we invite you to take Elastic Cloud for a spin. Try it out for 14 days, no credit card required. Enjoy.

** Here's a more specific example (using the AWS us-east region pricing): An i2.2xlarge instance in AWS Elasticsearch is $2.387 / hour which is ~$1,776 per month (averaging 744 hours). It has 61 GB of RAM, 8 vCPUs and 1600 GB of storage. The comparable Elastic Cloud cluster is 64GB in a single availability zone, which gives the user about ¼ of an i2.8xlarge instance, so very similar specs to what you get from an i2.2xlarge instance. This comes down to ~$1,369 per month before any discount applied for annual terms. This comparison becomes even more favorable when deploying a 2 AZ or 3 AZ cluster, since a 2 AZ cluster is only 1.9 times more expensive than a single AZ cluster, and a 3 AZ cluster is only 2.8 times more expensive.

Editor's Note, May 22, 2017: This post has been updated to include detail on how we estimate AWS pricing and to provide clarity and additional perspective where requested.