With simple REST APIs exposed over HTTP, Elasticsearch is a platform that encourages integration and automation. Whether using the percolator API, direct integration with infrastructure monitoring systems like Nagios, or cron-job scripts that run queries and take action, users have always been able to build notification and alerting systems on top of Elasticsearch.
As downloads of Elasticsearch have grown and its use cases have matured and expanded, we have heard frequent requests from our users and customers for integrated alerting and notification capabilities with a simple API to help them detect changes and anomalies in their growing, diverse data sets.
Today, after spending many hours with our customers understanding their needs, and many months designing and coding, we are thrilled to announce Watcher. Watcher is a flexible, powerful product that will allow Elasticsearch users to get insights and take action on changes in their data more efficiently across a wide range of use cases. Whether you're monitoring social media to detect earthquakes, looking for suspicious activity in your infrastructure, or even managing a rover on Mars, the ability to easily push notifications based on changes in your data will be a game changer.
Like all of our products, Watcher is built using public Elasticsearch extension points. This means you can install it on your existing Elasticsearch cluster and it's easy to get started.
How It Works: The Anatomy of a “Watch"
Define a Query
Interested in 404 errors? Low disk space? Or want to know the exact moment when social sentiment for your latest campaign takes takes off on Twitter? Just define it as a query using the full power of the Elasticsearch query language, including aggregations.
Set the Condition
Now that Watcher knows what to look for, set a threshold worthy of an alert — maybe you only care about 404 errors if they occur 50% more frequently than average. To craft more sophisticated conditions, scripting is supported.
Select a Schedule
Choose how often Watcher runs your queries and checks the condition. It's easy to define simple schedules — run every minute, hour, or day. For more complex scheduling needs, cron syntax is also supported.
Define the Actions
If your conditions are met, Watcher can send a custom email, push data to external systems like PagerDuty via WebHook, or take the results of your query and store them in Elasticsearch.
If you would like to learn more, please join us for the Watcher Webinar, happening on Wednesday, May 20. Uri Boness and Steve Kearns will give a detailed overview, including installation, configuration, and a live demo. And when you sign up, you can also opt into the Watcher beta.
Lastly, I am happy to announce that Watcher will be free for existing and future subscription customers. We invest heavily in the success of our customers, from insights provided by our developers during production and development support, to our growing family of products, and we couldn't be happier to provide them our latest product at no additional cost.