Alerting on cluster and index eventsedit
X-Pack alerting is a set of administrative features that enable you to watch for changes or anomalies in your data and perform the necessary actions in response. For example, you might want to:
- Monitor social media as another way to detect failures in user-facing automated systems like ATMs or ticketing systems. When the number of tweets and posts in an area exceeds a threshold of significance, notify a service technician.
- Monitor your infrastructure, tracking disk usage over time. Open a helpdesk ticket when any servers are likely to run out of free space in the next few days.
- Track network activity to detect malicious activity, and proactively change firewall configuration to reject the malicious user.
- Monitor Elasticsearch, and send immediate notification to the system administrator if nodes leave the cluster or query throughput exceeds an expected range.
- Track application response times and if page-load time exceeds SLAs for more than 5 minutes, open a helpdesk ticket. If SLAs are exceeded for an hour, page the administrator on duty.
All of these use-cases share a few key properties:
- The relevant data or changes in data can be identified with a periodic Elasticsearch query.
- The results of the query can be checked against a condition.
- One or more actions are taken if the condition is true — an email is sent, a 3rd party system is notified, or the query results are stored.
How Watches Workedit
X-Pack provides an API for creating, managing and testing watches. A watch describes a single alert and can contain multiple notification actions.
A watch is constructed from four simple building blocks:
- Schedule
- A schedule for running a query and checking the condition.
- Query
- The query to run as input to the condition. Watches support the full Elasticsearch query language, including aggregations.
- Condition
- A condition that determines whether or not to execute the actions. You can use simple conditions (always true), or use scripting for more sophisticated scenarios.
- Actions
- One or more actions, such as sending email, pushing data to 3rd party systems through a webhook, or indexing the results of the query.
A full history of all watches is maintained in an Elasticsearch index. This history keeps track of each time a watch is triggered and records the results from the query, whether the condition was met, and what actions were taken.