Enable Watcheredit

If you are looking for Kibana alerting, check Alerting and Actions in the Kibana Guide, available for version 7.7 and later.

Watcher lets you take action based on changes in your data. It is designed around the principle that, if you can query something in Elasticsearch, you can alert on it. Simply define a query, condition, schedule, the actions to take, and Watcher will do the rest.

Watcher can be enabled when configuring your cluster, available for clusters with version 1.7.2 or higher. You can run Alerting on a separate cluster from the cluster whose data you are actually watching.

Before you beginedit

Some restrictions apply when adding alerts. To learn more, check Restrictions for alerts (via Watcher).

To enable Watcher on a cluster, you may first need to perform one or several of the following steps. The options shown in the UI differ between stack versions; if an option is not available, you can skip it.

  • For Elasticsearch versions before 5.0: Enable authentication
  • To receive default Elasticsearch Watcher alerts (cluster status, nodes changed, version mismatch), you need to have monitoring enabled to send to the Admin email address specified in Kibana. To enable this, go to Advanced Settings > Admin email. Then, allowlist that same email in the Elasticsearch Service Console by updating the monitoring email allowlist.

To learn more about Kibana alerting and how to use it, check Alerting and Actions (version 7.7 and later), Alerting on cluster and index events (version 6.3 to 7.6), Alerting on cluster and index events (version 5.0 to 6.2), or Elasticsearch Watcher (all versions before 5.0).

Send alerts by emailedit

Alerting can send alerts by email. You can configure notifications similar to the operational emails that Elasticsearch Service sends automatically to alert you about performance issues in your clusters.

To send alerts by email:

  1. Log in to the Elasticsearch Service Console.
  2. From the user menu, select Organization and then Contacts, or visit the contacts page directly.
  3. Enter a recipient to be allowlisted and select Add.

    An email is sent to the email address.

  4. The email recipient must click a confirmation button to acknowledge the request.

    After the allowlist request is acknowledged, you are able to send alerts to the recipient address by email.

  5. Optional: You can add HTML sanitization settings under Elasticsearch User settings in the Elasticsearch Service Console so that HTML elements are sanitized in the email notification.

For more information on sending alerts by email, check Email action.

Elasticsearch Service lets you add destination email addresses to the allowlist as described in this section, but other configuration options are not supported. Specifically, configuring email accounts is not applicable to Elasticsearch Service.

Advanced usageedit

Slack and PagerDuty integrationedit

Under the hood, Alerting is configured through elasticsearch.yml. If you want to customize your Alerting settings, you can provide custom elasticsearch.yml snippet which is appended to your configuration.

To provide the custom snippet, you can use the console Elasticsearch settings editor for your deployment.

For example if you want to use the Slack integration:

There are three steps to integrate Elasticsearch 7.0 or greater with Slack:

  1. Generate a Webhook URL in Slack. It will look similar to https://hooks.slack.com/services/..
  2. Add a Slack account name to your Elasticsearch User settings
  3. Associate the Slack account with the Slack Webhook in the Elasticsearch keystore

To add a webhook in Slack, select the settings icon, then choose Add an app and search for webhook.

The following example shows a configuration with multiple Slack accounts (account1, account2, and account3) specified in elasticsearch.yml:

xpack.notification.slack:
  default_account: account1
  account:
    account1:
      message_defaults:
        from: account1
        to: channel1
    account2:
      message_defaults:
        from: account2
        to: channel2
    account3:
      message_defaults:
        from: account3
        to: channel3
Slack Webhook account settingsedit

The Slack Webhook is set for each account in the Elasticsearch Keystore with the following settings:

Setting name
xpack.notification.slack.account.ACCOUNT_NAME.secure_url where ACCOUNT_NAME is the Slack account, such as account1.
Type
Single string
Secret
The Webhook URL you generated in Slack earlier.

To specify a Slack account to use for a Watcher Alert that isn’t set as default_account, you must create an Advanced Watch and explicitly define which Slack account to use in the actions section.

If you have a Slack account that is not currently set as default_account, and you want to use this account for a Watcher Alert, you must create an Advanced Watch and explicitly define in the Actions section of the UI which Slack account to use.

PUT _watcher/watch/test-alarm
{
  "metadata" : {
    ...
  },
  "trigger" : {
    ...
  },
  "input" : {
    ...
  },
  "actions" : {
    "notify-slack" : {
      "throttle_period" : "10s",
      "slack" : {
        "account" : "account2",
        "message" : {
          "to" : [ "#testing-channel" ],
          "text" : "You Know, for Search"
        }
      }
    }
  }
}

In Elasticsearch versions before 7.0:, you are not required to use the Elasticsearch keystore. Instead, you can use the console Elasticsearch settings editor for your deployment.

Advanced Alerting configuration