Enable Alerting (Watcher)

Alerting lets you take action based on changes in your data. It is designed around the principle that, if you can query something in Elasticsearch, you can alert on it. Simply define a query, condition, schedule, the actions to take, and Alerting will do the rest.

Alerting (via Watcher) can be enabled when configuring your cluster, available for clusters with version 1.7.2 or higher. You can run Alerting on a separate cluster from the cluster whose data you are actually watching.

In Elasticsearch 5.x, Watcher was renamed to Alerting and became a part of X-Pack. If you’re using a version of Elasticsearch before 5.0, think Watcher every time you read about Alerting.

Before you begin

Some restrictions apply when adding alerts. To learn more, see Restrictions for alerts (via Watcher).

To enable alerting on a cluster, you also need to:

  • Enable automatic index creation on the Configuration page, if it isn’t enabled already
  • Enable scripting for most uses of Alerting, as Alerting uses the Elasticsearch script infrastructure
  • For Elasticsearch versions before 5.0: Enable authentication

To learn more about Alerting and how to use it, see Alerting on cluster and index events (version 6.3 and later), Alerting on cluster and index events (version 5.0 to 6.2), or Elasticsearch Watcher (all versions before 5.0).

Send alerts by email

Alerting can send alerts by email.

To send alerts by email:

  1. Sign in to the Elasticsearch Add-On for Heroku console.
  2. Go to Account and then Profile.
  3. Enter a recipient to be whitelisted under Monitoring email whitelist and click Add.

    An email is sent to the email address.

  4. The recipient must acknowledge the request by clicking Whitelist Email in the email.

    After the whitelist request is acknowledged, you are able to send alerts to the recipient address by email.

For more information on sending alerts by email, see Email Action.

Elasticsearch Add-On for Heroku lets you add destination email addresses to the whitelist as described in this section, but other configuration options are not supported. Specifically, Configuring email accounts is not applicable to Elasticsearch Add-On for Heroku.

Advanced usage

Slack, HipChat, and PagerDuty integration

Under the hood, Alerting is configured via elasticsearch.yml. If you want to customize your Alerting settings, you can provide custom elasticsearch.yml snippet which is appended to your configuration.

To provide the custom snippet, you can use the console Elasticsearch settings editor for your deployment.

For example if you want to use the Slack integration:

In Elasticsearch version 7.x and later, you can no longer configure Slack accounts using elasticsearch.yml settings. Your Slack account name and secure URL settings need to be configured in the Elasticsearch keystore, with a setting like the following:

xpack.notification.slack.account.<account_name>.secure_url: <Slack Webhook URL>

Once Slack accounts are configured in the secure keystore, you must set a default account in elasticsearch.yml to avoid seeing the following error when you configure Watcher actions in Kibana:

Kibana Watcher UI missing Slack default account warning

The following example shows a configuration with multiple Slack accounts specified in elasticsearch.yml

xpack.notification.slack:
  default_account: account1
  account:
    account1:
      message_defaults:
        from: account1
        to: channel1
    account2:
      message_defaults:
        from: account2
        to: channel2
    account3:
      message_defaults:
        from: account3
        to: channel3

To specify a Slack account to use for a Watcher Alert that isn’t set as default_account, you must create an Advanced Watch and explicitly define which Slack account to use in the actions section.

If you have a Slack account that is not currently set as default_account, and you want to use this account for a Watcher Alert, you must create an Advanced Watch and explicitly define in the Actions section of the UI which Slack account to use.

PUT _watcher/watch/test-alarm
{
  "metadata" : {
    ...
  },
  "trigger" : {
    ...
  },
  "input" : {
    ...
  },
  "actions" : {
    "notify-slack" : {
      "throttle_period" : "10s",
      "slack" : {
        "account" : "account2",
        "message" : {
          "to" : [ "#testing-channel" ],
          "text" : "You Know, for Search"
        }
      }
    }
  }
}

In Elasticsearch versions before 7.0:, you are not required to use the Elasticsearch keystore. Instead, you can use the console Elasticsearch settings editor for your deployment.

Advanced Alerting configuration