Elasticsearch queryedit

The Elasticsearch query rule type runs a user-configured Elasticsearch query, compares the number of matches to a configured threshold, and schedules actions to run when the threshold condition is met.

Create the ruleedit

Fill in the rule details, then select Elasticsearch query.

Define the conditionsedit

Define properties to detect the condition.

Five clauses define the condition to detect
Index
This clause requires an index or index pattern and a time field that will be used for the time window.
Size
This clause specifies the number of documents to pass to the configured actions when the the threshold condition is met.
Elasticsearch query
This clause specifies the ES DSL query to execute. The number of documents that match this query will be evaulated against the threshold condition. Aggregations are not supported at this time.
Threshold
This clause defines a threshold value and a comparison operator (is above, is above or equals, is below, is below or equals, or is between). The number of documents that match the specified query is compared to this threshold.
Time window
This clause determines how far back to search for documents, using the time field set in the index clause. Generally this value should be set to a value higher than the check every value in the general rule details, to avoid gaps in detection.

Add action variablesedit

Add an action to run when the rule condition is met. The following variables are specific to the Elasticsearch query rule. You can also specify variables common to all rules.

context.title
A preconstructed title for the rule. Example: rule term match alert query matched.
context.message
A preconstructed message for the rule. Example:
rule 'term match alert' is active:
- Value: 42
- Conditions Met: count greater than 4 over 5m
- Timestamp: 2020-01-01T00:00:00.000Z
context.group
The name of the action group associated with the condition. Example: query matched.
context.date
The date, in ISO format, that the rule met the condition. Example: 2020-01-01T00:00:00.000Z.
context.value
The value of the rule that met the condition.
context.conditions
A description of the condition. Example: count greater than 4.
context.hits

The most recent ES documents that matched the query. Using the Mustache template array syntax, you can iterate over these hits to get values from the ES documents into your actions.

Iterate over hits using Mustache template syntax

Test your queryedit

Use the Test query feature to verify that your query DSL is valid.

  • Valid queries are executed against the configured index using the configured time window. The number of documents that match the query will be displayed.

    Test Elasticsearch query returns number of matches when valid
  • An error message is shown if the query is invalid.

    Test Elasticsearch query shows error when invalid

Match de-duplicationedit

The Elasticsearch query rule type performs de-duplication of document matches across rule executions. If you configure the rule with a schedule interval smaller than the time window, and a document matches a query in multiple rule executions, it will be alerted on only once.

Suppose you have a rule configured to run every minute. The rule uses a time window of 1 hour and checks if there are more than 99 matches for the query. The Elasticsearch query rule type will do the following:

Execution 1 (0:00)

Rule finds 113 matches in the last hour: 113 > 99

Rule is active and user will be alerted.

Execution 2 (0:01)

Rule finds 127 matches in the last hour. 105 of the matches are duplicates that were alerted on in Execution 1, so you actually have 22 matches: 22 !> 99

No alert.

Execution 3 (0:02)

Rule finds 159 matches in the last hour. 88 of the matches are duplicates that were alerted on in Execution 1, so you actually have 71 matches: 71 !> 99

No alert.

Execution 4 (0:03)

Rule finds 190 matches in the last hour. 71 of them are duplicates that were alerted on in Exeuction 1, so you actually have 119 matches: 119 > 99

Rule is active and user will be alerted.