The NGINX OpenTelemetry Assets integration ships six alerting rule templates. Each one is an ES|QL query with a threshold already tuned. Install the integration, create a rule from one of the templates, and adjust the threshold to match your traffic. You get working alerts in minutes instead of writing them from scratch. This walkthrough covers the full setup, threshold tuning, and how to use an idle data streams rule to catch a service that stops sending data.
Prerequisites for Elastic integration alerting rule templates
Elastic Stack 9.4.0 or later.
Alerting rule templates have been available since 9.2.1, under the integration Assets tab. This article covers three things that need 9.4.0: the dedicated Alerting tab, idle data streams rules, and the NGINX OpenTelemetry Assets package, which is in technical preview.
Step 1: Send NGINX logs and metrics to Elasticsearch with OpenTelemetry
First, get NGINX metrics and logs into Elasticsearch.
Enable the NGINX stub_status module and make the access and error logs readable by the collector. Then, configure an EDOT or upstream OpenTelemetry Collector with the nginx and filelog receivers to export metrics and logs to Elasticsearch.
The integration setup has the full receiver and pipeline configuration.
If you want to reproduce this example, you can use the companion repository.
Both signals matter for alerting, and each group of templates reads a different data stream:
- The log-based templates (4xx and 5xx error rates, error log spike) query
logs-nginx.access.otel-*andlogs-nginx.error.otel-*, which come from thefilelogreceiver. - The metric-based templates (active connections, dropped connections) query
metrics-nginxreceiver.otel-*, which comes from thenginxreceiver.
This is easy to get wrong: the Fleet Nginx (OpenTelemetry) input package collects stub_status metrics only. Its companion for logs is the classic Nginx integration, which writes ECS-based nginx.access and nginx.error datasets, not the *.otel-* data streams the log-based templates query. If you rely on that pairing alone, the log-based rules have nothing to evaluate and silently never fire. Run the filelog receiver too, not just the nginx receiver.
Step 2: Install the NGINX OpenTelemetry Assets integration
NGINX OpenTelemetry Assets is a content-only package. It ships the dashboards, alerting rule templates, and SLO templates, but it does not collect data itself. The data comes from the collector you set up in Step 1.
You don't need to install it by hand. Once the NGINX OTel data from Step 1 starts arriving, Elastic detects it and installs the Assets package for you, which takes a minute or two. Confirm it under Management > Integrations > Installed integrations, where NGINX OpenTelemetry Assets should appear.
Step 3: Create Elasticsearch alert rules from a rule template
Open the integration and select the Alerting tab.
This package ships six templates: high 4xx and 5xx error rates, high active connections, an error log spike, dropped connections, and a generic High error rate by service template that points at a placeholder logs-myservicereceiver.otel-* index for you to repoint and rename. The five NGINX rules run ES|QL every minute and group results by host.name, so an alert points at the host with the problem. The generic template groups by service.name instead, since it is meant to be repointed at whichever service you choose.
Select a template, for example [Nginx OTel] High 5xx error rate. Kibana opens a prefilled Create rule form built on an Elasticsearch query rule. It runs the template's ES|QL on a schedule and alerts when the query returns rows.
The query looks like this:
FROM logs-nginx.access.otel-*
// Flag each access log entry as a server error (5xx) or not
| EVAL is_5xx = CASE(http.response.status_code >= 500, 1, 0)
// Aggregate total requests and 5xx count per NGINX host
| STATS total = COUNT(*), errors_5xx = SUM(is_5xx) BY host.name
// Minimum sample size to avoid noisy low-traffic hosts
| WHERE total > 50
// Calculate 5xx error rate as a percentage
| EVAL error_rate_pct = ROUND(TO_DOUBLE(errors_5xx) / TO_DOUBLE(total) * 100.0, 2)
// Alert threshold: adjust to tune sensitivity
| WHERE error_rate_pct > 5.0
| SORT error_rate_pct DESC
| LIMIT 10
It counts requests and 5xx responses per host, keeps hosts with enough traffic to matter, and returns those above five percent.
Three things to get right while the form is open:
- Send data first. ES|QL validates column names against the indices that exist when the query runs. Open a template before any NGINX data has been ingested and the editor reports
Unknown column "http.response.status_code"and the form shows errors. Once data is flowing (Step 1), the same query validates and the error clears, so collect data before you create the rule. - Set the time field to
@timestamp. - Leave "Create an alert for each row" selected. Because the query groups by
host.name, this makes every affected host raise its own alert.
Add a connector and an action so the alert reaches Slack, email, or PagerDuty, then save and enable the rule.
Step 4: Tune alerting rule template thresholds in ES|QL
The thresholds are starting points, so confirm them against your own traffic. The threshold lives in the ES|QL WHERE clause.
To make the rule stricter, change error_rate_pct > 5.0 to error_rate_pct > 2.0. To require more traffic before it fires, raise total > 50. Use Test query in the rule form to confirm the edited query parses and returns rows before you save.
Three more settings are worth a look:
- Time window: the look-back period the query runs over. A shorter window reacts faster but is noisier on bursty traffic.
- Rule schedule: how often the query runs, every minute by default.
- Alert delay: the number of consecutive runs the condition must hold before an alert is created, which filters out single-run blips.
How do you detect idle data streams in Elasticsearch?
Threshold rules only fire while data keeps arriving. When an agent goes offline or an output breaks, the data stops, and a threshold rule has nothing to evaluate.
Many Elastic integrations include a dynamically generated idle data streams template for exactly this case. It is named [{Integration name}] Idle data streams and appears in the same Alerting tab, though it is generated automatically rather than bundled with the integration. It alerts when no data is written to any of the integration's data stream patterns within a set period.
The NGINX OpenTelemetry packages do not include an idle data streams template, which is why no such template appears in the Alerting tab from Step 3. The end of this section covers what to do instead.
The default period is 24 hours, which is usually too long. A production service can go quiet for most of a day before you hear about it.
When you create the rule, drop the period to match how fast you need to know. Fifteen minutes to one hour works for a critical service. For a batch job, set a period comfortably longer than its run interval, so the quiet gaps between runs do not trigger it.
So why is this example left out? The template is generated from the data stream patterns an integration defines, and it is not generated for input-only packages. A content-only package like NGINX OpenTelemetry Assets defines no data streams of its own either. To catch silence in a collector-based setup like this, recreate the rule by hand with an Elasticsearch query rule. Use the query DSL or KQL variant rather than ES|QL, because an ES|QL rule fires on returned rows and so cannot alert on the absence of data. Point it at the OTel data streams (logs-nginx.access.otel-*, or metrics-nginxreceiver.otel-*) and set the condition to fire when the number of matching documents is below 1 over a window of, say, the last 15 minutes. That reproduces what an idle data streams template does, scoped to the data streams your collector writes.
Get started with Elastic integration alerting rule templates
Alerting rule templates turn alert setup into a few steps: send data, install the integration, create a rule from a template, and adjust the threshold. Treat the bundled thresholds as defaults to confirm, not numbers to trust blindly. And where an idle data streams template is available, reduce its 24-hour default so you find out quickly when a service goes silent.
Resources
- Companion repository, to generate the NGINX demo data used here
- Alerting rule templates
- NGINX OpenTelemetry Assets integration
- End-to-end tracing for NGINX with OpenTelemetry
- Elasticsearch query rule
- Elastic Agent built-in alerts, for monitoring the agents themselves