Exploring Nginx metrics with Elastic time series data streams

By

Lalit Satapathy

time-series-data-streams-blog-720x420-1.jpg

Elasticsearch® recently released time series data streams for metrics. This not only provides better metrics support in Elastic Observability, but it also helps reduce storage costs. We discussed this in a previous blog. 

In this blog, we dive into how to enable and use time series data streams by reviewing what a time series metrics document is and the mapping used for enabling time series. In particular, we will showcase this by using Elastic Observability’s Nginx integration. As Elastic® time series data stream (TSDS) metrics capabilities evolve, some of the scenarios below will change.

Elastic TSDS stores metrics in indices optimized for a time series database (TSDB), which is used to store time series metrics. Elastic’s TSDB also got a significant optimization in 8.7 by reducing storage costs by upward of 70%.

What is an Elastic time series data stream?

A time series data stream (TSDS) models timestamped metrics data as one or more time series. In a TSDS, each Elasticsearch document represents an observation or data point in a specific time series. Although a TSDS can contain multiple time series, a document can only belong to one time series. A time series can’t span multiple data streams.

A regular data stream can have different usages including logs. For metrics usage, however, a time series data stream is recommended. A time series data stream is different from a regular data stream in multiple ways. A TSDS contains more than one predefined dimension and multiple metrics.

Nginx metrics as an example

Integrations provide an easy way to ingest observability metrics for a large number of services and systems. We use the Nginx integration metrics data set as an example here. This is one of the integrations, on which time series has been recently enabled. 

Process of enabling TSDS on a package

Time series is enabled on a metrics data stream of an integration package, after adding the relevant time series metrics and dimension mappings. Existing integrations with metrics data streams will come with time series metrics enabled, so that users can use them as-is without any additional configuration.

The image below captures a high-level summary of a time series data stream, the corresponding index template, the time series indices and a single document. We will shortly dive into the details of each of the fields in the document.

time series data stream

TSDS metric document

Below we provide a snippet of an ingested Elastic document with time series metrics and dimension together.

{   
  "@timestamp": "2023-06-29T03:58:12.772Z",

   "nginx": {
      "stubstatus": {
        "accepts": 202,                                                                                                
        "active": 2,
        "current": 3,
        "dropped": 0,
        "handled": 202,
        "hostname": "host.docker.internal:80",                                      
        "reading": 0,
        "requests": 10217,
        "waiting": 1,                                                                       
        "writing": 1
      }
  }
}

Multiple metrics per document:
An ingested document has a collection of fields, including metrics fields. Multiple related metrics fields can be part of a single document. A document is part of a single data stream, and typically all the metrics it contains are related. All the metrics in a document are part of the same time series. 

Metric type and dimensions as mapping:
While the document contains the metrics details, the metric types and dimension details are defined as part of the field mapping. All the time series relevant field mappings are defined collectively for a given datastream, as part of the package development. All the integrations released with time series data stream, contain all the relevant time series field mappings, as part of the package release. There are two additional mappings needed in particular: time_series_metric mapping and time_series_dimension mapping.

Metrics types fields

A document contains the metric type fields (as shown above). The mappings for the metric type fields is done using time_series_metric mapping in the index templates as given below:

"nginx": {
    "properties": {
       "stubstatus": {
           "properties": {
                "accepts": {
                  "type": "long",
                  "time_series_metric": "counter"
                },
                "active": {
                  "type": "long",
                  "time_series_metric": "gauge"
                },
                "current": {
                  "type": "long",
                  "time_series_metric": "gauge"
                },
                "dropped": {
                  "type": "long",
                  "time_series_metric": "counter"
                },
                "handled": {
                  "type": "long",
                  "time_series_metric": "counter"
                },
                "reading": {
                  "type": "long",
                  "time_series_metric": "gauge"
                },
                "requests": {
                  "type": "long",
                  "time_series_metric": "counter"
                },
                "waiting": {
                  "type": "long",
                  "time_series_metric": "gauge"
                },
                "writing": {
                  "type": "long",
                  "time_series_metric": "gauge"
                }
           }
       }
    }
}

Dimension fields

Dimensions are field names and values that, in combination, identify a document’s time series. 

In Elastic time series, there are some additional considerations for dimensions:

  • Dimension fields need to be defined for each time series. There will be no time series with zero dimension fields.
  • Keyword (or similar) type fields can be defined as dimensions.
  • There is a current limit on the number of dimensions that can be defined in a data stream. The limit restrictions will likely be lifted going forward. 

Dimension is common for all the metrics in a single document, as part of a data stream. Each time series data stream of a package (example: Nginx) already comes with a predefined set of dimension fields as below.

The document would contain more than one dimension field. In the case of Nginx, agend.id and nginx.stubstatus.hostname are some of the dimension fields. The mappings for the dimension fields is done using time_series_dimension mapping as below:

"agent": {
   "properties": {
      "id": {
         "type": "keyword",
         "time_series_dimension": true
       }
    }
 },

"nginx": {
   "properties": {
       "stubstatus": {
            "properties": {
                "hostname": {
                  "type": "keyword",
                  "time_series_dimension": true
                },
            }
       }
    }
}

Meta fields

Documents ingested also have additional meta fields apart from the metric and dimension fields explained above. These additional fields provide richer query capabilities for the metrics.

Example Elastic meta fields

 "data_stream": {
      "dataset": "nginx.stubstatus",
      "namespace": "default",
      "type": "metrics"
 }

Discover and visualization in Kibana

Elastic provides comprehensive search and visualization for the time series metrics. Time series metrics can be searched as-is in Discover. In the search below, the counter and gauges metrics are captured as different icons. Below we also provide examples of visualization for the time series metrics using Lens and OOTB dashboard included as part of the Nginx integration package.

Discover search for TSDS metrics
Discover search for TSDS metrics
Maximum of counter field nginx.stubstatus.accepts visualized using Lens
Maximum of counter field nginx.stubstatus.accepts visualized using Lens
Median of gauge field nginx.stubstatus.active visualized using Lens
Median of gauge field nginx.stubstatus.active visualized using Lens
OOTB Nginx dashboard with the TSDS metrics visualizations
OOTB Nginx dashboard with the TSDS metrics visualizations

Try it out!

We have provided a detailed example of a time series document ingested by the Elastic Nginx integration. We have walked through how time series metrics are modeled in Elastic and the additional time series mappings with examples. We provided details of dimension requirements for Elastic time series, as well as brief examples of search/visualization/dashboard of TSDS metrics in Kibana®.

Don’t have an Elastic Cloud account yet? Sign up for Elastic Cloud and try out the auto-instrumentation capabilities that I discussed above. I would be interested in getting your feedback about your experience in gaining visibility into your application stack with Elastic.  

Additional resources

Sign up for Elastic Cloud free trial

Spin up a fully loaded deployment on the cloud provider you choose. As the company behind Elasticsearch, we bring our features and support to your Elastic clusters in the cloud.

Start free trial