Up to 70% metrics storage savings with TSDS enabled integrations in Elastic Observability


The latest versions of Elastic Observability’s most popular observability integrations now use the storage cost-efficient time series index mode for metrics by default. Kubernetes, Nginx, System, AWS, Azure, RabbitMQ, Redis, and more popular Elastic Observability integrations are time series data stream (TSDS) enabled integrations.

time series

Elastic® released the time series data stream functionality to GA in 8.7. Elastic time series data stream (TSDS) stores metrics in indices optimized for a time series database (TSDB), which is optimized to store time series metrics. Elastic’s also got a significant optimization in 8.7 with additional features, reducing storage costs for metrics in the region of 70%. Read how TSDS works and how we use it in Elastic Observability, as well as how you can use it for your own metrics.

The three features for TSDB that were made GA in 8.7 are:

  • Downsampling reduces the footprint of time series by storing them at lower granularity, giving you greater control in deciding how much historical metric data can be stored within your budget. And since downsampling “just works” in Kibana® with 8.7, it also speeds up visualizations using Lens, Timelion, or TSVB on large data sets.
  • Time series data streams (TSDS) leverage specific organization of time series data (e.g., by timestamp and dimension fields) to sort and store metric data more efficiently — 30% less disk space in our benchmarks!
  • Synthetic source reduces storage space by not saving the original document data in _source, and instead reconstructing it from doc_values when needed. We observed space saving of 40%, depending on index configuration.

By leveraging Elasticsearch's Time Series Data Stream (TSDS) for our integrations, Elastic Observability helps realize storage savings for your key metrics data being ingested.

At the time of the release of 8.9, upwards to 40% of the Elastic integrations have a version released to use time series index mode out of the box. These include but are not limited to: Kubernetes, Nginx, System, AWS, Kinesis, Lambda, and most of the integrations that collect large numbers of metrics.

index mode

When using the TSDS enabled version for an integration collecting metrics, the benefits that you can realize are the following:

  • Up to 70% less disk space: With TSDS seamlessly enabled in the integration for your platforms, you can enjoy up to a significant 70% reduction in disk storage for your metrics data. This reduction translates to cost savings and increased scalability for your infrastructure. 
  • Streamlined data management: TSDS simplifies the storage and retrieval of your time stamped metrics data, making it effortless to organize and analyze your valuable insights. 
  • Out-of-the-box functionality: Thanks to the native integration of TSDS, you can leverage its powerful features without any user input or additional configuration. The integration seamlessly optimizes storage efficiency for your metrics data, allowing you to focus on deriving meaningful insights.

This process of enabling integrations to use time series index mode will continue, and these will be released outside of the Elastic release cycle, essentially updated when ready. This will apply to Elastic Cloud, as any on-prem or self-managed cloud deployments will have to wait for the next release to upgrade.

Efficient storage of metrics in the region of 70% savings

When you use a time_series index mode enabled Elastic integration, your metrics data will be stored efficiently without need for you to manage your storage configuration and out of the box reduce your disk space requirements for storing metrics up to 70%.

Index size comparison
Index size comparison for metrics stored using standard (30.4GB) vs time series (5.9GB) mode

When you add a document to a TSDS, Elasticsearch adds the document to the appropriate backing index based on its @timestamp value. As a result, a TSDS can add documents to any TSDS backing index that can receive writes. This applies even if the index isn’t the most recent backing index.


Applications are scaling and data is growing at an exponential rate, along with the cost to store it. Most organizations need to make a difficult decision of what data to keep or delete to stay under budget. With Elastic’s optimizations, your usage of cloud storage (such as S3) will be lowered and reduce the need to move data to “cold” storage. 

This will naturally allow you to scale metrics storage for more longer-term metrics data, which will help analyze patterns (long-term analysis), reduce MTTx, and improve your application’s overall performance. Read more details on how TSDS works

Additional end-user benefits

In addition to the substantial storage costs savings, time series enabled integrations bring a new approach to metrics data storage, offering several distinctive advantages over regular data streams:

  • Efficient indexing: TSDS optimizes indexing and storage by utilizing dimension-based routing, internal index sorting, and time-bound backing indices. This intelligent organization allows for superior compression, faster retrieval, and improved performance.
  • Dimensions and metrics: TSDS introduces the concept of dimensions and metrics, allowing for more precise data organization. Dimensions capture specific aspects of the entity being measured, while metrics provide numeric measurements and aggregations over time.
  • Purpose-built for metrics: TSDS is designed exclusively for metrics data, enabling it to deliver exceptional efficiency and performance for your time series data storage needs.
  • Seamless integration: TSDS is seamlessly enabled into the latest integration version of popular platforms like Kubernetes, Nginx, System, AWS, Azure, and more, eliminating the need for manual setup and configuration.
  • Performance enhancements: Date histograms, range queries, term aggregations, and more are enhanced by upwards of a 15% enhancement.

How to use this in Kibana

To determine if a time series enabled version of an Elastic Agent metrics integration is available, use the integrations documentation to locate the integration, then scroll down to the changelog in its description page. 

If a time series enabled version has been released, there will be a release note like shown in the example below.


All you have to do is upgrade the integration version (with the upgrade integration policies selected) to the time series enabled version. This will unlock the time series indexing mode going forward!


The latest version of the following integrations is now TSDB enabled:

  • ActiveMQ
  • Apache
  • AWS
    • Billing, Dynamod, EBS, ECS, ELB, Firewall, Kinesis, Lambda, Nat Gateway, RDS, Redshift, S3, SNS, SQS, Transit Gateway, Usage, VPN, S3 Storage Lens
  • Couchdb
  • Docker
  • Elasticsearch
  • IBM MQ
  • IIS
  • Influxdb
  • Kafka
  • Kubernetes
  • Microsoft SQL server
  • Nginx
  • Oracle
  • RabbitMQ
  • Redis
  • System
  • Vsphere

Get started today and experience the benefits of storing your metrics in time series mode in the most popular and powerful store for logs!

For more insights, including a benchmark comparison for storing metrics using time series versus standard data stream, visit this article.


With the release of Elastic 8.9, we have started to provide time series index mode enabled Elastic integrations for storing metrics. More and more integrations will get time series enabled over time, but they will not be tied to Elastic releases. Currently available TSDS ready integrations include Kubernetes, Nginx, System, AWS, Kinesis, Lambda, and more. The following benefits can be realized:

  • Storage reduction in the region of 70%
  • Efficient indexing
  • More dimensions and metrics scale
  • Performance enhancements

Additional resources on TSDS for Elastic Observability:

Don’t have an Elastic Cloud account yet? Sign up for Elastic Cloud and try out the auto-instrumentation capabilities that I discussed above.  

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.