Reduce storage

The amount of storage for APM data depends on several factors: the number of services you are instrumenting, how much traffic the services see, agent and server settings, and the length of time you store your data.

The transaction sample rate directly influences the number of documents (more precisely, spans) to be indexed. It is the easiest way to reduce storage.

The transaction sample rate is a configuration setting of each agent. Reducing it does not affect the collection of metrics such as Transactions per second.

Elastic APM agents collect stacktrace information under certain circumstances. This can be very helpful in identifying issues in your code, but it also comes with an overhead at collection time and increases the storage usage.

Stacktrace collection settings are managed in each agent.

You might want to only keep data for a defined time period. This might mean deleting old documents periodically, deleting data collected for specific services or customers, or deleting specific indices.

Depending on your use case, you can delete data periodically with index lifecycle management, Curator, the Delete By Query API, or in the Kibana Index Management UI.

Index Lifecycle management (ILM) enables you to automate how you want to manage your indices over time. You can base actions on factors such as shard size and performance requirements. See Customize index lifecycle management to learn more.

To delete data periodically you can use Curator and set up a cron job to run it.

By default, APM indices have the pattern apm-%{[observer.version]}-{type}-%{+yyyy.MM.dd}. With the curator command line interface you can, for instance, see all your existing indices:

curator_cli --host localhost show_indices --filter_list '[{"filtertype":"pattern","kind":"prefix","value":"apm-"}]'

apm-7.17.24-error-2019.10.20
apm-7.17.24-error-2019.10.21
apm-7.17.24-error-2019.10.22
apm-7.17.24-sourcemap
apm-7.17.24-span-2019.10.20
apm-7.17.24-span-2019.10.21
apm-7.17.24-span-2019.10.22
apm-7.17.24-transaction-2019.10.20
apm-7.17.24-transaction-2019.10.21
apm-7.17.24-transaction-2019.10.22

And then delete any span indices older than 1 day:

curator_cli --host localhost delete_indices --filter_list '[{"filtertype":"pattern","kind":"prefix","value":"apm-7.17.24-span-"}, {"filtertype":"age","source":"name","timestring":"%Y.%m.%d","unit":"days","unit_count":1,"direction":"older"}]'

INFO      Deleting selected indices: [apm-7.17.24-span-2019.10.20, apm-7.17.24-span-2019.10.21]
INFO      ---deleting index apm-7.17.24-span-2019.10.20
INFO      ---deleting index apm-7.17.24-span-2019.10.21
INFO      "delete_indices" action completed.

You can delete documents matching a specific query. For example, all documents with a given context.service.name use the following request:

POST /apm-*/_delete_by_query
{
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "context.service.name": {
              "value": "old-service-name"
            }
          }
        }
      ]
    }
  }
}

See delete by query for further information on this topic.

In addition to trace event data, there are other APM indices that you may wish to delete data from.

APM custom links are stored in the .apm-custom-link index. To delete all custom links, run:

POST /.apm-custom-link/_delete_by_query
{
  "query": {
    "match_all": {}
  }
}

APM agent central configuration data is stored in the .apm-agent-configuration index. Follow these steps to delete all central agent configurations.

Select the indices you want to delete, and click Manage indices to see the available actions. Then click delete indices.