Configure Index lifecycle management (ILM)
editConfigure Index lifecycle management (ILM)
editDeprecated in 7.16.0.
Users should now use the Elastic APM integration. If you’re already using the APM integration
The index lifecycle management (ILM) feature in Elasticsearch allows you to automate the lifecycle of your APM Server indices as they grow and age.
ILM policies are only applied to indices created after the policies have taken effect. In other words, any indices present before ILM was configured still need to be managed manually.
To learn how to define your own index lifecycle policies or rollover aliases, see customize index lifecycle management.
Default policy
editILM is enabled by default, and the default policy will be applied to all APM indices as long as all of the following conditions are true:
- The Elasticsearch instance supports ILM.
-
output.elasticsearch
is enabled. -
Custom
index
orindices
settings are not configured.
The default ILM policy applies only a hot phase to all APM events:
span
, transaction
, error
, and metric
.
Warm, cold, and delete phases are not defined.
-
Hot — Rollover data when the index reaches a maximum size of 50gb or a maximum age of 30 days:
max_size: 50gb
,max_age: 30d
Default configuration
editapm-server: ilm: enabled: "auto" setup: enabled: true overwrite: false require_policy: true mapping: - event_type: "error" policy_name: "apm-rollover-30-days" index_suffix: "" - event_type: "span" policy_name: "apm-rollover-30-days" index_suffix: "" - event_type: "transaction" policy_name: "apm-rollover-30-days" index_suffix: "" - event_type: "metric" policy_name: "apm-rollover-30-days" index_suffix: "" policies: - name: "apm-rollover-30-days" policy: phases: hot: actions: rollover: max_size: "50gb" max_age: "30d" set_priority: priority: 100
Configuration scenarios
editThe following table can help you determine which options to configure when enabling, setting up, or disabling ILM:
Enable ILM and Enable ILM setup. |
|
Details: Create managed indices. Default policies and templates are written to Elasticsearch. Successful if: A policy and template do not exist. Example: Starting APM Server for the first time. |
Configure these options: |
Details: Create managed indices. Customized policies and templates are written to Elasticsearch. Successful if: A policy and template already exists. Example: You’re customizing a policy that’s already been sent to Elasticsearch. |
Configure these options: |
Enable ILM and Disable ILM setup. |
|
Details: Use managed indices. No policies, rollover aliases and templates will be created by APM Server. Example: You’re creating your policies, rollover aliases and templates elsewhere, e.g., with Elasticsearch APIs. |
Configure these options: |
Disable ILM to create unmanaged default indices. |
|
Details: Disable ILM to create unmanaged indices with default names. Ensure event specific templates are set up accordingly. |
Configure these options: |
Create unmanaged custom Indices |
|
Details: Disable ILM to create unmanaged indices with custom names. Ensure event specific templates are set up accordingly. |
Configure these options: |
Configuration reference
editilm.enabled
editCan be "auto"
, "true"
, or "false"
. Defaults to "auto"
.
"auto"
:
APM Server will enable ILM only if all three of the conditions below are true.
If any of the conditions are not met, ILM will be disabled.
- The Elasticsearch instance supports ILM.
-
output.elasticsearch
is enabled. -
Custom
index
orindices
settings are not configured.
"true"
:
APM Server will ignore any configured index settings and enable ILM if both of the conditions below are true.
If any of the conditions are not met, APM Server will disable ILM, create unmanaged indices, and an error will be logged.
-
output.elasticsearch
is enabled. - The Elasticsearch instance supports ILM.
"false"
:
Explicitly disables ILM.
apm-server.ilm.setup.enabled
editDefaults to true
.
When true
, APM Server will create an ILM specific index template for each APM event type.
This is required to map ILM aliases and policies to indices.
When false
, ILM setup is disabled. No policies, templates, or aliases will be created by APM Server.
Only disable ilm.setup
if you want to set up index management on your own.
If you simply want to disable ILM, use apm-server.ilm.enabled: false
instead.
apm-server.ilm.setup.overwrite
editDefaults to false
. When false
, APM Server will not overwrite any existing policies or ILM related templates.
When first setting up ILM, your initial template and policy will be applied.
You must set this to true
when customizing your policies and template for them to be applied,
or if you want to switch between managed and unmanaged indices.
apm-server.ilm.setup.require_policy
editDefaults to true
, which means that an ILM policy must be defined in apm-server.yml
.
Changing this to false
allows you to manually set up ILM policies and templates outside of APM Server,
e.g., with Elasticsearch APIs.
APM Server will still make use of ILM and connect your template with the defined mapping.
apm-server.ilm.setup.mapping
editMaps each event type to a corresponding policy_name
and index_suffix
.
APM event types can only be error
, span
, transaction
, and metric
.
If you attempt to map an index lifecycle policy to a different event type, APM Server will not start.
If you only map a subset of APM event types, the default values will be used for omitted event types.
If a policy is defined, it must be mapped to an event type to take effect.
By default, the APM Server creates a template without a custom index suffix per event type. When defining custom index suffixes, always ensure that templates, that might have been set up previously, are removed or do not conflict. See customizing an index suffix for more information.