Migrate to index lifecycle management | Elastic Cloud Enterprise Reference [2.3]

It is time to say goodbye: This version of Elastic Cloud Enterprise has reached end-of-life (EOL) and is no longer supported.

The documentation for this version is no longer being maintained. If you are running this version, we strongly advise you to upgrade. For the latest information, see the current release documentation.

› › ›

« Advanced cluster configuration Stop routing requests or pause nodes »

Migrate to index lifecycle managementedit

These instructions are provided as is. They have been tested on a development system, but might require additional assistance from Elastic, if they do not work for you. Elastic Cloud Enterprise 2.5.0 provides a wizard for this.

The index lifecycle management (ILM) feature of the Elastic Stack provides an integrated and streamlined way to manage time-based data, making it easier to follow best practices for managing your indices. Compared to index curation, migrating to ILM gives you more fine-grained control over the lifecycle of each index.

For existing hot-warm deployments that are currently using index curation, there are a couple of options for migrating to index lifecycle management (ILM). You can:

Restore your data from a snapshot to a new Elastic Stack 6.7+ deployment that has ILM enabled.
Manually make the configuration changes described in this section to migrate an existing deployment to ILM.

To learn more about configuring index lifecycle management for Elastic Cloud Enterprise or about all of the features that are available with ILM, see:

Before you beginedit

Making configuration changes manually requires that you edit the JSON that contains the cluster configuration plan for your deployment. Manually editing your deployment configuration plan can result in a boot-looping deployment that requires additional troubleshooting. If in doubt, consider restoring your data from a snapshot to a new 6.7+ deployment that has ILM enabled instead.

You can break things when editing the deployment configuration plan. Use this functionality only if you know what you are doing or if you are being directed by someone from Elastic.

Stepsedit

Log into the Cloud UI.
From the Deployments page, select your deployment.

Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters.
Temporarily disable shard allocation to prevent shards from moving back from warm nodes to hot nodes while you perform the subsequent steps:
1. From your deployment menu, go to Elasticsearch > API Console.
2. Update the Elasticsearch cluster settings to disable shard allocation:
```
PUT _cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.enable": "none"
  }
}
```
From your deployment menu, go to the Edit page.
At the bottom of the page, click to use the advanced Elasticsearch configuration editor.

In the Data section, turn off index curation by replacing the "curation" section with the following JSON snippet:

  "curation": {
    "enabled": false,
    "interval": "5 min",
    "to_instance_configuration_id": "",
    "from_instance_configuration_id": "",
    "specs": []
  },

Click Save.

In the Plan section, remove the entire "curation" section from the plan section. For example:

"curation": {
   "from_instance_configuration_id": "data.default",
   "to_instance_configuration_id": "data.highstorage"
}

For each data node in the cluster topology, find the "node_attributes": {} section and add the node attributes that will allow you to configure ILM in Kibana later, such as "data": "hot" and "data": "warm". For example:
```
"node_attributes": {
  "data": "hot"
 }
```
For the default hot-warm deployment template, you can identify the data node sections that you need to update by the instance configuration IDs data.default (hot nodes) and data.highstorage (warm nodes). Custom instance configurations that an ECE administrator created are currently identified by a UUID string.
Click Save changes to apply the new configuration settings.
After the configuration change is complete, edit your Kibana user settings and add xpack.ilm.enabled: true.
In Kibana, create your index lifecycle policies.
Reenable shard allocation:
1. From your deployment menu, go to Elasticsearch > API Console.
2. Update the Elasticsearch cluster settings:
```
PUT _cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.enable": "all"
  }
}
```

« Advanced cluster configuration Stop routing requests or pause nodes »