Migrate index allocation filters to node roles

edit

Migrate index allocation filters to node rolesedit

If you currently use custom node attributes and attribute-based allocation filters to move indices through data tiers in a hot-warm-cold architecture, we recommend that you switch to using the built-in node roles and automatic data tier allocation. Using node roles enables ILM to automatically move indices between data tiers.

While we recommend relying on automatic data tier allocation to manage your data in a hot-warm-cold architecture, you can still use attribute-based allocation filters to control shard allocation for other purposes.

Elasticsearch Service and Elastic Cloud Enterprise can perform the migration automatically. For self-managed deployments, you need to manually update your configuration, ILM policies, and indices to switch to node roles.

Automatically migrate to node roles on Elasticsearch Service or Elastic Cloud Enterpriseedit

If you are using node attributes from the default deployment template in Elasticsearch Service or Elastic Cloud Enterprise, you will be prompted to switch to node roles when you:

  • Upgrade to Elasticsearch 7.10 or higher
  • Deploy a warm, cold, or frozen data tier
  • Enable autoscaling

These actions automatically update your cluster configuration and ILM policies to use node roles. Additionally, upgrading to version 7.14 or higher automatically update ILM policies whenever any configuration change is applied to your deployment.

If you use custom index templates, check them after the automatic migration completes and remove any attribute-based allocation filters.

You do not need to take any further action after the automatic migration. The following manual steps are only necessary if you do not allow the automatic migration or have a self-managed deployment.

Migrate to node roles on self-managed deploymentsedit

To switch to using node roles:

  1. Assign data nodes to the appropriate data tier.
  2. Remove the attribute-based allocation settings from your index lifecycle management policy.
  3. Stop setting the custom hot attribute on new indices.
  4. Update existing indices to set a tier preference.

Assign data nodes to a data tieredit

Configure the appropriate roles for each data node to assign it to one or more data tiers: data_hot, data_content, data_warm, data_cold, or data_frozen. A node can also have other roles. By default, new nodes are configured with all roles.

When you add a data tier to an Elasticsearch Service deployment, one or more nodes are automatically configured with the corresponding role. To explicitly change the role of a node in an Elasticsearch Service deployment, use the Update deployment API. Replace the node’s node_type configuration with the appropriate node_roles. For example, the following configuration adds the node to the hot and content tiers, and enables it to act as an ingest node, remote, and transform node.

"node_roles": [
  "data_hot",
  "data_content",
  "ingest",
  "remote_cluster_client",
  "transform"
],

If you are directly managing your own cluster, configure the appropriate roles for each node in elasticsearch.yml. For example, the following setting configures a node to be a data-only node in the hot and content tiers.

node.roles [ data_hot, data_content ]

Remove custom allocation settings from existing ILM policiesedit

Update the allocate action for each lifecycle phase to remove the attribute-based allocation settings. ILM will inject a migrate action into each phase to automatically transition the indices through the data tiers.

If the allocate action does not set the number of replicas, remove the allocate action entirely. (An empty allocate action is invalid.)

The policy must specify the corresponding phase for each data tier in your architecture. Each phase must be present so ILM can inject the migrate action to move indices through the data tiers. If you don’t need to perform any other actions, the phase can be empty. For example, if you enable the warm and cold data tiers for a deployment, your policy must include the hot, warm, and cold phases.

Stop setting the custom hot attribute on new indicesedit

When you create a data stream, its first backing index is now automatically assigned to data_hot nodes. Similarly, when you directly create an index, it is automatically assigned to data_content nodes.

On Elasticsearch Service deployments, remove the cloud-hot-warm-allocation-0 index template that set the hot shard allocation attribute on all indices.

response = client.indices.delete_template(
  name: '.cloud-hot-warm-allocation-0'
)
puts response
DELETE _template/.cloud-hot-warm-allocation-0

If you’re using a custom index template, update it to remove the attribute-based allocation filters you used to assign new indices to the hot tier.

To completely avoid the issues that raise when mixing the tier preference and custom attribute routing setting we also recommend updating all the legacy, composable, and component templates to remove the attribute-based allocation filters from the settings they configure.

Set a tier preference for existing indicesedit

ILM automatically transitions managed indices through the available data tiers by automatically injecting a migrate action into each phase.

To enable ILM to move an existing managed index through the data tiers, update the index settings to:

  1. Remove the custom allocation filter by setting it to null.
  2. Set the tier preference.

For example, if your old template set the data attribute to hot to allocate shards to the hot tier, set the data attribute to null and set the _tier_preference to data_hot.

response = client.indices.put_settings(
  index: 'my-index',
  body: {
    "index.routing.allocation.require.data": nil,
    "index.routing.allocation.include._tier_preference": 'data_hot'
  }
)
puts response
PUT my-index/_settings
{
  "index.routing.allocation.require.data": null,
  "index.routing.allocation.include._tier_preference": "data_hot"
}

For indices that have already transitioned out of the hot phase, the tier preference should include the appropriate fallback tiers to ensure index shards can be allocated if the preferred tier is unavailable. For example, specify the hot tier as the fallback for indices already in the warm phase.

response = client.indices.put_settings(
  index: 'my-index',
  body: {
    "index.routing.allocation.require.data": nil,
    "index.routing.allocation.include._tier_preference": 'data_warm,data_hot'
  }
)
puts response
PUT my-index/_settings
{
  "index.routing.allocation.require.data": null,
  "index.routing.allocation.include._tier_preference": "data_warm,data_hot"
}

If an index is already in the cold phase, include the cold, warm, and hot tiers.

For indices that have both the _tier_preference and require.data configured but the _tier_preference is outdated (ie. the node attribute configuration is "colder" than the configured _tier_preference), the migration needs to remove the require.data attribute and update the _tier_preference to reflect the correct tiering.

eg. For an index with the following routing configuration:

{
  "index.routing.allocation.require.data": "warm",
  "index.routing.allocation.include._tier_preference": "data_hot"
}

The routing configuration should be fixed like so:

response = client.indices.put_settings(
  index: 'my-index',
  body: {
    "index.routing.allocation.require.data": nil,
    "index.routing.allocation.include._tier_preference": 'data_warm,data_hot'
  }
)
puts response
PUT my-index/_settings
{
  "index.routing.allocation.require.data": null,
  "index.routing.allocation.include._tier_preference": "data_warm,data_hot"
}

This situation can occur in a system that defaults to data tiers when, e.g., an ILM policy that uses node attributes is restored and transitions the managed indices from the hot phase into the warm phase. In this case the node attribute configuration indicates the correct tier where the index should be allocated.