Migrate the cross-cluster search deployment templateedit

The cross-cluster search deployment template is now deprecated and has been removed from the Elasticsearch Service user console. You no longer need to use the dedicated cross-cluster template to search across deployments. Instead, you can now use any template to configure remote clusters and search across them when using stack version 6.7 or above. Existing deployments created using this template are not affected, but they are required to migrate to another template before upgrading to the next 8.0 major version.

There are two different approaches to do this migration:

Use the API to migrate deployments that use the cross-cluster search templateedit

You can use a PUT request to update your deployment, changing both the deployment template ID and the instances required by the new template.

  1. First, choose the new template you want to use and obtain its ID. This template ID can be obtained from the Elasticsearch Service Console Create Deployment page by clicking Equivalent API request and inspecting the result for the field deployment_template. For example, we are going to use the "IO Optimized" deployment template, and in our GCP region the id is gcp-io-optimized.

You can also find the template in the list of templates available for each region.

+

Deployment Template ID

+

  1. Make a request to update your deployment with two changes:

    1. Under deployment_template, set id to the value obtained in the previous step.
    2. Change the cluster topology to match the new template that your deployment will migrate to.
curl -H 'Content-Type: application/json' -X PUT -H "Authorization: ApiKey $EC_API_KEY" https://api.elastic-cloud.com/api/v1/deployments/$DEPLOYMENT_ID -d "{
    "prune_orphans": false,
    "resources": {
        "elasticsearch": [
            {
                "region": "$REGION",
                "settings": {
                    "dedicated_masters_threshold": 6
                },
                "plan": {
                    "cluster_topology": [
                        {
                            "zone_count": 2,
                            "node_roles" : [
                              "master",
                              "ingest",
                              "remote_cluster_client",
                              "data_hot",
                              "transform",
                              "data_content"
                            ],
                            "elasticsearch": {
                                "enabled_built_in_plugins": [],
                                "node_attributes": {
                                    "data": "hot"
                                }
                            },
                            "autoscaling_max": {
                              "resource": "memory",
                              "value": 131072
                            },
                            "instance_configuration_id": "gcp.data.highio.1",
                            "id": "hot_content",
                            "size": {
                                "resource": "memory",
                                "value": 8192
                            }
                        },
                        {
                            "zone_count": 1,
                            "node_roles" : [
                              "ml",
                              "remote_cluster_client"
                            ],
                            "elasticsearch": {
                                "enabled_built_in_plugins": []
                            },
                            "autoscaling_max": {
                              "resource": "memory",
                              "value": 65536
                            },
                            "instance_configuration_id": "gcp.ml.1",
                            "id": "ml",
                            "size": {
                                "resource": "memory",
                                "value": 0
                            }
                        },
                        {
                            "zone_count": 2,
                            "elasticsearch": {
                                "enabled_built_in_plugins": []
                            },
                            "instance_configuration_id": "gcp.coordinating.1",
                            "node_roles": [
                                "ingest",
                                "remote_cluster_client"
                            ],
                            "id": "coordinating",
                            "size": {
                                "resource": "memory",
                                "value": 0
                            }
                        },
                        {
                            "zone_count": 2,
                            "elasticsearch": {
                                "enabled_built_in_plugins": [],
                                "node_attributes": {
                                    "data": "warm"
                                }
                            },
                            "autoscaling_max": {
                                "resource": "memory",
                                "value": 131072
                            },
                            "instance_configuration_id": "gcp.data.highstorage.1",
                            "node_roles": [
                                "data_warm",
                                "remote_cluster_client"
                            ],
                            "id": "warm",
                            "size": {
                                "resource": "memory",
                                "value": 0
                            }
                        },
                        {
                            "zone_count": 1,
                            "elasticsearch": {
                                "enabled_built_in_plugins": [],
                                "node_attributes": {
                                    "data": "cold"
                                }
                            },
                            "autoscaling_max": {
                                "resource": "memory",
                                "value": 65536
                            },
                            "instance_configuration_id": "gcp.data.highstorage.1",
                            "node_roles": [
                                "data_cold",
                                "remote_cluster_client"
                            ],
                            "id": "cold",
                            "size": {
                                "resource": "memory",
                                "value": 0
                            }
                        },
                        {
                            "zone_count": 3,
                            "elasticsearch": {
                                "enabled_built_in_plugins": []
                            },
                            "instance_configuration_id": "gcp.master.1",
                            "node_roles": [
                                "master"
                            ],
                            "id": "master",
                            "size": {
                                "resource": "memory",
                                "value": 0
                            }
                        }
                    ],
                    "elasticsearch": {
                        "version": "7.11.1"
                    },
                    "deployment_template": {
                        "id": "gcp-io-optimized"
                    }
                },
                "ref_id": "main-elasticsearch"
            }
        ],
        "enterprise_search": [
            {
                "elasticsearch_cluster_ref_id": "main-elasticsearch",
                "region": "$REGION",
                "plan": {
                    "cluster_topology": [
                        {
                            "node_type": {
                                "connector": true,
                                "appserver": true,
                                "worker": true
                            },
                            "instance_configuration_id": "gcp.enterprisesearch.1",
                            "zone_count": 2,
                            "size": {
                                "resource": "memory",
                                "value": 0
                            }
                        }
                    ],
                    "enterprise_search": {}
                },
                "ref_id": "enterprise_search-ref-id"
            }
        ],
        "kibana": [
            {
                "elasticsearch_cluster_ref_id": "main-elasticsearch",
                "region": "$REGION",
                "plan": {
                    "cluster_topology": [
                        {
                            "instance_configuration_id": "gcp.kibana.1",
                            "zone_count": 1,
                            "size": {
                                "resource": "memory",
                                "value": 1024
                            }
                        }
                    ],
                    "kibana": {
                        "version": "7.11.1"
                    }
                },
                "ref_id": "main-kibana"
            }
        ],
        "apm": [
            {
                "elasticsearch_cluster_ref_id": "main-elasticsearch",
                "region": "$REGION",
                "plan": {
                    "cluster_topology": [
                        {
                            "instance_configuration_id": "gcp.apm.1",
                            "zone_count": 1,
                            "size": {
                                "resource": "memory",
                                "value": 512
                            }
                        }
                    ],
                    "apm": {}
                },
                "ref_id": "apm-ref-id"
            }
        ]
    }
}"
DEPLOYMENT_ID
The ID of your deployment, as shown in the Cloud UI or obtained through the API.
REGION
The region of your deployment, as shown in the Cloud UI or obtained through the API.

Note that the ref_id and version numbers for your resources may not be the same as shown in this example. Make sure to use the ones your deployment has.

Use a snapshot to migrate deployments that use the cross-cluster search deployment templateedit

You can make this change in the user Elasticsearch Service Console. The only drawback of this method is that it changes the URL used to access the Elasticsearch cluster and Kibana.

  1. From the deployment menu, open the Snapshots page and click Take Snapshot now. Wait for the snapshot to finish.
  2. From the main Deployments page, click Create deployment and select any available template. Expand Deployment settings to choose a Snapshot source, and then select your deployment and the snapshot that you created.

    Create a Deployment using a snapshot