Reindex before upgradingedit

Elasticsearch can read indices created in the previous major version. Older indices must be reindexed or deleted. Elasticsearch 6.x can use indices created in Elasticsearch 5.x, but not those created in Elasticsearch 2.x or before. Elasticsearch 5.x can use indices created in Elasticsearch 2.x, but not those created in 1.x or before.

Elasticsearch nodes will fail to start if incompatible indices are present.

To upgrade an Elasticsearch 5.x cluster that contains indices created in 2.x, you must reindex or delete them before upgrading to 6.x. For more information, see Reindex in place.

To upgrade an Elasticsearch cluster running 2.x, you have two options:

  • Perform a full cluster restart upgrade to 5.6, reindex the 2.x indices, then perform a rolling upgrade to 6.x. If your Elasticsearch 2.x cluster contains indices that were created before 2.x, you must either delete or reindex them before upgrading to 5.6. For more information about upgrading from 2.x to 5.6, see Upgrading Elasticsearch in the Elasticsearch 5.6 Reference.
  • Create a new 6.x cluster and reindex from remote to import indices directly from the 2.x cluster.

To upgrade an Elasticsearch 1.x cluster, you have two options:

Reindex in placeedit

To manually reindex your old indices with the reindex API:

  1. Create a new index and copy the mappings and settings from the old index.
  2. Set the refresh_interval to -1 and the number_of_replicas to 0 for efficient reindexing.
  3. Reindex all documents from the old index into the new index using the reindex API.
  4. Reset the refresh_interval and number_of_replicas to the values used in the old index.
  5. Wait for the index status to change to green.
  6. In a single update aliases request:

    1. Delete the old index.
    2. Add an alias with the old index name to the new index.
    3. Add any aliases that existed on the old index to the new index.

Reindex from a remote clusteredit

You can use reindex from remote to migrate indices from your old cluster to a new 6.x cluster. This enables you move to 6.x from a pre-5.6 cluster without interrupting service.

Warning

Elasticsearch provides backwards compatibility support that enables indices from the previous major version to be upgraded to the current major version. Skipping a major version means that you must resolve any backward compatibility issues yourself.

To migrate your indices:

  1. Set up a new 6.x cluster alongside your old cluster. Enable it to access your old cluster by adding your old cluster to the reindex.remote.whitelist in elasticsearch.yml:

    reindex.remote.whitelist: oldhost:9200
    Note

    The new cluster doesn’t have to start fully-scaled out. As you migrate indices and shift the load to the new cluster, you can add nodes to the new cluster and remove nodes from the old one.

  2. For each index that you need to migrate to the 6.x cluster:

    1. Create a new index in 6.x with the appropriate mappings and settings. Set the refresh_interval to -1 and set number_of_replicas to 0 for faster reindexing.
    2. Reindex from remote to pull documents from the old index into the new 6.x index:

      POST _reindex
      {
        "source": {
          "remote": {
            "host": "http://oldhost:9200",
            "username": "user",
            "password": "pass"
          },
          "index": "source",
          "query": {
            "match": {
              "test": "data"
            }
          }
        },
        "dest": {
          "index": "dest"
        }
      }

      If you run the reindex job in the background by setting wait_for_completion to false, the reindex request returns a task_id you can use to monitor progress of the reindex job with the task API: GET _tasks/TASK_ID.

    3. When the reindex job completes, set the refresh_interval and number_of_replicas to the desired values (the default settings are 30s and 1).
    4. Once replication is complete and the status of the new index is green, you can delete the old index.