Upgrade Versions

When changing the version of an existing cluster, either a major or a minor upgrade is performed. The difference is that a minor upgrade takes you from 2.2 to 2.3, for example, and requires no downtime as a rolling upgrade is performed. A major upgrade takes you from from 2.3 to 5.0, for example, and requires a full cluster restart as part of the upgrade process.

Bug fix releases also require no downtime when upgrading. A bug fix release takes you from 2.3.4 to 2.3.5, for example.

To upgrade to a later Elasticsearch version:

  1. Sign in to the Elastic Cloud Console.
  2. Click Configuration for an existing cluster in the sidebar.
  3. In the Version section, select a new version.

    If you are performing a major version upgrade, the UI will offer you a link to our migration tool that can help you determine if a direct upgrade is feasible. You might also want to consider our Best Practices for Major Version Upgrades.

  4. Optional: Make any other changes that are needed, such as increasing the capacity or adding plugins.
  5. Save your changes. The new configuration takes a few moments to create.
  6. If you had Kibana enabled, the UI will prompt you to also upgrade Kibana.

Best Practices for Major Version Upgrades

For major version upgrades, we have to bring the cluster to a full stop before upgrading, as the nodes cannot communicate with each other. This is done by flushing all changes so we are sure we can recover them, then we start the cluster with the new version.

While Elasticsearch is working on making upgrades across major versions possible, major version upgrades often include so many changes that upgrades can be risky. This is usually true for any kind of software. Our recommended approach for major version upgrades is to simply make a new cluster with the latest major version, reindex everything and make sure index requests are temporarily sent to both clusters. With the new cluster ready, you can then do a hot swap and send requests to the new cluster. Since you are only billed for the hours a cluster is running, the few extra dollars added to your bill for having an extra cluster running for a while is money well spent. Since the cluster with the version known to work well is already running, you can quickly roll back if the new version has errors.

We make it easy to manage multiple clusters with different versions. We do not force customers to upgrade their clusters. If we need to end-of-life a very old version, you can expect to be notified in due time.

Upgrade to Elasticsearch 5.x

Elasticsearch 5.x provides major new features and improved usability, but there are a few things you need to keep in mind when upgrading to our latest and greatest software in Elastic Cloud.

Breaking Changes in Elasticsearch 5.x

A number of Elasticsearch queries were deprecated in version 2.0 or later and removed in version 5.x. Applications running deprecated queries might break after upgrading to 5.x.

Starting in Elasticsearch 2.3, a deprecation log became available that can help you determine if your applications are affected. Check this deprecation log before upgrading. If you are using deprecated queries, you will need to update your applications.

A number of other items were changed or removed in Elasticsearch 5.0 and later as well. For more information, see Breaking changes in 5.5.

Migrating Shield Configurations

In Elasticsearch 5.0 and later, the security features required to keep your Elastic Cloud clusters safe became part of X-Pack. With the move to X-Pack, the biggest changes to security features for the Elastic Stack include the names of the security configuration options, TLS/SSL configuration, and how roles are defined. A few privileges have also been removed.

When you upgrade an Elasticsearch cluster on Elastic Cloud to version 5.x, the upgrade process to the new X-Pack security features is handled for you. As part of the upgrade process, all users, roles, and user-role mappings that exist in your Shield configuration are upgraded to use the new X-Pack security features. In addition, two users are always created on version 5.x clusters, the elastic superuser and the anonymous user.

If your cluster never had an active Shield configuration or if you are not provided the password during the upgrade process, you might need to reset the password for the elastic user after upgrading to Elasticsearch 5.x. Users and applications will need to authenticate to be able to connect to your cluster.

After the upgrade is complete, you use the Kibana Management app for X-Pack to work with users and roles, which replaces the Shield Editor in previous releases:

User and role management in Kibana

To learn more about working with X-Pack security features in Elastic Cloud, see Securing your Cluster. For more background information about X-Pack for the Elastic Stack, see Securing Elasticsearch and Kibana.

Indices Originally Created in Older Versions of Elasticsearch

Indices created in Elasticsearch before version 2.0 are not compatible with version 5.x, even if you upgraded your cluster to version 2.0 or later at some point. In order to upgrade to Elasticsearch 5.x with these indices, you must perform some additional steps.

Important

If you upgrade to Elasticsearch 5.x with indices that are not supported, your Elasticsearch cluster will not start. Please make sure you follow the steps in this section first.

The following steps apply to you if your current cluster was originally created in a version of Elasticsearch before 2.0 or if you are not sure what version your cluster originally used:

  1. Check if you can upgrade to Elasticsearch 5.x directly.

    For all versions of Elasticsearch: You can find out what version of Elasticsearch an index was created with by querying the index settings and looking at the value of settings.index.version.created. The first number in the value indicates the major version of Elasticsearch and any value that is below 2000000 is not readable in Elasticsearch 5.x, even if settings.index.version.upgraded shows a value that is greater than or equal to 2000000.

    For example:

    - newdata: {
        aliases: { },
      + mappings: {…},
      - settings: {
          - index: {
              refresh_interval: "-1",
              number_of_shards: "5",
            - translog: {
                durability: "async"
            },
            creation_date: "1460043613621",
            number_of_replicas: "1",
            uuid: "n6kYaMYGT0OJx8McwGAzPw",
          - version: {
              created: "2030099"
              upgraded: "5000002"
            }
        }
      },
      warmers: { }
    },

    Here is how you read the example:

    • If created indicates a value that is greater than or equal to 2000000, you can upgrade to Elastisearch 5.x directly. In this example, the index was created with version 2.3.0 (created: "2030099"). You can skip the rest of the steps in this section and proceed to upgrade your cluster.
    • If created indicates a value that is smaller than 2000000, you cannot upgrade to Elastisearch 5.x directly. Follow the remaining steps in this section first before upgrading to Elastisearch 5.x.

      Examples of created values that prevent you from upgrading to to Elastisearch 5.x directly:

      1070199
      Indicates an index created with Elasticsearch 1.7.1.
      1060299
      Indicates an index created with Elasticsearch 1.6.2.

    For Elasticsearch 2.0 and later: You can see a human-friendly value for the version that an index was created with by appending ?human=true to your query URL.

    For example, for an Elasticsearch cluster a1b2c3d4a1b2c3d4a1b2c3d4a1b2c3d4 with the index named test, the URL https://a1b2c3d4a1b2c3d4a1b2c3d4a1b2c3d4.us-east-1.aws.found.io:9243/test?human=true returns a human-friendly version with the created_string (some output has been omitted for clarity):

    ...
    created_string: "2.3.2",
    created: "2030299"
    ...
  2. If you cannot upgrade to Elasticsearch 5.x directly, there are several options to get your cluster to version 5.x with minimal fuss:

    • Use the migration plugin that is available for Elasticsearch 2.3.0 and later by upgrading to 2.3 first. This means that you migrate in a two-step process, first by upgrading from 1.x to 2.3 and then by upgrading from 2.3 to 5.x with the migration plugin. This process performs an online reindex.

    • Use the upgrade functionality in Elastic Cloud built around the Upgrade API. This option performs a remote reindex of your indices and checks if the upgraded cluster will be able to start after upgrading.

      Tip

      There is additional overhead when you use this option. If your cluster is already running at capacity before the upgrade, Elastic recommends that you temporarily step up to the next available cluster size before upgrading, so that the reindex operation can complete in a reasonable amount of time. You can step down to the original cluster size after upgrading to avoid further costs.

    • If neither of those options work for you, you can also upgrade your indices on a cluster running Elasticsearch 2.3.x yourself with the Reindex API. For more information, see Reindex to upgrade.

A Note on Snapshots

Snapshots cannot be restored in Elasticsearch 5.x, if they contain indices created in an Elasticsearch version before 2.0. To work around this restriction, you either need to discard these snapshots or you need to open them on a cluster running Elasticsearch 2.3 before reindexing and creating new snapshots.