Elastic Docs ›Elasticsearch Service Documentation ›Snapshot and restore
« Configure a snapshot repository using Azure Blob storage Managing your organization »

Work with snapshotsedit

Snapshots provide a way to restore your Elasticsearch indices. They can be used to copy indices for testing, to recover from failures or accidental deletions, or to migrate data to other deployments.

By default, Elasticsearch Service takes a snapshot of all the indices in your Elasticsearch cluster every 30 minutes. You can set a different snapshot interval, if needed for your environment. You can also take snapshots on demand, without having to wait for the next interval. Taking a snapshot on demand does not affect the retention schedule for existing snapshots, it just adds an additional snapshot to the repository. This might be helpful if you are about to make a deployment change and you don’t have a current snapshot.

Snapshots back up only open indices. If you close an index, it is not included in snapshots and you will not be able to restore the data.

Beginning with Elastic Stack version 7.6, use Kibana to manage your snapshots. In Kibana, you can set up additional repositories where the snapshots are stored, other than the one currently managed by Elasticsearch Service. You can view and delete snapshots, and configure a snapshot lifecycle management (SLM) policy to automate when snapshots are created and deleted. To learn more, check the Snapshot and Restore documentation.

From within Elasticsearch Service you can restore a snapshot from a different deployment in the same region.

For deployments with Elastic Stack versions 7.5 and earlier, anywhere between a minimum of 12 and up to a maximum of the 100 most recent, successful snapshots may be stored. These tasks are available from the Elasticsearch Service UI:

List available snapshotsedit

To list all available snapshots:

  1. Log in to the Elasticsearch Service Console.

  2. Find your deployment on the home page in the Elasticsearch Service card and click the gear icon to access it directly. Or, select Elasticsearch Service to go to the deployments page to view all of your deployments.

    On the deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list.

  3. From your deployment menu, go to Elasticsearch and then Snapshots where the list of available snapshots is shown, newest first.
  4. Select a specific snapshot to get more details.

The list shows when the snapshot was completed, whether the snapshot succeeded or was only partially successful, along with the number of shards and indices in the snapshot.

Configure the snapshot retention periodedit

For Elastic Stack versions 7.6 and above, refer to the Snapshot and Restore documentation to learn about the snapshot management features.

For Elastic Stack versions 7.5 and below, from within Elasticsearch Service you can set the snapshot interval and count to have control over the retention period for your clusters. The calculation displayed is based on a projected estimate of the snapshot interval, expected snapshot count, and current cluster size. Changing any of these factors results in a different retention period.

Change the snapshot intervaledit

If you find the the default snapshot interval isn’t working for your environment, you can change it. It might make sense to use a 4 or 24 hour interval if the data in your index doesn’t change that often but you need more snapshot history.

  1. Log in to the Elasticsearch Service Console.

  2. Find your deployment on the home page in the Elasticsearch Service card and click the gear icon to access it directly. Or, select Elasticsearch Service to go to the deployments page to view all of your deployments.

    On the deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list.

  3. Open the Snapshot settings section.
  4. Make a selection from the Snapshot interval drop-down menu.
  5. Select Update settings.

When you change the interval, the interval timer begins after the next scheduled snapshot.

Change the snapshot countedit

You can get more control over your repository space by adjusting the number of snapshots that are stored for your cluster. The minimum number of snapshots must be at least 12, the maximum limit is 100.

  1. Log in to the Elasticsearch Service Console.

  2. Find your deployment on the home page in the Elasticsearch Service card and click the gear icon to access it directly. Or, select Elasticsearch Service to go to the deployments page to view all of your deployments.

    On the deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list.

  3. Open the Snapshot settings section.
  4. Enter a number in the Snapshot count field.
  5. Select Update settings.

Restore a snapshotedit

For Elastic Stack versions 7.6 and above, for details about restoring a snapshot, check Snapshot and Restore.

For Elastic Stack versions 7.5 and below, use these steps to restore a snapshot:

  1. List the available snapshots, and open the timestamp link of a snapshot to get the snapshot details.

  2. Optional: In the Restore Snapshot section, specify how you want to restore the snapshot. If you do not specify any options, all the indices of the snapshot are restored on the Elasticsearch cluster where the snapshot was taken.

    Snapshot restore

    • In the Specify Indices field, you can limit which indices get restored by specifying a data view (note that in Elastic Stack versions 8.0 and higher, index patterns have been renamed to data views). For example, entering logstash-*,.kibana restores any indices starting with logstash- as well as the .kibana index. All other indices are ignored.
    • You might need to specify values in the Match and Replace with fields, if you restore to the same cluster, as you cannot restore to an index that already exists.
  1. Select Restore snapshot.

After the snapshot is restored, a brief message indicates that the operation was successful.

Restore across clustersedit

Snapshots can be restored to either the same Elasticsearch cluster or to another cluster. If you are restoring all indices to another cluster, you can clone a cluster.

Users created using the X-Pack security features or using Shield are not included when you restore across clusters, only data from Elasticsearch indices is restored. If you do want to create a cloned cluster with the same users as your old cluster, you need to recreate the users manually on the new cluster.

Restoring to another cluster is useful for scenarios where isolating activities on a separate cluster is beneficial, such as:

Performing ad hoc analytics
For most logging and metrics use cases, it is cost prohibitive to have all the data in memory, even if it would provide the best performance for aggregations. Cloning the relevant data to an ad hoc analytics cluster that can be discarded after use is a cost effective way to experiment with your data, without risk to existing clusters used for production.
Enabling your developers
Realistic test data is crucial for uncovering unexpected errors early in the development cycle. What can be more realistic than actual data from a production cluster? Giving your developers access to real production data is a great way to break down silos.
Testing mapping changes
Mapping changes almost always require reindexing. Unless your data volume is trivial, reindexing requires time and tweaking the parameters to achieve the best reindexing performance usually takes a little trial and error. While this use case could also be handled by running the scan and scroll query directly against the source cluster, a long lived scroll has the side effect of blocking merges even if the scan query is very light weight.
Integration testing
Test your application against a real live Elasticsearch cluster with actual data. If you automate this, you could also aggregate performance metrics from the tests and use those metrics to detect if a change in your application has introduced a performance degradation.

A cluster is eligible as a destination for a snapshot restore, if it meets these criteria:

  • The cluster is in the same region. For example, a snapshot made in eu-west-1 cannot be restored to us-east-1 at this point. If you need to restore snapshots across regions, use a custom repository.
  • The destination cluster is able to read the indices. You can generally restore to your Elasticsearch cluster snapshots of indices created back to the previous major version, but see the version matrix for all the details.

To restore snapshots across clusters:

  1. First, create a new deployment and select Restore snapshot data. Select the deployment that you want to restore a snapshot from. If you don’t know the exact name, you can enter a few characters and then select from the list of matching deployments.

  2. Select the snapshot that you want to restore from. If none is chosen, the latest successful snapshot from the cluster you selected is restored on the new cluster when you create it.

    Restoring from a snapshot

    The cloud provider and region default to those of the deployment you selected to restore a snapshot from.

  3. Manually recreate users using the X-Pack security features or using Shield on the new cluster. User information is not included when you restore across clusters.

Restore snapshots containing searchable snapshots indices across clustersedit

If you are restoring a snapshot from one cluster to another, and that snaphot contains indices that are backed by searchable snapshots, there are a few more requirements to be aware of.

The first versions of Elasticsearch that supported searchable snapshots required that the repository name in the destination cluster, where the snapshot is to be restored from, match the repository name in the source cluster, where the snapshot was taken. Beginning with Elasticsearch version 7.12 this requirement is no longer present, but there are other prerequisites that need to be met.

Pre-requisites for restoring snapshots containing searchable snapshot indices across clusters:

  • The source cluster must have been created as version 7.12.0 or higher.

    To be more precise the requirement is on the found-snapshots repository settings at the time the snapshots were taken. The repository must have a uuid field, which is supported only in Elasticsearch versions 7.12.0 and higher. If the cluster was created with a previous version and later upgraded to 7.12.0 or higher, the repository may not have the required uuid field and therefore cannot be used to restore onto another cluster.

    To be sure, you can send a GET /_snapshot/found-snapshots request to your Elasticsearch cluster and check if the uuid field is present.

  • The destination cluster must be version 7.13.2 or higher.

    Previous versions had issues restoring the snapshot or recovering searchable snapshot indices in case of, for example, node failure.

The snapshot in the source cluster cannot be deleted after being successfully restored in the destination cluster. In fact, that’s also the case for the searchable snapshots in the source cluster for which there were indices backed by the restored snapshot. These snapshots are required for recovery of the searchable snapshot indices in case of, for example, node failure.

This means that until you delete the searchable snapshot indices in the destination cluster, you must not delete your source deployment, delete the restored snapshot, or delete any of the searchable snapshots used by the searchable snapshot indices.

« Configure a snapshot repository using Azure Blob storage Managing your organization »