Configure a snapshot repository using GCS | Elasticsearch Service Documentation

› › ›

« Configure a snapshot repository using AWS S3 Configure a snapshot repository using Azure Blob storage »

Configure a snapshot repository using GCSedit

Configure a custom snapshot repository using your Google Cloud Storage account.

This method only works for Elasticsearch clusters at version 6.x and higher. If you are using an earlier Elasticsearch version you must upgrade first.

Set up your service account credentialsedit

You’ll need to have an existing Google Cloud account and have the appropriate permissions to generate credentials.

Create a service account key in your Google Cloud project.

The service account should be configured to have permission to read, write, and list the bucket objects. For more information, refer to Recommended bucket permission in the Elasticsearch docs.
Save the service account key in JSON file format. You are going to use it later to configure your Elasticsearch deployment for snapshotting.

For more detailed information on the JSON account service key, refer to Using a Service Account.

Prepare a bucketedit

Follow the Google Cloud Storage documentation to create a GCS bucket.

Enable the `repository-gcs` plugin in Elastic Stack 7.17 and earlieredit

For deployments with Elastic Stack version 7.17 and earlier, you’ll need to enable the repository-gcs plugin to use the Google Cloud Storage repository type. On Google Cloud Platform, the plugin is enabled by default. If your deployment is on AWS or Azure, follow these steps to enable the repository-gcs plugin:

Refer to Google Cloud Storage Repository Plugin to download the version of the plugin that matches your Elastic Stack version.
Upload the plugin to your deployment:
1. Log in to the Elasticsearch Service Console.
2. Open the Features > Extensions page and select Upload extension.
3. Specify the plugin name (repository-gcs) and the version.
4. Select An installable plugin (Compiled, no source code).
5. Select Create extension.
6. Navigate back to the Features > Extensions page.
7. Select the extension name.
8. Drag and drop to upload the repository-gcs plugin zip file.

Configure the keystoreedit

Create an entry for the GCS client in the Elasticsearch keystore:

Log in to the Elasticsearch Service Console.
Navigate to the Security page of the deployment you wish to configure.
Locate Elasticsearch keystore and select Add settings.
Enter the Setting name gcs.client.secondary.credentials_file.
With Type set to JSON block / file, add your GCS service account key JSON file.
Select Save.

Create the repositoryedit

For deployments with Elastic Stack version 7.2 and higher:

Open Kibana and go to Management > Snapshot and Restore.
On the Repositories tab, select Register a repository.
Provide a name for your repository and select type Google Cloud Storage.
Provide the following settings:
- Client: secondary
- Bucket: Your GCS bucket name
- Base path: A directory to contain the snapshots
  - This setting is optional. Include a base_path if you have multiple clusters writing to the same GCS repository. This ensures that a snapshot won’t overwrite the snapshot metadata for another cluster.
Add any other settings that you wish to configure.
Select Register.
Select Verify to confirm that your settings are correct and the deployment can connect to your repository.
Your snapshot repository is now set up using GCS! You can use Kibana to manage your snapshots and begin sending Elasticsearch snapshots to your own bucket. For details, check the Snapshot and Restore documentation.

For deployments with Elastic Stack version 7.1 and lower:

With the bucket ready, you can create a repository with a PUT request to /_snapshot/repository_name:

PUT _snapshot/repository_name
{
  "type": "gcs",
  "settings": {
    "client": "secondary",
    "bucket": "name-of-gcs-bucket",
    "compress": true
  }
}