Add remote clusters using API key authenticationedit

This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.

API key authentication enables a local cluster to authenticate itself with a remote cluster via a cross-cluster API key. The API key needs to be created by an administrator of the remote cluster. The local cluster is configured to provide this API key on each request to the remote cluster. The remote cluster verifies the API key and grants access, based on the API key’s privileges.

All cross-cluster requests from the local cluster are bound by the API key’s privileges, regardless of local users associated with the requests. For example, if the API key only allows read access to my-index on the remote cluster, even a superuser from the local cluster is limited by this constraint. This mechanism enables the remote cluster’s administrator to have full control over who can access what data with cross-cluster search and/or cross-cluster replication. The remote cluster’s administrator can be confident that no access is possible beyond what is explicitly assigned to the API key.

On the local cluster side, not every local user needs to access every piece of data allowed by the API key. An administrator of the local cluster can further configure additional permission constraints on local users so each user only gets access to the necessary remote data. Note it is only possible to further reduce the permissions allowed by the API key for individual local users. It is impossible to increase the permissions to go beyond what is allowed by the API key.

To add a remote cluster using API key authentication:

If you run into any issues, refer to Troubleshooting.

Prerequisites and limitationsedit

  • The local and remote deployments must be on version 8.10 or later.
  • The local and remote deployments must be Elasticsearch Service deployments hosted on Elastic Cloud.
  • The local and remote deployments must be hosted in the same region.
  • API key authentication can’t be used in combination with traffic filters.
  • Contrary to the certificate based security model, the API key based security model does not require setting up trust between the local and remote clusters.

Create a cross-cluster API key on the remote deploymentedit

  • On the remote deployment, use the Elasticsearch Create cross-cluster key API or Kibana to create a cross-cluster API key. Configure it with access to the indices you want to use for cross-cluster search or cross-cluster replication.
  • Copy the encoded key (encoded in the response) to a safe location. You will need it in the next step.

Add the cross-cluster API key to the keystore of the local deploymentedit

  • Add a secret value to the keystore of the local deployment:

    1. Log in to the Elasticsearch Service Console.
    2. Find your deployment on the home page in the Elasticsearch Service card and select Manage to access it directly. Or, select Dedicated deployments to go to the deployments page to view all of your deployments.

      On the deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list.

    3. From the deployment menu, select Security.
    4. Locate Elasticsearch keystore.
    5. Add a setting:

      • Enter the name cluster.remote.ALIAS.credentials, replacing ALIAS with the alias you will use to connect to the remote cluster later.
      • Paste the encoded cross-cluster API key as the secret.
  • Restart the local deployment.

Connect to a remote clusteredit

On the local cluster, add the remote cluster using Kibana or the Elasticsearch API.

Using Kibanaedit
  1. Open the Kibana main menu, and select Stack Management > Data > Remote Clusters > Add a remote cluster.
  2. Enable Manually enter proxy address and server name.
  3. Fill in the following fields:

    • Name: This cluster alias is a unique identifier that represents the connection to the remote cluster and is used to distinguish between local and remote indices.
    • Proxy address: This value can be found on the Security page of the Elasticsearch Service deployment you want to use as a remote. Change the port into 9443.
    • Server name: This value can be found on the Security page of the Elasticsearch Service deployment you want to use as a remote.

      Remote Cluster Parameters in Deployment
  4. Click Next.
  5. Click Add remote cluster (you have already established trust in a previous step).
Using the Elasticsearch APIedit

To configure a deployment as a remote cluster, use the cluster update settings API. Configure the following fields:

  • mode: proxy
  • proxy_address: This value can be found on the Security page of the Elasticsearch Service deployment you want to use as a remote. Change the port into 9443. Also, using the API, this value can be obtained from the Elasticsearch resource info, concatenating the field metadata.endpoint and port 9443 using a semicolon.
  • server_name: This value can be found on the Security page of the Elasticsearch Service deployment you want to use as a remote. Also, using the API, this can be obtained from the Elasticsearch resource info field metadata.endpoint.

This is an example of the API call to _cluster/settings:

PUT /_cluster/settings
{
  "persistent": {
    "cluster": {
      "remote": {
        "alias-for-my-remote-cluster": {
          "mode":"proxy",
          "proxy_address": "a542184a7a7d45b88b83f95392f450ab.192.168.44.10.ip.es.io:9443",
          "server_name": "a542184a7a7d45b88b83f95392f450ab.192.168.44.10.ip.es.io"
        }
      }
    }
  }
}

Configure roles and usersedit

To use a remote cluster for cross-cluster replication or cross-cluster search, you need to create user roles with remote indices privileges on the local cluster. Refer to Configure roles and users.