Cluster state encryption

Some Elasticsearch features need to store sensitive values, such as credentials for ES|QL data federation, in cluster state. With cluster state encryption, this data is protected both at rest and in transit between nodes, so that secrets are never persisted or replicated in plain text.

These feature-managed secrets differ from secure settings in the Elasticsearch keystore. Secure settings are provided by the operator and stored locally on each node. They are not designed for secrets that features create or update through cluster APIs and then store in cluster state.

Elasticsearch generates a single, cluster-wide encryption key automatically. Features that need to persist secrets never handle the key directly. They call an internal encryption service that encrypts and decrypts values on their behalf. The key itself is never exposed through any API.

The first feature to use cluster state encryption is ES|QL data federation, which encrypts the credentials used to connect to external data sources.

Note

Cluster state encryption is distinct from customer-managed encryption keys. Customer-managed keys protect an Elastic Cloud Hosted deployment's files and snapshots at the infrastructure layer. Cluster state encryption protects feature-managed secrets before Elasticsearch stores or replicates them in cluster state. The two mechanisms are complementary.

How the cluster state encryption key works

Elasticsearch generates the cluster state encryption key automatically and stores it in cluster state, where it's distributed to every node. It's excluded from cluster state REST responses and from snapshots.

Each node keeps a copy of the key in memory. To survive a restart, a node also persists a copy of the key to local disk, protected by a password so it isn't stored in plain text.

Protect the cluster state encryption key with a password

To let a node persist the key to disk, it needs a password configured in the Elasticsearch keystore:

cluster.state.encryption.password.<id>: A secure setting holding an encryption password, identified by <id>.
cluster.state.encryption.active_password_id: The <id> of the password currently used to protect newly written keys.

These settings are node-local. Configure a different password value on each node. Password IDs can be the same across nodes.

On Elastic Cloud Hosted and Elastic Cloud Enterprise, the control plane supplies this password automatically.

On self-managed standalone Stack deployments, this password is generated by the automatic security setup on a node's first start, and stored in the keystore as cluster.state.encryption.password.autoconfigured.

Configure one yourself with elasticsearch-keystore if the automatic security setup is not executed.

cluster.state.encryption.required: Whether a password is required before Elasticsearch will store secrets using the cluster state encryption key. Defaults to true. Setting this to false is not recommended: Elasticsearch falls back to storing secrets in plain text instead, and logs a warning.

Upgrade existing deployments

You don't need to configure an encryption password to upgrade an existing deployment. If no password is configured after the upgrade, the cluster remains healthy and reports that cluster state encryption is not configured. Because encryption is required by default, Elasticsearch rejects attempts to store feature-managed secrets with an HTTP 503 response until a password is configured.

Self-managed deployments: Automatic security setup doesn't run again during an upgrade. Before a feature first stores a managed secret, configure cluster.state.encryption.password.<id> and cluster.state.encryption.active_password_id in the keystore on every node, then call POST /_nodes/reload_secure_settings. A restart isn't required.
Elastic Cloud on Kubernetes deployments: ECK doesn't provision the password automatically during an upgrade. Before a feature first stores a managed secret, configure cluster.state.encryption.password.<id> and cluster.state.encryption.active_password_id through ECK secure settings, then call POST /_nodes/reload_secure_settings. A restart isn't required.
Elastic Cloud Hosted and Elastic Cloud Enterprise deployments: When an existing deployment is upgraded to 9.5 or later, the control plane provisions the password and reloads the secure settings automatically.

Rotate the encryption password

You can rotate each node's password independently without restarting Elasticsearch. The following procedure rotates the password on one node. Repeat it for any other nodes whose passwords you want to rotate. Keep a node's old password available until that node has loaded the new password and rewrapped its on-disk copy of the cluster state encryption key.

Add the new password to the node's local Elasticsearch keystore. For example, to use the password ID v2:
```
bin/elasticsearch-keystore add cluster.state.encryption.password.v2
		
```

On the same node, set cluster.state.encryption.active_password_id to v2:

bin/elasticsearch-keystore add -f cluster.state.encryption.active_password_id

Reload the secure settings on that node:

				POST /_nodes/<node_id>/reload_secure_settings

Send the following request directly to the node's HTTP endpoint:
```
				GET /_health_report/cluster_state_encryption
		
```
Before proceeding, verify that the node reports a green status and that both active_password_id and metadata_password_id are v2.
Remove the old password from the node's keystore, then reload the secure settings on that node again.

Automatic key rotation

Elasticsearch rotates the cluster state encryption key (not the password in the keystore) automatically. You can control the schedule with:

xpack.encryption.key_rotation.interval: How often the key is rotated. Defaults to 30d. Set to 0 to disable automatic rotation.
xpack.encryption.key_rotation.check_interval: How often Elasticsearch checks whether rotation is due. Defaults to 1h, must be at least 1s, and can't be greater than key_rotation.interval.

Check cluster state encryption health

The cluster_state_encryption health indicator reports on the status of the cluster state encryption:

Green: encryption is either working normally, or hasn't been configured (which is expected on most self-managed clusters that haven't set a password).
Yellow: a password is missing where one is required, a node can't persist the key to disk, or Elasticsearch failed to decrypt the key.

Reset the cluster state encryption key

Warning

Resetting the cluster state encryption key is destructive and irreversible. Any data that was encrypted with the previous key is permanently lost.

As a last resort, if the encryption password is lost or corrupted and can't be recovered, you can discard the current cluster state encryption key and everything encrypted with it:

				POST /_encryption/_reset?accept_data_loss=true

The accept_data_loss=true query parameter is required. Elasticsearch generates a new cluster state encryption key on the next write.

The first Elastic feature to use cluster state encryption is ES|QL data federation, which encrypts the credentials used to connect to external data sources.

To learn more about the concepts and tools involved, refer to: