Get shard allocation information Generally available

GET /_cat/allocation/{node_id}

All methods and paths for this operation:

GET /_cat/allocation

GET /_cat/allocation/{node_id}

Get a snapshot of the number of shards allocated to each data node and their disk space.

IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications.

Required authorization

  • Cluster privileges: monitor

Path parameters

  • node_id string | array[string]

    A comma-separated list of node identifiers or names used to limit the returned information.

Query parameters

  • bytes string

    The unit used to display byte values.

    Values are b, kb, mb, gb, tb, or pb.

  • h string | array[string]

    A comma-separated list of columns names to display. It supports simple wildcards.

    Supported values include:

    • shards (or s): The number of shards on the node.
    • shards.undesired: The number of shards scheduled to be moved elsewhere in the cluster.
    • write_load.forecast (or wlf, writeLoadForecast): The sum of index write load forecasts.
    • disk.indices.forecast (or dif, diskIndicesForecast): The sum of shard size forecasts.
    • disk.indices (or di, diskIndices): The disk space used by Elasticsearch indices.
    • disk.used (or du, diskUsed): The total disk space used on the node.
    • disk.avail (or da, diskAvail): The available disk space on the node.
    • disk.total (or dt, diskTotal): The total disk capacity of all volumes on the node.
    • disk.percent (or dp, diskPercent): The percentage of disk space used on the node.
    • host (or h): IThe host of the node.
    • ip: The IP address of the node.
    • node (or n): The name of the node.
    • node.role (or r, role, nodeRole): The roles assigned to the node.

    Values are shards, s, shards.undesired, write_load.forecast, wlf, writeLoadForecast, disk.indices.forecast, dif, diskIndicesForecast, disk.indices, di, diskIndices, disk.used, du, diskUsed, disk.avail, da, diskAvail, disk.total, dt, diskTotal, disk.percent, dp, diskPercent, host, h, ip, node, n, node.role, r, role, or nodeRole.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

  • local boolean

    If true, the request computes the list of selected nodes from the local cluster state. If false the list of selected nodes are computed from the cluster state of the master node. In both cases the coordinating node will send requests for further information to each selected node.

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

Responses

GET /_cat/allocation/{node_id}
GET /_cat/allocation?v=true&format=json
resp = client.cat.allocation(
    v=True,
    format="json",
)
const response = await client.cat.allocation({
  v: "true",
  format: "json",
});
response = client.cat.allocation(
  v: "true",
  format: "json"
)
$resp = $client->cat()->allocation([
    "v" => "true",
    "format" => "json",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_cat/allocation?v=true&format=json"
client.cat().allocation();
Response examples (200)
A successful response from `GET /_cat/allocation?v=true&format=json`. It shows a single shard is allocated to the one node available.
[
  {
    "shards": "1",
    "shards.undesired": "0",
    "write_load.forecast": "0.0",
    "disk.indices.forecast": "260b",
    "disk.indices": "260b",
    "disk.used": "47.3gb",
    "disk.avail": "43.4gb",
    "disk.total": "100.7gb",
    "disk.percent": "46",
    "host": "127.0.0.1",
    "ip": "127.0.0.1",
    "node": "CSUXak2",
    "node.role": "himrst"
  }
]





































































































































































Reload the keystore on nodes in the cluster Generally available; Added in 6.5.0

POST /_nodes/{node_id}/reload_secure_settings

All methods and paths for this operation:

POST /_nodes/reload_secure_settings

POST /_nodes/{node_id}/reload_secure_settings

Secure settings are stored in an on-disk keystore. Certain of these settings are reloadable. That is, you can change them on disk and reload them without restarting any nodes in the cluster. When you have updated reloadable secure settings in your keystore, you can use this API to reload those settings on each node.

When the Elasticsearch keystore is password protected and not simply obfuscated, you must provide the password for the keystore when you reload the secure settings. Reloading the settings for the whole cluster assumes that the keystores for all nodes are protected with the same password; this method is allowed only when inter-node communications are encrypted. Alternatively, you can reload the secure settings on each node by locally accessing the API and passing the node-specific Elasticsearch keystore password.

Path parameters

  • node_id string | array[string] Required

    The names of particular nodes in the cluster to target.

Query parameters

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

application/json

Body

  • secure_settings_password string

    The password for the Elasticsearch keystore.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object

      Contains statistics about the number of nodes selected by the request’s node filters.

      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide failures attributes Show failures attributes object
        • type string Required

          The type of error

        • reason
        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by
        • root_cause array[object]
        • suppressed array[object]
      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string Required
    • nodes object Required
      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • name string Required
        • reload_exception object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide reload_exception attributes Show reload_exception attributes object
          • type string Required

            The type of error

          • reason
          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • root_cause array[object]
          • suppressed array[object]
POST /_nodes/{node_id}/reload_secure_settings
POST _nodes/reload_secure_settings
{
  "secure_settings_password": "keystore-password"
}
resp = client.nodes.reload_secure_settings(
    secure_settings_password="keystore-password",
)
const response = await client.nodes.reloadSecureSettings({
  secure_settings_password: "keystore-password",
});
response = client.nodes.reload_secure_settings(
  body: {
    "secure_settings_password": "keystore-password"
  }
)
$resp = $client->nodes()->reloadSecureSettings([
    "body" => [
        "secure_settings_password" => "keystore-password",
    ],
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"secure_settings_password":"keystore-password"}' "$ELASTICSEARCH_URL/_nodes/reload_secure_settings"
client.nodes().reloadSecureSettings(r -> r
    .secureSettingsPassword("keystore-password")
);
Request example
Run `POST _nodes/reload_secure_settings` to reload the keystore on nodes in the cluster.
{
  "secure_settings_password": "keystore-password"
}
Response examples (200)
A successful response when reloading keystore on nodes in your cluster.
{
  "_nodes": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "cluster_name": "my_cluster",
  "nodes": {
    "pQHNt5rXTTWNvUgOrdynKg": {
      "name": "node-0"
    }
  }
}


























































































Update the connector features Technical preview

PUT /_connector/{connector_id}/_features

Update the connector features in the connector document. This API can be used to control the following aspects of a connector:

  • document-level security
  • incremental syncs
  • advanced sync rules
  • basic sync rules

Normally, the running connector service automatically manages these features. However, you can use this API to override the default behavior.

To sync data using self-managed connectors, you need to deploy the Elastic connector service on your own infrastructure. This service runs automatically on Elastic Cloud for Elastic managed connectors.

Path parameters

  • connector_id string Required

    The unique identifier of the connector to be updated.

application/json

Body Required

  • features object Required
    Hide features attributes Show features attributes object
    • document_level_security object

      Indicates whether document-level security is enabled.

      Hide document_level_security attribute Show document_level_security attribute object
      • enabled boolean Required
    • incremental_sync object

      Indicates whether incremental syncs are enabled.

      Hide incremental_sync attribute Show incremental_sync attribute object
      • enabled boolean Required
    • native_connector_api_keys object

      Indicates whether managed connector API keys are enabled.

      Hide native_connector_api_keys attribute Show native_connector_api_keys attribute object
      • enabled boolean Required
    • sync_rules object
      Hide sync_rules attributes Show sync_rules attributes object
      • advanced object

        Indicates whether advanced sync rules are enabled.

        Hide advanced attribute Show advanced attribute object
        • enabled boolean Required
      • basic object

        Indicates whether basic sync rules are enabled.

        Hide basic attribute Show basic attribute object
        • enabled boolean Required

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.

PUT /_connector/{connector_id}/_features
PUT _connector/my-connector/_features
{
  "features": {
    "document_level_security": {
      "enabled": true
    },
    "incremental_sync": {
      "enabled": true
    },
    "sync_rules": {
      "advanced": {
        "enabled": false
      },
      "basic": {
        "enabled": true
      }
    }
  }
}
resp = client.connector.update_features(
    connector_id="my-connector",
    features={
        "document_level_security": {
            "enabled": True
        },
        "incremental_sync": {
            "enabled": True
        },
        "sync_rules": {
            "advanced": {
                "enabled": False
            },
            "basic": {
                "enabled": True
            }
        }
    },
)
const response = await client.connector.updateFeatures({
  connector_id: "my-connector",
  features: {
    document_level_security: {
      enabled: true,
    },
    incremental_sync: {
      enabled: true,
    },
    sync_rules: {
      advanced: {
        enabled: false,
      },
      basic: {
        enabled: true,
      },
    },
  },
});
response = client.connector.update_features(
  connector_id: "my-connector",
  body: {
    "features": {
      "document_level_security": {
        "enabled": true
      },
      "incremental_sync": {
        "enabled": true
      },
      "sync_rules": {
        "advanced": {
          "enabled": false
        },
        "basic": {
          "enabled": true
        }
      }
    }
  }
)
$resp = $client->connector()->updateFeatures([
    "connector_id" => "my-connector",
    "body" => [
        "features" => [
            "document_level_security" => [
                "enabled" => true,
            ],
            "incremental_sync" => [
                "enabled" => true,
            ],
            "sync_rules" => [
                "advanced" => [
                    "enabled" => false,
                ],
                "basic" => [
                    "enabled" => true,
                ],
            ],
        ],
    ],
]);
curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"features":{"document_level_security":{"enabled":true},"incremental_sync":{"enabled":true},"sync_rules":{"advanced":{"enabled":false},"basic":{"enabled":true}}}}' "$ELASTICSEARCH_URL/_connector/my-connector/_features"
client.connector().updateFeatures(u -> u
    .connectorId("my-connector")
    .features(f -> f
        .documentLevelSecurity(d -> d
            .enabled(true)
        )
        .incrementalSync(i -> i
            .enabled(true)
        )
        .syncRules(s -> s
            .advanced(a -> a
                .enabled(false)
            )
            .basic(b -> b
                .enabled(true)
            )
        )
    )
);
Request examples
{
  "features": {
    "document_level_security": {
      "enabled": true
    },
    "incremental_sync": {
      "enabled": true
    },
    "sync_rules": {
      "advanced": {
        "enabled": false
      },
      "basic": {
        "enabled": true
      }
    }
  }
}
{
  "features": {
    "document_level_security": {
      "enabled": true
    }
  }
}
Response examples (200)
{
  "result": "updated"
}



















































































































































Get a document by its ID Generally available

GET /{index}/_doc/{id}

Get a document and its source or stored fields from an index.

By default, this API is realtime and is not affected by the refresh rate of the index (when data will become visible for search). In the case where stored fields are requested with the stored_fields parameter and the document has been updated but is not yet refreshed, the API will have to parse and analyze the source to extract the stored fields. To turn off realtime behavior, set the realtime parameter to false.

Source filtering

By default, the API returns the contents of the _source field unless you have used the stored_fields parameter or the _source field is turned off. You can turn off _source retrieval by using the _source parameter:

GET my-index-000001/_doc/0?_source=false

If you only need one or two fields from the _source, use the _source_includes or _source_excludes parameters to include or filter out particular fields. This can be helpful with large documents where partial retrieval can save on network overhead Both parameters take a comma separated list of fields or wildcard expressions. For example:

GET my-index-000001/_doc/0?_source_includes=*.id&_source_excludes=entities

If you only want to specify includes, you can use a shorter notation:

GET my-index-000001/_doc/0?_source=*.id

Routing

If routing is used during indexing, the routing value also needs to be specified to retrieve a document. For example:

GET my-index-000001/_doc/2?routing=user1

This request gets the document with ID 2, but it is routed based on the user. The document is not fetched if the correct routing is not specified.

Distributed

The GET operation is hashed into a specific shard ID. It is then redirected to one of the replicas within that shard ID and returns the result. The replicas are the primary shard and its replicas within that shard ID group. This means that the more replicas you have, the better your GET scaling will be.

Versioning support

You can use the version parameter to retrieve the document only if its current version is equal to the specified one.

Internally, Elasticsearch has marked the old document as deleted and added an entirely new document. The old version of the document doesn't disappear immediately, although you won't be able to access it. Elasticsearch cleans up deleted documents in the background as you continue to index more data.

Required authorization

  • Index privileges: read

Path parameters

  • index string Required

    The name of the index that contains the document.

  • id string Required

    A unique document identifier.

Query parameters

  • preference string

    The node or shard the operation should be performed on. By default, the operation is randomized between the shard replicas.

    If it is set to _local, the operation will prefer to be run on a local allocated shard when possible. If it is set to a custom value, the value is used to guarantee that the same shards will be used for the same custom value. This can help with "jumping values" when hitting different shards in different refresh states. A sample value can be something like the web session ID or the user name.

  • realtime boolean

    If true, the request is real-time as opposed to near-real-time.

  • refresh boolean

    If true, the request refreshes the relevant shards before retrieving the document. Setting it to true should be done after careful thought and verification that this does not cause a heavy load on the system (and slow down indexing).

  • routing string

    A custom value used to route operations to a specific shard.

  • _source boolean | string | array[string]

    Indicates whether to return the _source field (true or false) or lists the fields to return.

  • _source_excludes string | array[string]

    A comma-separated list of source fields to exclude from the response. You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter. If the _source parameter is false, this parameter is ignored.

  • _source_includes string | array[string]

    A comma-separated list of source fields to include in the response. If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the _source_excludes query parameter. If the _source parameter is false, this parameter is ignored.

  • stored_fields string | array[string]

    A comma-separated list of stored fields to return as part of a hit. If no fields are specified, no stored fields are included in the response. If this field is specified, the _source parameter defaults to false. Only leaf fields can be retrieved with the stored_field option. Object fields can't be returned;​if specified, the request fails.

  • version number

    The version number for concurrency control. It must match the current version of the document for the request to succeed.

  • version_type string

    The version type.

    Supported values include:

    • internal: Use internal versioning that starts at 1 and increments with each update or delete.
    • external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.
    • external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document. NOTE: The external_gte version type is meant for special use cases and should be used with care. If used incorrectly, it can result in loss of data.
    • force: This option is deprecated because it can cause primary and replica shards to diverge.

    Values are internal, external, external_gte, or force.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _index string Required

      The name of the index the document belongs to.

    • fields object

      If the stored_fields parameter is set to true and found is true, it contains the document fields stored in the index.

      Hide fields attribute Show fields attribute object
      • * object Additional properties
    • _ignored array[string]
    • found boolean Required

      Indicates whether the document exists.

    • _id string Required

      The unique identifier for the document.

    • _primary_term number

      The primary term assigned to the document for the indexing operation.

    • _routing string

      The explicit routing, if set.

    • _seq_no number

      The sequence number assigned to the document for the indexing operation. Sequence numbers are used to ensure an older version of a document doesn't overwrite a newer version.

    • _source object

      If found is true, it contains the document data formatted in JSON. If the _source parameter is set to false or the stored_fields parameter is set to true, it is excluded.

    • _version number

      The document version, which is ncremented each time the document is updated.

GET /{index}/_doc/{id}
GET my-index-000001/_doc/1?stored_fields=tags,counter
resp = client.get(
    index="my-index-000001",
    id="1",
    stored_fields="tags,counter",
)
const response = await client.get({
  index: "my-index-000001",
  id: 1,
  stored_fields: "tags,counter",
});
response = client.get(
  index: "my-index-000001",
  id: "1",
  stored_fields: "tags,counter"
)
$resp = $client->get([
    "index" => "my-index-000001",
    "id" => "1",
    "stored_fields" => "tags,counter",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/my-index-000001/_doc/1?stored_fields=tags,counter"
Response examples (200)
A successful response from `GET my-index-000001/_doc/0`. It retrieves the JSON document with the `_id` 0 from the `my-index-000001` index.
{
  "_index": "my-index-000001",
  "_id": "0",
  "_version": 1,
  "_seq_no": 0,
  "_primary_term": 1,
  "found": true,
  "_source": {
    "@timestamp": "2099-11-15T14:12:12",
    "http": {
      "request": {
        "method": "get"
      },
      "response": {
        "status_code": 200,
        "bytes": 1070000
      },
      "version": "1.1"
    },
    "source": {
      "ip": "127.0.0.1"
    },
    "message": "GET /search HTTP/1.1 200 1070000",
    "user": {
      "id": "kimchy"
    }
  }
}
A successful response from `GET my-index-000001/_doc/1?stored_fields=tags,counter`, which retrieves a set of stored fields. Field values fetched from the document itself are always returned as an array. Any requested fields that are not stored (such as the counter field in this example) are ignored.
{
  "_index": "my-index-000001",
  "_id": "1",
  "_version": 1,
  "_seq_no" : 22,
  "_primary_term" : 1,
  "found": true,
  "fields": {
      "tags": [
        "production"
      ]
  }
}
A successful response from `GET my-index-000001/_doc/2?routing=user1&stored_fields=tags,counter`, which retrieves the `_routing` metadata field.
{
  "_index": "my-index-000001",
  "_id": "2",
  "_version": 1,
  "_seq_no" : 13,
  "_primary_term" : 1,
  "_routing": "user1",
  "found": true,
  "fields": {
      "tags": [
        "env2"
      ]
  }
}




























Get multiple documents Generally available; Added in 1.3.0

POST /{index}/_mget

All methods and paths for this operation:

GET /_mget

POST /_mget
GET /{index}/_mget
POST /{index}/_mget

Get multiple JSON documents by ID from one or more indices. If you specify an index in the request URI, you only need to specify the document IDs in the request body. To ensure fast responses, this multi get (mget) API responds with partial results if one or more shards fail.

Filter source fields

By default, the _source field is returned for every document (if stored). Use the _source and _source_include or source_exclude attributes to filter what fields are returned for a particular document. You can include the _source, _source_includes, and _source_excludes query parameters in the request URI to specify the defaults to use when there are no per-document instructions.

Get stored fields

Use the stored_fields attribute to specify the set of stored fields you want to retrieve. Any requested fields that are not stored are ignored. You can include the stored_fields query parameter in the request URI to specify the defaults to use when there are no per-document instructions.

Required authorization

  • Index privileges: read

Path parameters

  • index string Required

    Name of the index to retrieve documents from when ids are specified, or when a document in the docs array does not specify an index.

Query parameters

  • preference string

    Specifies the node or shard the operation should be performed on. Random by default.

  • realtime boolean

    If true, the request is real-time as opposed to near-real-time.

  • refresh boolean

    If true, the request refreshes relevant shards before retrieving documents.

  • routing string

    Custom value used to route operations to a specific shard.

  • _source boolean | string | array[string]

    True or false to return the _source field or not, or a list of fields to return.

  • _source_excludes string | array[string]

    A comma-separated list of source fields to exclude from the response. You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter.

  • _source_includes string | array[string]

    A comma-separated list of source fields to include in the response. If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the _source_excludes query parameter. If the _source parameter is false, this parameter is ignored.

  • stored_fields string | array[string]

    If true, retrieves the document fields stored in the index rather than the document _source.

application/json

Body Required

  • docs array[object]

    The documents you want to retrieve. Required if no index is specified in the request URI.

    Hide docs attributes Show docs attributes object
    • _id string Required

      The unique document ID.

    • _index string

      The index that contains the document.

    • routing string

      The key for the primary shard the document resides on. Required if routing is used during indexing.

    • _source boolean | object

      If false, excludes all _source fields.

      One of:

      If false, excludes all _source fields.

    • stored_fields string | array[string]

      The stored fields you want to retrieve.

    • version number
    • version_type string

      Supported values include:

      • internal: Use internal versioning that starts at 1 and increments with each update or delete.
      • external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.
      • external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document. NOTE: The external_gte version type is meant for special use cases and should be used with care. If used incorrectly, it can result in loss of data.
      • force: This option is deprecated because it can cause primary and replica shards to diverge.

      Values are internal, external, external_gte, or force.

  • ids string | array[string]

    The IDs of the documents you want to retrieve. Allowed when the index is specified in the request URI.

    One of:

    The IDs of the documents you want to retrieve. Allowed when the index is specified in the request URI.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • docs array[object] Required

      The response includes a docs array that contains the documents in the order specified in the request. The structure of the returned documents is similar to that returned by the get API. If there is a failure getting a particular document, the error is included in place of the document.

      One of:
      Hide attributes Show attributes
      • _index string Required

        The name of the index the document belongs to.

      • fields object

        If the stored_fields parameter is set to true and found is true, it contains the document fields stored in the index.

        Hide fields attribute Show fields attribute object
        • * object Additional properties
      • _ignored array[string]
      • found boolean Required

        Indicates whether the document exists.

      • _id string Required

        The unique identifier for the document.

      • _primary_term number

        The primary term assigned to the document for the indexing operation.

      • _routing string

        The explicit routing, if set.

      • _seq_no number

        The sequence number assigned to the document for the indexing operation. Sequence numbers are used to ensure an older version of a document doesn't overwrite a newer version.

      • _source object

        If found is true, it contains the document data formatted in JSON. If the _source parameter is set to false or the stored_fields parameter is set to true, it is excluded.

      • _version number

        The document version, which is ncremented each time the document is updated.

POST /{index}/_mget
GET /my-index-000001/_mget
{
  "docs": [
    {
      "_id": "1"
    },
    {
      "_id": "2"
    }
  ]
}
resp = client.mget(
    index="my-index-000001",
    docs=[
        {
            "_id": "1"
        },
        {
            "_id": "2"
        }
    ],
)
const response = await client.mget({
  index: "my-index-000001",
  docs: [
    {
      _id: "1",
    },
    {
      _id: "2",
    },
  ],
});
response = client.mget(
  index: "my-index-000001",
  body: {
    "docs": [
      {
        "_id": "1"
      },
      {
        "_id": "2"
      }
    ]
  }
)
$resp = $client->mget([
    "index" => "my-index-000001",
    "body" => [
        "docs" => array(
            [
                "_id" => "1",
            ],
            [
                "_id" => "2",
            ],
        ),
    ],
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"docs":[{"_id":"1"},{"_id":"2"}]}' "$ELASTICSEARCH_URL/my-index-000001/_mget"
client.mget(m -> m
    .docs(List.of(MultiGetOperation.of(mu -> mu
            .id("1")),MultiGetOperation.of(mu -> mu
            .id("2"))))
    .index("my-index-000001")
);
Request examples
Run `GET /my-index-000001/_mget`. When you specify an index in the request URI, only the document IDs are required in the request body.
{
  "docs": [
    {
      "_id": "1"
    },
    {
      "_id": "2"
    }
  ]
}
Run `GET /_mget`. This request sets `_source` to `false` for document 1 to exclude the source entirely. It retrieves `field3` and `field4` from document 2. It retrieves the `user` field from document 3 but filters out the `user.location` field.
{
  "docs": [
    {
      "_index": "test",
      "_id": "1",
      "_source": false
    },
    {
      "_index": "test",
      "_id": "2",
      "_source": [ "field3", "field4" ]
    },
    {
      "_index": "test",
      "_id": "3",
      "_source": {
        "include": [ "user" ],
        "exclude": [ "user.location" ]
      }
    }
  ]
}
Run `GET /_mget`. This request retrieves `field1` and `field2` from document 1 and `field3` and `field4` from document 2.
{
  "docs": [
    {
      "_index": "test",
      "_id": "1",
      "stored_fields": [ "field1", "field2" ]
    },
    {
      "_index": "test",
      "_id": "2",
      "stored_fields": [ "field3", "field4" ]
    }
  ]
}
Run `GET /_mget?routing=key1`. If routing is used during indexing, you need to specify the routing value to retrieve documents. This request fetches `test/_doc/2` from the shard corresponding to routing key `key1`. It fetches `test/_doc/1` from the shard corresponding to routing key `key2`.
{
  "docs": [
    {
      "_index": "test",
      "_id": "1",
      "routing": "key2"
    },
    {
      "_index": "test",
      "_id": "2"
    }
  ]
}


























































Get the async EQL status Generally available; Added in 7.9.0

GET /_eql/search/status/{id}

Get the current status for an async EQL search or a stored synchronous EQL search without returning results.

Path parameters

  • id string Required

    Identifier for the search.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • id string Required

      Identifier for the search.

    • is_partial boolean Required

      If true, the search request is still executing. If false, the search is completed.

    • is_running boolean Required

      If true, the response does not contain complete search results. This could be because either the search is still running (is_running status is false), or because it is already completed (is_running status is true) and results are partial due to failures or timeouts.

    • start_time_in_millis number

      Time unit for milliseconds

    • expiration_time_in_millis number

      Time unit for milliseconds

    • completion_status number

      For a completed search shows the http status code of the completed search.

GET /_eql/search/status/{id}
GET /_eql/search/status/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=
resp = client.eql.get_status(
    id="FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
)
const response = await client.eql.getStatus({
  id: "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
});
response = client.eql.get_status(
  id: "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE="
)
$resp = $client->eql()->getStatus([
    "id" => "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_eql/search/status/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE="
client.eql().getStatus(g -> g
    .id("FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=")
);
Response examples (200)
A successful response for getting status information for an async EQL search.
{
  "id": "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
  "is_running" : true,
  "is_partial" : true,
  "start_time_in_millis" : 1611690235000,
  "expiration_time_in_millis" : 1611690295000
}





























































































































































































































































































































































































































































































































































































































































































Get anomaly detection job stats Generally available; Added in 5.5.0

GET /_ml/anomaly_detectors/{job_id}/_stats

All methods and paths for this operation:

GET /_ml/anomaly_detectors/_stats

GET /_ml/anomaly_detectors/{job_id}/_stats

Required authorization

  • Cluster privileges: monitor_ml

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job. It can be a job identifier, a group name, a comma-separated list of jobs, or a wildcard expression. If you do not specify one of these options, the API returns information for all anomaly detection jobs.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    1. Contains wildcard expressions and there are no jobs that match.
    2. Contains the _all string or no identifiers and there are no matches.
    3. Contains wildcard expressions and there are only partial matches.

    If true, the API returns an empty jobs array when there are no matches and the subset of results when there are partial matches. If false, the API returns a 404 status code when there are no matches or only partial matches.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • count number Required
    • jobs array[object] Required
      Hide jobs attributes Show jobs attributes object
      • assignment_explanation string

        For open anomaly detection jobs only, contains messages relating to the selection of a node to run the job.

      • data_counts object Required

        An object that describes the quantity of input to the job and any related error counts. The data_count values are cumulative for the lifetime of a job. If a model snapshot is reverted or old results are deleted, the job counts are not reset.

        Hide data_counts attributes Show data_counts attributes object
        • bucket_count number Required
        • earliest_record_timestamp number
        • empty_bucket_count number Required
        • input_bytes number Required
        • input_field_count number Required
        • input_record_count number Required
        • invalid_date_count number Required
        • job_id string Required
        • last_data_time number
        • latest_empty_bucket_timestamp number
        • latest_record_timestamp number
        • latest_sparse_bucket_timestamp number
        • latest_bucket_timestamp number
        • log_time number
        • missing_field_count number Required
        • out_of_order_timestamp_count number Required
        • processed_field_count number Required
        • processed_record_count number Required
        • sparse_bucket_count number Required
      • forecasts_stats object Required

        An object that provides statistical information about forecasts belonging to this job. Some statistics are omitted if no forecasts have been made.

        Hide forecasts_stats attributes Show forecasts_stats attributes object
        • memory_bytes object
        • processing_time_ms object
        • records object
        • status object
          Hide status attribute Show status attribute object
          • * number Additional properties
        • total number Required
        • forecasted_jobs number Required
      • job_id string Required

        Identifier for the anomaly detection job.

      • model_size_stats object Required

        An object that provides information about the size and contents of the model.

        Hide model_size_stats attributes Show model_size_stats attributes object
        • bucket_allocation_failures_count number Required
        • job_id string Required
        • log_time
        • memory_status string Required

          Values are ok, soft_limit, or hard_limit.

        • model_bytes
        • model_bytes_exceeded
        • model_bytes_memory_limit
        • output_memory_allocator_bytes
        • peak_model_bytes
        • assignment_memory_basis string
        • result_type string Required
        • total_by_field_count number Required
        • total_over_field_count number Required
        • total_partition_field_count number Required
        • categorization_status string Required

          Values are ok or warn.

        • categorized_doc_count number Required
        • dead_category_count number Required
        • failed_category_count number Required
        • frequent_category_count number Required
        • rare_category_count number Required
        • total_category_count number Required
        • timestamp number
      • node object

        Contains properties for the node that runs the job. This information is available only for open jobs.

        Hide node attributes Show node attributes object
        • name string Required
        • ephemeral_id string Required
        • id string Required
        • transport_address string Required
        • attributes object Required
          Hide attributes attribute Show attributes attribute object
          • * string Additional properties
      • open_time string | number

        For open jobs only, the elapsed time for which the job has been open.

        One of:

        For open jobs only, the elapsed time for which the job has been open.

      • state string Required

        The status of the anomaly detection job, which can be one of the following values: closed, closing, failed, opened, opening.

        Supported values include:

        • closing: The job close action is in progress and has not yet completed. A closing job cannot accept further data.
        • closed: The job finished successfully with its model state persisted. The job must be opened before it can accept further data.
        • opened: The job is available to receive and process data.
        • failed: The job did not finish successfully due to an error. This situation can occur due to invalid input data, a fatal error occurring during the analysis, or an external interaction such as the process being killed by the Linux out of memory (OOM) killer. If the job had irrevocably failed, it must be force closed and then deleted. If the datafeed can be corrected, the job can be closed and then re-opened.
        • opening: The job open action is in progress and has not yet completed.

        Values are closing, closed, opened, failed, or opening.

      • timing_stats object Required

        An object that provides statistical information about timing aspect of this job.

        Hide timing_stats attributes Show timing_stats attributes object
        • bucket_count number Required
        • job_id string Required
      • deleting boolean

        Indicates that the process of deleting the job is in progress but not yet completed. It is only reported when true.

GET /_ml/anomaly_detectors/{job_id}/_stats
GET _ml/anomaly_detectors/low_request_rate/_stats
resp = client.ml.get_job_stats(
    job_id="low_request_rate",
)
const response = await client.ml.getJobStats({
  job_id: "low_request_rate",
});
response = client.ml.get_job_stats(
  job_id: "low_request_rate"
)
$resp = $client->ml()->getJobStats([
    "job_id" => "low_request_rate",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_ml/anomaly_detectors/low_request_rate/_stats"
client.ml().getJobStats(g -> g
    .jobId("low_request_rate")
);













































































































































































































































































































































































































Get searchable snapshot statistics Generally available; Added in 7.10.0

GET /{index}/_searchable_snapshots/stats

All methods and paths for this operation:

GET /_searchable_snapshots/stats

GET /{index}/_searchable_snapshots/stats

Required authorization

  • Index privileges: manage
  • Cluster privileges: manage

Path parameters

  • index string | array[string] Required

    A comma-separated list of data streams and indices to retrieve statistics for.

Query parameters

  • level string

    Return stats aggregated at cluster, index or shard level

    Values are cluster, indices, or shards.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • stats object Required
    • total object Required
GET /{index}/_searchable_snapshots/stats
GET /my-index/_searchable_snapshots/stats
resp = client.searchable_snapshots.stats(
    index="my-index",
)
const response = await client.searchableSnapshots.stats({
  index: "my-index",
});
response = client.searchable_snapshots.stats(
  index: "my-index"
)
$resp = $client->searchableSnapshots()->stats([
    "index" => "my-index",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/my-index/_searchable_snapshots/stats"
client.searchableSnapshots().stats(s -> s
    .index("my-index")
);





















































Invalidate API keys Generally available; Added in 6.7.0

DELETE /_security/api_key

This API invalidates API keys created by the create API key or grant API key APIs. Invalidated API keys fail authentication, but they can still be viewed using the get API key information and query API key information APIs, for at least the configured retention period, until they are automatically deleted.

To use this API, you must have at least the manage_security, manage_api_key, or manage_own_api_key cluster privileges. The manage_security privilege allows deleting any API key, including both REST and cross cluster API keys. The manage_api_key privilege allows deleting any REST API key, but not cross cluster API keys. The manage_own_api_key only allows deleting REST API keys that are owned by the user. In addition, with the manage_own_api_key privilege, an invalidation request must be issued in one of the three formats:

  • Set the parameter owner=true.
  • Or, set both username and realm_name to match the user's identity.
  • Or, if the request is issued by an API key, that is to say an API key invalidates itself, specify its ID in the ids field.

Required authorization

  • Cluster privileges: manage_api_key,manage_own_api_key
application/json

Body Required

  • id string
  • ids array[string]

    A list of API key ids. This parameter cannot be used with any of name, realm_name, or username.

  • name string

    An API key name. This parameter cannot be used with any of ids, realm_name or username.

  • owner boolean

    Query API keys owned by the currently authenticated user. The realm_name or username parameters cannot be specified when this parameter is set to true as they are assumed to be the currently authenticated ones.

    NOTE: At least one of ids, name, username, and realm_name must be specified if owner is false.

    Default value is false.

  • realm_name string

    The name of an authentication realm. This parameter cannot be used with either ids or name, or when owner flag is set to true.

  • username string

    The username of a user. This parameter cannot be used with either ids or name or when owner flag is set to true.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • error_count number Required

      The number of errors that were encountered when invalidating the API keys.

    • error_details array[object]

      Details about the errors. This field is not present in the response when error_count is 0.

      Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      Hide error_details attributes Show error_details attributes object
      • type string Required

        The type of error

      • reason string | null

        A human-readable explanation of the error, in English.

      • stack_trace string

        The server stack trace. Present only if the error_trace=true parameter was sent with the request.

      • caused_by object

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • root_cause array[object]

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • suppressed array[object]

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

    • invalidated_api_keys array[string] Required

      The IDs of the API keys that were invalidated as part of this request.

    • previously_invalidated_api_keys array[string] Required

      The IDs of the API keys that were already invalidated.

DELETE /_security/api_key
DELETE /_security/api_key
{
  "ids" : [ "VuaCfGcBCdbkQm-e5aOx" ]
}
resp = client.security.invalidate_api_key(
    ids=[
        "VuaCfGcBCdbkQm-e5aOx"
    ],
)
const response = await client.security.invalidateApiKey({
  ids: ["VuaCfGcBCdbkQm-e5aOx"],
});
response = client.security.invalidate_api_key(
  body: {
    "ids": [
      "VuaCfGcBCdbkQm-e5aOx"
    ]
  }
)
$resp = $client->security()->invalidateApiKey([
    "body" => [
        "ids" => array(
            "VuaCfGcBCdbkQm-e5aOx",
        ),
    ],
]);
curl -X DELETE -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"ids":["VuaCfGcBCdbkQm-e5aOx"]}' "$ELASTICSEARCH_URL/_security/api_key"
client.security().invalidateApiKey(i -> i
    .ids("VuaCfGcBCdbkQm-e5aOx")
);
Request examples
Run `DELETE /_security/api_key` to invalidate the API keys identified by ID.
{
  "ids" : [ "VuaCfGcBCdbkQm-e5aOx" ]
}
Run `DELETE /_security/api_key` to invalidate the API keys identified by name.
{
  "name" : "my-api-key"
}
Run `DELETE /_security/api_key` to invalidate all API keys for the `native1` realm.
{
  "realm_name" : "native1"
}
Run `DELETE /_security/api_key` to invalidate all API keys for the user `myuser` in all realms.
{
  "username" : "myuser"
}
Run `DELETE /_security/api_key` to invalidate the API keys identified by ID if they are owned by the currently authenticated user.
{
  "ids" : ["VuaCfGcBCdbkQm-e5aOx"],
  "owner" : "true"
}
Run `DELETE /_security/api_key` to invalidate all API keys for the user `myuser` in the `native1` realm .
{
  "username" : "myuser",
  "realm_name" : "native1"
}
Response examples (200)
A successful response from `DELETE /_security/api_key`.
{
  "invalidated_api_keys": [ 
    "api-key-id-1"
  ],
  "previously_invalidated_api_keys": [ 
    "api-key-id-2",
    "api-key-id-3"
  ],
  "error_count": 2, 
  "error_details": [ 
    {
      "type": "exception",
      "reason": "error occurred while invalidating api keys",
      "caused_by": {
        "type": "illegal_argument_exception",
        "reason": "invalid api key id"
      }
    },
    {
      "type": "exception",
      "reason": "error occurred while invalidating api keys",
      "caused_by": {
        "type": "illegal_argument_exception",
        "reason": "invalid api key id"
      }
    }
  ]
}








































































































Get a token Generally available; Added in 5.5.0

POST /_security/oauth2/token

Create a bearer token for access without requiring basic authentication. The tokens are created by the Elasticsearch Token Service, which is automatically enabled when you configure TLS on the HTTP interface. Alternatively, you can explicitly enable the xpack.security.authc.token.enabled setting. When you are running in production mode, a bootstrap check prevents you from enabling the token service unless you also enable TLS on the HTTP interface.

The get token API takes the same parameters as a typical OAuth 2.0 token API except for the use of a JSON request body.

A successful get token API call returns a JSON structure that contains the access token, the amount of time (seconds) that the token expires in, the type, and the scope if available.

The tokens returned by the get token API have a finite period of time for which they are valid and after that time period, they can no longer be used. That time period is defined by the xpack.security.authc.token.timeout setting. If you want to invalidate a token immediately, you can do so by using the invalidate token API.

Required authorization

  • Cluster privileges: manage_token
External documentation
application/json

Body Required

  • grant_type string

    The type of grant. Supported grant types are: password, _kerberos, client_credentials, and refresh_token.

    Supported values include:

    • password: This grant type implements the Resource Owner Password Credentials Grant of OAuth2. In this grant, a trusted client exchanges the end user's credentials for an access token and (possibly) a refresh token. The request needs to be made by an authenticated user but happens on behalf of another authenticated user (the one whose credentials are passed as request parameters). This grant type is not suitable or designed for the self-service user creation of tokens.
    • client_credentials: This grant type implements the Client Credentials Grant of OAuth2. It is geared for machine to machine communication and is not suitable or designed for the self-service user creation of tokens. It generates only access tokens that cannot be refreshed. The premise is that the entity that uses client_credentials has constant access to a set of (client, not end-user) credentials and can authenticate itself at will.
    • _kerberos: This grant type is supported internally and implements SPNEGO based Kerberos support. The _kerberos grant type may change from version to version.
    • refresh_token: This grant type implements the Refresh Token Grant of OAuth2. In this grant a user exchanges a previously issued refresh token for a new access token and a new refresh token.

    Values are password, client_credentials, _kerberos, or refresh_token.

  • scope string

    The scope of the token. Currently tokens are only issued for a scope of FULL regardless of the value sent with the request.

  • password string

    The user's password. If you specify the password grant type, this parameter is required. This parameter is not valid with any other supported grant type.

  • kerberos_ticket string

    The base64 encoded kerberos ticket. If you specify the _kerberos grant type, this parameter is required. This parameter is not valid with any other supported grant type.

  • refresh_token string

    The string that was returned when you created the token, which enables you to extend its life. If you specify the refresh_token grant type, this parameter is required. This parameter is not valid with any other supported grant type.

  • username string

    The username that identifies the user. If you specify the password grant type, this parameter is required. This parameter is not valid with any other supported grant type.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • access_token string Required
    • expires_in number Required
    • scope string
    • type string Required
    • refresh_token string
    • kerberos_authentication_response_token string
    • authentication object Additional properties
      Hide authentication attributes Show authentication attributes object
      • full_name string | null

      • metadata object Required
        Hide metadata attribute Show metadata attribute object
        • * object Additional properties
      • roles array[string] Required
      • username string Required
      • enabled boolean Required
      • profile_uid string
      • authentication_realm object Required
        Hide authentication_realm attribute Show authentication_realm attribute object
        • type string Required
      • lookup_realm object Required
        Hide lookup_realm attribute Show lookup_realm attribute object
        • type string Required
      • authentication_provider object
        Hide authentication_provider attribute Show authentication_provider attribute object
        • type string Required
      • authentication_type string Required
POST /_security/oauth2/token
POST /_security/oauth2/token
{
  "grant_type" : "client_credentials"
}
resp = client.security.get_token(
    grant_type="client_credentials",
)
const response = await client.security.getToken({
  grant_type: "client_credentials",
});
response = client.security.get_token(
  body: {
    "grant_type": "client_credentials"
  }
)
$resp = $client->security()->getToken([
    "body" => [
        "grant_type" => "client_credentials",
    ],
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"grant_type":"client_credentials"}' "$ELASTICSEARCH_URL/_security/oauth2/token"
client.security().getToken(g -> g
    .grantType(AccessTokenGrantType.ClientCredentials)
);
Request examples
Run `POST /_security/oauth2/token` to obtain a token using the `client_credentials` grant type, which simply creates a token as the authenticated user.
{
  "grant_type" : "client_credentials"
}
Run `POST /_security/oauth2/token` to obtain a token for the `test_admin` user using the password grant type. This request needs to be made by an authenticated user with sufficient privileges that may or may not be the same as the one whose username is passed in the `username` parameter.
{
  "grant_type" : "password",
  "username" : "test_admin",
  "password" : "x-pack-test-password"
}
Response examples (200)
A successful response from `POST /_security/oauth2/token`.
{
  "access_token" : "dGhpcyBpcyBub3QgYSByZWFsIHRva2VuIGJ1dCBpdCBpcyBvbmx5IHRlc3QgZGF0YS4gZG8gbm90IHRyeSB0byByZWFkIHRva2VuIQ==",
  "type" : "Bearer",
  "expires_in" : 1200,
  "authentication" : {
    "username" : "test_admin",
    "roles" : [
      "superuser"
    ],
    "full_name" : null,
    "email" : null,
    "metadata" : { },
    "enabled" : true,
    "authentication_realm" : {
      "name" : "file",
      "type" : "file"
    },
    "lookup_realm" : {
      "name" : "file",
      "type" : "file"
    },
    "authentication_type" : "realm"
  }
}
A successful response from `POST /_security/oauth2/token`.
{
  "access_token" : "dGhpcyBpcyBub3QgYSByZWFsIHRva2VuIGJ1dCBpdCBpcyBvbmx5IHRlc3QgZGF0YS4gZG8gbm90IHRyeSB0byByZWFkIHRva2VuIQ==",
  "type" : "Bearer",
  "expires_in" : 1200,
  "authentication" : {
    "username" : "test_admin",
    "roles" : [
      "superuser"
    ],
    "full_name" : null,
    "email" : null,
    "metadata" : { },
    "enabled" : true,
    "authentication_realm" : {
      "name" : "file",
      "type" : "file"
    },
    "lookup_realm" : {
      "name" : "file",
      "type" : "file"
    },
    "authentication_type" : "realm"
  }
}

































































































































Analyze a snapshot repository Generally available; Added in 7.12.0

POST /_snapshot/{repository}/_analyze

Analyze the performance characteristics and any incorrect behaviour found in a repository.

The response exposes implementation details of the analysis which may change from version to version. The response body format is therefore not considered stable and may be different in newer versions.

There are a large number of third-party storage systems available, not all of which are suitable for use as a snapshot repository by Elasticsearch. Some storage systems behave incorrectly, or perform poorly, especially when accessed concurrently by multiple clients as the nodes of an Elasticsearch cluster do. This API performs a collection of read and write operations on your repository which are designed to detect incorrect behaviour and to measure the performance characteristics of your storage system.

The default values for the parameters are deliberately low to reduce the impact of running an analysis inadvertently and to provide a sensible starting point for your investigations. Run your first analysis with the default parameter values to check for simple problems. If successful, run a sequence of increasingly large analyses until you encounter a failure or you reach a blob_count of at least 2000, a max_blob_size of at least 2gb, a max_total_data_size of at least 1tb, and a register_operation_count of at least 100. Always specify a generous timeout, possibly 1h or longer, to allow time for each analysis to run to completion. Perform the analyses using a multi-node cluster of a similar size to your production cluster so that it can detect any problems that only arise when the repository is accessed by many nodes at once.

If the analysis fails, Elasticsearch detected that your repository behaved unexpectedly. This usually means you are using a third-party storage system with an incorrect or incompatible implementation of the API it claims to support. If so, this storage system is not suitable for use as a snapshot repository. You will need to work with the supplier of your storage system to address the incompatibilities that Elasticsearch detects.

If the analysis is successful, the API returns details of the testing process, optionally including how long each operation took. You can use this information to determine the performance of your storage system. If any operation fails or returns an incorrect result, the API returns an error. If the API returns an error, it may not have removed all the data it wrote to the repository. The error will indicate the location of any leftover data and this path is also recorded in the Elasticsearch logs. You should verify that this location has been cleaned up correctly. If there is still leftover data at the specified location, you should manually remove it.

If the connection from your client to Elasticsearch is closed while the client is waiting for the result of the analysis, the test is cancelled. Some clients are configured to close their connection if no response is received within a certain timeout. An analysis takes a long time to complete so you might need to relax any such client-side timeouts. On cancellation the analysis attempts to clean up the data it was writing, but it may not be able to remove it all. The path to the leftover data is recorded in the Elasticsearch logs. You should verify that this location has been cleaned up correctly. If there is still leftover data at the specified location, you should manually remove it.

If the analysis is successful then it detected no incorrect behaviour, but this does not mean that correct behaviour is guaranteed. The analysis attempts to detect common bugs but it does not offer 100% coverage. Additionally, it does not test the following:

  • Your repository must perform durable writes. Once a blob has been written it must remain in place until it is deleted, even after a power loss or similar disaster.
  • Your repository must not suffer from silent data corruption. Once a blob has been written, its contents must remain unchanged until it is deliberately modified or deleted.
  • Your repository must behave correctly even if connectivity from the cluster is disrupted. Reads and writes may fail in this case, but they must not return incorrect results.

IMPORTANT: An analysis writes a substantial amount of data to your repository and then reads it back again. This consumes bandwidth on the network between the cluster and the repository, and storage space and I/O bandwidth on the repository itself. You must ensure this load does not affect other users of these systems. Analyses respect the repository settings max_snapshot_bytes_per_sec and max_restore_bytes_per_sec if available and the cluster setting indices.recovery.max_bytes_per_sec which you can use to limit the bandwidth they consume.

NOTE: This API is intended for exploratory use by humans. You should expect the request parameters and the response format to vary in future versions.

NOTE: Different versions of Elasticsearch may perform different checks for repository compatibility, with newer versions typically being stricter than older ones. A storage system that passes repository analysis with one version of Elasticsearch may fail with a different version. This indicates it behaves incorrectly in ways that the former version did not detect. You must work with the supplier of your storage system to address the incompatibilities detected by the repository analysis API in any version of Elasticsearch.

NOTE: This API may not work correctly in a mixed-version cluster.

Implementation details

NOTE: This section of documentation describes how the repository analysis API works in this version of Elasticsearch, but you should expect the implementation to vary between versions. The request parameters and response format depend on details of the implementation so may also be different in newer versions.

The analysis comprises a number of blob-level tasks, as set by the blob_count parameter and a number of compare-and-exchange operations on linearizable registers, as set by the register_operation_count parameter. These tasks are distributed over the data and master-eligible nodes in the cluster for execution.

For most blob-level tasks, the executing node first writes a blob to the repository and then instructs some of the other nodes in the cluster to attempt to read the data it just wrote. The size of the blob is chosen randomly, according to the max_blob_size and max_total_data_size parameters. If any of these reads fails then the repository does not implement the necessary read-after-write semantics that Elasticsearch requires.

For some blob-level tasks, the executing node will instruct some of its peers to attempt to read the data before the writing process completes. These reads are permitted to fail, but must not return partial data. If any read returns partial data then the repository does not implement the necessary atomicity semantics that Elasticsearch requires.

For some blob-level tasks, the executing node will overwrite the blob while its peers are reading it. In this case the data read may come from either the original or the overwritten blob, but the read operation must not return partial data or a mix of data from the two blobs. If any of these reads returns partial data or a mix of the two blobs then the repository does not implement the necessary atomicity semantics that Elasticsearch requires for overwrites.

The executing node will use a variety of different methods to write the blob. For instance, where applicable, it will use both single-part and multi-part uploads. Similarly, the reading nodes will use a variety of different methods to read the data back again. For instance they may read the entire blob from start to end or may read only a subset of the data.

For some blob-level tasks, the executing node will cancel the write before it is complete. In this case, it still instructs some of the other nodes in the cluster to attempt to read the blob but all of these reads must fail to find the blob.

Linearizable registers are special blobs that Elasticsearch manipulates using an atomic compare-and-exchange operation. This operation ensures correct and strongly-consistent behavior even when the blob is accessed by multiple nodes at the same time. The detailed implementation of the compare-and-exchange operation on linearizable registers varies by repository type. Repository analysis verifies that that uncontended compare-and-exchange operations on a linearizable register blob always succeed. Repository analysis also verifies that contended operations either succeed or report the contention but do not return incorrect results. If an operation fails due to contention, Elasticsearch retries the operation until it succeeds. Most of the compare-and-exchange operations performed by repository analysis atomically increment a counter which is represented as an 8-byte blob. Some operations also verify the behavior on small blobs with sizes other than 8 bytes.

Required authorization

  • Cluster privileges: manage

Path parameters

  • repository string Required

    The name of the repository.

Query parameters

  • blob_count number

    The total number of blobs to write to the repository during the test. For realistic experiments, you should set it to at least 2000.

  • concurrency number

    The number of operations to run concurrently during the test.

  • detailed boolean

    Indicates whether to return detailed results, including timing information for every operation performed during the analysis. If false, it returns only a summary of the analysis.

  • early_read_node_count number

    The number of nodes on which to perform an early read operation while writing each blob. Early read operations are only rarely performed.

  • max_blob_size number | string

    The maximum size of a blob to be written during the test. For realistic experiments, you should set it to at least 2gb.

  • max_total_data_size number | string

    An upper limit on the total size of all the blobs written during the test. For realistic experiments, you should set it to at least 1tb.

  • rare_action_probability number

    The probability of performing a rare action such as an early read, an overwrite, or an aborted write on each blob.

  • rarely_abort_writes boolean

    Indicates whether to rarely cancel writes before they complete.

  • read_node_count number

    The number of nodes on which to read a blob after writing.

  • register_operation_count number

    The minimum number of linearizable register operations to perform in total. For realistic experiments, you should set it to at least 100.

  • seed number

    The seed for the pseudo-random number generator used to generate the list of operations performed during the test. To repeat the same set of operations in multiple experiments, use the same seed in each experiment. Note that the operations are performed concurrently so might not always happen in the same order on each run.

  • timeout string

    The period of time to wait for the test to complete. If no response is received before the timeout expires, the test is cancelled and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • blob_count number Required

      The number of blobs written to the repository during the test.

    • blob_path string Required

      The path in the repository under which all the blobs were written during the test.

    • concurrency number Required

      The number of write operations performed concurrently during the test.

    • coordinating_node object Required

      The node that coordinated the analysis and performed the final cleanup.

      Hide coordinating_node attributes Show coordinating_node attributes object
      • id string Required
      • name string Required
    • delete_elapsed string Required

      The time it took to delete all the blobs in the container.

    • delete_elapsed_nanos number

      Time unit for nanoseconds

    • details object Required

      A description of every read and write operation performed during the test.

      Hide details attributes Show details attributes object
      • blob object Required

        A description of the blob that was written and read.

        Hide blob attributes Show blob attributes object
        • name string Required

          The name of the blob.

        • overwritten boolean Required

          Indicates whether the blob was overwritten while the read operations were ongoing. /**

        • read_early boolean Required
        • read_end number Required

          The position, in bytes, at which read operations completed.

        • read_start number Required

          The position, in bytes, at which read operations started.

        • reads object Required

          A description of every read operation performed on the blob.

        • size
        • size_bytes number Required

          The size of the blob in bytes.

      • overwrite_elapsed string

        The elapsed time spent overwriting the blob. If the blob was not overwritten, this information is omitted.

      • Time unit for nanoseconds

      • write_elapsed string Required

        The elapsed time spent writing the blob.

      • Time unit for nanoseconds

      • write_throttled string Required

        The length of time spent waiting for the max_snapshot_bytes_per_sec (or indices.recovery.max_bytes_per_sec if the recovery settings for managed services are set) throttle while writing the blob.

      • Time unit for nanoseconds

      • writer_node object Required

        The node which wrote the blob and coordinated the read operations.

        Hide writer_node attributes Show writer_node attributes object
        • id string Required
        • name string Required
    • early_read_node_count number Required

      The limit on the number of nodes on which early read operations were performed after writing each blob.

    • issues_detected array[string] Required

      A list of correctness issues detected, which is empty if the API succeeded. It is included to emphasize that a successful response does not guarantee correct behaviour in future.

    • listing_elapsed string Required

      The time it took to retrieve a list of all the blobs in the container.

    • listing_elapsed_nanos number

      Time unit for nanoseconds

    • max_blob_size number | string

      The limit on the size of a blob written during the test.

      One of:

      The limit on the size of a blob written during the test.

    • max_blob_size_bytes number Required

      The limit, in bytes, on the size of a blob written during the test.

    • max_total_data_size number | string

      The limit on the total size of all blob written during the test.

      One of:

      The limit on the total size of all blob written during the test.

    • max_total_data_size_bytes number Required

      The limit, in bytes, on the total size of all blob written during the test.

    • rare_action_probability number Required

      The probability of performing rare actions during the test.

    • read_node_count number Required

      The limit on the number of nodes on which read operations were performed after writing each blob.

    • repository string Required

      The name of the repository that was the subject of the analysis.

    • seed number Required

      The seed for the pseudo-random number generator used to generate the operations used during the test.

    • summary object Required

      A collection of statistics that summarize the results of the test.

      Hide summary attributes Show summary attributes object
      • read object Required

        A collection of statistics that summarise the results of the read operations in the test.

        Hide read attributes Show read attributes object
        • count number Required

          The number of read operations performed in the test.

        • max_wait string Required

          The maximum time spent waiting for the first byte of any read request to be received.

        • total_elapsed string Required

          The total elapsed time spent on reading blobs in the test.

        • total_size
        • total_size_bytes number Required

          The total size of all the blobs or partial blobs read in the test, in bytes.

        • total_throttled string Required

          The total time spent waiting due to the max_restore_bytes_per_sec or indices.recovery.max_bytes_per_sec throttles.

        • total_wait string Required

          The total time spent waiting for the first byte of each read request to be received.

      • write object Required

        A collection of statistics that summarise the results of the write operations in the test.

        Hide write attributes Show write attributes object
        • count number Required

          The number of write operations performed in the test.

        • total_elapsed string Required

          The total elapsed time spent on writing blobs in the test.

        • total_size
        • total_size_bytes number Required

          The total size of all the blobs written in the test, in bytes.

        • total_throttled string Required

          The total time spent waiting due to the max_snapshot_bytes_per_sec throttle.

        • total_throttled_nanos number Required

          The total time spent waiting due to the max_snapshot_bytes_per_sec throttle, in nanoseconds.

POST /_snapshot/{repository}/_analyze
POST /_snapshot/my_repository/_analyze?blob_count=10&max_blob_size=1mb&timeout=120s
resp = client.snapshot.repository_analyze(
    name="my_repository",
    blob_count="10",
    max_blob_size="1mb",
    timeout="120s",
)
const response = await client.snapshot.repositoryAnalyze({
  name: "my_repository",
  blob_count: 10,
  max_blob_size: "1mb",
  timeout: "120s",
});
response = client.snapshot.repository_analyze(
  repository: "my_repository",
  blob_count: "10",
  max_blob_size: "1mb",
  timeout: "120s"
)
$resp = $client->snapshot()->repositoryAnalyze([
    "repository" => "my_repository",
    "blob_count" => "10",
    "max_blob_size" => "1mb",
    "timeout" => "120s",
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_snapshot/my_repository/_analyze?blob_count=10&max_blob_size=1mb&timeout=120s"
client.snapshot().repositoryAnalyze(r -> r
    .blobCount(10)
    .maxBlobSize("1mb")
    .name("my_repository")
    .timeout(t -> t
        .offset(120)
    )
);




Get the snapshot status Generally available; Added in 7.8.0

GET /_snapshot/{repository}/{snapshot}/_status

All methods and paths for this operation:

GET /_snapshot/_status

GET /_snapshot/{repository}/_status
GET /_snapshot/{repository}/{snapshot}/_status

Get a detailed description of the current state for each shard participating in the snapshot.

Note that this API should be used only to obtain detailed shard-level information for ongoing snapshots. If this detail is not needed or you want to obtain information about one or more existing snapshots, use the get snapshot API.

If you omit the <snapshot> request path parameter, the request retrieves information only for currently running snapshots. This usage is preferred. If needed, you can specify <repository> and <snapshot> to retrieve information for specific snapshots, even if they're not currently running.

WARNING: Using the API to return the status of any snapshots other than currently running snapshots can be expensive. The API requires a read from the repository for each shard in each snapshot. For example, if you have 100 snapshots with 1,000 shards each, an API request that includes all snapshots will require 100,000 reads (100 snapshots x 1,000 shards).

Depending on the latency of your storage, such requests can take an extremely long time to return results. These requests can also tax machine resources and, when using cloud storage, incur high processing costs.

Required authorization

  • Cluster privileges: monitor_snapshot

Path parameters

  • repository string Required

    The snapshot repository name used to limit the request. It supports wildcards (*) if <snapshot> isn't specified.

  • snapshot string | array[string] Required

    A comma-separated list of snapshots to retrieve status for. The default is currently running snapshots. Wildcards (*) are not supported.

Query parameters

  • ignore_unavailable boolean

    If false, the request returns an error for any snapshots that are unavailable. If true, the request ignores snapshots that are unavailable, such as those that are corrupted or temporarily cannot be returned.

  • master_timeout string

    The period to wait for the master node. If the master node is not available before the timeout expires, the request fails and returns an error. To indicate that the request should never timeout, set it to -1.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • snapshots array[object] Required
      Hide snapshots attributes Show snapshots attributes object
      • include_global_state boolean Required

        Indicates whether the current cluster state is included in the snapshot.

      • indices object Required
        Hide indices attribute Show indices attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • shards object Required
            Hide shards attribute Show shards attribute object
            • * object Additional properties
          • shards_stats object Required
          • stats object Required
      • repository string Required

        The name of the repository that includes the snapshot.

      • shards_stats object Required

        Statistics for the shards in the snapshot.

        Hide shards_stats attributes Show shards_stats attributes object
        • done number Required

          The number of shards that initialized, started, and finalized successfully.

        • failed number Required

          The number of shards that failed to be included in the snapshot.

        • finalizing number Required

          The number of shards that are finalizing but are not done.

        • initializing number Required

          The number of shards that are still initializing.

        • started number Required

          The number of shards that have started but are not finalized.

        • total number Required

          The total number of shards included in the snapshot.

      • snapshot string Required

        The name of the snapshot.

      • state string Required

        The current snapshot state:

        • FAILED: The snapshot finished with an error and failed to store any data.
        • STARTED: The snapshot is currently running.
        • SUCCESS: The snapshot completed.
      • stats object Required

        Details about the number (file_count) and size (size_in_bytes) of files included in the snapshot.

        Hide stats attributes Show stats attributes object
        • incremental object Required

          The number and size of files that still need to be copied as part of the incremental snapshot. For completed snapshots, this property indicates the number and size of files that were not already in the repository and were copied as part of the incremental snapshot.

        • time string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • total object Required

          The total number and size of files that are referenced by the snapshot.

      • uuid string Required

        The universally unique identifier (UUID) for the snapshot.

GET /_snapshot/{repository}/{snapshot}/_status
GET _snapshot/my_repository/snapshot_2/_status
resp = client.snapshot.status(
    repository="my_repository",
    snapshot="snapshot_2",
)
const response = await client.snapshot.status({
  repository: "my_repository",
  snapshot: "snapshot_2",
});
response = client.snapshot.status(
  repository: "my_repository",
  snapshot: "snapshot_2"
)
$resp = $client->snapshot()->status([
    "repository" => "my_repository",
    "snapshot" => "snapshot_2",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_snapshot/my_repository/snapshot_2/_status"
client.snapshot().status(s -> s
    .repository("my_repository")
    .snapshot("snapshot_2")
);
Response examples (200)
A successful response from `GET _snapshot/my_repository/snapshot_2/_status`. The response contains detailed status information for `snapshot_2` in the `my_repository` repository.
{
  "snapshots" : [
    {
      "snapshot" : "snapshot_2",
      "repository" : "my_repository",
      "uuid" : "lNeQD1SvTQCqqJUMQSwmGg",
      "state" : "SUCCESS",
      "include_global_state" : false,
      "shards_stats" : {
        "initializing" : 0,
        "started" : 0,
        "finalizing" : 0,
        "done" : 1,
        "failed" : 0,
        "total" : 1
      },
      "stats" : {
        "incremental" : {
          "file_count" : 3,
          "size_in_bytes" : 5969
        },
        "total" : {
          "file_count" : 4,
          "size_in_bytes" : 6024
        },
        "start_time_in_millis" : 1594829326691,
        "time_in_millis" : 205
      },
      "indices" : {
        "index_1" : {
          "shards_stats" : {
            "initializing" : 0,
            "started" : 0,
            "finalizing" : 0,
            "done" : 1,
            "failed" : 0,
            "total" : 1
          },
          "stats" : {
            "incremental" : {
              "file_count" : 3,
              "size_in_bytes" : 5969
            },
            "total" : {
              "file_count" : 4,
              "size_in_bytes" : 6024
            },
            "start_time_in_millis" : 1594829326896,
            "time_in_millis" : 0
          },
          "shards" : {
            "0" : {
              "stage" : "DONE",
              "stats" : {
                "incremental" : {
                  "file_count" : 3,
                  "size_in_bytes" : 5969
                },
                "total" : {
                  "file_count" : 4,
                  "size_in_bytes" : 6024
                },
                "start_time_in_millis" : 1594829326896,
                "time_in_millis" : 0
              }
            }
          }
        }
      }
    }
  ]
}





















Run a retention policy Generally available; Added in 7.5.0

POST /_slm/_execute_retention

Manually apply the retention policy to force immediate removal of snapshots that are expired according to the snapshot lifecycle policy retention rules. The retention policy is normally applied according to its schedule.

Required authorization

  • Cluster privileges: manage_slm

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

POST /_slm/_execute_retention
POST _slm/_execute_retention
resp = client.slm.execute_retention()
const response = await client.slm.executeRetention();
response = client.slm.execute_retention
$resp = $client->slm()->executeRetention();
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_slm/_execute_retention"
client.slm().executeRetention(e -> e);


















































Delete a synonym set Generally available; Added in 8.10.0

DELETE /_synonyms/{id}

You can only delete a synonyms set that is not in use by any index analyzer.

Synonyms sets can be used in synonym graph token filters and synonym token filters. These synonym filters can be used as part of search analyzers.

Analyzers need to be loaded when an index is restored (such as when a node starts, or the index becomes open). Even if the analyzer is not used on any field mapping, it still needs to be loaded on the index recovery phase.

If any analyzers cannot be loaded, the index becomes unavailable and the cluster status becomes red or yellow as index shards are not available. To prevent that, synonyms sets that are used in analyzers can't be deleted. A delete request in this case will return a 400 response code.

To remove a synonyms set, you must first remove all indices that contain analyzers using it. You can migrate an index by creating a new index that does not contain the token filter with the synonyms set, and use the reindex API in order to copy over the index data. Once finished, you can delete the index. When the synonyms set is not used in analyzers, you will be able to delete it.

Required authorization

  • Cluster privileges: manage_search_synonyms

Path parameters

  • id string Required

    The synonyms set identifier to delete.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

DELETE /_synonyms/{id}
DELETE _synonyms/my-synonyms-set
resp = client.synonyms.delete_synonym(
    id="my-synonyms-set",
)
const response = await client.synonyms.deleteSynonym({
  id: "my-synonyms-set",
});
response = client.synonyms.delete_synonym(
  id: "my-synonyms-set"
)
$resp = $client->synonyms()->deleteSynonym([
    "id" => "my-synonyms-set",
]);
curl -X DELETE -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_synonyms/my-synonyms-set"
client.synonyms().deleteSynonym(d -> d
    .id("my-synonyms-set")
);