Authentication

The API accepts 3 different authentication methods:

Api key auth (http_api_key)

Elasticsearch APIs support key-based authentication. You must create an API key and use the encoded value in the request header. For example:

curl -X GET "${ES_URL}/_cat/indices?v=true" \
  -H "Authorization: ApiKey ${API_KEY}"

To get API keys, use the /_security/api_key APIs.

Basic auth (http)

Basic auth tokens are constructed with the Basic keyword, followed by a space, followed by a base64-encoded string of your username:password (separated by a : colon).

Example: send a Authorization: Basic aGVsbG86aGVsbG8= HTTP header with your requests to authenticate with the API.

Bearer auth (http)

Elasticsearch APIs support the use of bearer tokens in the Authorization HTTP header to authenticate with the API. For examples, refer to Token-based authentication services























Get shard allocation information Generally available

GET /_cat/allocation/{node_id}

All methods and paths for this operation:

GET /_cat/allocation

GET /_cat/allocation/{node_id}

Get a snapshot of the number of shards allocated to each data node and their disk space.

IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications.

Required authorization

  • Cluster privileges: monitor

Path parameters

  • node_id string | array[string]

    A comma-separated list of node identifiers or names used to limit the returned information.

Query parameters

  • bytes string

    The unit used to display byte values.

    Values are b, kb, mb, gb, tb, or pb.

  • h string | array[string]

    A comma-separated list of columns names to display. It supports simple wildcards.

    Supported values include:

    • shards (or s): The number of shards on the node.
    • shards.undesired: The number of shards scheduled to be moved elsewhere in the cluster.
    • write_load.forecast (or wlf, writeLoadForecast): The sum of index write load forecasts.
    • disk.indices.forecast (or dif, diskIndicesForecast): The sum of shard size forecasts.
    • disk.indices (or di, diskIndices): The disk space used by Elasticsearch indices.
    • disk.used (or du, diskUsed): The total disk space used on the node.
    • disk.avail (or da, diskAvail): The available disk space on the node.
    • disk.total (or dt, diskTotal): The total disk capacity of all volumes on the node.
    • disk.percent (or dp, diskPercent): The percentage of disk space used on the node.
    • host (or h): IThe host of the node.
    • ip: The IP address of the node.
    • node (or n): The name of the node.
    • node.role (or r, role, nodeRole): The roles assigned to the node.

    Values are shards, s, shards.undesired, write_load.forecast, wlf, writeLoadForecast, disk.indices.forecast, dif, diskIndicesForecast, disk.indices, di, diskIndices, disk.used, du, diskUsed, disk.avail, da, diskAvail, disk.total, dt, diskTotal, disk.percent, dp, diskPercent, host, h, ip, node, n, node.role, r, role, or nodeRole.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

  • local boolean

    If true, the request computes the list of selected nodes from the local cluster state. If false the list of selected nodes are computed from the cluster state of the master node. In both cases the coordinating node will send requests for further information to each selected node.

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

Responses

GET /_cat/allocation/{node_id}
GET /_cat/allocation?v=true&format=json
resp = client.cat.allocation(
    v=True,
    format="json",
)
const response = await client.cat.allocation({
  v: "true",
  format: "json",
});
response = client.cat.allocation(
  v: "true",
  format: "json"
)
$resp = $client->cat()->allocation([
    "v" => "true",
    "format" => "json",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_cat/allocation?v=true&format=json"
client.cat().allocation();
Response examples (200)
A successful response from `GET /_cat/allocation?v=true&format=json`. It shows a single shard is allocated to the one node available.
[
  {
    "shards": "1",
    "shards.undesired": "0",
    "write_load.forecast": "0.0",
    "disk.indices.forecast": "260b",
    "disk.indices": "260b",
    "disk.used": "47.3gb",
    "disk.avail": "43.4gb",
    "disk.total": "100.7gb",
    "disk.percent": "46",
    "host": "127.0.0.1",
    "ip": "127.0.0.1",
    "node": "CSUXak2",
    "node.role": "himrst"
  }
]




















Get index information Generally available

GET /_cat/indices/{index}

All methods and paths for this operation:

GET /_cat/indices

GET /_cat/indices/{index}

Get high-level information about indices in a cluster, including backing indices for data streams.

Use this request to get the following information for each index in a cluster:

  • shard count
  • document count
  • deleted document count
  • primary store size
  • total store size of all shards, including shard replicas

These metrics are retrieved directly from Lucene, which Elasticsearch uses internally to power indexing and search. As a result, all document counts include hidden nested documents. To get an accurate count of Elasticsearch documents, use the cat count or count APIs.

CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use an index endpoint.

Required authorization

  • Index privileges: monitor
  • Cluster privileges: monitor

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.

Query parameters

  • bytes string

    The unit used to display byte values.

    Values are b, kb, mb, gb, tb, or pb.

  • expand_wildcards string | array[string]

    The type of index that wildcard patterns can match.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.

    Values are all, open, closed, hidden, or none.

  • health string

    The health status used to limit returned indices. By default, the response includes indices of any health status.

    Supported values include:

    • green (or GREEN): All shards are assigned.
    • yellow (or YELLOW): All primary shards are assigned, but one or more replica shards are unassigned. If a node in the cluster fails, some data could be unavailable until that node is repaired.
    • red (or RED): One or more primary shards are unassigned, so some data is unavailable. This can occur briefly during cluster startup as primary shards are assigned.
    • unknown
    • unavailable

    Values are green, GREEN, yellow, YELLOW, red, RED, unknown, or unavailable.

  • include_unloaded_segments boolean

    If true, the response includes information from segments that are not loaded into memory.

  • pri boolean

    If true, the response only includes information from primary shards.

  • time string

    The unit used to display time values.

    Values are nanos, micros, ms, s, m, h, or d.

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • health string

      current health status

    • status string

      open/close status

    • index string

      index name

    • uuid string

      index uuid

    • pri string

      number of primary shards

    • rep string

      number of replica shards

    • docs.count string | null

      available docs

    • docs.deleted string | null

      deleted docs

    • creation.date string

      index creation date (millisecond value)

    • creation.date.string string

      index creation date (as string)

    • store.size string | null

      store size of primaries & replicas

    • pri.store.size string | null

      store size of primaries

    • dataset.size string | null

      total size of dataset (including the cache for partially mounted indices)

    • completion.size string

      size of completion

    • pri.completion.size string

      size of completion

    • fielddata.memory_size string

      used fielddata cache

    • pri.fielddata.memory_size string

      used fielddata cache

    • fielddata.evictions string

      fielddata evictions

    • pri.fielddata.evictions string

      fielddata evictions

    • query_cache.memory_size string

      used query cache

    • pri.query_cache.memory_size string

      used query cache

    • query_cache.evictions string

      query cache evictions

    • pri.query_cache.evictions string

      query cache evictions

    • request_cache.memory_size string

      used request cache

    • pri.request_cache.memory_size string

      used request cache

    • request_cache.evictions string

      request cache evictions

    • pri.request_cache.evictions string

      request cache evictions

    • request_cache.hit_count string

      request cache hit count

    • pri.request_cache.hit_count string

      request cache hit count

    • request_cache.miss_count string

      request cache miss count

    • pri.request_cache.miss_count string

      request cache miss count

    • flush.total string

      number of flushes

    • pri.flush.total string

      number of flushes

    • flush.total_time string

      time spent in flush

    • pri.flush.total_time string

      time spent in flush

    • get.current string

      number of current get ops

    • pri.get.current string

      number of current get ops

    • get.time string

      time spent in get

    • pri.get.time string

      time spent in get

    • get.total string

      number of get ops

    • pri.get.total string

      number of get ops

    • get.exists_time string

      time spent in successful gets

    • pri.get.exists_time string

      time spent in successful gets

    • get.exists_total string

      number of successful gets

    • pri.get.exists_total string

      number of successful gets

    • get.missing_time string

      time spent in failed gets

    • pri.get.missing_time string

      time spent in failed gets

    • get.missing_total string

      number of failed gets

    • pri.get.missing_total string

      number of failed gets

    • indexing.delete_current string

      number of current deletions

    • pri.indexing.delete_current string

      number of current deletions

    • indexing.delete_time string

      time spent in deletions

    • pri.indexing.delete_time string

      time spent in deletions

    • indexing.delete_total string

      number of delete ops

    • pri.indexing.delete_total string

      number of delete ops

    • indexing.index_current string

      number of current indexing ops

    • pri.indexing.index_current string

      number of current indexing ops

    • indexing.index_time string

      time spent in indexing

    • pri.indexing.index_time string

      time spent in indexing

    • indexing.index_total string

      number of indexing ops

    • pri.indexing.index_total string

      number of indexing ops

    • indexing.index_failed string

      number of failed indexing ops

    • pri.indexing.index_failed string

      number of failed indexing ops

    • merges.current string

      number of current merges

    • pri.merges.current string

      number of current merges

    • merges.current_docs string

      number of current merging docs

    • pri.merges.current_docs string

      number of current merging docs

    • merges.current_size string

      size of current merges

    • pri.merges.current_size string

      size of current merges

    • merges.total string

      number of completed merge ops

    • pri.merges.total string

      number of completed merge ops

    • merges.total_docs string

      docs merged

    • pri.merges.total_docs string

      docs merged

    • merges.total_size string

      size merged

    • pri.merges.total_size string

      size merged

    • merges.total_time string

      time spent in merges

    • pri.merges.total_time string

      time spent in merges

    • refresh.total string

      total refreshes

    • pri.refresh.total string

      total refreshes

    • refresh.time string

      time spent in refreshes

    • pri.refresh.time string

      time spent in refreshes

    • refresh.external_total string

      total external refreshes

    • pri.refresh.external_total string

      total external refreshes

    • refresh.external_time string

      time spent in external refreshes

    • pri.refresh.external_time string

      time spent in external refreshes

    • refresh.listeners string

      number of pending refresh listeners

    • pri.refresh.listeners string

      number of pending refresh listeners

    • search.fetch_current string

      current fetch phase ops

    • pri.search.fetch_current string

      current fetch phase ops

    • search.fetch_time string

      time spent in fetch phase

    • pri.search.fetch_time string

      time spent in fetch phase

    • search.fetch_total string

      total fetch ops

    • pri.search.fetch_total string

      total fetch ops

    • search.open_contexts string

      open search contexts

    • pri.search.open_contexts string

      open search contexts

    • search.query_current string

      current query phase ops

    • pri.search.query_current string

      current query phase ops

    • search.query_time string

      time spent in query phase

    • pri.search.query_time string

      time spent in query phase

    • search.query_total string

      total query phase ops

    • pri.search.query_total string

      total query phase ops

    • search.scroll_current string

      open scroll contexts

    • pri.search.scroll_current string

      open scroll contexts

    • search.scroll_time string

      time scroll contexts held open

    • pri.search.scroll_time string

      time scroll contexts held open

    • search.scroll_total string

      completed scroll contexts

    • pri.search.scroll_total string

      completed scroll contexts

    • segments.count string

      number of segments

    • pri.segments.count string

      number of segments

    • segments.memory string

      memory used by segments

    • pri.segments.memory string

      memory used by segments

    • segments.index_writer_memory string

      memory used by index writer

    • pri.segments.index_writer_memory string

      memory used by index writer

    • segments.version_map_memory string

      memory used by version map

    • pri.segments.version_map_memory string

      memory used by version map

    • segments.fixed_bitset_memory string

      memory used by fixed bit sets for nested object field types and export type filters for types referred in _parent fields

    • pri.segments.fixed_bitset_memory string

      memory used by fixed bit sets for nested object field types and export type filters for types referred in _parent fields

    • warmer.current string

      current warmer ops

    • pri.warmer.current string

      current warmer ops

    • warmer.total string

      total warmer ops

    • pri.warmer.total string

      total warmer ops

    • warmer.total_time string

      time spent in warmers

    • pri.warmer.total_time string

      time spent in warmers

    • suggest.current string

      number of current suggest ops

    • pri.suggest.current string

      number of current suggest ops

    • suggest.time string

      time spend in suggest

    • pri.suggest.time string

      time spend in suggest

    • suggest.total string

      number of suggest ops

    • pri.suggest.total string

      number of suggest ops

    • memory.total string

      total used memory

    • pri.memory.total string

      total user memory

    • search.throttled string

      indicates if the index is search throttled

    • bulk.total_operations string

      number of bulk shard ops

    • pri.bulk.total_operations string

      number of bulk shard ops

    • bulk.total_time string

      time spend in shard bulk

    • pri.bulk.total_time string

      time spend in shard bulk

    • bulk.total_size_in_bytes string

      total size in bytes of shard bulk

    • pri.bulk.total_size_in_bytes string

      total size in bytes of shard bulk

    • bulk.avg_time string

      average time spend in shard bulk

    • pri.bulk.avg_time string

      average time spend in shard bulk

    • bulk.avg_size_in_bytes string

      average size in bytes of shard bulk

    • pri.bulk.avg_size_in_bytes string

      average size in bytes of shard bulk

GET /_cat/indices/{index}
GET /_cat/indices/my-index-*?v=true&s=index&format=json
resp = client.cat.indices(
    index="my-index-*",
    v=True,
    s="index",
    format="json",
)
const response = await client.cat.indices({
  index: "my-index-*",
  v: "true",
  s: "index",
  format: "json",
});
response = client.cat.indices(
  index: "my-index-*",
  v: "true",
  s: "index",
  format: "json"
)
$resp = $client->cat()->indices([
    "index" => "my-index-*",
    "v" => "true",
    "s" => "index",
    "format" => "json",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_cat/indices/my-index-*?v=true&s=index&format=json"
client.cat().indices();
Response examples (200)
A successful response from `GET /_cat/indices/my-index-*?v=true&s=index&format=json`.
[
  {
    "health": "yellow",
    "status": "open",
    "index": "my-index-000001",
    "uuid": "u8FNjxh8Rfy_awN11oDKYQ",
    "pri": "1",
    "rep": "1",
    "docs.count": "1200",
    "docs.deleted": "0",
    "store.size": "88.1kb",
    "pri.store.size": "88.1kb",
    "dataset.size": "88.1kb"
  },
  {
    "health": "green",
    "status": "open",
    "index": "my-index-000002",
    "uuid": "nYFWZEO7TUiOjLQXBaYJpA ",
    "pri": "1",
    "rep": "0",
    "docs.count": "0",
    "docs.deleted": "0",
    "store.size": "260b",
    "pri.store.size": "260b",
    "dataset.size": "260b"
  }
]








































































Cluster

















































Ping the cluster Generally available

HEAD /

Get information about whether the cluster is running.

Responses

  • 200 application/json
HEAD /
curl \
 --request HEAD 'http://api.example.com/' \
 --header "Authorization: $API_KEY"

Clear the archived repositories metering Technical preview; Added in 7.16.0

DELETE /_nodes/{node_id}/_repositories_metering/{max_archive_version}

Clear the archived repositories metering information in the cluster.

Required authorization

  • Cluster privileges: monitor,manage

Path parameters

  • node_id string | array[string] Required

    Comma-separated list of node IDs or names used to limit returned information.

  • max_archive_version number Required

    Specifies the maximum archive_version to be cleared from the archive.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object

      Contains statistics about the number of nodes selected by the request’s node filters.

      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide failures attributes Show failures attributes object
        • type string Required

          The type of error

        • reason
        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by
        • root_cause array[object]
        • suppressed array[object]
      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string Required

      Name of the cluster. Based on the cluster.name setting.

    • nodes object Required

      Contains repositories metering information for the nodes selected by the request.

      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • repository_name string Required

          Repository name.

        • repository_type string Required

          Repository type.

        • repository_location object Required

          Represents an unique location within the repository.

          Hide repository_location attributes Show repository_location attributes object
          • base_path string Required
          • container string

            Container name (Azure)

          • bucket string

            Bucket name (GCP, S3)

        • repository_ephemeral_id string Required

          An identifier that changes every time the repository is updated.

        • Time unit for milliseconds

        • Time unit for milliseconds

        • archived boolean Required

          A flag that tells whether or not this object has been archived. When a repository is closed or updated the repository metering information is archived and kept for a certain period of time. This allows retrieving the repository metering information of previous repository instantiations.

        • cluster_version number

          The cluster state version when this object was archived, this field can be used as a logical timestamp to delete all the archived metrics up to an observed version. This field is only present for archived repository metering information objects. The main purpose of this field is to avoid possible race conditions during repository metering information deletions, i.e. deleting archived repositories metering information that we haven’t observed yet.

        • request_counts object Required

          An object with the number of request performed against the repository grouped by request type.

          Hide request_counts attributes Show request_counts attributes object
          • GetBlobProperties number

            Number of Get Blob Properties requests (Azure)

          • GetBlob number

            Number of Get Blob requests (Azure)

          • ListBlobs number

            Number of List Blobs requests (Azure)

          • PutBlob number

            Number of Put Blob requests (Azure)

          • PutBlock number

            Number of Put Block (Azure)

          • PutBlockList number

            Number of Put Block List requests

          • GetObject number

            Number of get object requests (GCP, S3)

          • ListObjects number

            Number of list objects requests (GCP, S3)

          • InsertObject number

            Number of insert object requests, including simple, multipart and resumable uploads. Resumable uploads can perform multiple http requests to insert a single object but they are considered as a single request since they are billed as an individual operation. (GCP)

          • PutObject number

            Number of PutObject requests (S3)

          • PutMultipartObject number

            Number of Multipart requests, including CreateMultipartUpload, UploadPart and CompleteMultipartUpload requests (S3)

DELETE /_nodes/{node_id}/_repositories_metering/{max_archive_version}
curl \
 --request DELETE 'http://api.example.com/_nodes/{node_id}/_repositories_metering/{max_archive_version}' \
 --header "Authorization: $API_KEY"

Get cluster repositories metering Technical preview; Added in 7.16.0

GET /_nodes/{node_id}/_repositories_metering

Get repositories metering information for a cluster. This API exposes monotonically non-decreasing counters and it is expected that clients would durably store the information needed to compute aggregations over a period of time. Additionally, the information exposed by this API is volatile, meaning that it will not be present after node restarts.

Required authorization

  • Cluster privileges: monitor,manage

Path parameters

  • node_id string | array[string] Required

    Comma-separated list of node IDs or names used to limit returned information. For more information about the nodes selective options, refer to the node specification documentation.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object

      Contains statistics about the number of nodes selected by the request’s node filters.

      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide failures attributes Show failures attributes object
        • type string Required

          The type of error

        • reason
        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by
        • root_cause array[object]
        • suppressed array[object]
      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string Required

      Name of the cluster. Based on the cluster.name setting.

    • nodes object Required

      Contains repositories metering information for the nodes selected by the request.

      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • repository_name string Required

          Repository name.

        • repository_type string Required

          Repository type.

        • repository_location object Required

          Represents an unique location within the repository.

          Hide repository_location attributes Show repository_location attributes object
          • base_path string Required
          • container string

            Container name (Azure)

          • bucket string

            Bucket name (GCP, S3)

        • repository_ephemeral_id string Required

          An identifier that changes every time the repository is updated.

        • Time unit for milliseconds

        • Time unit for milliseconds

        • archived boolean Required

          A flag that tells whether or not this object has been archived. When a repository is closed or updated the repository metering information is archived and kept for a certain period of time. This allows retrieving the repository metering information of previous repository instantiations.

        • cluster_version number

          The cluster state version when this object was archived, this field can be used as a logical timestamp to delete all the archived metrics up to an observed version. This field is only present for archived repository metering information objects. The main purpose of this field is to avoid possible race conditions during repository metering information deletions, i.e. deleting archived repositories metering information that we haven’t observed yet.

        • request_counts object Required

          An object with the number of request performed against the repository grouped by request type.

          Hide request_counts attributes Show request_counts attributes object
          • GetBlobProperties number

            Number of Get Blob Properties requests (Azure)

          • GetBlob number

            Number of Get Blob requests (Azure)

          • ListBlobs number

            Number of List Blobs requests (Azure)

          • PutBlob number

            Number of Put Blob requests (Azure)

          • PutBlock number

            Number of Put Block (Azure)

          • PutBlockList number

            Number of Put Block List requests

          • GetObject number

            Number of get object requests (GCP, S3)

          • ListObjects number

            Number of list objects requests (GCP, S3)

          • InsertObject number

            Number of insert object requests, including simple, multipart and resumable uploads. Resumable uploads can perform multiple http requests to insert a single object but they are considered as a single request since they are billed as an individual operation. (GCP)

          • PutObject number

            Number of PutObject requests (S3)

          • PutMultipartObject number

            Number of Multipart requests, including CreateMultipartUpload, UploadPart and CompleteMultipartUpload requests (S3)

GET /_nodes/{node_id}/_repositories_metering
curl \
 --request GET 'http://api.example.com/_nodes/{node_id}/_repositories_metering' \
 --header "Authorization: $API_KEY"




















Cluster - Health


















Delete a connector Beta; Added in 8.12.0

DELETE /_connector/{connector_id}

Removes a connector and associated sync jobs. This is a destructive action that is not recoverable. NOTE: This action doesn’t delete any API keys, ingest pipelines, or data indices associated with the connector. These need to be removed manually.

Path parameters

  • connector_id string Required

    The unique identifier of the connector to be deleted

Query parameters

  • delete_sync_jobs boolean

    A flag indicating if associated sync jobs should be also removed. Defaults to false.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

DELETE /_connector/{connector_id}
DELETE _connector/my-connector-id&delete_sync_jobs=true
resp = client.connector.delete(
    connector_id="my-connector-id&delete_sync_jobs=true",
)
const response = await client.connector.delete({
  connector_id: "my-connector-id&delete_sync_jobs=true",
});
response = client.connector.delete(
  connector_id: "my-connector-id&delete_sync_jobs=true"
)
$resp = $client->connector()->delete([
    "connector_id" => "my-connector-id&delete_sync_jobs=true",
]);
curl -X DELETE -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_connector/my-connector-id&delete_sync_jobs=true"
client.connector().delete(d -> d
    .connectorId("my-connector-id&delete_sync_jobs=true")
);
Response examples (200)
{
    "acknowledged": true
}




Create a connector Beta; Added in 8.12.0

POST /_connector

Connectors are Elasticsearch integrations that bring content from third-party data sources, which can be deployed on Elastic Cloud or hosted on your own infrastructure. Elastic managed connectors (Native connectors) are a managed service on Elastic Cloud. Self-managed connectors (Connector clients) are self-managed on your infrastructure.

application/json

Body

  • description string
  • index_name string
  • is_native boolean
  • language string
  • name string
  • service_type string

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.

    • id string Required
POST /_connector
curl \
 --request POST 'http://api.example.com/_connector' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"description":"string","index_name":"string","is_native":true,"language":"string","name":"string","service_type":"string"}'




Check in a connector sync job Technical preview

PUT /_connector/_sync_job/{connector_sync_job_id}/_check_in

Check in a connector sync job and set the last_seen field to the current time before updating it in the internal index.

To sync data using self-managed connectors, you need to deploy the Elastic connector service on your own infrastructure. This service runs automatically on Elastic Cloud for Elastic managed connectors.

Path parameters

  • connector_sync_job_id string Required

    The unique identifier of the connector sync job to be checked in.

Responses

  • 200 application/json
PUT /_connector/_sync_job/{connector_sync_job_id}/_check_in
PUT _connector/_sync_job/my-connector-sync-job/_check_in
resp = client.connector.sync_job_check_in(
    connector_sync_job_id="my-connector-sync-job",
)
const response = await client.connector.syncJobCheckIn({
  connector_sync_job_id: "my-connector-sync-job",
});
response = client.connector.sync_job_check_in(
  connector_sync_job_id: "my-connector-sync-job"
)
$resp = $client->connector()->syncJobCheckIn([
    "connector_sync_job_id" => "my-connector-sync-job",
]);
curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_connector/_sync_job/my-connector-sync-job/_check_in"
client.connector().syncJobCheckIn(s -> s
    .connectorSyncJobId("my-connector-sync-job")
);
































Update the connector API key ID Beta; Added in 8.12.0

PUT /_connector/{connector_id}/_api_key_id

Update the api_key_id and api_key_secret_id fields of a connector. You can specify the ID of the API key used for authorization and the ID of the connector secret where the API key is stored. The connector secret ID is required only for Elastic managed (native) connectors. Self-managed connectors (connector clients) do not use this field.

Path parameters

  • connector_id string Required

    The unique identifier of the connector to be updated

application/json

Body Required

  • api_key_id string
  • api_key_secret_id string

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.

PUT /_connector/{connector_id}/_api_key_id
PUT _connector/my-connector/_api_key_id
{
    "api_key_id": "my-api-key-id",
    "api_key_secret_id": "my-connector-secret-id"
}
resp = client.connector.update_api_key_id(
    connector_id="my-connector",
    api_key_id="my-api-key-id",
    api_key_secret_id="my-connector-secret-id",
)
const response = await client.connector.updateApiKeyId({
  connector_id: "my-connector",
  api_key_id: "my-api-key-id",
  api_key_secret_id: "my-connector-secret-id",
});
response = client.connector.update_api_key_id(
  connector_id: "my-connector",
  body: {
    "api_key_id": "my-api-key-id",
    "api_key_secret_id": "my-connector-secret-id"
  }
)
$resp = $client->connector()->updateApiKeyId([
    "connector_id" => "my-connector",
    "body" => [
        "api_key_id" => "my-api-key-id",
        "api_key_secret_id" => "my-connector-secret-id",
    ],
]);
curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"api_key_id":"my-api-key-id","api_key_secret_id":"my-connector-secret-id"}' "$ELASTICSEARCH_URL/_connector/my-connector/_api_key_id"
client.connector().updateApiKeyId(u -> u
    .apiKeyId("my-api-key-id")
    .apiKeySecretId("my-connector-secret-id")
    .connectorId("my-connector")
);
Request example
{
    "api_key_id": "my-api-key-id",
    "api_key_secret_id": "my-connector-secret-id"
}
Response examples (200)
{
  "result": "updated"
}
























Update the connector name and description Beta; Added in 8.12.0

PUT /_connector/{connector_id}/_name

Path parameters

  • connector_id string Required

    The unique identifier of the connector to be updated

application/json

Body Required

  • name string
  • description string

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.

PUT /_connector/{connector_id}/_name
PUT _connector/my-connector/_name
{
    "name": "Custom connector",
    "description": "This is my customized connector"
}
resp = client.connector.update_name(
    connector_id="my-connector",
    name="Custom connector",
    description="This is my customized connector",
)
const response = await client.connector.updateName({
  connector_id: "my-connector",
  name: "Custom connector",
  description: "This is my customized connector",
});
response = client.connector.update_name(
  connector_id: "my-connector",
  body: {
    "name": "Custom connector",
    "description": "This is my customized connector"
  }
)
$resp = $client->connector()->updateName([
    "connector_id" => "my-connector",
    "body" => [
        "name" => "Custom connector",
        "description" => "This is my customized connector",
    ],
]);
curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"name":"Custom connector","description":"This is my customized connector"}' "$ELASTICSEARCH_URL/_connector/my-connector/_name"
client.connector().updateName(u -> u
    .connectorId("my-connector")
    .description("This is my customized connector")
    .name("Custom connector")
);
Request example
{
    "name": "Custom connector",
    "description": "This is my customized connector"
}
Response examples (200)
{
  "result": "updated"
}








Update the connector scheduling Beta; Added in 8.12.0

PUT /_connector/{connector_id}/_scheduling

Path parameters

  • connector_id string Required

    The unique identifier of the connector to be updated

application/json

Body Required

  • scheduling object Required
    Hide scheduling attributes Show scheduling attributes object
    • access_control object
      Hide access_control attributes Show access_control attributes object
      • enabled boolean Required
      • interval string Required

        The interval is expressed using the crontab syntax

    • full object
      Hide full attributes Show full attributes object
      • enabled boolean Required
      • interval string Required

        The interval is expressed using the crontab syntax

    • incremental object
      Hide incremental attributes Show incremental attributes object
      • enabled boolean Required
      • interval string Required

        The interval is expressed using the crontab syntax

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.

PUT /_connector/{connector_id}/_scheduling
PUT _connector/my-connector/_scheduling
{
    "scheduling": {
        "access_control": {
            "enabled": true,
            "interval": "0 10 0 * * ?"
        },
        "full": {
            "enabled": true,
            "interval": "0 20 0 * * ?"
        },
        "incremental": {
            "enabled": false,
            "interval": "0 30 0 * * ?"
        }
    }
}
resp = client.connector.update_scheduling(
    connector_id="my-connector",
    scheduling={
        "access_control": {
            "enabled": True,
            "interval": "0 10 0 * * ?"
        },
        "full": {
            "enabled": True,
            "interval": "0 20 0 * * ?"
        },
        "incremental": {
            "enabled": False,
            "interval": "0 30 0 * * ?"
        }
    },
)
const response = await client.connector.updateScheduling({
  connector_id: "my-connector",
  scheduling: {
    access_control: {
      enabled: true,
      interval: "0 10 0 * * ?",
    },
    full: {
      enabled: true,
      interval: "0 20 0 * * ?",
    },
    incremental: {
      enabled: false,
      interval: "0 30 0 * * ?",
    },
  },
});
response = client.connector.update_scheduling(
  connector_id: "my-connector",
  body: {
    "scheduling": {
      "access_control": {
        "enabled": true,
        "interval": "0 10 0 * * ?"
      },
      "full": {
        "enabled": true,
        "interval": "0 20 0 * * ?"
      },
      "incremental": {
        "enabled": false,
        "interval": "0 30 0 * * ?"
      }
    }
  }
)
$resp = $client->connector()->updateScheduling([
    "connector_id" => "my-connector",
    "body" => [
        "scheduling" => [
            "access_control" => [
                "enabled" => true,
                "interval" => "0 10 0 * * ?",
            ],
            "full" => [
                "enabled" => true,
                "interval" => "0 20 0 * * ?",
            ],
            "incremental" => [
                "enabled" => false,
                "interval" => "0 30 0 * * ?",
            ],
        ],
    ],
]);
curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"scheduling":{"access_control":{"enabled":true,"interval":"0 10 0 * * ?"},"full":{"enabled":true,"interval":"0 20 0 * * ?"},"incremental":{"enabled":false,"interval":"0 30 0 * * ?"}}}' "$ELASTICSEARCH_URL/_connector/my-connector/_scheduling"
client.connector().updateScheduling(u -> u
    .connectorId("my-connector")
    .scheduling(s -> s
        .accessControl(a -> a
            .enabled(true)
            .interval("0 10 0 * * ?")
        )
        .full(f -> f
            .enabled(true)
            .interval("0 20 0 * * ?")
        )
        .incremental(i -> i
            .enabled(false)
            .interval("0 30 0 * * ?")
        )
    )
);
Request examples
{
    "scheduling": {
        "access_control": {
            "enabled": true,
            "interval": "0 10 0 * * ?"
        },
        "full": {
            "enabled": true,
            "interval": "0 20 0 * * ?"
        },
        "incremental": {
            "enabled": false,
            "interval": "0 30 0 * * ?"
        }
    }
}
{
    "scheduling": {
        "full": {
            "enabled": true,
            "interval": "0 10 0 * * ?"
        }
    }
}
Response examples (200)
{
  "result": "updated"
}









































Pause a follower Generally available; Added in 6.5.0

POST /{index}/_ccr/pause_follow

Pause a cross-cluster replication follower index. The follower index will not fetch any additional operations from the leader index. You can resume following with the resume follower API. You can pause and resume a follower index to change the configuration of the following task.

Required authorization

  • Cluster privileges: manage_ccr

Path parameters

  • index string Required

    The name of the follower index.

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

POST /{index}/_ccr/pause_follow
POST /follower_index/_ccr/pause_follow
resp = client.ccr.pause_follow(
    index="follower_index",
)
const response = await client.ccr.pauseFollow({
  index: "follower_index",
});
response = client.ccr.pause_follow(
  index: "follower_index"
)
$resp = $client->ccr()->pauseFollow([
    "index" => "follower_index",
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/follower_index/_ccr/pause_follow"
client.ccr().pauseFollow(p -> p
    .index("follower_index")
);
Response examples (200)
A successful response from `POST /follower_index/_ccr/pause_follow`, which pauses a follower index.
{
  "acknowledged" : true
}

































Get data stream lifecycles Generally available; Added in 8.11.0

GET /_data_stream/{name}/_lifecycle

Get the data stream lifecycle configuration of one or more data streams.

External documentation

Path parameters

  • name string | array[string] Required

    Comma-separated list of data streams to limit the request. Supports wildcards (*). To target all data streams, omit this parameter or use * or _all.

Query parameters

  • expand_wildcards string | array[string]

    Type of data stream that wildcard patterns can match. Supports comma-separated values, such as open,hidden.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.

    Values are all, open, closed, hidden, or none.

  • include_defaults boolean

    If true, return all default settings in the response.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • data_streams array[object] Required
      Hide data_streams attributes Show data_streams attributes object
      • name string Required
      • lifecycle object

        Data stream lifecycle with rollover can be used to display the configuration including the default rollover conditions, if asked.

        Hide lifecycle attribute Show lifecycle attribute object
        • enabled boolean

          If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle that's disabled (enabled: false) will have no effect on the data stream.

          Default value is true.

GET /_data_stream/{name}/_lifecycle
GET /_data_stream/{name}/_lifecycle?human&pretty
resp = client.indices.get_data_lifecycle(
    name="{name}",
    human=True,
    pretty=True,
)
const response = await client.indices.getDataLifecycle({
  name: "{name}",
  human: "true",
  pretty: "true",
});
response = client.indices.get_data_lifecycle(
  name: "{name}",
  human: "true",
  pretty: "true"
)
$resp = $client->indices()->getDataLifecycle([
    "name" => "{name}",
    "human" => "true",
    "pretty" => "true",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_data_stream/%7Bname%7D/_lifecycle?human&pretty"
Response examples (200)
A successful response from `GET /_data_stream/{name}/_lifecycle?human&pretty`.
{
  "data_streams": [
    {
      "name": "my-data-stream-1",
      "lifecycle": {
        "enabled": true,
        "data_retention": "7d"
      }
    },
    {
      "name": "my-data-stream-2",
      "lifecycle": {
        "enabled": true,
        "data_retention": "7d"
      }
    }
  ]
}




Downsample an index Technical preview; Added in 8.5.0

POST /{index}/_downsample/{target_index}

Aggregate a time series (TSDS) index and store pre-computed statistical summaries (min, max, sum, value_count and avg) for each metric field grouped by a configured time interval. For example, a TSDS index that contains metrics sampled every 10 seconds can be downsampled to an hourly index. All documents within an hour interval are summarized and stored as a single document in the downsample index.

NOTE: Only indices in a time series data stream are supported. Neither field nor document level security can be defined on the source index. The source index must be read only (index.blocks.write: true).

Path parameters

  • index string Required

    Name of the time series index to downsample.

  • target_index string Required

    Name of the index to create.

application/json

Body Required

  • fixed_interval string Required

    The interval at which to aggregate the original time series index.

Responses

  • 200 application/json
POST /{index}/_downsample/{target_index}
POST /my-time-series-index/_downsample/my-downsampled-time-series-index
{
  "fixed_interval": "1d"
}
resp = client.indices.downsample(
    index="my-time-series-index",
    target_index="my-downsampled-time-series-index",
    config={
        "fixed_interval": "1d"
    },
)
const response = await client.indices.downsample({
  index: "my-time-series-index",
  target_index: "my-downsampled-time-series-index",
  config: {
    fixed_interval: "1d",
  },
});
response = client.indices.downsample(
  index: "my-time-series-index",
  target_index: "my-downsampled-time-series-index",
  body: {
    "fixed_interval": "1d"
  }
)
$resp = $client->indices()->downsample([
    "index" => "my-time-series-index",
    "target_index" => "my-downsampled-time-series-index",
    "body" => [
        "fixed_interval" => "1d",
    ],
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"fixed_interval":"1d"}' "$ELASTICSEARCH_URL/my-time-series-index/_downsample/my-downsampled-time-series-index"
client.indices().downsample(d -> d
    .index("my-time-series-index")
    .targetIndex("my-downsampled-time-series-index")
    .config(c -> c
        .fixedInterval(f -> f
            .time("1d")
        )
    )
);
Request example
{
  "fixed_interval": "1d"
}

Get the status for a data stream lifecycle Generally available; Added in 8.11.0

GET /{index}/_lifecycle/explain

Get information about an index or data stream's current data stream lifecycle status, such as time since index creation, time since rollover, the lifecycle configuration managing the index, or any errors encountered during lifecycle execution.

External documentation

Path parameters

  • index string | array[string] Required

    The name of the index to explain

Query parameters

  • include_defaults boolean

    indicates if the API should return the default values the system uses for the index's lifecycle

  • master_timeout string

    Specify timeout for connection to master

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • indices object Required
      Hide indices attribute Show indices attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • index string Required
        • managed_by_lifecycle boolean Required
        • index_creation_date_millis number

          Time unit for milliseconds

        • time_since_index_creation string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • rollover_date_millis number

          Time unit for milliseconds

        • time_since_rollover string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • lifecycle object

          Data stream lifecycle with rollover can be used to display the configuration including the default rollover conditions, if asked.

          Hide lifecycle attribute Show lifecycle attribute object
          • enabled boolean

            If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle that's disabled (enabled: false) will have no effect on the data stream.

            Default value is true.

        • generation_time string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • error string
GET /{index}/_lifecycle/explain
GET .ds-metrics-2023.03.22-000001/_lifecycle/explain
resp = client.indices.explain_data_lifecycle(
    index=".ds-metrics-2023.03.22-000001",
)
const response = await client.indices.explainDataLifecycle({
  index: ".ds-metrics-2023.03.22-000001",
});
response = client.indices.explain_data_lifecycle(
  index: ".ds-metrics-2023.03.22-000001"
)
$resp = $client->indices()->explainDataLifecycle([
    "index" => ".ds-metrics-2023.03.22-000001",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/.ds-metrics-2023.03.22-000001/_lifecycle/explain"
client.indices().explainDataLifecycle(e -> e
    .index(".ds-metrics-2023.03.22-000001")
);
Response examples (200)
A successful response from `GET .ds-metrics-2023.03.22-000001/_lifecycle/explain`, which retrieves the lifecycle status for a data stream backing index. If the index is managed by a data stream lifecycle, the API will show the `managed_by_lifecycle` field set to `true` and the rest of the response will contain information about the lifecycle execution status for this index.
{
  "indices": {
    ".ds-metrics-2023.03.22-000001": {
      "index" : ".ds-metrics-2023.03.22-000001",
      "managed_by_lifecycle" : true,
      "index_creation_date_millis" : 1679475563571,
      "time_since_index_creation" : "843ms",
      "rollover_date_millis" : 1679475564293,
      "time_since_rollover" : "121ms",
      "lifecycle" : { },
      "generation_time" : "121ms"
  }
}
The API reports any errors related to the lifecycle execution for the target index.
{
  "indices": {
    ".ds-metrics-2023.03.22-000001": {
      "index" : ".ds-metrics-2023.03.22-000001",
      "managed_by_lifecycle" : true,
      "index_creation_date_millis" : 1679475563571,
      "time_since_index_creation" : "843ms",
      "lifecycle" : {
        "enabled": true
      },
      "error": "{\"type\":\"validation_exception\",\"reason\":\"Validation Failed: 1: this action would add [2] shards, but this cluster
currently has [4]/[3] maximum normal shards open;\"}"
  }
}








Update data streams Generally available; Added in 7.16.0

POST /_data_stream/_modify

Performs one or more data stream modification actions in a single atomic operation.

application/json

Body Required

  • actions array[object] Required

    Actions to perform.

    Hide actions attributes Show actions attributes object
    • add_backing_index object

      Adds an existing index as a backing index for a data stream. The index is hidden as part of this operation. WARNING: Adding indices with the add_backing_index action can potentially result in improper data stream behavior. This should be considered an expert level API.

      Hide add_backing_index attributes Show add_backing_index attributes object
      • data_stream string Required

        Data stream targeted by the action.

      • index string Required

        Index for the action.

    • remove_backing_index object

      Removes a backing index from a data stream. The index is unhidden as part of this operation. A data stream’s write index cannot be removed.

      Hide remove_backing_index attributes Show remove_backing_index attributes object
      • data_stream string Required

        Data stream targeted by the action.

      • index string Required

        Index for the action.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

POST /_data_stream/_modify
POST _data_stream/_modify
{
  "actions": [
    {
      "remove_backing_index": {
        "data_stream": "my-data-stream",
        "index": ".ds-my-data-stream-2023.07.26-000001"
      }
    },
    {
      "add_backing_index": {
        "data_stream": "my-data-stream",
        "index": ".ds-my-data-stream-2023.07.26-000001-downsample"
      }
    }
  ]
}
resp = client.indices.modify_data_stream(
    actions=[
        {
            "remove_backing_index": {
                "data_stream": "my-data-stream",
                "index": ".ds-my-data-stream-2023.07.26-000001"
            }
        },
        {
            "add_backing_index": {
                "data_stream": "my-data-stream",
                "index": ".ds-my-data-stream-2023.07.26-000001-downsample"
            }
        }
    ],
)
const response = await client.indices.modifyDataStream({
  actions: [
    {
      remove_backing_index: {
        data_stream: "my-data-stream",
        index: ".ds-my-data-stream-2023.07.26-000001",
      },
    },
    {
      add_backing_index: {
        data_stream: "my-data-stream",
        index: ".ds-my-data-stream-2023.07.26-000001-downsample",
      },
    },
  ],
});
response = client.indices.modify_data_stream(
  body: {
    "actions": [
      {
        "remove_backing_index": {
          "data_stream": "my-data-stream",
          "index": ".ds-my-data-stream-2023.07.26-000001"
        }
      },
      {
        "add_backing_index": {
          "data_stream": "my-data-stream",
          "index": ".ds-my-data-stream-2023.07.26-000001-downsample"
        }
      }
    ]
  }
)
$resp = $client->indices()->modifyDataStream([
    "body" => [
        "actions" => array(
            [
                "remove_backing_index" => [
                    "data_stream" => "my-data-stream",
                    "index" => ".ds-my-data-stream-2023.07.26-000001",
                ],
            ],
            [
                "add_backing_index" => [
                    "data_stream" => "my-data-stream",
                    "index" => ".ds-my-data-stream-2023.07.26-000001-downsample",
                ],
            ],
        ),
    ],
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"actions":[{"remove_backing_index":{"data_stream":"my-data-stream","index":".ds-my-data-stream-2023.07.26-000001"}},{"add_backing_index":{"data_stream":"my-data-stream","index":".ds-my-data-stream-2023.07.26-000001-downsample"}}]}' "$ELASTICSEARCH_URL/_data_stream/_modify"
client.indices().modifyDataStream(m -> m
    .actions(List.of(Action.of(a -> a
            .removeBackingIndex(r -> r
                .dataStream("my-data-stream")
                .index(".ds-my-data-stream-2023.07.26-000001")
        )),Action.of(ac -> ac
            .addBackingIndex(ad -> ad
                .dataStream("my-data-stream")
                .index(".ds-my-data-stream-2023.07.26-000001-downsample")
        ))))
);
Request example
An example body for a `POST _data_stream/_modify` request.
{
  "actions": [
    {
      "remove_backing_index": {
        "data_stream": "my-data-stream",
        "index": ".ds-my-data-stream-2023.07.26-000001"
      }
    },
    {
      "add_backing_index": {
        "data_stream": "my-data-stream",
        "index": ".ds-my-data-stream-2023.07.26-000001-downsample"
      }
    }
  ]
}

























Check a document Generally available

HEAD /{index}/_doc/{id}

Verify that a document exists. For example, check to see if a document with the _id 0 exists:

HEAD my-index-000001/_doc/0

If the document exists, the API returns a status code of 200 - OK. If the document doesn’t exist, the API returns 404 - Not Found.

Versioning support

You can use the version parameter to check the document only if its current version is equal to the specified one.

Internally, Elasticsearch has marked the old document as deleted and added an entirely new document. The old version of the document doesn't disappear immediately, although you won't be able to access it. Elasticsearch cleans up deleted documents in the background as you continue to index more data.

Path parameters

  • index string Required

    A comma-separated list of data streams, indices, and aliases. It supports wildcards (*).

  • id string Required

    A unique document identifier.

Query parameters

  • preference string

    The node or shard the operation should be performed on. By default, the operation is randomized between the shard replicas.

    If it is set to _local, the operation will prefer to be run on a local allocated shard when possible. If it is set to a custom value, the value is used to guarantee that the same shards will be used for the same custom value. This can help with "jumping values" when hitting different shards in different refresh states. A sample value can be something like the web session ID or the user name.

  • realtime boolean

    If true, the request is real-time as opposed to near-real-time.

  • refresh boolean

    If true, the request refreshes the relevant shards before retrieving the document. Setting it to true should be done after careful thought and verification that this does not cause a heavy load on the system (and slow down indexing).

  • routing string

    A custom value used to route operations to a specific shard.

  • _source boolean | string | array[string]

    Indicates whether to return the _source field (true or false) or lists the fields to return.

  • _source_excludes string | array[string]

    A comma-separated list of source fields to exclude from the response. You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter. If the _source parameter is false, this parameter is ignored.

  • _source_includes string | array[string]

    A comma-separated list of source fields to include in the response. If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the _source_excludes query parameter. If the _source parameter is false, this parameter is ignored.

  • stored_fields string | array[string]

    A comma-separated list of stored fields to return as part of a hit. If no fields are specified, no stored fields are included in the response. If this field is specified, the _source parameter defaults to false.

  • version number

    Explicit version number for concurrency control. The specified version must match the current version of the document for the request to succeed.

  • version_type string

    The version type.

    Supported values include:

    • internal: Use internal versioning that starts at 1 and increments with each update or delete.
    • external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.
    • external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document. NOTE: The external_gte version type is meant for special use cases and should be used with care. If used incorrectly, it can result in loss of data.
    • force: This option is deprecated because it can cause primary and replica shards to diverge.

    Values are internal, external, external_gte, or force.

Responses

  • 200 application/json
HEAD /{index}/_doc/{id}
HEAD my-index-000001/_doc/0
resp = client.exists(
    index="my-index-000001",
    id="0",
)
const response = await client.exists({
  index: "my-index-000001",
  id: 0,
});
response = client.exists(
  index: "my-index-000001",
  id: "0"
)
$resp = $client->exists([
    "index" => "my-index-000001",
    "id" => "0",
]);
curl --head -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/my-index-000001/_doc/0"
client.exists(e -> e
    .id("0")
    .index("my-index-000001")
);








Get a document's source Generally available

GET /{index}/_source/{id}

Get the source of a document. For example:

GET my-index-000001/_source/1

You can use the source filtering parameters to control which parts of the _source are returned:

GET my-index-000001/_source/1/?_source_includes=*.id&_source_excludes=entities

Required authorization

  • Index privileges: read
External documentation

Path parameters

  • index string Required

    The name of the index that contains the document.

  • id string Required

    A unique document identifier.

Query parameters

  • preference string

    The node or shard the operation should be performed on. By default, the operation is randomized between the shard replicas.

  • realtime boolean

    If true, the request is real-time as opposed to near-real-time.

  • refresh boolean

    If true, the request refreshes the relevant shards before retrieving the document. Setting it to true should be done after careful thought and verification that this does not cause a heavy load on the system (and slow down indexing).

  • routing string

    A custom value used to route operations to a specific shard.

  • _source boolean | string | array[string]

    Indicates whether to return the _source field (true or false) or lists the fields to return.

  • _source_excludes string | array[string]

    A comma-separated list of source fields to exclude in the response.

  • _source_includes string | array[string]

    A comma-separated list of source fields to include in the response.

  • version number

    The version number for concurrency control. It must match the current version of the document for the request to succeed.

  • version_type string

    The version type.

    Supported values include:

    • internal: Use internal versioning that starts at 1 and increments with each update or delete.
    • external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.
    • external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document. NOTE: The external_gte version type is meant for special use cases and should be used with care. If used incorrectly, it can result in loss of data.
    • force: This option is deprecated because it can cause primary and replica shards to diverge.

    Values are internal, external, external_gte, or force.

Responses

  • 200 application/json
GET /{index}/_source/{id}
GET my-index-000001/_source/1
resp = client.get_source(
    index="my-index-000001",
    id="1",
)
const response = await client.getSource({
  index: "my-index-000001",
  id: 1,
});
response = client.get_source(
  index: "my-index-000001",
  id: "1"
)
$resp = $client->getSource([
    "index" => "my-index-000001",
    "id" => "1",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/my-index-000001/_source/1"
client.getSource(g -> g
    .id("1")
    .index("my-index-000001")
);




Get multiple documents Generally available; Added in 1.3.0

POST /{index}/_mget

All methods and paths for this operation:

GET /_mget

POST /_mget
GET /{index}/_mget
POST /{index}/_mget

Get multiple JSON documents by ID from one or more indices. If you specify an index in the request URI, you only need to specify the document IDs in the request body. To ensure fast responses, this multi get (mget) API responds with partial results if one or more shards fail.

Filter source fields

By default, the _source field is returned for every document (if stored). Use the _source and _source_include or source_exclude attributes to filter what fields are returned for a particular document. You can include the _source, _source_includes, and _source_excludes query parameters in the request URI to specify the defaults to use when there are no per-document instructions.

Get stored fields

Use the stored_fields attribute to specify the set of stored fields you want to retrieve. Any requested fields that are not stored are ignored. You can include the stored_fields query parameter in the request URI to specify the defaults to use when there are no per-document instructions.

Required authorization

  • Index privileges: read

Path parameters

  • index string Required

    Name of the index to retrieve documents from when ids are specified, or when a document in the docs array does not specify an index.

Query parameters

  • preference string

    Specifies the node or shard the operation should be performed on. Random by default.

  • realtime boolean

    If true, the request is real-time as opposed to near-real-time.

  • refresh boolean

    If true, the request refreshes relevant shards before retrieving documents.

  • routing string

    Custom value used to route operations to a specific shard.

  • _source boolean | string | array[string]

    True or false to return the _source field or not, or a list of fields to return.

  • _source_excludes string | array[string]

    A comma-separated list of source fields to exclude from the response. You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter.

  • _source_includes string | array[string]

    A comma-separated list of source fields to include in the response. If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the _source_excludes query parameter. If the _source parameter is false, this parameter is ignored.

  • stored_fields string | array[string]

    If true, retrieves the document fields stored in the index rather than the document _source.

application/json

Body Required

  • docs array[object]

    The documents you want to retrieve. Required if no index is specified in the request URI.

    Hide docs attributes Show docs attributes object
    • _id string Required

      The unique document ID.

    • _index string

      The index that contains the document.

    • routing string

      The key for the primary shard the document resides on. Required if routing is used during indexing.

    • _source boolean | object

      If false, excludes all _source fields.

      One of:

      If false, excludes all _source fields.

    • stored_fields string | array[string]

      The stored fields you want to retrieve.

    • version number
    • version_type string

      Supported values include:

      • internal: Use internal versioning that starts at 1 and increments with each update or delete.
      • external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.
      • external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document. NOTE: The external_gte version type is meant for special use cases and should be used with care. If used incorrectly, it can result in loss of data.
      • force: This option is deprecated because it can cause primary and replica shards to diverge.

      Values are internal, external, external_gte, or force.

  • ids string | array[string]

    The IDs of the documents you want to retrieve. Allowed when the index is specified in the request URI.

    One of:

    The IDs of the documents you want to retrieve. Allowed when the index is specified in the request URI.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • docs array[object] Required

      The response includes a docs array that contains the documents in the order specified in the request. The structure of the returned documents is similar to that returned by the get API. If there is a failure getting a particular document, the error is included in place of the document.

      One of:
      Hide attributes Show attributes
      • _index string Required

        The name of the index the document belongs to.

      • fields object

        If the stored_fields parameter is set to true and found is true, it contains the document fields stored in the index.

        Hide fields attribute Show fields attribute object
        • * object Additional properties
      • _ignored array[string]
      • found boolean Required

        Indicates whether the document exists.

      • _id string Required

        The unique identifier for the document.

      • _primary_term number

        The primary term assigned to the document for the indexing operation.

      • _routing string

        The explicit routing, if set.

      • _seq_no number

        The sequence number assigned to the document for the indexing operation. Sequence numbers are used to ensure an older version of a document doesn't overwrite a newer version.

      • _source object

        If found is true, it contains the document data formatted in JSON. If the _source parameter is set to false or the stored_fields parameter is set to true, it is excluded.

      • _version number

        The document version, which is ncremented each time the document is updated.

POST /{index}/_mget
GET /my-index-000001/_mget
{
  "docs": [
    {
      "_id": "1"
    },
    {
      "_id": "2"
    }
  ]
}
resp = client.mget(
    index="my-index-000001",
    docs=[
        {
            "_id": "1"
        },
        {
            "_id": "2"
        }
    ],
)
const response = await client.mget({
  index: "my-index-000001",
  docs: [
    {
      _id: "1",
    },
    {
      _id: "2",
    },
  ],
});
response = client.mget(
  index: "my-index-000001",
  body: {
    "docs": [
      {
        "_id": "1"
      },
      {
        "_id": "2"
      }
    ]
  }
)
$resp = $client->mget([
    "index" => "my-index-000001",
    "body" => [
        "docs" => array(
            [
                "_id" => "1",
            ],
            [
                "_id" => "2",
            ],
        ),
    ],
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"docs":[{"_id":"1"},{"_id":"2"}]}' "$ELASTICSEARCH_URL/my-index-000001/_mget"
client.mget(m -> m
    .docs(List.of(MultiGetOperation.of(mu -> mu
            .id("1")),MultiGetOperation.of(mu -> mu
            .id("2"))))
    .index("my-index-000001")
);
Request examples
Run `GET /my-index-000001/_mget`. When you specify an index in the request URI, only the document IDs are required in the request body.
{
  "docs": [
    {
      "_id": "1"
    },
    {
      "_id": "2"
    }
  ]
}
Run `GET /_mget`. This request sets `_source` to `false` for document 1 to exclude the source entirely. It retrieves `field3` and `field4` from document 2. It retrieves the `user` field from document 3 but filters out the `user.location` field.
{
  "docs": [
    {
      "_index": "test",
      "_id": "1",
      "_source": false
    },
    {
      "_index": "test",
      "_id": "2",
      "_source": [ "field3", "field4" ]
    },
    {
      "_index": "test",
      "_id": "3",
      "_source": {
        "include": [ "user" ],
        "exclude": [ "user.location" ]
      }
    }
  ]
}
Run `GET /_mget`. This request retrieves `field1` and `field2` from document 1 and `field3` and `field4` from document 2.
{
  "docs": [
    {
      "_index": "test",
      "_id": "1",
      "stored_fields": [ "field1", "field2" ]
    },
    {
      "_index": "test",
      "_id": "2",
      "stored_fields": [ "field3", "field4" ]
    }
  ]
}
Run `GET /_mget?routing=key1`. If routing is used during indexing, you need to specify the routing value to retrieve documents. This request fetches `test/_doc/2` from the shard corresponding to routing key `key1`. It fetches `test/_doc/1` from the shard corresponding to routing key `key2`.
{
  "docs": [
    {
      "_index": "test",
      "_id": "1",
      "routing": "key2"
    },
    {
      "_index": "test",
      "_id": "2"
    }
  ]
}


























































Get the async EQL status Generally available; Added in 7.9.0

GET /_eql/search/status/{id}

Get the current status for an async EQL search or a stored synchronous EQL search without returning results.

Path parameters

  • id string Required

    Identifier for the search.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • id string Required

      Identifier for the search.

    • is_partial boolean Required

      If true, the search request is still executing. If false, the search is completed.

    • is_running boolean Required

      If true, the response does not contain complete search results. This could be because either the search is still running (is_running status is false), or because it is already completed (is_running status is true) and results are partial due to failures or timeouts.

    • start_time_in_millis number

      Time unit for milliseconds

    • expiration_time_in_millis number

      Time unit for milliseconds

    • completion_status number

      For a completed search shows the http status code of the completed search.

GET /_eql/search/status/{id}
GET /_eql/search/status/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=
resp = client.eql.get_status(
    id="FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
)
const response = await client.eql.getStatus({
  id: "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
});
response = client.eql.get_status(
  id: "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE="
)
$resp = $client->eql()->getStatus([
    "id" => "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_eql/search/status/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE="
client.eql().getStatus(g -> g
    .id("FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=")
);
Response examples (200)
A successful response for getting status information for an async EQL search.
{
  "id": "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
  "is_running" : true,
  "is_partial" : true,
  "start_time_in_millis" : 1611690235000,
  "expiration_time_in_millis" : 1611690295000
}

Get EQL search results Generally available; Added in 7.9.0

POST /{index}/_eql/search

All methods and paths for this operation:

GET /{index}/_eql/search

POST /{index}/_eql/search

Returns search results for an Event Query Language (EQL) query. EQL assumes each document in a data stream or index corresponds to an event.

External documentation

Path parameters

  • index string | array[string] Required

    The name of the index to scope the operation

Query parameters

  • allow_no_indices boolean

    Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes _all string or when no indices have been specified)

  • allow_partial_search_results boolean

    If true, returns partial results if there are shard failures. If false, returns an error with no partial results.

  • allow_partial_sequence_results boolean

    If true, sequence queries will return partial results in case of shard failures. If false, they will return no results at all. This flag has effect only if allow_partial_search_results is true.

  • expand_wildcards string | array[string]

    Whether to expand wildcard expression to concrete indices that are open, closed or both.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.

    Values are all, open, closed, hidden, or none.

  • ccs_minimize_roundtrips boolean

    Indicates whether network round-trips should be minimized as part of cross-cluster search requests execution

  • ignore_unavailable boolean

    If true, missing or closed indices are not included in the response.

  • keep_alive string

    Period for which the search and its results are stored on the cluster.

    Values are -1 or 0.

  • keep_on_completion boolean

    If true, the search and its results are stored on the cluster.

  • wait_for_completion_timeout string

    Timeout duration to wait for the request to finish. Defaults to no timeout, meaning the request waits for complete search results.

    Values are -1 or 0.

application/json

Body Required

  • query string Required

    EQL query you wish to run.

  • case_sensitive boolean
  • event_category_field string

    Field containing the event classification, such as process, file, or network.

  • tiebreaker_field string

    Field used to sort hits with the same timestamp in ascending order

  • timestamp_field string

    Field containing event timestamp. Default "@timestamp"

  • fetch_size number

    Maximum number of events to search at a time for sequence queries.

  • filter object | array[object]

    Query, written in Query DSL, used to filter the events on which the EQL query runs.

    One of:

    An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

    External documentation
  • keep_alive string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • keep_on_completion boolean
  • wait_for_completion_timeout string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • allow_partial_search_results boolean

    Allow query execution also in case of shard failures. If true, the query will keep running and will return results based on the available shards. For sequences, the behavior can be further refined using allow_partial_sequence_results

    Default value is true.

  • allow_partial_sequence_results boolean

    This flag applies only to sequences and has effect only if allow_partial_search_results=true. If true, the sequence query will return results based on the available shards, ignoring the others. If false, the sequence query will return successfully, but will always have empty results.

    Default value is false.

  • size number

    For basic queries, the maximum number of matching events to return. Defaults to 10

  • fields object | array[object]

    Array of wildcard (*) patterns. The response returns values for field names matching these patterns in the fields property of each hit.

    One of:

    A reference to a field with formatting instructions on how to return the value

    Hide attributes Show attributes
    • field string Required

      A wildcard pattern. The request returns values for field names matching this pattern.

    • format string

      The format in which the values are returned.

    • include_unmapped boolean
  • result_position string

    Supported values include:

    • tail: Return the most recent matches, similar to the Unix tail command.
    • head: Return the earliest matches, similar to the Unix head command.

    Values are tail or head.

  • runtime_mappings object
    Hide runtime_mappings attribute Show runtime_mappings attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • fields object

        For type composite

        Hide fields attribute Show fields attribute object
        • * object Additional properties
          Hide * attribute Show * attribute object
          • type string Required

            Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

      • fetch_fields array[object]

        For type lookup

        Hide fetch_fields attributes Show fetch_fields attributes object
        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • format string
      • format string

        A custom format for date type runtime fields.

      • input_field string

        For type lookup

      • target_field string

        For type lookup

      • target_index string

        For type lookup

      • script object

        Painless script executed at query time.

        Hide script attributes Show script attributes object
        • source string

          The script source.

        • id string

          The id for a stored script.

        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          Hide params attribute Show params attribute object
          • * object Additional properties
        • lang
        • options object
          Hide options attribute Show options attribute object
          • * string Additional properties
      • type string Required

        Field type, which can be: boolean, composite, date, double, geo_point, ip,keyword, long, or lookup.

        Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

  • max_samples_per_key number

    By default, the response of a sample query contains up to 10 samples, with one sample per unique set of join keys. Use the size parameter to get a smaller or larger set of samples. To retrieve more than one sample per set of join keys, use the max_samples_per_key parameter. Pipes are not supported for sample queries.

    Default value is 1.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • id string

      Identifier for the search.

    • is_partial boolean

      If true, the response does not contain complete search results.

    • is_running boolean

      If true, the search request is still executing.

    • took number

      Time unit for milliseconds

    • timed_out boolean

      If true, the request timed out before completion.

    • hits object Required

      Contains matching events and sequences. Also contains related metadata.

      Hide hits attributes Show hits attributes object
      • total object

        Metadata about the number of matching events or sequences.

        Hide total attributes Show total attributes object
        • relation string Required

          Supported values include:

          • eq: Accurate
          • gte: Lower bound, including returned events or sequences

          Values are eq or gte.

        • value number Required
      • events array[object]

        Contains events matching the query. Each object represents a matching event.

        Hide events attributes Show events attributes object
        • _index string Required

          Name of the index containing the event.

        • _id string Required

          Unique identifier for the event. This ID is only unique within the index.

        • _source object Required

          Original JSON body passed for the event at index time.

        • missing boolean

          Set to true for events in a timespan-constrained sequence that do not meet a given condition.

        • fields object
          Hide fields attribute Show fields attribute object
          • * array[object] Additional properties
      • sequences array[object]

        Contains event sequences matching the query. Each object represents a matching sequence. This parameter is only returned for EQL queries containing a sequence.

        Hide sequences attributes Show sequences attributes object
        • events array[object] Required

          Contains events matching the query. Each object represents a matching event.

        • join_keys array[object]

          Shared field values used to constrain matches in the sequence. These are defined using the by keyword in the EQL query syntax.

    • shard_failures array[object]

      Contains information about shard failures (if any), in case allow_partial_search_results=true

      Hide shard_failures attributes Show shard_failures attributes object
      • index string
      • node string
      • reason object Required

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide reason attributes Show reason attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • shard number
      • status string
      • primary boolean
POST /{index}/_eql/search
GET /my-data-stream/_eql/search
{
  "query": """
    process where (process.name == "cmd.exe" and process.pid != 2013)
  """
}
resp = client.eql.search(
    index="my-data-stream",
    query="\n    process where (process.name == \"cmd.exe\" and process.pid != 2013)\n  ",
)
const response = await client.eql.search({
  index: "my-data-stream",
  query:
    '\n    process where (process.name == "cmd.exe" and process.pid != 2013)\n  ',
});
response = client.eql.search(
  index: "my-data-stream",
  body: {
    "query": "\n    process where (process.name == \"cmd.exe\" and process.pid != 2013)\n  "
  }
)
$resp = $client->eql()->search([
    "index" => "my-data-stream",
    "body" => [
        "query" => "\n    process where (process.name == \"cmd.exe\" and process.pid != 2013)\n  ",
    ],
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"query":"\n    process where (process.name == \"cmd.exe\" and process.pid != 2013)\n  "}' "$ELASTICSEARCH_URL/my-data-stream/_eql/search"
client.eql().search(s -> s
    .index("my-data-stream")
    .query(" process where (process.name == \"cmd.exe\" and process.pid != 2013) ")
);
Request examples
Run `GET /my-data-stream/_eql/search` to search for events that have a `process.name` of `cmd.exe` and a `process.pid` other than `2013`.
{
  "query": """
    process where (process.name == "cmd.exe" and process.pid != 2013)
  """
}
Run `GET /my-data-stream/_eql/search` to search for a sequence of events. The sequence starts with an event with an `event.category` of `file`, a `file.name` of `cmd.exe`, and a `process.pid` other than `2013`. It is followed by an event with an `event.category` of `process` and a `process.executable` that contains the substring `regsvr32`. These events must also share the same `process.pid` value.
{
  "query": """
    sequence by process.pid
      [ file where file.name == "cmd.exe" and process.pid != 2013 ]
      [ process where stringContains(process.executable, "regsvr32") ]
  """
}
Response examples (200)
{
  "is_partial": false,
  "is_running": false,
  "took": 6,
  "timed_out": false,
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "sequences": [
      {
        "join_keys": [
          2012
        ],
        "events": [
          {
            "_index": ".ds-my-data-stream-2099.12.07-000001",
            "_id": "AtOJ4UjUBAAx3XR5kcCM",
            "_source": {
              "@timestamp": "2099-12-06T11:04:07.000Z",
              "event": {
                "category": "file",
                "id": "dGCHwoeS",
                "sequence": 2
              },
              "file": {
                "accessed": "2099-12-07T11:07:08.000Z",
                "name": "cmd.exe",
                "path": "C:\\Windows\\System32\\cmd.exe",
                "type": "file",
                "size": 16384
              },
              "process": {
                "pid": 2012,
                "name": "cmd.exe",
                "executable": "C:\\Windows\\System32\\cmd.exe"
              }
            }
          },
          {
            "_index": ".ds-my-data-stream-2099.12.07-000001",
            "_id": "OQmfCaduce8zoHT93o4H",
            "_source": {
              "@timestamp": "2099-12-07T11:07:09.000Z",
              "event": {
                "category": "process",
                "id": "aR3NWVOs",
                "sequence": 4
              },
              "process": {
                "pid": 2012,
                "name": "regsvr32.exe",
                "command_line": "regsvr32.exe  /s /u /i:https://...RegSvr32.sct scrobj.dll",
                "executable": "C:\\Windows\\System32\\regsvr32.exe"
              }
            }
          }
        ]
      }
    ]
  }
}





















Features

The feature APIs enable you to introspect and manage features provided by Elasticsearch and Elasticsearch plugins.






















Graph explore

The graph explore API enables you to extract and summarize information about the documents and terms in an Elasticsearch data stream or index.

Get started with Graph





Get component templates Generally available; Added in 7.8.0

GET /_component_template/{name}

All methods and paths for this operation:

GET /_component_template

GET /_component_template/{name}

Get information about component templates.

Required authorization

  • Cluster privileges: manage_index_templates

Path parameters

  • name string Required

    Comma-separated list of component template names used to limit the request. Wildcard (*) expressions are supported.

Query parameters

  • flat_settings boolean

    If true, returns settings in flat format.

  • settings_filter string | array[string]

    Filter out results, for example to filter out sensitive information. Supports wildcards or full settings keys

  • include_defaults boolean Generally available; Added in 8.11.0

    Return all default configurations for the component template (default: false)

  • local boolean

    If true, the request retrieves information from the local node only. If false, information is retrieved from the master node.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • component_templates array[object] Required
      Hide component_templates attributes Show component_templates attributes object
      • name string Required
      • component_template object Required
        Hide component_template attributes Show component_template attributes object
        • template object Required
        • version number
        • _meta object
        • deprecated boolean
GET /_component_template/{name}
GET /_component_template/template_1
resp = client.cluster.get_component_template(
    name="template_1",
)
const response = await client.cluster.getComponentTemplate({
  name: "template_1",
});
response = client.cluster.get_component_template(
  name: "template_1"
)
$resp = $client->cluster()->getComponentTemplate([
    "name" => "template_1",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_component_template/template_1"
client.cluster().getComponentTemplate(g -> g
    .name("template_1")
);








Check component templates Generally available; Added in 7.8.0

HEAD /_component_template/{name}

Returns information about whether a particular component template exists.

Path parameters

  • name string | array[string] Required

    Comma-separated list of component template names used to limit the request. Wildcard (*) expressions are supported.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • local boolean

    If true, the request retrieves information from the local node only. Defaults to false, which means information is retrieved from the master node.

Responses

  • 200 application/json
HEAD /_component_template/{name}
curl \
 --request HEAD 'http://api.example.com/_component_template/{name}' \
 --header "Authorization: $API_KEY"








Get the dangling indices Generally available; Added in 7.9.0

GET /_dangling

If Elasticsearch encounters index data that is absent from the current cluster state, those indices are considered to be dangling. For example, this can happen if you delete more than cluster.indices.tombstones.size indices while an Elasticsearch node is offline.

Use this API to list dangling indices, which you can then import or delete.

Required authorization

  • Cluster privileges: manage

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • dangling_indices array[object] Required
      Hide dangling_indices attributes Show dangling_indices attributes object
GET /_dangling
GET /_dangling
resp = client.dangling_indices.list_dangling_indices()
const response = await client.danglingIndices.listDanglingIndices();
response = client.dangling_indices.list_dangling_indices
$resp = $client->danglingIndices()->listDanglingIndices();
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_dangling"
client.danglingIndices().listDanglingIndices();
Response examples (200)
{
  "dangling_indices": [
   {
    "index_name": "my-index-000001",
    "index_uuid": "zmM4e0JtBkeUjiHD-MihPQ",
    "creation_date_millis": 1589414451372,
    "node_ids": [
      "pL47UN3dAb2d5RCWP6lQ3e"
    ]
   }
  ]
}
































Check indices Generally available

HEAD /{index}

Check if one or more indices, index aliases, or data streams exist.

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams, indices, and aliases. Supports wildcards (*).

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.

    Values are all, open, closed, hidden, or none.

  • flat_settings boolean

    If true, returns settings in flat format.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • include_defaults boolean

    If true, return all default settings in the response.

  • local boolean

    If true, the request retrieves information from the local node only.

Responses

  • 200 application/json
HEAD /{index}
HEAD my-data-stream
resp = client.indices.exists(
    index="my-data-stream",
)
const response = await client.indices.exists({
  index: "my-data-stream",
});
response = client.indices.exists(
  index: "my-data-stream"
)
$resp = $client->indices()->exists([
    "index" => "my-data-stream",
]);
curl --head -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/my-data-stream"
client.indices().exists(e -> e
    .index("my-data-stream")
);

Create or update an alias Generally available

POST /{index}/_aliases/{name}

All methods and paths for this operation:

PUT /{index}/_alias/{name}

POST /{index}/_alias/{name}
PUT /{index}/_aliases/{name}
POST /{index}/_aliases/{name}

Adds a data stream or index to an alias.

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams or indices to add. Supports wildcards (*). Wildcard patterns that match both data streams and indices return an error.

  • name string Required

    Alias to update. If the alias doesn’t exist, the request creates it. Index alias names support date math.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

application/json

Body

  • filter object

    Query used to limit documents the alias can access.

    External documentation
  • index_routing string

    Value used to route indexing operations to a specific shard. If specified, this overwrites the routing value for indexing operations. Data stream aliases don’t support this parameter.

  • is_write_index boolean

    If true, sets the write index or data stream for the alias. If an alias points to multiple indices or data streams and is_write_index isn’t set, the alias rejects write requests. If an index alias points to one index and is_write_index isn’t set, the index automatically acts as the write index. Data stream aliases don’t automatically set a write data stream, even if the alias points to one data stream.

  • routing string

    Value used to route indexing and search operations to a specific shard. Data stream aliases don’t support this parameter.

  • search_routing string

    Value used to route search operations to a specific shard. If specified, this overwrites the routing value for search operations. Data stream aliases don’t support this parameter.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

POST /{index}/_aliases/{name}
POST _aliases
{
  "actions": [
    {
      "add": {
        "index": "my-data-stream",
        "alias": "my-alias"
      }
    }
  ]
}
resp = client.indices.update_aliases(
    actions=[
        {
            "add": {
                "index": "my-data-stream",
                "alias": "my-alias"
            }
        }
    ],
)
const response = await client.indices.updateAliases({
  actions: [
    {
      add: {
        index: "my-data-stream",
        alias: "my-alias",
      },
    },
  ],
});
response = client.indices.update_aliases(
  body: {
    "actions": [
      {
        "add": {
          "index": "my-data-stream",
          "alias": "my-alias"
        }
      }
    ]
  }
)
$resp = $client->indices()->updateAliases([
    "body" => [
        "actions" => array(
            [
                "add" => [
                    "index" => "my-data-stream",
                    "alias" => "my-alias",
                ],
            ],
        ),
    ],
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"actions":[{"add":{"index":"my-data-stream","alias":"my-alias"}}]}' "$ELASTICSEARCH_URL/_aliases"
client.indices().updateAliases(u -> u
    .actions(a -> a
        .add(ad -> ad
            .alias("my-alias")
            .index("my-data-stream")
        )
    )
);
Request example
{
  "actions": [
    {
      "add": {
        "index": "my-data-stream",
        "alias": "my-alias"
      }
    }
  ]
}
















































Check aliases Generally available

HEAD /{index}/_alias/{name}

All methods and paths for this operation:

HEAD /_alias/{name}

HEAD /{index}/_alias/{name}

Check if one or more data stream or index aliases exist.

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams or indices used to limit the request. Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.

  • name string | array[string] Required

    Comma-separated list of aliases to check. Supports wildcards (*).

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.

    Values are all, open, closed, hidden, or none.

  • ignore_unavailable boolean

    If false, requests that include a missing data stream or index in the target indices or data streams return an error.

  • local boolean Deprecated

    If true, the request retrieves information from the local node only.

Responses

  • 200 application/json
HEAD /{index}/_alias/{name}
HEAD _alias/my-alias
resp = client.indices.exists_alias(
    name="my-alias",
)
const response = await client.indices.existsAlias({
  name: "my-alias",
});
response = client.indices.exists_alias(
  name: "my-alias"
)
$resp = $client->indices()->existsAlias([
    "name" => "my-alias",
]);
curl --head -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_alias/my-alias"
client.indices().existsAlias(e -> e
    .name("my-alias")
);
























Get index settings Generally available

GET /{index}/_settings/{name}

All methods and paths for this operation:

GET /_settings

GET /_settings/{name}
GET /{index}/_settings
GET /{index}/_settings/{name}

Get setting information for one or more indices. For data streams, it returns setting information for the stream's backing indices.

Required authorization

  • Index privileges: view_index_metadata

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.

  • name string | array[string] Required

    Comma-separated list or wildcard expression of settings to retrieve.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.

    Values are all, open, closed, hidden, or none.

  • flat_settings boolean

    If true, returns settings in flat format.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • include_defaults boolean

    If true, return all default settings in the response.

  • local boolean

    If true, the request retrieves information from the local node only. If false, information is retrieved from the master node.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object
      Hide * attributes Show * attributes object
      • aliases object
        Hide aliases attribute Show aliases attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • filter object

            Query used to limit documents the alias can access.

            External documentation
          • index_routing string

            Value used to route indexing operations to a specific shard. If specified, this overwrites the routing value for indexing operations.

          • is_hidden boolean

            If true, the alias is hidden. All indices for the alias must have the same is_hidden value.

            Default value is false.

          • is_write_index boolean

            If true, the index is the write index for the alias.

            Default value is false.

          • routing string

            Value used to route indexing and search operations to a specific shard.

          • search_routing string

            Value used to route search operations to a specific shard. If specified, this overwrites the routing value for search operations.

      • mappings object
        Hide mappings attributes Show mappings attributes object
        • all_field object
          Hide all_field attributes Show all_field attributes object
          • analyzer string Required
          • enabled boolean Required
          • omit_norms boolean Required
          • search_analyzer string Required
          • similarity string Required
          • store boolean Required
          • store_term_vector_offsets boolean Required
          • store_term_vector_payloads boolean Required
          • store_term_vector_positions boolean Required
          • store_term_vectors boolean Required
        • date_detection boolean
        • dynamic string

          Values are strict, runtime, true, or false.

        • dynamic_date_formats array[string]
        • dynamic_templates array[object]
        • _field_names object
          Hide _field_names attribute Show _field_names attribute object
          • enabled boolean Required
        • index_field object
          Hide index_field attribute Show index_field attribute object
          • enabled boolean Required
        • _meta object
          Hide _meta attribute Show _meta attribute object
          • * object Additional properties
        • numeric_detection boolean
        • properties object
        • _routing object
          Hide _routing attribute Show _routing attribute object
          • required boolean Required
        • _size object
          Hide _size attribute Show _size attribute object
          • enabled boolean Required
        • _source object
          Hide _source attributes Show _source attributes object
          • compress boolean
          • compress_threshold string
          • enabled boolean
          • excludes array[string]
          • includes array[string]
        • runtime object
          Hide runtime attribute Show runtime attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • fields object

              For type composite

            • fetch_fields array[object]

              For type lookup

            • format string

              A custom format for date type runtime fields.

        • enabled boolean
        • subobjects string

          Values are true or false.

        • _data_stream_timestamp object
          Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
          • enabled boolean Required
      • settings object
        Index settings
      • defaults object

        Default settings, included when the request's include_default is true.

        Index settings
      • data_stream string
      • lifecycle object

        Data stream lifecycle applicable if this is a data stream.

        Hide lifecycle attributes Show lifecycle attributes object
        • data_retention string

          If defined, every document added to this data stream will be stored at least for this time frame. Any time after this duration the document could be deleted. When empty, every document in this data stream will be stored indefinitely.

        • downsampling object

          The downsampling configuration to execute for the managed backing index after rollover.

          Hide downsampling attribute Show downsampling attribute object
          • rounds array[object] Required

            The list of downsampling rounds to execute as part of this downsampling configuration

        • enabled boolean

          If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle that's disabled (enabled: false) will have no effect on the data stream.

          Default value is true.

GET /{index}/_settings/{name}
GET _all/_settings?expand_wildcards=all&filter_path=*.settings.index.*.slowlog
resp = client.indices.get_settings(
    index="_all",
    expand_wildcards="all",
    filter_path="*.settings.index.*.slowlog",
)
const response = await client.indices.getSettings({
  index: "_all",
  expand_wildcards: "all",
  filter_path: "*.settings.index.*.slowlog",
});
response = client.indices.get_settings(
  index: "_all",
  expand_wildcards: "all",
  filter_path: "*.settings.index.*.slowlog"
)
$resp = $client->indices()->getSettings([
    "index" => "_all",
    "expand_wildcards" => "all",
    "filter_path" => "*.settings.index.*.slowlog",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_all/_settings?expand_wildcards=all&filter_path=*.settings.index.*.slowlog"

Open a closed index Generally available

POST /{index}/_open

For data streams, the API opens any closed backing indices.

A closed index is blocked for read/write operations and does not allow all operations that opened indices allow. It is not possible to index documents or to search for documents in a closed index. This allows closed indices to not have to maintain internal data structures for indexing or searching documents, resulting in a smaller overhead on the cluster.

When opening or closing an index, the master is responsible for restarting the index shards to reflect the new state of the index. The shards will then go through the normal recovery process. The data of opened or closed indices is automatically replicated by the cluster to ensure that enough shard copies are safely kept around at all times.

You can open and close multiple indices. An error is thrown if the request explicitly refers to a missing index. This behavior can be turned off by using the ignore_unavailable=true parameter.

By default, you must explicitly name the indices you are opening or closing. To open or close indices with _all, *, or other wildcard expressions, change the action.destructive_requires_name setting to false. This setting can also be changed with the cluster update settings API.

Closed indices consume a significant amount of disk-space which can cause problems in managed environments. Closing indices can be turned off with the cluster settings API by setting cluster.indices.close.enable to false.

Because opening or closing an index allocates its shards, the wait_for_active_shards setting on index creation applies to the _open and _close index actions as well.

Required authorization

  • Index privileges: manage

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (*). By default, you must explicitly name the indices you using to limit the request. To limit a request using _all, *, or other wildcard expressions, change the action.destructive_requires_name setting to false. You can update this setting in the elasticsearch.yml file or using the cluster update settings API.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.

    Values are all, open, closed, hidden, or none.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).

    Values are all or index-setting.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • acknowledged boolean Required
    • shards_acknowledged boolean Required
POST /{index}/_open
POST /.ds-my-data-stream-2099.03.07-000001/_open/
resp = client.indices.open(
    index=".ds-my-data-stream-2099.03.07-000001",
)
const response = await client.indices.open({
  index: ".ds-my-data-stream-2099.03.07-000001",
});
response = client.indices.open(
  index: ".ds-my-data-stream-2099.03.07-000001"
)
$resp = $client->indices()->open([
    "index" => ".ds-my-data-stream-2099.03.07-000001",
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/.ds-my-data-stream-2099.03.07-000001/_open/"
client.indices().open(o -> o
    .index(".ds-my-data-stream-2099.03.07-000001")
);
Response examples (200)
A successful response for opening an index.
{
  "acknowledged" : true,
  "shards_acknowledged" : true
}

Update index settings Generally available

PUT /{index}/_settings

All methods and paths for this operation:

PUT /_settings

PUT /{index}/_settings

Changes dynamic index settings in real time. For data streams, index setting changes are applied to all backing indices by default.

To revert a setting to the default value, use a null value. The list of per-index settings that can be updated dynamically on live indices can be found in index settings documentation. To preserve existing settings from being updated, set the preserve_existing parameter to true.

There are multiple valid ways to represent index settings in the request body. You can specify only the setting, for example:

{
  "number_of_replicas": 1
}

Or you can use an index setting object:

{
  "index": {
    "number_of_replicas": 1
  }
}

Or you can use dot annotation:

{
  "index.number_of_replicas": 1
}

Or you can embed any of the aforementioned options in a settings object. For example:

{
  "settings": {
    "index": {
      "number_of_replicas": 1
    }
  }
}

NOTE: You can only define new analyzers on closed indices. To add an analyzer, you must close the index, define the analyzer, and reopen the index. You cannot close the write index of a data stream. To update the analyzer for a data stream's write index and future backing indices, update the analyzer in the index template used by the stream. Then roll over the data stream to apply the new analyzer to the stream's write index and future backing indices. This affects searches and any new data added to the stream after the rollover. However, it does not affect the data stream's backing indices or their existing data. To change the analyzer for existing backing indices, you must create a new data stream and reindex your data into it.

Required authorization

  • Index privileges: manage
External documentation

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.

    Values are all, open, closed, hidden, or none.

  • flat_settings boolean

    If true, returns settings in flat format.

  • ignore_unavailable boolean

    If true, returns settings in flat format.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • preserve_existing boolean

    If true, existing index settings remain unchanged.

  • reopen boolean

    Whether to close and reopen the index to apply non-dynamic settings. If set to true the indices to which the settings are being applied will be closed temporarily and then reopened in order to apply the changes.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

application/json

Body Required

object object
Index settings

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /{index}/_settings
PUT /my-index-000001/_settings
{
  "index" : {
    "number_of_replicas" : 2
  }
}
resp = client.indices.put_settings(
    index="my-index-000001",
    settings={
        "index": {
            "number_of_replicas": 2
        }
    },
)
const response = await client.indices.putSettings({
  index: "my-index-000001",
  settings: {
    index: {
      number_of_replicas: 2,
    },
  },
});
response = client.indices.put_settings(
  index: "my-index-000001",
  body: {
    "index": {
      "number_of_replicas": 2
    }
  }
)
$resp = $client->indices()->putSettings([
    "index" => "my-index-000001",
    "body" => [
        "index" => [
            "number_of_replicas" => 2,
        ],
    ],
]);
curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"index":{"number_of_replicas":2}}' "$ELASTICSEARCH_URL/my-index-000001/_settings"
client.indices().putSettings(p -> p
    .index("my-index-000001")
    .settings(s -> s
        .index(i -> i
            .numberOfReplicas("2")
        )
    )
);
Request examples
{
  "index" : {
    "number_of_replicas" : 2
  }
}
To revert a setting to the default value, use `null`.
{
  "index" : {
    "refresh_interval" : null
  }
}
To add an analyzer, you must close the index (`POST /my-index-000001/_close`), define the analyzer, then reopen the index (`POST /my-index-000001/_open`).
{
  "analysis": {
    "analyzer": {
      "content": {
        "type": "custom",
        "tokenizer": "whitespace"
      }
    }
  }
}




Refresh an index Generally available

GET /{index}/_refresh

All methods and paths for this operation:

POST /_refresh

GET /_refresh
POST /{index}/_refresh
GET /{index}/_refresh

A refresh makes recent operations performed on one or more indices available for search. For data streams, the API runs the refresh operation on the stream’s backing indices.

By default, Elasticsearch periodically refreshes indices every second, but only on indices that have received one search request or more in the last 30 seconds. You can change this default interval with the index.refresh_interval setting.

Refresh requests are synchronous and do not return a response until the refresh operation completes.

Refreshes are resource-intensive. To ensure good cluster performance, it's recommended to wait for Elasticsearch's periodic refresh rather than performing an explicit refresh when possible.

If your application workflow indexes documents and then runs a search to retrieve the indexed document, it's recommended to use the index API's refresh=wait_for query parameter option. This option ensures the indexing operation waits for a periodic refresh before running the search.

Required authorization

  • Index privileges: maintenance

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.

    Values are all, open, closed, hidden, or none.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • _shards object
      Hide _shards attributes Show _shards attributes object
      • failed number Required

        The number of shards the operation or search attempted to run on but failed.

      • successful number Required

        The number of shards the operation or search succeeded on.

      • total number Required

        The number of shards the operation or search will run on overall.

      • failures array[object]
        Hide failures attributes Show failures attributes object
        • index string
        • node string
        • reason object Required

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • shard number
        • status string
        • primary boolean
      • skipped number
GET /{index}/_refresh
GET _refresh
resp = client.indices.refresh()
const response = await client.indices.refresh();
response = client.indices.refresh
$resp = $client->indices()->refresh();
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_refresh"
client.indices().refresh(r -> r);
















































Create or update an alias Generally available; Added in 1.3.0

POST /_aliases

Adds a data stream or index to an alias.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

application/json

Body Required

  • actions array[object]

    Actions to perform.

    Hide actions attributes Show actions attributes object
    • add object

      Adds a data stream or index to an alias. If the alias doesn’t exist, the add action creates it.

      Hide add attributes Show add attributes object
      • alias string

        Alias for the action. Index alias names support date math.

      • aliases string | array[string]

        Aliases for the action. Index alias names support date math.

      • filter object

        Query used to limit documents the alias can access.

        External documentation
      • index string

        Data stream or index for the action. Supports wildcards (*).

      • indices string | array[string]

        Data streams or indices for the action. Supports wildcards (*).

      • index_routing string

        Value used to route indexing operations to a specific shard. If specified, this overwrites the routing value for indexing operations. Data stream aliases don’t support this parameter.

      • is_hidden boolean

        If true, the alias is hidden.

        Default value is false.

      • is_write_index boolean

        If true, sets the write index or data stream for the alias.

      • routing string

        Value used to route indexing and search operations to a specific shard. Data stream aliases don’t support this parameter.

      • search_routing string

        Value used to route search operations to a specific shard. If specified, this overwrites the routing value for search operations. Data stream aliases don’t support this parameter.

      • must_exist boolean

        If true, the alias must exist to perform the action.

        Default value is false.

    • remove object

      Removes a data stream or index from an alias.

      Hide remove attributes Show remove attributes object
      • alias string

        Alias for the action. Index alias names support date math.

      • aliases string | array[string]

        Aliases for the action. Index alias names support date math.

      • index string

        Data stream or index for the action. Supports wildcards (*).

      • indices string | array[string]

        Data streams or indices for the action. Supports wildcards (*).

      • must_exist boolean

        If true, the alias must exist to perform the action.

        Default value is false.

    • remove_index object

      Deletes an index. You cannot use this action on aliases or data streams.

      Hide remove_index attributes Show remove_index attributes object
      • index string

        Data stream or index for the action. Supports wildcards (*).

      • indices string | array[string]

        Data streams or indices for the action. Supports wildcards (*).

      • must_exist boolean

        If true, the alias must exist to perform the action.

        Default value is false.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

POST /_aliases
POST _aliases
{
  "actions": [
    {
      "add": {
        "index": "logs-nginx.access-prod",
        "alias": "logs"
      }
    }
  ]
}
resp = client.indices.update_aliases(
    actions=[
        {
            "add": {
                "index": "logs-nginx.access-prod",
                "alias": "logs"
            }
        }
    ],
)
const response = await client.indices.updateAliases({
  actions: [
    {
      add: {
        index: "logs-nginx.access-prod",
        alias: "logs",
      },
    },
  ],
});
response = client.indices.update_aliases(
  body: {
    "actions": [
      {
        "add": {
          "index": "logs-nginx.access-prod",
          "alias": "logs"
        }
      }
    ]
  }
)
$resp = $client->indices()->updateAliases([
    "body" => [
        "actions" => array(
            [
                "add" => [
                    "index" => "logs-nginx.access-prod",
                    "alias" => "logs",
                ],
            ],
        ),
    ],
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"actions":[{"add":{"index":"logs-nginx.access-prod","alias":"logs"}}]}' "$ELASTICSEARCH_URL/_aliases"
client.indices().updateAliases(u -> u
    .actions(a -> a
        .add(ad -> ad
            .alias("logs")
            .index("logs-nginx.access-prod")
        )
    )
);
Request example
An example body for a `POST _aliases` request.
{
  "actions": [
    {
      "add": {
        "index": "logs-nginx.access-prod",
        "alias": "logs"
      }
    }
  ]
}

























Migrate to data tiers routing Generally available; Added in 7.14.0

POST /_ilm/migrate_to_data_tiers

Switch the indices, ILM policies, and legacy, composable, and component templates from using custom node attributes and attribute-based allocation filters to using data tiers. Optionally, delete one legacy index template. Using node roles enables ILM to automatically move the indices between data tiers.

Migrating away from custom node attributes routing can be manually performed. This API provides an automated way of performing three out of the four manual steps listed in the migration guide:

  1. Stop setting the custom hot attribute on new indices.
  2. Remove custom allocation settings from existing ILM policies.
  3. Replace custom allocation settings from existing indices with the corresponding tier preference.

ILM must be stopped before performing the migration. Use the stop ILM and get ILM status APIs to wait until the reported operation mode is STOPPED.

External documentation

Query parameters

  • dry_run boolean

    If true, simulates the migration from node attributes based allocation filters to data tiers, but does not perform the migration. This provides a way to retrieve the indices and ILM policies that need to be migrated.

application/json

Body

  • legacy_template_to_delete string
  • node_attribute string

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • dry_run boolean Required
    • removed_legacy_template string Required

      The name of the legacy index template that was deleted. This information is missing if no legacy index templates were deleted.

    • migrated_ilm_policies array[string] Required

      The ILM policies that were updated.

    • migrated_indices string | array[string] Required

      The indices that were migrated to tier preference routing.

    • migrated_legacy_templates array[string] Required

      The legacy index templates that were updated to not contain custom routing settings for the provided data attribute.

    • migrated_composable_templates array[string] Required

      The composable index templates that were updated to not contain custom routing settings for the provided data attribute.

    • migrated_component_templates array[string] Required

      The component templates that were updated to not contain custom routing settings for the provided data attribute.

POST /_ilm/migrate_to_data_tiers
POST /_ilm/migrate_to_data_tiers
{
  "legacy_template_to_delete": "global-template",
  "node_attribute": "custom_attribute_name"
}
resp = client.ilm.migrate_to_data_tiers(
    legacy_template_to_delete="global-template",
    node_attribute="custom_attribute_name",
)
const response = await client.ilm.migrateToDataTiers({
  legacy_template_to_delete: "global-template",
  node_attribute: "custom_attribute_name",
});
response = client.ilm.migrate_to_data_tiers(
  body: {
    "legacy_template_to_delete": "global-template",
    "node_attribute": "custom_attribute_name"
  }
)
$resp = $client->ilm()->migrateToDataTiers([
    "body" => [
        "legacy_template_to_delete" => "global-template",
        "node_attribute" => "custom_attribute_name",
    ],
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"legacy_template_to_delete":"global-template","node_attribute":"custom_attribute_name"}' "$ELASTICSEARCH_URL/_ilm/migrate_to_data_tiers"
client.ilm().migrateToDataTiers(m -> m
    .legacyTemplateToDelete("global-template")
    .nodeAttribute("custom_attribute_name")
);
Request example
Run `POST /_ilm/migrate_to_data_tiers` to migrate the indices, ILM policies, legacy templates, composable, and component templates away from defining custom allocation filtering using the `custom_attribute_name` node attribute. It also deletes the legacy template with name `global-template` if it exists in the system.
{
  "legacy_template_to_delete": "global-template",
  "node_attribute": "custom_attribute_name"
}
Response examples (200)
A successful response when migrating indices, ILMs, and templates from custom node attributes to data tiers.
{
  "dry_run": false,
  "removed_legacy_template":"global-template",
  "migrated_ilm_policies":["policy_with_allocate_action"],
  "migrated_indices":["warm-index-to-migrate-000001"],
  "migrated_legacy_templates":["a-legacy-template"],
  "migrated_composable_templates":["a-composable-template"],
  "migrated_component_templates":["a-component-template"]
}





































































































Create an JinaAI inference endpoint Generally available; Added in 8.18.0

PUT /_inference/{task_type}/{jinaai_inference_id}

Create an inference endpoint to perform an inference task with the jinaai service.

To review the available rerank models, refer to https://jina.ai/reranker. To review the available text_embedding models, refer to the https://jina.ai/embeddings/.

Required authorization

  • Cluster privileges: manage_inference

Path parameters

  • task_type string

    The type of the inference task that the model will perform.

    Values are rerank or text_embedding.

  • jinaai_inference_id string Required

    The unique identifier of the inference endpoint.

Query parameters

  • timeout string

    Specifies the amount of time to wait for the inference endpoint to be created.

    Values are -1 or 0.

application/json

Body

  • chunking_settings object

    The chunking configuration object.

    Hide chunking_settings attributes Show chunking_settings attributes object
    • max_chunk_size number

      The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

      Default value is 250.

    • overlap number

      The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

      Default value is 100.

    • sentence_overlap number

      The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

      Default value is 1.

    • strategy string

      The chunking strategy: sentence or word.

      Default value is sentence.

  • service string Required

    The type of service supported for the specified task type. In this case, jinaai.

    Value is jinaai.

  • service_settings object Required

    Settings used to install the inference model. These settings are specific to the jinaai service.

    Hide service_settings attributes Show service_settings attributes object
    • api_key string Required

      A valid API key of your JinaAI account.

      IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key. After creating the inference model, you cannot change the associated API key. If you want to use a different API key, delete the inference model and recreate it with the same name and the updated API key.

      External documentation
    • model_id string

      The name of the model to use for the inference task. For a rerank task, it is required. For a text_embedding task, it is optional.

    • rate_limit object

      This setting helps to minimize the number of rate limit errors returned from JinaAI. By default, the jinaai service sets the number of requests allowed per minute to 2000 for all task types.

      Hide rate_limit attribute Show rate_limit attribute object
      • requests_per_minute number

        The number of requests allowed per minute.

    • similarity string

      For a text_embedding task, the similarity measure. One of cosine, dot_product, l2_norm. The default values varies with the embedding type. For example, a float embedding type uses a dot_product similarity measure by default.

      Values are cosine, dot_product, or l2_norm.

  • task_settings object

    Settings to configure the inference task. These settings are specific to the task type you specified.

    Hide task_settings attributes Show task_settings attributes object
    • return_documents boolean

      For a rerank task, return the doc text within the results.

    • task string

      For a text_embedding task, the task passed to the model. Valid values are:

      • classification: Use it for embeddings passed through a text classifier.
      • clustering: Use it for the embeddings run through a clustering algorithm.
      • ingest: Use it for storing document embeddings in a vector database.
      • search: Use it for storing embeddings of search queries run against a vector database to find relevant documents.

      Values are classification, clustering, ingest, or search.

    • top_n number

      For a rerank task, the number of most relevant documents to return. It defaults to the number of the documents. If this inference endpoint is used in a text_similarity_reranker retriever query and top_n is set, it must be greater than or equal to rank_window_size in the query.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • chunking_settings object

      Chunking configuration object

      Hide chunking_settings attributes Show chunking_settings attributes object
      • max_chunk_size number

        The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

        Default value is 250.

      • overlap number

        The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

        Default value is 100.

      • sentence_overlap number

        The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

        Default value is 1.

      • strategy string

        The chunking strategy: sentence or word.

        Default value is sentence.

    • service string Required

      The service type

    • service_settings object Required

      Settings specific to the service

    • task_settings object

      Task settings specific to the service and task type

    • inference_id string Required

      The inference Id

    • task_type string Required

      The task type

      Values are text_embedding or rerank.

PUT /_inference/{task_type}/{jinaai_inference_id}
PUT _inference/text_embedding/jinaai-embeddings
{
    "service": "jinaai",
    "service_settings": {
        "model_id": "jina-embeddings-v3",
        "api_key": "JinaAi-Api-key"
    }
}
resp = client.inference.put(
    task_type="text_embedding",
    inference_id="jinaai-embeddings",
    inference_config={
        "service": "jinaai",
        "service_settings": {
            "model_id": "jina-embeddings-v3",
            "api_key": "JinaAi-Api-key"
        }
    },
)
const response = await client.inference.put({
  task_type: "text_embedding",
  inference_id: "jinaai-embeddings",
  inference_config: {
    service: "jinaai",
    service_settings: {
      model_id: "jina-embeddings-v3",
      api_key: "JinaAi-Api-key",
    },
  },
});
response = client.inference.put(
  task_type: "text_embedding",
  inference_id: "jinaai-embeddings",
  body: {
    "service": "jinaai",
    "service_settings": {
      "model_id": "jina-embeddings-v3",
      "api_key": "JinaAi-Api-key"
    }
  }
)
$resp = $client->inference()->put([
    "task_type" => "text_embedding",
    "inference_id" => "jinaai-embeddings",
    "body" => [
        "service" => "jinaai",
        "service_settings" => [
            "model_id" => "jina-embeddings-v3",
            "api_key" => "JinaAi-Api-key",
        ],
    ],
]);
curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"service":"jinaai","service_settings":{"model_id":"jina-embeddings-v3","api_key":"JinaAi-Api-key"}}' "$ELASTICSEARCH_URL/_inference/text_embedding/jinaai-embeddings"
client.inference().put(p -> p
    .inferenceId("jinaai-embeddings")
    .taskType(TaskType.TextEmbedding)
    .inferenceConfig(i -> i
        .service("jinaai")
        .serviceSettings(JsonData.fromJson("{\"model_id\":\"jina-embeddings-v3\",\"api_key\":\"JinaAi-Api-key\"}"))
    )
);
Request examples
Run `PUT _inference/text_embedding/jinaai-embeddings` to create an inference endpoint for text embedding tasks using the JinaAI service.
{
    "service": "jinaai",
    "service_settings": {
        "model_id": "jina-embeddings-v3",
        "api_key": "JinaAi-Api-key"
    }
}
Run `PUT _inference/rerank/jinaai-rerank` to create an inference endpoint for rerank tasks using the JinaAI service.
{
    "service": "jinaai",
    "service_settings": {
        "api_key": "JinaAI-Api-key",
        "model_id": "jina-reranker-v2-base-multilingual"
    },
    "task_settings": {
        "top_n": 10,
        "return_documents": true
    }
}




Create an OpenAI inference endpoint Generally available; Added in 8.12.0

PUT /_inference/{task_type}/{openai_inference_id}

Create an inference endpoint to perform an inference task with the openai service or openai compatible APIs.

Required authorization

  • Cluster privileges: manage_inference

Path parameters

  • task_type string

    The type of the inference task that the model will perform. NOTE: The chat_completion task type only supports streaming and only through the _stream API.

    Values are chat_completion, completion, or text_embedding.

  • openai_inference_id string Required

    The unique identifier of the inference endpoint.

Query parameters

  • timeout string

    Specifies the amount of time to wait for the inference endpoint to be created.

    Values are -1 or 0.

application/json

Body

  • chunking_settings object

    The chunking configuration object.

    Hide chunking_settings attributes Show chunking_settings attributes object
    • max_chunk_size number

      The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

      Default value is 250.

    • overlap number

      The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

      Default value is 100.

    • sentence_overlap number

      The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

      Default value is 1.

    • strategy string

      The chunking strategy: sentence or word.

      Default value is sentence.

  • service string Required

    The type of service supported for the specified task type. In this case, openai.

    Value is openai.

  • service_settings object Required

    Settings used to install the inference model. These settings are specific to the openai service.

    Hide service_settings attributes Show service_settings attributes object
    • api_key string Required

      A valid API key of your OpenAI account. You can find your OpenAI API keys in your OpenAI account under the API keys section.

      IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key. After creating the inference model, you cannot change the associated API key. If you want to use a different API key, delete the inference model and recreate it with the same name and the updated API key.

      External documentation
    • dimensions number

      The number of dimensions the resulting output embeddings should have. It is supported only in text-embedding-3 and later models. If it is not set, the OpenAI defined default for the model is used.

    • model_id string Required

      The name of the model to use for the inference task. Refer to the OpenAI documentation for the list of available text embedding models.

      External documentation
    • organization_id string

      The unique identifier for your organization. You can find the Organization ID in your OpenAI account under Settings > Organizations.

    • rate_limit object

      This setting helps to minimize the number of rate limit errors returned from OpenAI. The openai service sets a default number of requests allowed per minute depending on the task type. For text_embedding, it is set to 3000. For completion, it is set to 500.

      Hide rate_limit attribute Show rate_limit attribute object
      • requests_per_minute number

        The number of requests allowed per minute.

    • url string

      The URL endpoint to use for the requests. It can be changed for testing purposes.

      Default value is https://api.openai.com/v1/embeddings..

  • task_settings object

    Settings to configure the inference task. These settings are specific to the task type you specified.

    Hide task_settings attribute Show task_settings attribute object
    • user string

      For a completion or text_embedding task, specify the user issuing the request. This information can be used for abuse detection.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • chunking_settings object

      Chunking configuration object

      Hide chunking_settings attributes Show chunking_settings attributes object
      • max_chunk_size number

        The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

        Default value is 250.

      • overlap number

        The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

        Default value is 100.

      • sentence_overlap number

        The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

        Default value is 1.

      • strategy string

        The chunking strategy: sentence or word.

        Default value is sentence.

    • service string Required

      The service type

    • service_settings object Required

      Settings specific to the service

    • task_settings object

      Task settings specific to the service and task type

    • inference_id string Required

      The inference Id

    • task_type string Required

      The task type

      Values are text_embedding, chat_completion, or completion.

PUT /_inference/{task_type}/{openai_inference_id}
PUT _inference/text_embedding/openai-embeddings
{
    "service": "openai",
    "service_settings": {
        "api_key": "OpenAI-API-Key",
        "model_id": "text-embedding-3-small",
        "dimensions": 128
    }
}
resp = client.inference.put(
    task_type="text_embedding",
    inference_id="openai-embeddings",
    inference_config={
        "service": "openai",
        "service_settings": {
            "api_key": "OpenAI-API-Key",
            "model_id": "text-embedding-3-small",
            "dimensions": 128
        }
    },
)
const response = await client.inference.put({
  task_type: "text_embedding",
  inference_id: "openai-embeddings",
  inference_config: {
    service: "openai",
    service_settings: {
      api_key: "OpenAI-API-Key",
      model_id: "text-embedding-3-small",
      dimensions: 128,
    },
  },
});
response = client.inference.put(
  task_type: "text_embedding",
  inference_id: "openai-embeddings",
  body: {
    "service": "openai",
    "service_settings": {
      "api_key": "OpenAI-API-Key",
      "model_id": "text-embedding-3-small",
      "dimensions": 128
    }
  }
)
$resp = $client->inference()->put([
    "task_type" => "text_embedding",
    "inference_id" => "openai-embeddings",
    "body" => [
        "service" => "openai",
        "service_settings" => [
            "api_key" => "OpenAI-API-Key",
            "model_id" => "text-embedding-3-small",
            "dimensions" => 128,
        ],
    ],
]);
curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"service":"openai","service_settings":{"api_key":"OpenAI-API-Key","model_id":"text-embedding-3-small","dimensions":128}}' "$ELASTICSEARCH_URL/_inference/text_embedding/openai-embeddings"
client.inference().put(p -> p
    .inferenceId("openai-embeddings")
    .taskType(TaskType.TextEmbedding)
    .inferenceConfig(i -> i
        .service("openai")
        .serviceSettings(JsonData.fromJson("{\"api_key\":\"OpenAI-API-Key\",\"model_id\":\"text-embedding-3-small\",\"dimensions\":128}"))
    )
);
Request examples
Run `PUT _inference/text_embedding/openai-embeddings` to create an inference endpoint that performs a `text_embedding` task. The embeddings created by requests to this endpoint will have 128 dimensions.
{
    "service": "openai",
    "service_settings": {
        "api_key": "OpenAI-API-Key",
        "model_id": "text-embedding-3-small",
        "dimensions": 128
    }
}
Run `PUT _inference/completion/amazon_bedrock_completion` to create an inference endpoint to perform a completion task.
{
    "service": "amazonbedrock",
    "service_settings": {
        "access_key": "AWS-access-key",
        "secret_key": "AWS-secret-key",
        "region": "us-east-1",
        "provider": "amazontitan",
        "model": "amazon.titan-text-premier-v1:0"
    }
}

Create a VoyageAI inference endpoint Generally available; Added in 8.19.0

PUT /_inference/{task_type}/{voyageai_inference_id}

Create an inference endpoint to perform an inference task with the voyageai service.

Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.

Required authorization

  • Cluster privileges: manage_inference

Path parameters

  • task_type string

    The type of the inference task that the model will perform.

    Values are text_embedding or rerank.

  • voyageai_inference_id string Required

    The unique identifier of the inference endpoint.

Query parameters

  • timeout string

    Specifies the amount of time to wait for the inference endpoint to be created.

    Values are -1 or 0.

application/json

Body

  • chunking_settings object

    The chunking configuration object.

    Hide chunking_settings attributes Show chunking_settings attributes object
    • max_chunk_size number

      The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

      Default value is 250.

    • overlap number

      The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

      Default value is 100.

    • sentence_overlap number

      The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

      Default value is 1.

    • strategy string

      The chunking strategy: sentence or word.

      Default value is sentence.

  • service string Required

    The type of service supported for the specified task type. In this case, voyageai.

    Value is voyageai.

  • service_settings object Required

    Settings used to install the inference model. These settings are specific to the voyageai service.

    Hide service_settings attributes Show service_settings attributes object
    • dimensions number

      The number of dimensions for resulting output embeddings. This setting maps to output_dimension in the VoyageAI documentation. Only for the text_embedding task type.

      External documentation
    • model_id string Required

      The name of the model to use for the inference task. Refer to the VoyageAI documentation for the list of available text embedding and rerank models.

      External documentation
    • rate_limit object

      This setting helps to minimize the number of rate limit errors returned from VoyageAI. The voyageai service sets a default number of requests allowed per minute depending on the task type. For both text_embedding and rerank, it is set to 2000.

      Hide rate_limit attribute Show rate_limit attribute object
      • requests_per_minute number

        The number of requests allowed per minute.

    • embedding_type number

      The data type for the embeddings to be returned. This setting maps to output_dtype in the VoyageAI documentation. Permitted values: float, int8, bit. int8 is a synonym of byte in the VoyageAI documentation. bit is a synonym of binary in the VoyageAI documentation. Only for the text_embedding task type.

      External documentation
  • task_settings object

    Settings to configure the inference task. These settings are specific to the task type you specified.

    Hide task_settings attributes Show task_settings attributes object
    • input_type string

      Type of the input text. Permitted values: ingest (maps to document in the VoyageAI documentation), search (maps to query in the VoyageAI documentation). Only for the text_embedding task type.

    • return_documents boolean

      Whether to return the source documents in the response. Only for the rerank task type.

      Default value is false.

    • top_k number

      The number of most relevant documents to return. If not specified, the reranking results of all documents will be returned. Only for the rerank task type.

    • truncation boolean

      Whether to truncate the input texts to fit within the context length.

      Default value is true.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • chunking_settings object

      Chunking configuration object

      Hide chunking_settings attributes Show chunking_settings attributes object
      • max_chunk_size number

        The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

        Default value is 250.

      • overlap number

        The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

        Default value is 100.

      • sentence_overlap number

        The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

        Default value is 1.

      • strategy string

        The chunking strategy: sentence or word.

        Default value is sentence.

    • service string Required

      The service type

    • service_settings object Required

      Settings specific to the service

    • task_settings object

      Task settings specific to the service and task type

    • inference_id string Required

      The inference Id

    • task_type string Required

      The task type

      Values are text_embedding or rerank.

PUT /_inference/{task_type}/{voyageai_inference_id}
PUT _inference/text_embedding/openai-embeddings
{
    "service": "voyageai",
    "service_settings": {
        "model_id": "voyage-3-large",
        "dimensions": 512
    }
}
resp = client.inference.put(
    task_type="text_embedding",
    inference_id="openai-embeddings",
    inference_config={
        "service": "voyageai",
        "service_settings": {
            "model_id": "voyage-3-large",
            "dimensions": 512
        }
    },
)
const response = await client.inference.put({
  task_type: "text_embedding",
  inference_id: "openai-embeddings",
  inference_config: {
    service: "voyageai",
    service_settings: {
      model_id: "voyage-3-large",
      dimensions: 512,
    },
  },
});
response = client.inference.put(
  task_type: "text_embedding",
  inference_id: "openai-embeddings",
  body: {
    "service": "voyageai",
    "service_settings": {
      "model_id": "voyage-3-large",
      "dimensions": 512
    }
  }
)
$resp = $client->inference()->put([
    "task_type" => "text_embedding",
    "inference_id" => "openai-embeddings",
    "body" => [
        "service" => "voyageai",
        "service_settings" => [
            "model_id" => "voyage-3-large",
            "dimensions" => 512,
        ],
    ],
]);
curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"service":"voyageai","service_settings":{"model_id":"voyage-3-large","dimensions":512}}' "$ELASTICSEARCH_URL/_inference/text_embedding/openai-embeddings"
client.inference().put(p -> p
    .inferenceId("openai-embeddings")
    .taskType(TaskType.TextEmbedding)
    .inferenceConfig(i -> i
        .service("voyageai")
        .serviceSettings(JsonData.fromJson("{\"model_id\":\"voyage-3-large\",\"dimensions\":512}"))
    )
);
Request examples
Run `PUT _inference/text_embedding/voyageai-embeddings` to create an inference endpoint that performs a `text_embedding` task. The embeddings created by requests to this endpoint will have 512 dimensions.
{
    "service": "voyageai",
    "service_settings": {
        "model_id": "voyage-3-large",
        "dimensions": 512
    }
}
Run `PUT _inference/rerank/voyageai-rerank` to create an inference endpoint that performs a `rerank` task.
{
    "service": "voyageai",
    "service_settings": {
        "model_id": "rerank-2"
    }
}






















































Get pipelines Generally available; Added in 5.0.0

GET /_ingest/pipeline/{id}

All methods and paths for this operation:

GET /_ingest/pipeline

GET /_ingest/pipeline/{id}

Get information about one or more ingest pipelines. This API returns a local reference of the pipeline.

External documentation

Path parameters

  • id string Required

    Comma-separated list of pipeline IDs to retrieve. Wildcard (*) expressions are supported. To get all ingest pipelines, omit this parameter or use *.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • summary boolean

    Return pipelines without their definitions (default: false)

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • description string

        Description of the ingest pipeline.

      • on_failure array[object]

        Processors to run immediately after a processor failure.

        Hide on_failure attributes Show on_failure attributes object
        • append object
        • attachment object
        • bytes object
        • circle object
        • community_id object
        • convert object
        • csv object
        • date object
        • date_index_name object
        • dissect object
        • dot_expander object
        • drop object
        • enrich object
        • fail object
        • fingerprint object
        • foreach object
        • ip_location object
        • geo_grid object
        • geoip object
        • grok object
        • gsub object
        • html_strip object
        • inference object
        • join object
        • json object
        • kv object
        • lowercase object
        • network_direction object
        • pipeline object
        • redact object
        • registered_domain object
        • remove object
        • rename object
        • reroute object
        • script object
        • set object
        • set_security_user object
        • sort object
        • split object
        • terminate object
        • trim object
        • uppercase object
        • urldecode object
        • uri_parts object
        • user_agent object
      • processors array[object]

        Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.

        Hide processors attributes Show processors attributes object
        • append object
        • attachment object
        • bytes object
        • circle object
        • community_id object
        • convert object
        • csv object
        • date object
        • date_index_name object
        • dissect object
        • dot_expander object
        • drop object
        • enrich object
        • fail object
        • fingerprint object
        • foreach object
        • ip_location object
        • geo_grid object
        • geoip object
        • grok object
        • gsub object
        • html_strip object
        • inference object
        • join object
        • json object
        • kv object
        • lowercase object
        • network_direction object
        • pipeline object
        • redact object
        • registered_domain object
        • remove object
        • rename object
        • reroute object
        • script object
        • set object
        • set_security_user object
        • sort object
        • split object
        • terminate object
        • trim object
        • uppercase object
        • urldecode object
        • uri_parts object
        • user_agent object
      • version number

        Version number used by external systems to track ingest pipelines.

      • deprecated boolean

        Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.

        Default value is false.

      • _meta object

        Arbitrary metadata about the ingest pipeline. This map is not automatically generated by Elasticsearch.

        Hide _meta attribute Show _meta attribute object
        • * object Additional properties
GET /_ingest/pipeline/{id}
GET /_ingest/pipeline/my-pipeline-id
resp = client.ingest.get_pipeline(
    id="my-pipeline-id",
)
const response = await client.ingest.getPipeline({
  id: "my-pipeline-id",
});
response = client.ingest.get_pipeline(
  id: "my-pipeline-id"
)
$resp = $client->ingest()->getPipeline([
    "id" => "my-pipeline-id",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_ingest/pipeline/my-pipeline-id"
client.ingest().getPipeline(g -> g
    .id("my-pipeline-id")
);
Response examples (200)
A successful response for retrieving information about an ingest pipeline.
{
  "my-pipeline-id" : {
    "description" : "describe pipeline",
    "version" : 123,
    "processors" : [
      {
        "set" : {
          "field" : "foo",
          "value" : "bar"
        }
      }
    ]
  }
}













































Start a basic license Generally available; Added in 6.3.0

POST /_license/start_basic

Start an indefinite basic license, which gives access to all the basic features.

NOTE: In order to start a basic license, you must not currently have a basic license.

If the basic license does not support all of the features that are available with your current license, however, you are notified in the response. You must then re-submit the API request with the acknowledge parameter set to true.

To check the status of your basic license, use the get basic license API.

Required authorization

  • Cluster privileges: manage

Query parameters

  • acknowledge boolean

    whether the user has acknowledged acknowledge messages (default: false)

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • acknowledged boolean Required
    • basic_was_started boolean Required
    • error_message string
    • type string

      Values are missing, trial, basic, standard, dev, silver, gold, platinum, or enterprise.

    • acknowledge object
POST /_license/start_basic
POST /_license/start_basic?acknowledge=true
resp = client.license.post_start_basic(
    acknowledge=True,
)
const response = await client.license.postStartBasic({
  acknowledge: "true",
});
response = client.license.post_start_basic(
  acknowledge: "true"
)
$resp = $client->license()->postStartBasic([
    "acknowledge" => "true",
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_license/start_basic?acknowledge=true"
client.license().postStartBasic(p -> p
    .acknowledge(true)
);
Response examples (200)
A successful response from `POST /_license/start_basic?acknowledge=true`. If you currently have a license with more features than a basic license and you start a basic license, you must pass the acknowledge parameter.
{
  "basic_was_started": true,
  "acknowledged": true
}

Start a trial Generally available; Added in 6.1.0

POST /_license/start_trial

Start a 30-day trial, which gives access to all subscription features.

NOTE: You are allowed to start a trial only if your cluster has not already activated a trial for the current major product version. For example, if you have already activated a trial for v8.0, you cannot start a new trial until v9.0. You can, however, request an extended trial at https://www.elastic.co/trialextension.

To check the status of your trial, use the get trial status API.

Required authorization

  • Cluster privileges: manage

Query parameters

  • acknowledge boolean

    whether the user has acknowledged acknowledge messages (default: false)

  • type string

    The type of trial license to generate (default: "trial")

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • acknowledged boolean Required
    • error_message string
    • trial_was_started boolean Required
    • type string

      Values are missing, trial, basic, standard, dev, silver, gold, platinum, or enterprise.

POST /_license/start_trial
POST /_license/start_trial?acknowledge=true
resp = client.license.post_start_trial(
    acknowledge=True,
)
const response = await client.license.postStartTrial({
  acknowledge: "true",
});
response = client.license.post_start_trial(
  acknowledge: "true"
)
$resp = $client->license()->postStartTrial([
    "acknowledge" => "true",
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_license/start_trial?acknowledge=true"
client.license().postStartTrial(p -> p
    .acknowledge(true)
);
Response examples (200)
A successful response from `POST /_license/start_trial?acknowledge=true`.
{
  "trial_was_started": true,
  "acknowledged": true
}

Logstash

Logstash APIs enable you to manage pipelines that are used by Logstash Central Management.

Learn more about centralized pipeline management












Machine learning


































Add anomaly detection job to calendar Generally available; Added in 6.2.0

PUT /_ml/calendars/{calendar_id}/jobs/{job_id}

Required authorization

  • Cluster privileges: manage_ml

Path parameters

  • calendar_id string Required

    A string that uniquely identifies a calendar.

  • job_id string | array[string] Required

    An identifier for the anomaly detection jobs. It can be a job identifier, a group name, or a comma-separated list of jobs or groups.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • calendar_id string Required

      A string that uniquely identifies a calendar.

    • description string

      A description of the calendar.

    • job_ids string | array[string]

      A list of anomaly detection job identifiers or group names.

      One of:

      A list of anomaly detection job identifiers or group names.

PUT /_ml/calendars/{calendar_id}/jobs/{job_id}
PUT _ml/calendars/planned-outages/jobs/total-requests
resp = client.ml.put_calendar_job(
    calendar_id="planned-outages",
    job_id="total-requests",
)
const response = await client.ml.putCalendarJob({
  calendar_id: "planned-outages",
  job_id: "total-requests",
});
response = client.ml.put_calendar_job(
  calendar_id: "planned-outages",
  job_id: "total-requests"
)
$resp = $client->ml()->putCalendarJob([
    "calendar_id" => "planned-outages",
    "job_id" => "total-requests",
]);
curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_ml/calendars/planned-outages/jobs/total-requests"
client.ml().putCalendarJob(p -> p
    .calendarId("planned-outages")
    .jobId("total-requests")
);




















Get filters Generally available; Added in 5.5.0

GET /_ml/filters/{filter_id}

All methods and paths for this operation:

GET /_ml/filters

GET /_ml/filters/{filter_id}

You can get a single filter or all filters.

Required authorization

  • Cluster privileges: manage_ml

Path parameters

  • filter_id string | array[string] Required

    A string that uniquely identifies a filter.

Query parameters

  • from number

    Skips the specified number of filters.

  • size number

    Specifies the maximum number of filters to obtain.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • count number Required
    • filters array[object] Required
      Hide filters attributes Show filters attributes object
      • description string

        A description of the filter.

      • filter_id string Required

        A string that uniquely identifies a filter.

      • items array[string] Required

        An array of strings which is the filter item list.

GET /_ml/filters/{filter_id}
GET _ml/filters/safe_domains
resp = client.ml.get_filters(
    filter_id="safe_domains",
)
const response = await client.ml.getFilters({
  filter_id: "safe_domains",
});
response = client.ml.get_filters(
  filter_id: "safe_domains"
)
$resp = $client->ml()->getFilters([
    "filter_id" => "safe_domains",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_ml/filters/safe_domains"
client.ml().getFilters(g -> g
    .filterId("safe_domains")
);












Get anomaly detection jobs configuration info Generally available; Added in 5.5.0

GET /_ml/anomaly_detectors/{job_id}

All methods and paths for this operation:

GET /_ml/anomaly_detectors

GET /_ml/anomaly_detectors/{job_id}

You can get information for multiple anomaly detection jobs in a single API request by using a group name, a comma-separated list of jobs, or a wildcard expression. You can get information for all anomaly detection jobs by using _all, by specifying * as the <job_id>, or by omitting the <job_id>.

Required authorization

  • Cluster privileges: monitor_ml

Path parameters

  • job_id string | array[string] Required

    Identifier for the anomaly detection job. It can be a job identifier, a group name, or a wildcard expression. If you do not specify one of these options, the API returns information for all anomaly detection jobs.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    1. Contains wildcard expressions and there are no jobs that match.
    2. Contains the _all string or no identifiers and there are no matches.
    3. Contains wildcard expressions and there are only partial matches.

    The default value is true, which returns an empty jobs array when there are no matches and the subset of results when there are partial matches. If this parameter is false, the request returns a 404 status code when there are no matches or only partial matches.

  • exclude_generated boolean

    Indicates if certain fields should be removed from the configuration on retrieval. This allows the configuration to be in an acceptable format to be retrieved and then added to another cluster.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • count number Required
    • jobs array[object] Required
      Hide jobs attributes Show jobs attributes object
      • allow_lazy_open boolean Required

        Advanced configuration option. Specifies whether this job can open when there is insufficient machine learning node capacity for it to be immediately assigned to a node.

      • analysis_config object Required

        The analysis configuration, which specifies how to analyze the data. After you create a job, you cannot change the analysis configuration; all the properties are informational.

        Hide analysis_config attributes Show analysis_config attributes object
        • bucket_span string

          The size of the interval that the analysis is aggregated into, typically between 5m and 1h. This value should be either a whole number of days or equate to a whole number of buckets in one day. If the anomaly detection job uses a datafeed with aggregations, this value must also be divisible by the interval of the date histogram aggregation.

        • categorization_analyzer
        • categorization_field_name string

          If this property is specified, the values of the specified field will be categorized. The resulting categories must be used in a detector by setting by_field_name, over_field_name, or partition_field_name to the keyword mlcategory.

        • categorization_filters array[string]

          If categorization_field_name is specified, you can also define optional filters. This property expects an array of regular expressions. The expressions are used to filter out matching sequences from the categorization field values. You can use this functionality to fine tune the categorization by excluding sequences from consideration when categories are defined. For example, you can exclude SQL statements that appear in your log files. This property cannot be used at the same time as categorization_analyzer. If you only want to define simple regular expression filters that are applied prior to tokenization, setting this property is the easiest method. If you also want to customize the tokenizer or post-tokenization filtering, use the categorization_analyzer property instead and include the filters as pattern_replace character filters. The effect is exactly the same.

        • detectors array[object] Required

          Detector configuration objects specify which data fields a job analyzes. They also specify which analytical functions are used. You can specify multiple detectors for a job. If the detectors array does not contain at least one detector, no analysis can occur and an error is returned.

        • influencers array[string]

          A comma separated list of influencer field names. Typically these can be the by, over, or partition fields that are used in the detector configuration. You might also want to use a field name that is not specifically named in a detector, but is available as part of the input data. When you use multiple detectors, the use of influencers is recommended as it aggregates results for each influencer entity.

        • latency string

          The size of the window in which to expect data that is out of time order. If you specify a non-zero value, it must be greater than or equal to one second. NOTE: Latency is applicable only when you send data by using the post data API.

        • model_prune_window string

          Advanced configuration option. Affects the pruning of models that have not been updated for the given time duration. The value must be set to a multiple of the bucket_span. If set too low, important information may be removed from the model. For jobs created in 8.1 and later, the default value is the greater of 30d or 20 times bucket_span.

        • multivariate_by_fields boolean

          This functionality is reserved for internal use. It is not supported for use in customer environments and is not subject to the support SLA of official GA features. If set to true, the analysis will automatically find correlations between metrics for a given by field value and report anomalies when those correlations cease to hold. For example, suppose CPU and memory usage on host A is usually highly correlated with the same metrics on host B. Perhaps this correlation occurs because they are running a load-balanced application. If you enable this property, anomalies will be reported when, for example, CPU usage on host A is high and the value of CPU usage on host B is low. That is to say, you’ll see an anomaly when the CPU of host A is unusual given the CPU of host B. To use the multivariate_by_fields property, you must also specify by_field_name in your detector.

        • per_partition_categorization object

          Settings related to how categorization interacts with partition fields.

        • summary_count_field_name string

          If this property is specified, the data that is fed to the job is expected to be pre-summarized. This property value is the name of the field that contains the count of raw data points that have been summarized. The same summary_count_field_name applies to all detectors in the job. NOTE: The summary_count_field_name property cannot be used with the metric function.

      • analysis_limits object

        Limits can be applied for the resources required to hold the mathematical models in memory. These limits are approximate and can be set per job. They do not control the memory used by other processes, for example the Elasticsearch Java processes.

        Hide analysis_limits attributes Show analysis_limits attributes object
        • categorization_examples_limit number

          The maximum number of examples stored per category in memory and in the results data store. If you increase this value, more examples are available, however it requires that you have more storage available. If you set this value to 0, no examples are stored. NOTE: The categorization_examples_limit applies only to analysis that uses categorization.

          Default value is 4.

        • model_memory_limit
      • background_persist_interval string

        Advanced configuration option. The time between each periodic persistence of the model. The default value is a randomized value between 3 to 4 hours, which avoids all jobs persisting at exactly the same time. The smallest allowed value is 1 hour.

      • blocked object
        Hide blocked attributes Show blocked attributes object
        • reason string Required

          Values are delete, reset, or revert.

        • task_id
      • create_time string | number

        One of:
      • custom_settings object

        Advanced configuration option. Contains custom metadata about the job.

      • daily_model_snapshot_retention_after_days number

        Advanced configuration option, which affects the automatic removal of old model snapshots for this job. It specifies a period of time (in days) after which only the first snapshot per day is retained. This period is relative to the timestamp of the most recent snapshot for this job. Valid values range from 0 to model_snapshot_retention_days.

        Default value is 1.

      • data_description object Required

        The data description defines the format of the input data when you send data to the job by using the post data API. Note that when configuring a datafeed, these properties are automatically set. When data is received via the post data API, it is not stored in Elasticsearch. Only the results for anomaly detection are retained.

        Hide data_description attributes Show data_description attributes object
        • format string

          Only JSON format is supported at this time.

        • time_field string

          The name of the field that contains the timestamp.

        • time_format string

          The time format, which can be epoch, epoch_ms, or a custom pattern. The value epoch refers to UNIX or Epoch time (the number of seconds since 1 Jan 1970). The value epoch_ms indicates that time is measured in milliseconds since the epoch. The epoch and epoch_ms time formats accept either integer or real values. Custom patterns must conform to the Java DateTimeFormatter class. When you use date-time formatting patterns, it is recommended that you provide the full date, time and time zone. For example: yyyy-MM-dd'T'HH:mm:ssX. If the pattern that you specify is not sufficient to produce a complete timestamp, job creation fails.

          Default value is epoch.

        • field_delimiter string
      • datafeed_config object

        The datafeed, which retrieves data from Elasticsearch for analysis by the job. You can associate only one datafeed with each anomaly detection job.

        Hide datafeed_config attributes Show datafeed_config attributes object
        • aggregations object
        • authorization object

          The security privileges that the datafeed uses to run its queries. If Elastic Stack security features were disabled at the time of the most recent update to the datafeed, this property is omitted.

        • chunking_config object
        • datafeed_id string Required
        • frequency string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • indices array[string] Required
        • indexes array[string]
        • job_id string Required
        • max_empty_searches number
        • query_delay string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • script_fields object
          Hide script_fields attribute Show script_fields attribute object
          • * object Additional properties
        • scroll_size number
        • delayed_data_check_config object Required
        • runtime_mappings object
        • indices_options object

          Controls how to deal with unavailable concrete indices (closed or missing), how wildcard expressions are expanded to actual indices (all, closed or open indices) and how to deal with wildcard expressions that resolve to no indices.

        • query object Required

          The Elasticsearch query domain-specific language (DSL). This value corresponds to the query object in an Elasticsearch search POST body. All the options that are supported by Elasticsearch can be used, as this object is passed verbatim to Elasticsearch. By default, this property has the following value: {"match_all": {"boost": 1}}.

          Query DSL
      • deleting boolean

        Indicates that the process of deleting the job is in progress but not yet completed. It is only reported when true.

      • description string

        A description of the job.

      • finished_time string | number

        If the job closed or failed, this is the time the job finished, otherwise it is null. This property is informational; you cannot change its value.

        One of:

        If the job closed or failed, this is the time the job finished, otherwise it is null. This property is informational; you cannot change its value.

      • groups array[string]

        A list of job groups. A job can belong to no groups or many.

      • job_id string Required

        Identifier for the anomaly detection job. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.

      • job_type string

        Reserved for future use, currently set to anomaly_detector.

      • job_version string

        The machine learning configuration version number at which the the job was created.

      • model_plot_config object

        This advanced configuration option stores model information along with the results. It provides a more detailed view into anomaly detection. Model plot provides a simplified and indicative view of the model and its bounds.

        Hide model_plot_config attributes Show model_plot_config attributes object
        • annotations_enabled boolean Generally available; Added in 7.9.0

          If true, enables calculation and storage of the model change annotations for each entity that is being analyzed.

          Default value is true.

        • enabled boolean

          If true, enables calculation and storage of the model bounds for each entity that is being analyzed.

          Default value is false.

        • terms string

          Limits data collection to this comma separated list of partition or by field values. If terms are not specified or it is an empty string, no filtering is applied. Wildcards are not supported. Only the specified terms can be viewed when using the Single Metric Viewer.

      • model_snapshot_id string
      • model_snapshot_retention_days number Required

        Advanced configuration option, which affects the automatic removal of old model snapshots for this job. It specifies the maximum period of time (in days) that snapshots are retained. This period is relative to the timestamp of the most recent snapshot for this job. By default, snapshots ten days older than the newest snapshot are deleted.

      • renormalization_window_days number

        Advanced configuration option. The period over which adjustments to the score are applied, as new data is seen. The default value is the longer of 30 days or 100 bucket_spans.

      • results_index_name string Required

        A text string that affects the name of the machine learning results index. The default value is shared, which generates an index named .ml-anomalies-shared.

      • results_retention_days number

        Advanced configuration option. The period of time (in days) that results are retained. Age is calculated relative to the timestamp of the latest bucket result. If this property has a non-null value, once per day at 00:30 (server time), results that are the specified number of days older than the latest bucket result are deleted from Elasticsearch. The default value is null, which means all results are retained. Annotations generated by the system also count as results for retention purposes; they are deleted after the same number of days as results. Annotations added by users are retained forever.

GET /_ml/anomaly_detectors/{job_id}
GET _ml/anomaly_detectors/high_sum_total_sales
resp = client.ml.get_jobs(
    job_id="high_sum_total_sales",
)
const response = await client.ml.getJobs({
  job_id: "high_sum_total_sales",
});
response = client.ml.get_jobs(
  job_id: "high_sum_total_sales"
)
$resp = $client->ml()->getJobs([
    "job_id" => "high_sum_total_sales",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_ml/anomaly_detectors/high_sum_total_sales"
client.ml().getJobs(g -> g
    .jobId("high_sum_total_sales")
);

Create an anomaly detection job Generally available; Added in 5.4.0

PUT /_ml/anomaly_detectors/{job_id}

If you include a datafeed_config, you must have read index privileges on the source index. If you include a datafeed_config but do not provide a query, the datafeed uses {"match_all": {"boost": 1}}.

Required authorization

  • Index privileges: read
  • Cluster privileges: manage_ml

Path parameters

  • job_id string Required

    The identifier for the anomaly detection job. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.

Query parameters

  • allow_no_indices boolean

    If true, wildcard indices expressions that resolve into no concrete indices are ignored. This includes the _all string or when no indices are specified.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.

    Values are all, open, closed, hidden, or none.

  • ignore_throttled boolean Deprecated

    If true, concrete, expanded or aliased indices are ignored when frozen.

  • ignore_unavailable boolean

    If true, unavailable indices (missing or closed) are ignored.

application/json

Body Required

  • allow_lazy_open boolean

    Advanced configuration option. Specifies whether this job can open when there is insufficient machine learning node capacity for it to be immediately assigned to a node. By default, if a machine learning node with capacity to run the job cannot immediately be found, the open anomaly detection jobs API returns an error. However, this is also subject to the cluster-wide xpack.ml.max_lazy_ml_nodes setting. If this option is set to true, the open anomaly detection jobs API does not return an error and the job waits in the opening state until sufficient machine learning node capacity is available.

    Default value is false.

  • analysis_config object Required

    Specifies how to analyze the data. After you create a job, you cannot change the analysis configuration; all the properties are informational.

    Hide analysis_config attributes Show analysis_config attributes object
    • bucket_span string

      The size of the interval that the analysis is aggregated into, typically between 5m and 1h. This value should be either a whole number of days or equate to a whole number of buckets in one day. If the anomaly detection job uses a datafeed with aggregations, this value must also be divisible by the interval of the date histogram aggregation.

    • categorization_analyzer string | object

      If categorization_field_name is specified, you can also define the analyzer that is used to interpret the categorization field. This property cannot be used at the same time as categorization_filters. The categorization analyzer specifies how the categorization_field is interpreted by the categorization process. The categorization_analyzer field can be specified either as a string or as an object. If it is a string, it must refer to a built-in analyzer or one added by another plugin.

      One of:

      If categorization_field_name is specified, you can also define the analyzer that is used to interpret the categorization field. This property cannot be used at the same time as categorization_filters. The categorization analyzer specifies how the categorization_field is interpreted by the categorization process. The categorization_analyzer field can be specified either as a string or as an object. If it is a string, it must refer to a built-in analyzer or one added by another plugin.

    • categorization_field_name string

      If this property is specified, the values of the specified field will be categorized. The resulting categories must be used in a detector by setting by_field_name, over_field_name, or partition_field_name to the keyword mlcategory.

    • categorization_filters array[string]

      If categorization_field_name is specified, you can also define optional filters. This property expects an array of regular expressions. The expressions are used to filter out matching sequences from the categorization field values. You can use this functionality to fine tune the categorization by excluding sequences from consideration when categories are defined. For example, you can exclude SQL statements that appear in your log files. This property cannot be used at the same time as categorization_analyzer. If you only want to define simple regular expression filters that are applied prior to tokenization, setting this property is the easiest method. If you also want to customize the tokenizer or post-tokenization filtering, use the categorization_analyzer property instead and include the filters as pattern_replace character filters. The effect is exactly the same.

    • detectors array[object] Required

      Detector configuration objects specify which data fields a job analyzes. They also specify which analytical functions are used. You can specify multiple detectors for a job. If the detectors array does not contain at least one detector, no analysis can occur and an error is returned.

      Hide detectors attributes Show detectors attributes object
      • by_field_name string

        The field used to split the data. In particular, this property is used for analyzing the splits with respect to their own history. It is used for finding unusual values in the context of the split.

      • custom_rules array[object]

        Custom rules enable you to customize the way detectors operate. For example, a rule may dictate conditions under which results should be skipped. Kibana refers to custom rules as job rules.

        Hide custom_rules attributes Show custom_rules attributes object
        • actions array[string]

          The set of actions to be triggered when the rule applies. If more than one action is specified the effects of all actions are combined.

          Supported values include:

          • skip_result: The result will not be created. Unless you also specify skip_model_update, the model will be updated as usual with the corresponding series value.
          • skip_model_update: The value for that series will not be used to update the model. Unless you also specify skip_result, the results will be created as usual. This action is suitable when certain values are expected to be consistently anomalous and they affect the model in a way that negatively impacts the rest of the results.

          Values are skip_result or skip_model_update. Default value is ["skip_result"].

        • conditions array[object]

          An array of numeric conditions when the rule applies. A rule must either have a non-empty scope or at least one condition. Multiple conditions are combined together with a logical AND.

        • scope object

          A scope of series where the rule applies. A rule must either have a non-empty scope or at least one condition. By default, the scope includes all series. Scoping is allowed for any of the fields that are also specified in by_field_name, over_field_name, or partition_field_name.

      • detector_description string

        A description of the detector.

      • detector_index number

        A unique identifier for the detector. This identifier is based on the order of the detectors in the analysis_config, starting at zero. If you specify a value for this property, it is ignored.

      • exclude_frequent string

        If set, frequent entities are excluded from influencing the anomaly results. Entities can be considered frequent over time or frequent in a population. If you are working with both over and by fields, you can set exclude_frequent to all for both fields, or to by or over for those specific fields.

        Values are all, none, by, or over.

      • field_name string

        The field that the detector uses in the function. If you use an event rate function such as count or rare, do not specify this field. The field_name cannot contain double quotes or backslashes.

      • function string

        The analysis function that is used. For example, count, rare, mean, min, max, or sum.

      • over_field_name string

        The field used to split the data. In particular, this property is used for analyzing the splits with respect to the history of all splits. It is used for finding unusual values in the population of all splits.

      • partition_field_name string

        The field used to segment the analysis. When you use this property, you have completely independent baselines for each value of this field.

      • use_null boolean

        Defines whether a new series is used as the null series when there is no value for the by or partition fields.

        Default value is false.

    • influencers array[string]

      A comma separated list of influencer field names. Typically these can be the by, over, or partition fields that are used in the detector configuration. You might also want to use a field name that is not specifically named in a detector, but is available as part of the input data. When you use multiple detectors, the use of influencers is recommended as it aggregates results for each influencer entity.

    • latency string

      The size of the window in which to expect data that is out of time order. If you specify a non-zero value, it must be greater than or equal to one second. NOTE: Latency is applicable only when you send data by using the post data API.

    • model_prune_window string

      Advanced configuration option. Affects the pruning of models that have not been updated for the given time duration. The value must be set to a multiple of the bucket_span. If set too low, important information may be removed from the model. For jobs created in 8.1 and later, the default value is the greater of 30d or 20 times bucket_span.

    • multivariate_by_fields boolean

      This functionality is reserved for internal use. It is not supported for use in customer environments and is not subject to the support SLA of official GA features. If set to true, the analysis will automatically find correlations between metrics for a given by field value and report anomalies when those correlations cease to hold. For example, suppose CPU and memory usage on host A is usually highly correlated with the same metrics on host B. Perhaps this correlation occurs because they are running a load-balanced application. If you enable this property, anomalies will be reported when, for example, CPU usage on host A is high and the value of CPU usage on host B is low. That is to say, you’ll see an anomaly when the CPU of host A is unusual given the CPU of host B. To use the multivariate_by_fields property, you must also specify by_field_name in your detector.

    • per_partition_categorization object

      Settings related to how categorization interacts with partition fields.

      Hide per_partition_categorization attributes Show per_partition_categorization attributes object
      • enabled boolean

        To enable this setting, you must also set the partition_field_name property to the same value in every detector that uses the keyword mlcategory. Otherwise, job creation fails.

      • stop_on_warn boolean

        This setting can be set to true only if per-partition categorization is enabled. If true, both categorization and subsequent anomaly detection stops for partitions where the categorization status changes to warn. This setting makes it viable to have a job where it is expected that categorization works well for some partitions but not others; you do not pay the cost of bad categorization forever in the partitions where it works badly.

    • summary_count_field_name string

      If this property is specified, the data that is fed to the job is expected to be pre-summarized. This property value is the name of the field that contains the count of raw data points that have been summarized. The same summary_count_field_name applies to all detectors in the job. NOTE: The summary_count_field_name property cannot be used with the metric function.

  • analysis_limits object

    Limits can be applied for the resources required to hold the mathematical models in memory. These limits are approximate and can be set per job. They do not control the memory used by other processes, for example the Elasticsearch Java processes.

    Hide analysis_limits attributes Show analysis_limits attributes object
    • categorization_examples_limit number

      The maximum number of examples stored per category in memory and in the results data store. If you increase this value, more examples are available, however it requires that you have more storage available. If you set this value to 0, no examples are stored. NOTE: The categorization_examples_limit applies only to analysis that uses categorization.

      Default value is 4.

    • model_memory_limit number | string

      The approximate maximum amount of memory resources that are required for analytical processing. Once this limit is approached, data pruning becomes more aggressive. Upon exceeding this limit, new entities are not modeled. If the xpack.ml.max_model_memory_limit setting has a value greater than 0 and less than 1024mb, that value is used instead of the default. The default value is relatively small to ensure that high resource usage is a conscious decision. If you have jobs that are expected to analyze high cardinality fields, you will likely need to use a higher value. If you specify a number instead of a string, the units are assumed to be MiB. Specifying a string is recommended for clarity. If you specify a byte size unit of b or kb and the number does not equate to a discrete number of megabytes, it is rounded down to the closest MiB. The minimum valid value is 1 MiB. If you specify a value less than 1 MiB, an error occurs. If you specify a value for the xpack.ml.max_model_memory_limit setting, an error occurs when you try to create jobs that have model_memory_limit values greater than that setting value.

      One of:

      The approximate maximum amount of memory resources that are required for analytical processing. Once this limit is approached, data pruning becomes more aggressive. Upon exceeding this limit, new entities are not modeled. If the xpack.ml.max_model_memory_limit setting has a value greater than 0 and less than 1024mb, that value is used instead of the default. The default value is relatively small to ensure that high resource usage is a conscious decision. If you have jobs that are expected to analyze high cardinality fields, you will likely need to use a higher value. If you specify a number instead of a string, the units are assumed to be MiB. Specifying a string is recommended for clarity. If you specify a byte size unit of b or kb and the number does not equate to a discrete number of megabytes, it is rounded down to the closest MiB. The minimum valid value is 1 MiB. If you specify a value less than 1 MiB, an error occurs. If you specify a value for the xpack.ml.max_model_memory_limit setting, an error occurs when you try to create jobs that have model_memory_limit values greater than that setting value.

  • background_persist_interval string

    Advanced configuration option. The time between each periodic persistence of the model. The default value is a randomized value between 3 to 4 hours, which avoids all jobs persisting at exactly the same time. The smallest allowed value is 1 hour. For very large models (several GB), persistence could take 10-20 minutes, so do not set the background_persist_interval value too low.

  • custom_settings object

    Advanced configuration option. Contains custom meta data about the job.

  • daily_model_snapshot_retention_after_days number

    Advanced configuration option, which affects the automatic removal of old model snapshots for this job. It specifies a period of time (in days) after which only the first snapshot per day is retained. This period is relative to the timestamp of the most recent snapshot for this job. Valid values range from 0 to model_snapshot_retention_days.

    Default value is 1.

  • data_description object Required

    Defines the format of the input data when you send data to the job by using the post data API. Note that when configure a datafeed, these properties are automatically set. When data is received via the post data API, it is not stored in Elasticsearch. Only the results for anomaly detection are retained.

    Hide data_description attributes Show data_description attributes object
    • format string

      Only JSON format is supported at this time.

    • time_field string

      The name of the field that contains the timestamp.

    • time_format string

      The time format, which can be epoch, epoch_ms, or a custom pattern. The value epoch refers to UNIX or Epoch time (the number of seconds since 1 Jan 1970). The value epoch_ms indicates that time is measured in milliseconds since the epoch. The epoch and epoch_ms time formats accept either integer or real values. Custom patterns must conform to the Java DateTimeFormatter class. When you use date-time formatting patterns, it is recommended that you provide the full date, time and time zone. For example: yyyy-MM-dd'T'HH:mm:ssX. If the pattern that you specify is not sufficient to produce a complete timestamp, job creation fails.

      Default value is epoch.

    • field_delimiter string
  • datafeed_config object

    Defines a datafeed for the anomaly detection job. If Elasticsearch security features are enabled, your datafeed remembers which roles the user who created it had at the time of creation and runs the query using those same roles. If you provide secondary authorization headers, those credentials are used instead.

    Hide datafeed_config attributes Show datafeed_config attributes object
    • aggregations object

      If set, the datafeed performs aggregation searches. Support for aggregations is limited and should be used only with low cardinality data.

    • chunking_config object

      Datafeeds might be required to search over long time periods, for several months or years. This search is split into time chunks in order to ensure the load on Elasticsearch is managed. Chunking configuration controls how the size of these time chunks are calculated and is an advanced configuration option.

      Hide chunking_config attributes Show chunking_config attributes object
      • mode string Required

        If the mode is auto, the chunk size is dynamically calculated; this is the recommended value when the datafeed does not use aggregations. If the mode is manual, chunking is applied according to the specified time_span; use this mode when the datafeed uses aggregations. If the mode is off, no chunking is applied.

        Values are auto, manual, or off.

      • time_span string

        The time span that each search will be querying. This setting is applicable only when the mode is set to manual.

    • datafeed_id string

      A numerical character string that uniquely identifies the datafeed. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters. The default value is the job identifier.

    • delayed_data_check_config object

      Specifies whether the datafeed checks for missing data and the size of the window. The datafeed can optionally search over indices that have already been read in an effort to determine whether any data has subsequently been added to the index. If missing data is found, it is a good indication that the query_delay option is set too low and the data is being indexed after the datafeed has passed that moment in time. This check runs only on real-time datafeeds.

      Hide delayed_data_check_config attributes Show delayed_data_check_config attributes object
      • check_window string

        The window of time that is searched for late data. This window of time ends with the latest finalized bucket. It defaults to null, which causes an appropriate check_window to be calculated when the real-time datafeed runs. In particular, the default check_window span calculation is based on the maximum of 2h or 8 * bucket_span.

      • enabled boolean Required

        Specifies whether the datafeed periodically checks for delayed data.

    • frequency string

      The interval at which scheduled queries are made while the datafeed runs in real time. The default value is either the bucket span for short bucket spans, or, for longer bucket spans, a sensible fraction of the bucket span. For example: 150s. When frequency is shorter than the bucket span, interim results for the last (partial) bucket are written then eventually overwritten by the full bucket results. If the datafeed uses aggregations, this value must be divisible by the interval of the date histogram aggregation.

    • indices string | array[string]

      An array of index names. Wildcards are supported. If any indices are in remote clusters, the machine learning nodes must have the remote_cluster_client role.

    • indices_options object

      Specifies index expansion options that are used during search.

      Hide indices_options attributes Show indices_options attributes object
      • allow_no_indices boolean

        If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

      • expand_wildcards string | array[string]

        Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

        Supported values include:

        • all: Match any data stream or index, including hidden ones.
        • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
        • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
        • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
        • none: Wildcard expressions are not accepted.
      • ignore_unavailable boolean

        If true, missing or closed indices are not included in the response.

        Default value is false.

      • ignore_throttled boolean

        If true, concrete, expanded or aliased indices are ignored when frozen.

        Default value is true.

    • job_id string
    • max_empty_searches number

      If a real-time datafeed has never seen any data (including during any initial training period) then it will automatically stop itself and close its associated job after this many real-time searches that return no documents. In other words, it will stop after frequency times max_empty_searches of real-time operation. If not set then a datafeed with no end time that sees no data will remain started until it is explicitly stopped.

    • query object

      The Elasticsearch query domain-specific language (DSL). This value corresponds to the query object in an Elasticsearch search POST body. All the options that are supported by Elasticsearch can be used, as this object is passed verbatim to Elasticsearch.

      External documentation
    • query_delay string

      The number of seconds behind real time that data is queried. For example, if data from 10:04 a.m. might not be searchable in Elasticsearch until 10:06 a.m., set this property to 120 seconds. The default value is randomly selected between 60s and 120s. This randomness improves the query performance when there are multiple jobs running on the same node.

    • runtime_mappings object

      Specifies runtime fields for the datafeed search.

      Hide runtime_mappings attribute Show runtime_mappings attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • fields object

          For type composite

          Hide fields attribute Show fields attribute object
          • * object Additional properties
        • fetch_fields array[object]

          For type lookup

        • format string

          A custom format for date type runtime fields.

        • input_field string

          For type lookup

        • target_field string

          For type lookup

        • target_index string

          For type lookup

        • script object

          Painless script executed at query time.

        • type string Required

          Field type, which can be: boolean, composite, date, double, geo_point, ip,keyword, long, or lookup.

          Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

    • script_fields object

      Specifies scripts that evaluate custom expressions and returns script fields to the datafeed. The detector configuration objects in a job can contain functions that use these script fields.

      Hide script_fields attribute Show script_fields attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • script object Required
          Hide script attributes Show script attributes object
          • source string

            The script source.

          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • ignore_failure boolean
    • scroll_size number

      The size parameter that is used in Elasticsearch searches when the datafeed does not use aggregations. The maximum value is the value of index.max_result_window, which is 10,000 by default.

      Default value is 1000.

  • description string

    A description of the job.

  • job_id string

    The identifier for the anomaly detection job. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.

  • groups array[string]

    A list of job groups. A job can belong to no groups or many.

  • model_plot_config object

    This advanced configuration option stores model information along with the results. It provides a more detailed view into anomaly detection. If you enable model plot it can add considerable overhead to the performance of the system; it is not feasible for jobs with many entities. Model plot provides a simplified and indicative view of the model and its bounds. It does not display complex features such as multivariate correlations or multimodal data. As such, anomalies may occasionally be reported which cannot be seen in the model plot. Model plot config can be configured when the job is created or updated later. It must be disabled if performance issues are experienced.

    Hide model_plot_config attributes Show model_plot_config attributes object
    • annotations_enabled boolean Generally available; Added in 7.9.0

      If true, enables calculation and storage of the model change annotations for each entity that is being analyzed.

      Default value is true.

    • enabled boolean

      If true, enables calculation and storage of the model bounds for each entity that is being analyzed.

      Default value is false.

    • terms string

      Limits data collection to this comma separated list of partition or by field values. If terms are not specified or it is an empty string, no filtering is applied. Wildcards are not supported. Only the specified terms can be viewed when using the Single Metric Viewer.

  • model_snapshot_retention_days number

    Advanced configuration option, which affects the automatic removal of old model snapshots for this job. It specifies the maximum period of time (in days) that snapshots are retained. This period is relative to the timestamp of the most recent snapshot for this job. By default, snapshots ten days older than the newest snapshot are deleted.

    Default value is 10.

  • renormalization_window_days number

    Advanced configuration option. The period over which adjustments to the score are applied, as new data is seen. The default value is the longer of 30 days or 100 bucket spans.

  • results_index_name string

    A text string that affects the name of the machine learning results index. By default, the job generates an index named .ml-anomalies-shared.

  • results_retention_days number

    Advanced configuration option. The period of time (in days) that results are retained. Age is calculated relative to the timestamp of the latest bucket result. If this property has a non-null value, once per day at 00:30 (server time), results that are the specified number of days older than the latest bucket result are deleted from Elasticsearch. The default value is null, which means all results are retained. Annotations generated by the system also count as results for retention purposes; they are deleted after the same number of days as results. Annotations added by users are retained forever.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • allow_lazy_open boolean Required
    • analysis_config object Required
      Hide analysis_config attributes Show analysis_config attributes object
      • bucket_span string Required

        The size of the interval that the analysis is aggregated into, typically between 5m and 1h.

      • categorization_analyzer string | object

        If categorization_field_name is specified, you can also define the analyzer that is used to interpret the categorization field. This property cannot be used at the same time as categorization_filters. The categorization analyzer specifies how the categorization_field is interpreted by the categorization process.

        One of:

        If categorization_field_name is specified, you can also define the analyzer that is used to interpret the categorization field. This property cannot be used at the same time as categorization_filters. The categorization analyzer specifies how the categorization_field is interpreted by the categorization process.

      • categorization_field_name string

        If this property is specified, the values of the specified field will be categorized. The resulting categories must be used in a detector by setting by_field_name, over_field_name, or partition_field_name to the keyword mlcategory.

      • categorization_filters array[string]

        If categorization_field_name is specified, you can also define optional filters. This property expects an array of regular expressions. The expressions are used to filter out matching sequences from the categorization field values.

      • detectors array[object] Required

        An array of detector configuration objects. Detector configuration objects specify which data fields a job analyzes. They also specify which analytical functions are used. You can specify multiple detectors for a job.

        Hide detectors attributes Show detectors attributes object
        • by_field_name string

          The field used to split the data. In particular, this property is used for analyzing the splits with respect to their own history. It is used for finding unusual values in the context of the split.

        • custom_rules array[object]

          An array of custom rule objects, which enable you to customize the way detectors operate. For example, a rule may dictate to the detector conditions under which results should be skipped. Kibana refers to custom rules as job rules.

        • detector_description string

          A description of the detector.

        • detector_index number

          A unique identifier for the detector. This identifier is based on the order of the detectors in the analysis_config, starting at zero.

        • exclude_frequent string

          Contains one of the following values: all, none, by, or over. If set, frequent entities are excluded from influencing the anomaly results. Entities can be considered frequent over time or frequent in a population. If you are working with both over and by fields, then you can set exclude_frequent to all for both fields, or to by or over for those specific fields.

          Values are all, none, by, or over.

        • field_name string

          The field that the detector uses in the function. If you use an event rate function such as count or rare, do not specify this field.

        • function string Required

          The analysis function that is used. For example, count, rare, mean, min, max, and sum.

        • over_field_name string

          The field used to split the data. In particular, this property is used for analyzing the splits with respect to the history of all splits. It is used for finding unusual values in the population of all splits.

        • partition_field_name string

          The field used to segment the analysis. When you use this property, you have completely independent baselines for each value of this field.

        • use_null boolean

          Defines whether a new series is used as the null series when there is no value for the by or partition fields.

          Default value is false.

      • influencers array[string] Required

        A comma separated list of influencer field names. Typically these can be the by, over, or partition fields that are used in the detector configuration. You might also want to use a field name that is not specifically named in a detector, but is available as part of the input data. When you use multiple detectors, the use of influencers is recommended as it aggregates results for each influencer entity.

      • model_prune_window string

        Advanced configuration option. Affects the pruning of models that have not been updated for the given time duration. The value must be set to a multiple of the bucket_span. If set too low, important information may be removed from the model. Typically, set to 30d or longer. If not set, model pruning only occurs if the model memory status reaches the soft limit or the hard limit. For jobs created in 8.1 and later, the default value is the greater of 30d or 20 times bucket_span.

      • latency string

        The size of the window in which to expect data that is out of time order. Defaults to no latency. If you specify a non-zero value, it must be greater than or equal to one second.

      • multivariate_by_fields boolean

        This functionality is reserved for internal use. It is not supported for use in customer environments and is not subject to the support SLA of official GA features. If set to true, the analysis will automatically find correlations between metrics for a given by field value and report anomalies when those correlations cease to hold.

      • per_partition_categorization object

        Settings related to how categorization interacts with partition fields.

        Hide per_partition_categorization attributes Show per_partition_categorization attributes object
        • enabled boolean

          To enable this setting, you must also set the partition_field_name property to the same value in every detector that uses the keyword mlcategory. Otherwise, job creation fails.

        • stop_on_warn boolean

          This setting can be set to true only if per-partition categorization is enabled. If true, both categorization and subsequent anomaly detection stops for partitions where the categorization status changes to warn. This setting makes it viable to have a job where it is expected that categorization works well for some partitions but not others; you do not pay the cost of bad categorization forever in the partitions where it works badly.

      • summary_count_field_name string

        If this property is specified, the data that is fed to the job is expected to be pre-summarized. This property value is the name of the field that contains the count of raw data points that have been summarized. The same summary_count_field_name applies to all detectors in the job.

    • analysis_limits object Required
      Hide analysis_limits attributes Show analysis_limits attributes object
      • categorization_examples_limit number

        The maximum number of examples stored per category in memory and in the results data store. If you increase this value, more examples are available, however it requires that you have more storage available. If you set this value to 0, no examples are stored. NOTE: The categorization_examples_limit applies only to analysis that uses categorization.

        Default value is 4.

      • model_memory_limit number | string

        The approximate maximum amount of memory resources that are required for analytical processing. Once this limit is approached, data pruning becomes more aggressive. Upon exceeding this limit, new entities are not modeled. If the xpack.ml.max_model_memory_limit setting has a value greater than 0 and less than 1024mb, that value is used instead of the default. The default value is relatively small to ensure that high resource usage is a conscious decision. If you have jobs that are expected to analyze high cardinality fields, you will likely need to use a higher value. If you specify a number instead of a string, the units are assumed to be MiB. Specifying a string is recommended for clarity. If you specify a byte size unit of b or kb and the number does not equate to a discrete number of megabytes, it is rounded down to the closest MiB. The minimum valid value is 1 MiB. If you specify a value less than 1 MiB, an error occurs. If you specify a value for the xpack.ml.max_model_memory_limit setting, an error occurs when you try to create jobs that have model_memory_limit values greater than that setting value.

        One of:

        The approximate maximum amount of memory resources that are required for analytical processing. Once this limit is approached, data pruning becomes more aggressive. Upon exceeding this limit, new entities are not modeled. If the xpack.ml.max_model_memory_limit setting has a value greater than 0 and less than 1024mb, that value is used instead of the default. The default value is relatively small to ensure that high resource usage is a conscious decision. If you have jobs that are expected to analyze high cardinality fields, you will likely need to use a higher value. If you specify a number instead of a string, the units are assumed to be MiB. Specifying a string is recommended for clarity. If you specify a byte size unit of b or kb and the number does not equate to a discrete number of megabytes, it is rounded down to the closest MiB. The minimum valid value is 1 MiB. If you specify a value less than 1 MiB, an error occurs. If you specify a value for the xpack.ml.max_model_memory_limit setting, an error occurs when you try to create jobs that have model_memory_limit values greater than that setting value.

    • background_persist_interval string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • create_time string | number

      One of:
    • custom_settings object

      Custom metadata about the job

    • daily_model_snapshot_retention_after_days number Required
    • data_description object Required
      Hide data_description attributes Show data_description attributes object
      • format string

        Only JSON format is supported at this time.

      • time_field string

        The name of the field that contains the timestamp.

      • time_format string

        The time format, which can be epoch, epoch_ms, or a custom pattern. The value epoch refers to UNIX or Epoch time (the number of seconds since 1 Jan 1970). The value epoch_ms indicates that time is measured in milliseconds since the epoch. The epoch and epoch_ms time formats accept either integer or real values. Custom patterns must conform to the Java DateTimeFormatter class. When you use date-time formatting patterns, it is recommended that you provide the full date, time and time zone. For example: yyyy-MM-dd'T'HH:mm:ssX. If the pattern that you specify is not sufficient to produce a complete timestamp, job creation fails.

        Default value is epoch.

      • field_delimiter string
    • datafeed_config object
      Hide datafeed_config attributes Show datafeed_config attributes object
      • aggregations object
      • authorization object

        The security privileges that the datafeed uses to run its queries. If Elastic Stack security features were disabled at the time of the most recent update to the datafeed, this property is omitted.

        Hide authorization attributes Show authorization attributes object
        • api_key object

          If an API key was used for the most recent update to the datafeed, its name and identifier are listed in the response.

        • roles array[string]

          If a user ID was used for the most recent update to the datafeed, its roles at the time of the update are listed in the response.

        • service_account string

          If a service account was used for the most recent update to the datafeed, the account name is listed in the response.

      • chunking_config object
        Hide chunking_config attributes Show chunking_config attributes object
        • mode string Required

          If the mode is auto, the chunk size is dynamically calculated; this is the recommended value when the datafeed does not use aggregations. If the mode is manual, chunking is applied according to the specified time_span; use this mode when the datafeed uses aggregations. If the mode is off, no chunking is applied.

          Values are auto, manual, or off.

        • time_span string

          The time span that each search will be querying. This setting is applicable only when the mode is set to manual.

      • datafeed_id string Required
      • frequency string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • indices array[string] Required
      • indexes array[string]
      • job_id string Required
      • max_empty_searches number
      • query_delay string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • script_fields object
        Hide script_fields attribute Show script_fields attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • script object Required
          • ignore_failure boolean
      • scroll_size number
      • delayed_data_check_config object Required
        Hide delayed_data_check_config attributes Show delayed_data_check_config attributes object
        • check_window string

          The window of time that is searched for late data. This window of time ends with the latest finalized bucket. It defaults to null, which causes an appropriate check_window to be calculated when the real-time datafeed runs. In particular, the default check_window span calculation is based on the maximum of 2h or 8 * bucket_span.

        • enabled boolean Required

          Specifies whether the datafeed periodically checks for delayed data.

      • runtime_mappings object
        Hide runtime_mappings attribute Show runtime_mappings attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • fields object

            For type composite

          • fetch_fields array[object]

            For type lookup

          • format string

            A custom format for date type runtime fields.

      • indices_options object

        Controls how to deal with unavailable concrete indices (closed or missing), how wildcard expressions are expanded to actual indices (all, closed or open indices) and how to deal with wildcard expressions that resolve to no indices.

        Hide indices_options attributes Show indices_options attributes object
        • allow_no_indices boolean

          If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

        • expand_wildcards string | array[string]

          Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

          Supported values include:

          • all: Match any data stream or index, including hidden ones.
          • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
          • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
          • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
          • none: Wildcard expressions are not accepted.
        • ignore_unavailable boolean

          If true, missing or closed indices are not included in the response.

          Default value is false.

        • ignore_throttled boolean

          If true, concrete, expanded or aliased indices are ignored when frozen.

          Default value is true.

      • query object Required

        The Elasticsearch query domain-specific language (DSL). This value corresponds to the query object in an Elasticsearch search POST body. All the options that are supported by Elasticsearch can be used, as this object is passed verbatim to Elasticsearch. By default, this property has the following value: {"match_all": {"boost": 1}}.

        Query DSL
    • description string
    • groups array[string]
    • job_id string Required
    • job_type string Required
    • job_version string Required
    • model_plot_config object
      Hide model_plot_config attributes Show model_plot_config attributes object
      • annotations_enabled boolean Generally available; Added in 7.9.0

        If true, enables calculation and storage of the model change annotations for each entity that is being analyzed.

        Default value is true.

      • enabled boolean

        If true, enables calculation and storage of the model bounds for each entity that is being analyzed.

        Default value is false.

      • terms string

        Limits data collection to this comma separated list of partition or by field values. If terms are not specified or it is an empty string, no filtering is applied. Wildcards are not supported. Only the specified terms can be viewed when using the Single Metric Viewer.

    • model_snapshot_id string
    • model_snapshot_retention_days number Required
    • renormalization_window_days number
    • results_index_name string Required
    • results_retention_days number
PUT /_ml/anomaly_detectors/{job_id}
PUT /_ml/anomaly_detectors/job-01
{
  "analysis_config": {
    "bucket_span": "15m",
    "detectors": [
      {
        "detector_description": "Sum of bytes",
        "function": "sum",
        "field_name": "bytes"
      }
    ]
  },
  "data_description": {
    "time_field": "timestamp",
    "time_format": "epoch_ms"
  },
  "analysis_limits": {
    "model_memory_limit": "11MB"
  },
  "model_plot_config": {
    "enabled": true,
    "annotations_enabled": true
  },
  "results_index_name": "test-job1",
  "datafeed_config": {
    "indices": [
      "kibana_sample_data_logs"
    ],
    "query": {
      "bool": {
        "must": [
          {
            "match_all": {}
          }
        ]
      }
    },
    "runtime_mappings": {
      "hour_of_day": {
        "type": "long",
        "script": {
          "source": "emit(doc['timestamp'].value.getHour());"
        }
      }
    },
    "datafeed_id": "datafeed-test-job1"
  }
}
resp = client.ml.put_job(
    job_id="job-01",
    analysis_config={
        "bucket_span": "15m",
        "detectors": [
            {
                "detector_description": "Sum of bytes",
                "function": "sum",
                "field_name": "bytes"
            }
        ]
    },
    data_description={
        "time_field": "timestamp",
        "time_format": "epoch_ms"
    },
    analysis_limits={
        "model_memory_limit": "11MB"
    },
    model_plot_config={
        "enabled": True,
        "annotations_enabled": True
    },
    results_index_name="test-job1",
    datafeed_config={
        "indices": [
            "kibana_sample_data_logs"
        ],
        "query": {
            "bool": {
                "must": [
                    {
                        "match_all": {}
                    }
                ]
            }
        },
        "runtime_mappings": {
            "hour_of_day": {
                "type": "long",
                "script": {
                    "source": "emit(doc['timestamp'].value.getHour());"
                }
            }
        },
        "datafeed_id": "datafeed-test-job1"
    },
)
const response = await client.ml.putJob({
  job_id: "job-01",
  analysis_config: {
    bucket_span: "15m",
    detectors: [
      {
        detector_description: "Sum of bytes",
        function: "sum",
        field_name: "bytes",
      },
    ],
  },
  data_description: {
    time_field: "timestamp",
    time_format: "epoch_ms",
  },
  analysis_limits: {
    model_memory_limit: "11MB",
  },
  model_plot_config: {
    enabled: true,
    annotations_enabled: true,
  },
  results_index_name: "test-job1",
  datafeed_config: {
    indices: ["kibana_sample_data_logs"],
    query: {
      bool: {
        must: [
          {
            match_all: {},
          },
        ],
      },
    },
    runtime_mappings: {
      hour_of_day: {
        type: "long",
        script: {
          source: "emit(doc['timestamp'].value.getHour());",
        },
      },
    },
    datafeed_id: "datafeed-test-job1",
  },
});
response = client.ml.put_job(
  job_id: "job-01",
  body: {
    "analysis_config": {
      "bucket_span": "15m",
      "detectors": [
        {
          "detector_description": "Sum of bytes",
          "function": "sum",
          "field_name": "bytes"
        }
      ]
    },
    "data_description": {
      "time_field": "timestamp",
      "time_format": "epoch_ms"
    },
    "analysis_limits": {
      "model_memory_limit": "11MB"
    },
    "model_plot_config": {
      "enabled": true,
      "annotations_enabled": true
    },
    "results_index_name": "test-job1",
    "datafeed_config": {
      "indices": [
        "kibana_sample_data_logs"
      ],
      "query": {
        "bool": {
          "must": [
            {
              "match_all": {}
            }
          ]
        }
      },
      "runtime_mappings": {
        "hour_of_day": {
          "type": "long",
          "script": {
            "source": "emit(doc['timestamp'].value.getHour());"
          }
        }
      },
      "datafeed_id": "datafeed-test-job1"
    }
  }
)
$resp = $client->ml()->putJob([
    "job_id" => "job-01",
    "body" => [
        "analysis_config" => [
            "bucket_span" => "15m",
            "detectors" => array(
                [
                    "detector_description" => "Sum of bytes",
                    "function" => "sum",
                    "field_name" => "bytes",
                ],
            ),
        ],
        "data_description" => [
            "time_field" => "timestamp",
            "time_format" => "epoch_ms",
        ],
        "analysis_limits" => [
            "model_memory_limit" => "11MB",
        ],
        "model_plot_config" => [
            "enabled" => true,
            "annotations_enabled" => true,
        ],
        "results_index_name" => "test-job1",
        "datafeed_config" => [
            "indices" => array(
                "kibana_sample_data_logs",
            ),
            "query" => [
                "bool" => [
                    "must" => array(
                        [
                            "match_all" => new ArrayObject([]),
                        ],
                    ),
                ],
            ],
            "runtime_mappings" => [
                "hour_of_day" => [
                    "type" => "long",
                    "script" => [
                        "source" => "emit(doc['timestamp'].value.getHour());",
                    ],
                ],
            ],
            "datafeed_id" => "datafeed-test-job1",
        ],
    ],
]);
curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"analysis_config":{"bucket_span":"15m","detectors":[{"detector_description":"Sum of bytes","function":"sum","field_name":"bytes"}]},"data_description":{"time_field":"timestamp","time_format":"epoch_ms"},"analysis_limits":{"model_memory_limit":"11MB"},"model_plot_config":{"enabled":true,"annotations_enabled":true},"results_index_name":"test-job1","datafeed_config":{"indices":["kibana_sample_data_logs"],"query":{"bool":{"must":[{"match_all":{}}]}},"runtime_mappings":{"hour_of_day":{"type":"long","script":{"source":"emit(doc['"'"'timestamp'"'"'].value.getHour());"}}},"datafeed_id":"datafeed-test-job1"}}' "$ELASTICSEARCH_URL/_ml/anomaly_detectors/job-01"
client.ml().putJob(p -> p
    .analysisConfig(a -> a
        .bucketSpan(b -> b
            .time("15m")
        )
        .detectors(d -> d
            .detectorDescription("Sum of bytes")
            .fieldName("bytes")
            .function("sum")
        )
    )
    .analysisLimits(an -> an
        .modelMemoryLimit("11MB")
    )
    .dataDescription(d -> d
        .timeField("timestamp")
        .timeFormat("epoch_ms")
    )
    .datafeedConfig(d -> d
        .datafeedId("datafeed-test-job1")
        .indices("kibana_sample_data_logs")
        .query(q -> q
            .bool(b -> b
                .must(m -> m
                    .matchAll(ma -> ma)
                )
            )
        )
        .runtimeMappings("hour_of_day", r -> r
            .script(s -> s
                .source(so -> so
                    .scriptString("emit(doc['timestamp'].value.getHour());")
                )
            )
            .type(RuntimeFieldType.Long)
        )
    )
    .jobId("job-01")
    .modelPlotConfig(m -> m
        .annotationsEnabled(true)
        .enabled(true)
    )
    .resultsIndexName("test-job1")
);
Request example
A request to create an anomaly detection job and datafeed.
{
  "analysis_config": {
    "bucket_span": "15m",
    "detectors": [
      {
        "detector_description": "Sum of bytes",
        "function": "sum",
        "field_name": "bytes"
      }
    ]
  },
  "data_description": {
    "time_field": "timestamp",
    "time_format": "epoch_ms"
  },
  "analysis_limits": {
    "model_memory_limit": "11MB"
  },
  "model_plot_config": {
    "enabled": true,
    "annotations_enabled": true
  },
  "results_index_name": "test-job1",
  "datafeed_config": {
    "indices": [
      "kibana_sample_data_logs"
    ],
    "query": {
      "bool": {
        "must": [
          {
            "match_all": {}
          }
        ]
      }
    },
    "runtime_mappings": {
      "hour_of_day": {
        "type": "long",
        "script": {
          "source": "emit(doc['timestamp'].value.getHour());"
        }
      }
    },
    "datafeed_id": "datafeed-test-job1"
  }
}
Response examples (200)
A successful response when creating an anomaly detection job and datafeed.
{
  "job_id": "test-job1",
  "job_type": "anomaly_detector",
  "job_version": "8.4.0",
  "create_time": 1656087283340,
  "datafeed_config": {
    "datafeed_id": "datafeed-test-job1",
    "job_id": "test-job1",
    "authorization": {
      "roles": [
        "superuser"
      ]
    },
    "query_delay": "61499ms",
    "chunking_config": {
      "mode": "auto"
    },
    "indices_options": {
      "expand_wildcards": [
        "open"
      ],
      "ignore_unavailable": false,
      "allow_no_indices": true,
      "ignore_throttled": true
    },
    "query": {
      "bool": {
        "must": [
          {
            "match_all": {}
          }
        ]
      }
    },
    "indices": [
      "kibana_sample_data_logs"
    ],
    "scroll_size": 1000,
    "delayed_data_check_config": {
      "enabled": true
    },
    "runtime_mappings": {
      "hour_of_day": {
        "type": "long",
        "script": {
          "source": "emit(doc['timestamp'].value.getHour());"
        }
      }
    }
  },
  "analysis_config": {
    "bucket_span": "15m",
    "detectors": [
      {
        "detector_description": "Sum of bytes",
        "function": "sum",
        "field_name": "bytes",
        "detector_index": 0
      }
    ],
    "influencers": [],
    "model_prune_window": "30d"
  },
  "analysis_limits": {
    "model_memory_limit": "11mb",
    "categorization_examples_limit": 4
  },
  "data_description": {
    "time_field": "timestamp",
    "time_format": "epoch_ms"
  },
  "model_plot_config": {
    "enabled": true,
    "annotations_enabled": true
  },
  "model_snapshot_retention_days": 10,
  "daily_model_snapshot_retention_after_days": 1,
  "results_index_name": "custom-test-job1",
  "allow_lazy_open": false
}

Delete an anomaly detection job Generally available; Added in 5.4.0

DELETE /_ml/anomaly_detectors/{job_id}

All job configuration, model state and results are deleted. It is not currently possible to delete multiple jobs using wildcards or a comma separated list. If you delete a job that has a datafeed, the request first tries to delete the datafeed. This behavior is equivalent to calling the delete datafeed API with the same timeout and force parameters as the delete job request.

Required authorization

  • Cluster privileges: manage_ml

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job.

Query parameters

  • force boolean

    Use to forcefully delete an opened job; this method is quicker than closing and deleting the job.

  • delete_user_annotations boolean

    Specifies whether annotations that have been added by the user should be deleted along with any auto-generated annotations when the job is reset.

  • wait_for_completion boolean

    Specifies whether the request should return immediately or wait until the job deletion completes.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

DELETE /_ml/anomaly_detectors/{job_id}
DELETE _ml/anomaly_detectors/total-requests
resp = client.ml.delete_job(
    job_id="total-requests",
)
const response = await client.ml.deleteJob({
  job_id: "total-requests",
});
response = client.ml.delete_job(
  job_id: "total-requests"
)
$resp = $client->ml()->deleteJob([
    "job_id" => "total-requests",
]);
curl -X DELETE -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_ml/anomaly_detectors/total-requests"
client.ml().deleteJob(d -> d
    .jobId("total-requests")
);
Response examples (200)
A successful response when deleting an anomaly detection job.
{
  "acknowledged": true
}
A successful response when deleting an anomaly detection job asynchronously. When the `wait_for_completion` query parameter is set to `false`, the response contains an identifier for the job deletion task.
{
  "task": "oTUltX4IQMOUUVeiohTt8A:39"
}




































Get datafeed stats Generally available; Added in 5.5.0

GET /_ml/datafeeds/{datafeed_id}/_stats

All methods and paths for this operation:

GET /_ml/datafeeds/_stats

GET /_ml/datafeeds/{datafeed_id}/_stats

You can get statistics for multiple datafeeds in a single API request by using a comma-separated list of datafeeds or a wildcard expression. You can get statistics for all datafeeds by using _all, by specifying * as the <feed_id>, or by omitting the <feed_id>. If the datafeed is stopped, the only information you receive is the datafeed_id and the state. This API returns a maximum of 10,000 datafeeds.

Required authorization

  • Cluster privileges: monitor_ml

Path parameters

  • datafeed_id string | array[string] Required

    Identifier for the datafeed. It can be a datafeed identifier or a wildcard expression. If you do not specify one of these options, the API returns information about all datafeeds.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    1. Contains wildcard expressions and there are no datafeeds that match.
    2. Contains the _all string or no identifiers and there are no matches.
    3. Contains wildcard expressions and there are only partial matches.

    The default value is true, which returns an empty datafeeds array when there are no matches and the subset of results when there are partial matches. If this parameter is false, the request returns a 404 status code when there are no matches or only partial matches.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • count number Required
    • datafeeds array[object] Required
      Hide datafeeds attributes Show datafeeds attributes object
      • assignment_explanation string

        For started datafeeds only, contains messages relating to the selection of a node.

      • datafeed_id string Required

        A numerical character string that uniquely identifies the datafeed. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.

      • node object

        For started datafeeds only, this information pertains to the node upon which the datafeed is started.

        Hide node attributes Show node attributes object
        • name string Required
        • ephemeral_id string Required
        • id string Required
        • transport_address string Required
        • attributes object Required
          Hide attributes attribute Show attributes attribute object
          • * string Additional properties
      • state string Required

        The status of the datafeed, which can be one of the following values: starting, started, stopping, stopped.

        Values are started, stopped, starting, or stopping.

      • timing_stats object

        An object that provides statistical information about timing aspect of this datafeed.

        Hide timing_stats attributes Show timing_stats attributes object
        • bucket_count number Required

          The number of buckets processed.

        • exponential_average_calculation_context object
        • job_id string Required

          Identifier for the anomaly detection job.

        • search_count number Required

          The number of searches run by the datafeed.

      • running_state object

        An object containing the running state for this datafeed. It is only provided if the datafeed is started.

        Hide running_state attributes Show running_state attributes object
        • real_time_configured boolean Required

          Indicates if the datafeed is "real-time"; meaning that the datafeed has no configured end time.

        • real_time_running boolean Required

          Indicates whether the datafeed has finished running on the available past data. For datafeeds without a configured end time, this means that the datafeed is now running on "real-time" data.

        • search_interval object

          Provides the latest time interval the datafeed has searched.

GET /_ml/datafeeds/{datafeed_id}/_stats
GET _ml/datafeeds/datafeed-high_sum_total_sales/_stats
resp = client.ml.get_datafeed_stats(
    datafeed_id="datafeed-high_sum_total_sales",
)
const response = await client.ml.getDatafeedStats({
  datafeed_id: "datafeed-high_sum_total_sales",
});
response = client.ml.get_datafeed_stats(
  datafeed_id: "datafeed-high_sum_total_sales"
)
$resp = $client->ml()->getDatafeedStats([
    "datafeed_id" => "datafeed-high_sum_total_sales",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_ml/datafeeds/datafeed-high_sum_total_sales/_stats"
client.ml().getDatafeedStats(g -> g
    .datafeedId("datafeed-high_sum_total_sales")
);

Get anomaly detection job results for influencers Generally available; Added in 5.4.0

POST /_ml/anomaly_detectors/{job_id}/results/influencers

All methods and paths for this operation:

GET /_ml/anomaly_detectors/{job_id}/results/influencers

POST /_ml/anomaly_detectors/{job_id}/results/influencers

Influencers are the entities that have contributed to, or are to blame for, the anomalies. Influencer results are available only if an influencer_field_name is specified in the job configuration.

Required authorization

  • Cluster privileges: monitor_ml

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job.

Query parameters

  • desc boolean

    If true, the results are sorted in descending order.

  • end string | number

    Returns influencers with timestamps earlier than this time. The default value means it is unset and results are not limited to specific timestamps.

  • exclude_interim boolean

    If true, the output excludes interim results. By default, interim results are included.

  • influencer_score number

    Returns influencers with anomaly scores greater than or equal to this value.

  • from number

    Skips the specified number of influencers.

  • size number

    Specifies the maximum number of influencers to obtain.

  • sort string

    Specifies the sort field for the requested influencers. By default, the influencers are sorted by the influencer_score value.

  • start string | number

    Returns influencers with timestamps after this time. The default value means it is unset and results are not limited to specific timestamps.

application/json

Body

  • page object

    Configures pagination. This parameter has the from and size properties.

    Hide page attributes Show page attributes object
    • from number

      Skips the specified number of items.

      Default value is 0.

    • size number

      Specifies the maximum number of items to obtain.

      Default value is 10000.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • count number Required
    • influencers array[object] Required

      Array of influencer objects

      Hide influencers attributes Show influencers attributes object
      • bucket_span number

        Time unit for seconds

      • influencer_score number Required

        A normalized score between 0-100, which is based on the probability of the influencer in this bucket aggregated across detectors. Unlike initial_influencer_score, this value is updated by a re-normalization process as new data is analyzed.

      • influencer_field_name string Required

        The field name of the influencer.

      • influencer_field_value string Required

        The entity that influenced, contributed to, or was to blame for the anomaly.

      • initial_influencer_score number Required

        A normalized score between 0-100, which is based on the probability of the influencer aggregated across detectors. This is the initial value that was calculated at the time the bucket was processed.

      • is_interim boolean Required

        If true, this is an interim result. In other words, the results are calculated based on partial input data.

      • job_id string Required

        Identifier for the anomaly detection job.

      • probability number Required

        The probability that the influencer has this behavior, in the range 0 to 1. This value can be held to a high precision of over 300 decimal places, so the influencer_score is provided as a human-readable and friendly interpretation of this value.

      • result_type string Required

        Internal. This value is always set to influencer.

      • timestamp number

        Time unit for milliseconds

      • foo string

        Additional influencer properties are added, depending on the fields being analyzed. For example, if it’s analyzing user_name as an influencer, a field user_name is added to the result document. This information enables you to filter the anomaly results more easily.

POST /_ml/anomaly_detectors/{job_id}/results/influencers
GET _ml/anomaly_detectors/high_sum_total_sales/results/influencers
{
  "sort": "influencer_score",
  "desc": true
}
resp = client.ml.get_influencers(
    job_id="high_sum_total_sales",
    sort="influencer_score",
    desc=True,
)
const response = await client.ml.getInfluencers({
  job_id: "high_sum_total_sales",
  sort: "influencer_score",
  desc: true,
});
response = client.ml.get_influencers(
  job_id: "high_sum_total_sales",
  body: {
    "sort": "influencer_score",
    "desc": true
  }
)
$resp = $client->ml()->getInfluencers([
    "job_id" => "high_sum_total_sales",
    "body" => [
        "sort" => "influencer_score",
        "desc" => true,
    ],
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"sort":"influencer_score","desc":true}' "$ELASTICSEARCH_URL/_ml/anomaly_detectors/high_sum_total_sales/results/influencers"
client.ml().getInfluencers(g -> g
    .jobId("high_sum_total_sales")
);
Request example
An example body for a `GET _ml/anomaly_detectors/high_sum_total_sales/results/influencers` request.
{
  "sort": "influencer_score",
  "desc": true
}




















































Update an anomaly detection job Generally available; Added in 5.5.0

POST /_ml/anomaly_detectors/{job_id}/_update

Updates certain properties of an anomaly detection job.

Required authorization

  • Cluster privileges: manage_ml

Path parameters

  • job_id string Required

    Identifier for the job.

application/json

Body Required

  • allow_lazy_open boolean

    Advanced configuration option. Specifies whether this job can open when there is insufficient machine learning node capacity for it to be immediately assigned to a node. If false and a machine learning node with capacity to run the job cannot immediately be found, the open anomaly detection jobs API returns an error. However, this is also subject to the cluster-wide xpack.ml.max_lazy_ml_nodes setting. If this option is set to true, the open anomaly detection jobs API does not return an error and the job waits in the opening state until sufficient machine learning node capacity is available.

    Default value is false.

  • analysis_limits object
    Hide analysis_limits attribute Show analysis_limits attribute object
    • model_memory_limit string Required

      Limits can be applied for the resources required to hold the mathematical models in memory. These limits are approximate and can be set per job. They do not control the memory used by other processes, for example the Elasticsearch Java processes.

  • background_persist_interval string

    Advanced configuration option. The time between each periodic persistence of the model. The default value is a randomized value between 3 to 4 hours, which avoids all jobs persisting at exactly the same time. The smallest allowed value is 1 hour. For very large models (several GB), persistence could take 10-20 minutes, so do not set the value too low. If the job is open when you make the update, you must stop the datafeed, close the job, then reopen the job and restart the datafeed for the changes to take effect.

  • custom_settings object

    Advanced configuration option. Contains custom meta data about the job. For example, it can contain custom URL information as shown in Adding custom URLs to machine learning results.

    Hide custom_settings attribute Show custom_settings attribute object
    • * object Additional properties
  • categorization_filters array[string]
  • description string

    A description of the job.

  • model_plot_config object
    Hide model_plot_config attributes Show model_plot_config attributes object
    • annotations_enabled boolean Generally available; Added in 7.9.0

      If true, enables calculation and storage of the model change annotations for each entity that is being analyzed.

      Default value is true.

    • enabled boolean

      If true, enables calculation and storage of the model bounds for each entity that is being analyzed.

      Default value is false.

    • terms string

      Limits data collection to this comma separated list of partition or by field values. If terms are not specified or it is an empty string, no filtering is applied. Wildcards are not supported. Only the specified terms can be viewed when using the Single Metric Viewer.

  • model_prune_window string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • daily_model_snapshot_retention_after_days number

    Advanced configuration option, which affects the automatic removal of old model snapshots for this job. It specifies a period of time (in days) after which only the first snapshot per day is retained. This period is relative to the timestamp of the most recent snapshot for this job. Valid values range from 0 to model_snapshot_retention_days. For jobs created before version 7.8.0, the default value matches model_snapshot_retention_days.

    Default value is 1.

  • model_snapshot_retention_days number

    Advanced configuration option, which affects the automatic removal of old model snapshots for this job. It specifies the maximum period of time (in days) that snapshots are retained. This period is relative to the timestamp of the most recent snapshot for this job.

    Default value is 10.

  • renormalization_window_days number

    Advanced configuration option. The period over which adjustments to the score are applied, as new data is seen.

  • results_retention_days number

    Advanced configuration option. The period of time (in days) that results are retained. Age is calculated relative to the timestamp of the latest bucket result. If this property has a non-null value, once per day at 00:30 (server time), results that are the specified number of days older than the latest bucket result are deleted from Elasticsearch. The default value is null, which means all results are retained.

  • groups array[string]

    A list of job groups. A job can belong to no groups or many.

  • detectors array[object]

    An array of detector update objects.

    Hide detectors attributes Show detectors attributes object
    • detector_index number Required

      A unique identifier for the detector. This identifier is based on the order of the detectors in the analysis_config, starting at zero.

    • description string

      A description of the detector.

    • custom_rules array[object]

      An array of custom rule objects, which enable you to customize the way detectors operate. For example, a rule may dictate to the detector conditions under which results should be skipped. Kibana refers to custom rules as job rules.

      Hide custom_rules attributes Show custom_rules attributes object
      • actions array[string]

        The set of actions to be triggered when the rule applies. If more than one action is specified the effects of all actions are combined.

        Supported values include:

        • skip_result: The result will not be created. Unless you also specify skip_model_update, the model will be updated as usual with the corresponding series value.
        • skip_model_update: The value for that series will not be used to update the model. Unless you also specify skip_result, the results will be created as usual. This action is suitable when certain values are expected to be consistently anomalous and they affect the model in a way that negatively impacts the rest of the results.

        Values are skip_result or skip_model_update. Default value is ["skip_result"].

      • conditions array[object]

        An array of numeric conditions when the rule applies. A rule must either have a non-empty scope or at least one condition. Multiple conditions are combined together with a logical AND.

        Hide conditions attributes Show conditions attributes object
        • applies_to
        • operator
        • value number Required

          The value that is compared against the applies_to field using the operator.

      • scope object

        A scope of series where the rule applies. A rule must either have a non-empty scope or at least one condition. By default, the scope includes all series. Scoping is allowed for any of the fields that are also specified in by_field_name, over_field_name, or partition_field_name.

        Hide scope attribute Show scope attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • filter_id
          • filter_type
  • per_partition_categorization object

    Settings related to how categorization interacts with partition fields.

    Hide per_partition_categorization attributes Show per_partition_categorization attributes object
    • enabled boolean

      To enable this setting, you must also set the partition_field_name property to the same value in every detector that uses the keyword mlcategory. Otherwise, job creation fails.

    • stop_on_warn boolean

      This setting can be set to true only if per-partition categorization is enabled. If true, both categorization and subsequent anomaly detection stops for partitions where the categorization status changes to warn. This setting makes it viable to have a job where it is expected that categorization works well for some partitions but not others; you do not pay the cost of bad categorization forever in the partitions where it works badly.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • allow_lazy_open boolean Required
    • analysis_config object Required
      Hide analysis_config attributes Show analysis_config attributes object
      • bucket_span string Required

        The size of the interval that the analysis is aggregated into, typically between 5m and 1h.

      • categorization_analyzer string | object

        If categorization_field_name is specified, you can also define the analyzer that is used to interpret the categorization field. This property cannot be used at the same time as categorization_filters. The categorization analyzer specifies how the categorization_field is interpreted by the categorization process.

        One of:

        If categorization_field_name is specified, you can also define the analyzer that is used to interpret the categorization field. This property cannot be used at the same time as categorization_filters. The categorization analyzer specifies how the categorization_field is interpreted by the categorization process.

      • categorization_field_name string

        If this property is specified, the values of the specified field will be categorized. The resulting categories must be used in a detector by setting by_field_name, over_field_name, or partition_field_name to the keyword mlcategory.

      • categorization_filters array[string]

        If categorization_field_name is specified, you can also define optional filters. This property expects an array of regular expressions. The expressions are used to filter out matching sequences from the categorization field values.

      • detectors array[object] Required

        An array of detector configuration objects. Detector configuration objects specify which data fields a job analyzes. They also specify which analytical functions are used. You can specify multiple detectors for a job.

        Hide detectors attributes Show detectors attributes object
        • by_field_name string

          The field used to split the data. In particular, this property is used for analyzing the splits with respect to their own history. It is used for finding unusual values in the context of the split.

        • custom_rules array[object]

          An array of custom rule objects, which enable you to customize the way detectors operate. For example, a rule may dictate to the detector conditions under which results should be skipped. Kibana refers to custom rules as job rules.

        • detector_description string

          A description of the detector.

        • detector_index number

          A unique identifier for the detector. This identifier is based on the order of the detectors in the analysis_config, starting at zero.

        • exclude_frequent string

          Contains one of the following values: all, none, by, or over. If set, frequent entities are excluded from influencing the anomaly results. Entities can be considered frequent over time or frequent in a population. If you are working with both over and by fields, then you can set exclude_frequent to all for both fields, or to by or over for those specific fields.

          Values are all, none, by, or over.

        • field_name string

          The field that the detector uses in the function. If you use an event rate function such as count or rare, do not specify this field.

        • function string Required

          The analysis function that is used. For example, count, rare, mean, min, max, and sum.

        • over_field_name string

          The field used to split the data. In particular, this property is used for analyzing the splits with respect to the history of all splits. It is used for finding unusual values in the population of all splits.

        • partition_field_name string

          The field used to segment the analysis. When you use this property, you have completely independent baselines for each value of this field.

        • use_null boolean

          Defines whether a new series is used as the null series when there is no value for the by or partition fields.

          Default value is false.

      • influencers array[string] Required

        A comma separated list of influencer field names. Typically these can be the by, over, or partition fields that are used in the detector configuration. You might also want to use a field name that is not specifically named in a detector, but is available as part of the input data. When you use multiple detectors, the use of influencers is recommended as it aggregates results for each influencer entity.

      • model_prune_window string

        Advanced configuration option. Affects the pruning of models that have not been updated for the given time duration. The value must be set to a multiple of the bucket_span. If set too low, important information may be removed from the model. Typically, set to 30d or longer. If not set, model pruning only occurs if the model memory status reaches the soft limit or the hard limit. For jobs created in 8.1 and later, the default value is the greater of 30d or 20 times bucket_span.

      • latency string

        The size of the window in which to expect data that is out of time order. Defaults to no latency. If you specify a non-zero value, it must be greater than or equal to one second.

      • multivariate_by_fields boolean

        This functionality is reserved for internal use. It is not supported for use in customer environments and is not subject to the support SLA of official GA features. If set to true, the analysis will automatically find correlations between metrics for a given by field value and report anomalies when those correlations cease to hold.

      • per_partition_categorization object

        Settings related to how categorization interacts with partition fields.

        Hide per_partition_categorization attributes Show per_partition_categorization attributes object
        • enabled boolean

          To enable this setting, you must also set the partition_field_name property to the same value in every detector that uses the keyword mlcategory. Otherwise, job creation fails.

        • stop_on_warn boolean

          This setting can be set to true only if per-partition categorization is enabled. If true, both categorization and subsequent anomaly detection stops for partitions where the categorization status changes to warn. This setting makes it viable to have a job where it is expected that categorization works well for some partitions but not others; you do not pay the cost of bad categorization forever in the partitions where it works badly.

      • summary_count_field_name string

        If this property is specified, the data that is fed to the job is expected to be pre-summarized. This property value is the name of the field that contains the count of raw data points that have been summarized. The same summary_count_field_name applies to all detectors in the job.

    • analysis_limits object Required
      Hide analysis_limits attributes Show analysis_limits attributes object
      • categorization_examples_limit number

        The maximum number of examples stored per category in memory and in the results data store. If you increase this value, more examples are available, however it requires that you have more storage available. If you set this value to 0, no examples are stored. NOTE: The categorization_examples_limit applies only to analysis that uses categorization.

        Default value is 4.

      • model_memory_limit number | string

        The approximate maximum amount of memory resources that are required for analytical processing. Once this limit is approached, data pruning becomes more aggressive. Upon exceeding this limit, new entities are not modeled. If the xpack.ml.max_model_memory_limit setting has a value greater than 0 and less than 1024mb, that value is used instead of the default. The default value is relatively small to ensure that high resource usage is a conscious decision. If you have jobs that are expected to analyze high cardinality fields, you will likely need to use a higher value. If you specify a number instead of a string, the units are assumed to be MiB. Specifying a string is recommended for clarity. If you specify a byte size unit of b or kb and the number does not equate to a discrete number of megabytes, it is rounded down to the closest MiB. The minimum valid value is 1 MiB. If you specify a value less than 1 MiB, an error occurs. If you specify a value for the xpack.ml.max_model_memory_limit setting, an error occurs when you try to create jobs that have model_memory_limit values greater than that setting value.

        One of:

        The approximate maximum amount of memory resources that are required for analytical processing. Once this limit is approached, data pruning becomes more aggressive. Upon exceeding this limit, new entities are not modeled. If the xpack.ml.max_model_memory_limit setting has a value greater than 0 and less than 1024mb, that value is used instead of the default. The default value is relatively small to ensure that high resource usage is a conscious decision. If you have jobs that are expected to analyze high cardinality fields, you will likely need to use a higher value. If you specify a number instead of a string, the units are assumed to be MiB. Specifying a string is recommended for clarity. If you specify a byte size unit of b or kb and the number does not equate to a discrete number of megabytes, it is rounded down to the closest MiB. The minimum valid value is 1 MiB. If you specify a value less than 1 MiB, an error occurs. If you specify a value for the xpack.ml.max_model_memory_limit setting, an error occurs when you try to create jobs that have model_memory_limit values greater than that setting value.

    • background_persist_interval string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • create_time number

      Time unit for milliseconds

    • finished_time number

      Time unit for milliseconds

    • custom_settings object
      Hide custom_settings attribute Show custom_settings attribute object
      • * string Additional properties
    • daily_model_snapshot_retention_after_days number Required
    • data_description object Required
      Hide data_description attributes Show data_description attributes object
      • format string

        Only JSON format is supported at this time.

      • time_field string

        The name of the field that contains the timestamp.

      • time_format string

        The time format, which can be epoch, epoch_ms, or a custom pattern. The value epoch refers to UNIX or Epoch time (the number of seconds since 1 Jan 1970). The value epoch_ms indicates that time is measured in milliseconds since the epoch. The epoch and epoch_ms time formats accept either integer or real values. Custom patterns must conform to the Java DateTimeFormatter class. When you use date-time formatting patterns, it is recommended that you provide the full date, time and time zone. For example: yyyy-MM-dd'T'HH:mm:ssX. If the pattern that you specify is not sufficient to produce a complete timestamp, job creation fails.

        Default value is epoch.

      • field_delimiter string
    • datafeed_config object
      Hide datafeed_config attributes Show datafeed_config attributes object
      • aggregations object
      • authorization object

        The security privileges that the datafeed uses to run its queries. If Elastic Stack security features were disabled at the time of the most recent update to the datafeed, this property is omitted.

        Hide authorization attributes Show authorization attributes object
        • api_key object

          If an API key was used for the most recent update to the datafeed, its name and identifier are listed in the response.

        • roles array[string]

          If a user ID was used for the most recent update to the datafeed, its roles at the time of the update are listed in the response.

        • service_account string

          If a service account was used for the most recent update to the datafeed, the account name is listed in the response.

      • chunking_config object
        Hide chunking_config attributes Show chunking_config attributes object
        • mode string Required

          If the mode is auto, the chunk size is dynamically calculated; this is the recommended value when the datafeed does not use aggregations. If the mode is manual, chunking is applied according to the specified time_span; use this mode when the datafeed uses aggregations. If the mode is off, no chunking is applied.

          Values are auto, manual, or off.

        • time_span string

          The time span that each search will be querying. This setting is applicable only when the mode is set to manual.

      • datafeed_id string Required
      • frequency string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • indices array[string] Required
      • indexes array[string]
      • job_id string Required
      • max_empty_searches number
      • query_delay string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • script_fields object
        Hide script_fields attribute Show script_fields attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • script object Required
          • ignore_failure boolean
      • scroll_size number
      • delayed_data_check_config object Required
        Hide delayed_data_check_config attributes Show delayed_data_check_config attributes object
        • check_window string

          The window of time that is searched for late data. This window of time ends with the latest finalized bucket. It defaults to null, which causes an appropriate check_window to be calculated when the real-time datafeed runs. In particular, the default check_window span calculation is based on the maximum of 2h or 8 * bucket_span.

        • enabled boolean Required

          Specifies whether the datafeed periodically checks for delayed data.

      • runtime_mappings object
        Hide runtime_mappings attribute Show runtime_mappings attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • fields object

            For type composite

          • fetch_fields array[object]

            For type lookup

          • format string

            A custom format for date type runtime fields.

      • indices_options object

        Controls how to deal with unavailable concrete indices (closed or missing), how wildcard expressions are expanded to actual indices (all, closed or open indices) and how to deal with wildcard expressions that resolve to no indices.

        Hide indices_options attributes Show indices_options attributes object
        • allow_no_indices boolean

          If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

        • expand_wildcards string | array[string]

          Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

          Supported values include:

          • all: Match any data stream or index, including hidden ones.
          • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
          • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
          • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
          • none: Wildcard expressions are not accepted.
        • ignore_unavailable boolean

          If true, missing or closed indices are not included in the response.

          Default value is false.

        • ignore_throttled boolean

          If true, concrete, expanded or aliased indices are ignored when frozen.

          Default value is true.

      • query object Required

        The Elasticsearch query domain-specific language (DSL). This value corresponds to the query object in an Elasticsearch search POST body. All the options that are supported by Elasticsearch can be used, as this object is passed verbatim to Elasticsearch. By default, this property has the following value: {"match_all": {"boost": 1}}.

        Query DSL
    • description string
    • groups array[string]
    • job_id string Required
    • job_type string Required
    • job_version string Required
    • model_plot_config object
      Hide model_plot_config attributes Show model_plot_config attributes object
      • annotations_enabled boolean Generally available; Added in 7.9.0

        If true, enables calculation and storage of the model change annotations for each entity that is being analyzed.

        Default value is true.

      • enabled boolean

        If true, enables calculation and storage of the model bounds for each entity that is being analyzed.

        Default value is false.

      • terms string

        Limits data collection to this comma separated list of partition or by field values. If terms are not specified or it is an empty string, no filtering is applied. Wildcards are not supported. Only the specified terms can be viewed when using the Single Metric Viewer.

    • model_snapshot_id string
    • model_snapshot_retention_days number Required
    • renormalization_window_days number
    • results_index_name string Required
    • results_retention_days number
POST /_ml/anomaly_detectors/{job_id}/_update
POST _ml/anomaly_detectors/low_request_rate/_update
{
  "description":"An updated job",
  "detectors": {
    "detector_index": 0,
    "description": "An updated detector description"
  },
  "groups": ["kibana_sample_data","kibana_sample_web_logs"],
  "model_plot_config": {
    "enabled": true
  },
  "renormalization_window_days": 30,
  "background_persist_interval": "2h",
  "model_snapshot_retention_days": 7,
  "results_retention_days": 60
}
resp = client.ml.update_job(
    job_id="low_request_rate",
    description="An updated job",
    detectors={
        "detector_index": 0,
        "description": "An updated detector description"
    },
    groups=[
        "kibana_sample_data",
        "kibana_sample_web_logs"
    ],
    model_plot_config={
        "enabled": True
    },
    renormalization_window_days=30,
    background_persist_interval="2h",
    model_snapshot_retention_days=7,
    results_retention_days=60,
)
const response = await client.ml.updateJob({
  job_id: "low_request_rate",
  description: "An updated job",
  detectors: {
    detector_index: 0,
    description: "An updated detector description",
  },
  groups: ["kibana_sample_data", "kibana_sample_web_logs"],
  model_plot_config: {
    enabled: true,
  },
  renormalization_window_days: 30,
  background_persist_interval: "2h",
  model_snapshot_retention_days: 7,
  results_retention_days: 60,
});
response = client.ml.update_job(
  job_id: "low_request_rate",
  body: {
    "description": "An updated job",
    "detectors": {
      "detector_index": 0,
      "description": "An updated detector description"
    },
    "groups": [
      "kibana_sample_data",
      "kibana_sample_web_logs"
    ],
    "model_plot_config": {
      "enabled": true
    },
    "renormalization_window_days": 30,
    "background_persist_interval": "2h",
    "model_snapshot_retention_days": 7,
    "results_retention_days": 60
  }
)
$resp = $client->ml()->updateJob([
    "job_id" => "low_request_rate",
    "body" => [
        "description" => "An updated job",
        "detectors" => [
            "detector_index" => 0,
            "description" => "An updated detector description",
        ],
        "groups" => array(
            "kibana_sample_data",
            "kibana_sample_web_logs",
        ),
        "model_plot_config" => [
            "enabled" => true,
        ],
        "renormalization_window_days" => 30,
        "background_persist_interval" => "2h",
        "model_snapshot_retention_days" => 7,
        "results_retention_days" => 60,
    ],
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"description":"An updated job","detectors":{"detector_index":0,"description":"An updated detector description"},"groups":["kibana_sample_data","kibana_sample_web_logs"],"model_plot_config":{"enabled":true},"renormalization_window_days":30,"background_persist_interval":"2h","model_snapshot_retention_days":7,"results_retention_days":60}' "$ELASTICSEARCH_URL/_ml/anomaly_detectors/low_request_rate/_update"
client.ml().updateJob(u -> u
    .backgroundPersistInterval(b -> b
        .time("2h")
    )
    .description("An updated job")
    .detectors(d -> d
        .detectorIndex(0)
        .description("An updated detector description")
    )
    .groups(List.of("kibana_sample_data","kibana_sample_web_logs"))
    .jobId("low_request_rate")
    .modelPlotConfig(m -> m
        .enabled(true)
    )
    .modelSnapshotRetentionDays(7L)
    .renormalizationWindowDays(30L)
    .resultsRetentionDays(60L)
);
Request example
An example body for a `POST _ml/anomaly_detectors/low_request_rate/_update` request.
{
  "description":"An updated job",
  "detectors": {
    "detector_index": 0,
    "description": "An updated detector description"
  },
  "groups": ["kibana_sample_data","kibana_sample_web_logs"],
  "model_plot_config": {
    "enabled": true
  },
  "renormalization_window_days": 30,
  "background_persist_interval": "2h",
  "model_snapshot_retention_days": 7,
  "results_retention_days": 60
}





















Evaluate data frame analytics Generally available; Added in 7.3.0

POST /_ml/data_frame/_evaluate

The API packages together commonly used evaluation metrics for various types of machine learning features. This has been designed for use on indexes created by data frame analytics. Evaluation requires both a ground truth field and an analytics result field to be present.

Required authorization

  • Cluster privileges: monitor_ml
application/json

Body Required

  • evaluation object Required

    Defines the type of evaluation you want to perform.

    Hide evaluation attributes Show evaluation attributes object
    • classification object

      Classification evaluation evaluates the results of a classification analysis which outputs a prediction that identifies to which of the classes each document belongs.

      Hide classification attributes Show classification attributes object
      • actual_field string Required

        The field of the index which contains the ground truth. The data type of this field can be boolean or integer. If the data type is integer, the value has to be either 0 (false) or 1 (true).

      • predicted_field string

        The field in the index which contains the predicted value, in other words the results of the classification analysis.

      • top_classes_field string

        The field of the index which is an array of documents of the form { "class_name": XXX, "class_probability": YYY }. This field must be defined as nested in the mappings.

    • outlier_detection object

      Outlier detection evaluates the results of an outlier detection analysis which outputs the probability that each document is an outlier.

      Hide outlier_detection attributes Show outlier_detection attributes object
      • actual_field string Required

        The field of the index which contains the ground truth. The data type of this field can be boolean or integer. If the data type is integer, the value has to be either 0 (false) or 1 (true).

      • predicted_probability_field string Required

        The field of the index that defines the probability of whether the item belongs to the class in question or not. It’s the field that contains the results of the analysis.

    • regression object

      Regression evaluation evaluates the results of a regression analysis which outputs a prediction of values.

      Hide regression attributes Show regression attributes object
      • actual_field string Required

        The field of the index which contains the ground truth. The data type of this field must be numerical.

      • predicted_field string Required

        The field in the index that contains the predicted value, in other words the results of the regression analysis.

      • metrics object

        Specifies the metrics that are used for the evaluation. For more information on mse, msle, and huber, consult the Jupyter notebook on regression loss functions.

        Hide metrics attributes Show metrics attributes object
        • mse object

          Average squared difference between the predicted values and the actual (ground truth) value. For more information, read this wiki article.

        • r_squared object

          Proportion of the variance in the dependent variable that is predictable from the independent variables.

  • index string Required

    Defines the index in which the evaluation will be performed.

  • query object

    A query clause that retrieves a subset of data from the source index.

    External documentation

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • classification object

      Evaluation results for a classification analysis. It outputs a prediction that identifies to which of the classes each document belongs.

      Hide classification attributes Show classification attributes object
      • auc_roc object

        The AUC ROC (area under the curve of the receiver operating characteristic) score and optionally the curve. It is calculated for a specific class (provided as "class_name") treated as positive.

        Hide auc_roc attributes Show auc_roc attributes object
        • value number Required
        • curve array[object]
      • accuracy object

        Accuracy of predictions (per-class and overall).

        Hide accuracy attributes Show accuracy attributes object
        • classes array[object] Required
        • overall_accuracy number Required
      • multiclass_confusion_matrix object

        Multiclass confusion matrix.

        Hide multiclass_confusion_matrix attributes Show multiclass_confusion_matrix attributes object
        • confusion_matrix array[object] Required
        • other_actual_class_count number Required
      • precision object

        Precision of predictions (per-class and average).

        Hide precision attributes Show precision attributes object
        • classes array[object] Required
        • avg_precision number Required
      • recall object

        Recall of predictions (per-class and average).

        Hide recall attributes Show recall attributes object
        • classes array[object] Required
        • avg_recall number Required
    • outlier_detection object

      Evaluation results for an outlier detection analysis. It outputs the probability that each document is an outlier.

      Hide outlier_detection attributes Show outlier_detection attributes object
      • auc_roc object

        The AUC ROC (area under the curve of the receiver operating characteristic) score and optionally the curve.

        Hide auc_roc attributes Show auc_roc attributes object
        • value number Required
        • curve array[object]
      • precision object

        Set the different thresholds of the outlier score at where the metric is calculated.

        Hide precision attribute Show precision attribute object
        • * number Additional properties
      • recall object

        Set the different thresholds of the outlier score at where the metric is calculated.

        Hide recall attribute Show recall attribute object
        • * number Additional properties
      • confusion_matrix object

        Set the different thresholds of the outlier score at where the metrics (tp - true positive, fp - false positive, tn - true negative, fn - false negative) are calculated.

        Hide confusion_matrix attribute Show confusion_matrix attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • tp number Required

            True Positive

          • fp number Required

            False Positive

          • tn number Required

            True Negative

          • fn number Required

            False Negative

    • regression object

      Evaluation results for a regression analysis which outputs a prediction of values.

      Hide regression attributes Show regression attributes object
      • huber object

        Pseudo Huber loss function.

        Hide huber attribute Show huber attribute object
        • value number Required
      • mse object

        Average squared difference between the predicted values and the actual (ground truth) value.

        Hide mse attribute Show mse attribute object
        • value number Required
      • msle object

        Average squared difference between the logarithm of the predicted values and the logarithm of the actual (ground truth) value.

        Hide msle attribute Show msle attribute object
        • value number Required
      • r_squared object

        Proportion of the variance in the dependent variable that is predictable from the independent variables.

        Hide r_squared attribute Show r_squared attribute object
        • value number Required
POST /_ml/data_frame/_evaluate
POST _ml/data_frame/_evaluate
{
  "index": "animal_classification",
  "evaluation": {
    "classification": {
      "actual_field": "animal_class",
      "predicted_field": "ml.animal_class_prediction",
      "metrics": {
        "multiclass_confusion_matrix": {}
      }
    }
  }
}
resp = client.ml.evaluate_data_frame(
    index="animal_classification",
    evaluation={
        "classification": {
            "actual_field": "animal_class",
            "predicted_field": "ml.animal_class_prediction",
            "metrics": {
                "multiclass_confusion_matrix": {}
            }
        }
    },
)
const response = await client.ml.evaluateDataFrame({
  index: "animal_classification",
  evaluation: {
    classification: {
      actual_field: "animal_class",
      predicted_field: "ml.animal_class_prediction",
      metrics: {
        multiclass_confusion_matrix: {},
      },
    },
  },
});
response = client.ml.evaluate_data_frame(
  body: {
    "index": "animal_classification",
    "evaluation": {
      "classification": {
        "actual_field": "animal_class",
        "predicted_field": "ml.animal_class_prediction",
        "metrics": {
          "multiclass_confusion_matrix": {}
        }
      }
    }
  }
)
$resp = $client->ml()->evaluateDataFrame([
    "body" => [
        "index" => "animal_classification",
        "evaluation" => [
            "classification" => [
                "actual_field" => "animal_class",
                "predicted_field" => "ml.animal_class_prediction",
                "metrics" => [
                    "multiclass_confusion_matrix" => new ArrayObject([]),
                ],
            ],
        ],
    ],
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"index":"animal_classification","evaluation":{"classification":{"actual_field":"animal_class","predicted_field":"ml.animal_class_prediction","metrics":{"multiclass_confusion_matrix":{}}}}}' "$ELASTICSEARCH_URL/_ml/data_frame/_evaluate"
client.ml().evaluateDataFrame(e -> e
    .evaluation(ev -> ev
        .classification(c -> c
            .actualField("animal_class")
            .predictedField("ml.animal_class_prediction")
            .metrics(m -> m)
        )
    )
    .index("animal_classification")
);
Request examples
Run `POST _ml/data_frame/_evaluate` to evaluate a a classification job for an annotated index. The `actual_field` contains the ground truth for classification. The `predicted_field` contains the predicted value calculated by the classification analysis.
{
  "index": "animal_classification",
  "evaluation": {
    "classification": {
      "actual_field": "animal_class",
      "predicted_field": "ml.animal_class_prediction",
      "metrics": {
        "multiclass_confusion_matrix": {}
      }
    }
  }
}
Run `POST _ml/data_frame/_evaluate` to evaluate a classification job with AUC ROC metrics for an annotated index. The `actual_field` contains the ground truth value for the actual animal classification. This is required in order to evaluate results. The `class_name` specifies the class name that is treated as positive during the evaluation, all the other classes are treated as negative.
{
  "index": "animal_classification",
  "evaluation": {
    "classification": {
      "actual_field": "animal_class",
      "metrics": {
        "auc_roc": {
          "class_name": "dog"
        }
      }
    }
  }
}
Run `POST _ml/data_frame/_evaluate` to evaluate an outlier detection job for an annotated index.
{
  "index": "my_analytics_dest_index",
  "evaluation": {
    "outlier_detection": {
      "actual_field": "is_outlier",
      "predicted_probability_field": "ml.outlier_score"
    }
  }
}
Run `POST _ml/data_frame/_evaluate` to evaluate the testing error of a regression job for an annotated index. The term query in the body limits evaluation to be performed on the test split only. The `actual_field` contains the ground truth for house prices. The `predicted_field` contains the house price calculated by the regression analysis.
{
  "index": "house_price_predictions",
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "ml.is_training": false
          }
        }
      ]
    }
  },
  "evaluation": {
    "regression": {
      "actual_field": "price",
      "predicted_field": "ml.price_prediction",
      "metrics": {
        "r_squared": {},
        "mse": {},
        "msle": {
          "offset": 10
        },
        "huber": {
          "delta": 1.5
        }
      }
    }
  }
}
Run `POST _ml/data_frame/_evaluate` to evaluate the training error of a regression job for an annotated index. The term query in the body limits evaluation to be performed on the training split only. The `actual_field` contains the ground truth for house prices. The `predicted_field` contains the house price calculated by the regression analysis.
{
  "index": "house_price_predictions",
  "query": {
    "term": {
      "ml.is_training": {
        "value": true
      }
    }
  },
  "evaluation": {
    "regression": {
      "actual_field": "price",
      "predicted_field": "ml.price_prediction",
      "metrics": {
        "r_squared": {},
        "mse": {},
        "msle": {},
        "huber": {}
      }
    }
  }
}
Response examples (200)
A succesful response from `POST _ml/data_frame/_evaluate` to evaluate a classification analysis job for an annotated index. The `actual_class` contains the name of the class the analysis tried to predict. The `actual_class_doc_count` is the number of documents in the index belonging to the `actual_class`. The `predicted_classes` object contains the list of the predicted classes and the number of predictions associated with the class.
{
  "classification": {
    "multiclass_confusion_matrix": {
      "confusion_matrix": [
        {
          "actual_class": "cat",
          "actual_class_doc_count": 12,
          "predicted_classes": [
            {
              "predicted_class": "cat",
              "count": 12
            },
            {
              "predicted_class": "dog",
              "count": 0
            }
          ],
          "other_predicted_class_doc_count": 0
        },
        {
          "actual_class": "dog",
          "actual_class_doc_count": 11,
          "predicted_classes": [
            {
              "predicted_class": "dog",
              "count": 7
            },
            {
              "predicted_class": "cat",
              "count": 4
            }
          ],
          "other_predicted_class_doc_count": 0
        }
      ],
      "other_actual_class_count": 0
    }
  }
}
A succesful response from `POST _ml/data_frame/_evaluate` to evaluate a classification analysis job with the AUC ROC metrics for an annotated index.
{
  "classification": {
    "auc_roc": {
      "value": 0.8941788639536681
    }
  }
}
A successful response from `POST _ml/data_frame/_evaluate` to evaluate an outlier detection job.
{
  "outlier_detection": {
    "auc_roc": {
      "value": 0.9258475774641445
    },
    "confusion_matrix": {
      "0.25": {
        "tp": 5,
        "fp": 9,
        "tn": 204,
        "fn": 5
      },
      "0.5": {
        "tp": 1,
        "fp": 5,
        "tn": 208,
        "fn": 9
      },
      "0.75": {
        "tp": 0,
        "fp": 4,
        "tn": 209,
        "fn": 10
      }
    },
    "precision": {
      "0.25": 0.35714285714285715,
      "0.5": 0.16666666666666666,
      "0.75": 0
    },
    "recall": {
      "0.25": 0.5,
      "0.5": 0.1,
      "0.75": 0
    }
  }
}
















Stop data frame analytics jobs Generally available; Added in 7.3.0

POST /_ml/data_frame/analytics/{id}/_stop

A data frame analytics job can be started and stopped multiple times throughout its lifecycle.

Required authorization

  • Cluster privileges: manage_ml

Path parameters

  • id string Required

    Identifier for the data frame analytics job. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    1. Contains wildcard expressions and there are no data frame analytics jobs that match.
    2. Contains the _all string or no identifiers and there are no matches.
    3. Contains wildcard expressions and there are only partial matches.

    The default value is true, which returns an empty data_frame_analytics array when there are no matches and the subset of results when there are partial matches. If this parameter is false, the request returns a 404 status code when there are no matches or only partial matches.

  • force boolean

    If true, the data frame analytics job is stopped forcefully.

  • timeout string

    Controls the amount of time to wait until the data frame analytics job stops. Defaults to 20 seconds.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • stopped boolean Required
POST /_ml/data_frame/analytics/{id}/_stop
POST _ml/data_frame/analytics/loganalytics/_stop
resp = client.ml.stop_data_frame_analytics(
    id="loganalytics",
)
const response = await client.ml.stopDataFrameAnalytics({
  id: "loganalytics",
});
response = client.ml.stop_data_frame_analytics(
  id: "loganalytics"
)
$resp = $client->ml()->stopDataFrameAnalytics([
    "id" => "loganalytics",
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_ml/data_frame/analytics/loganalytics/_stop"
client.ml().stopDataFrameAnalytics(s -> s
    .id("loganalytics")
);













Create a trained model Generally available; Added in 7.10.0

PUT /_ml/trained_models/{model_id}

Enable you to supply a trained model that is not created by data frame analytics.

Required authorization

  • Cluster privileges: manage_ml

Path parameters

  • model_id string Required

    The unique identifier of the trained model.

Query parameters

  • defer_definition_decompression boolean Generally available; Added in 8.0.0

    If set to true and a compressed_definition is provided, the request defers definition decompression and skips relevant validations.

  • wait_for_completion boolean Generally available; Added in 8.8.0

    Whether to wait for all child operations (e.g. model download) to complete.

application/json

Body Required

  • compressed_definition string

    The compressed (GZipped and Base64 encoded) inference definition of the model. If compressed_definition is specified, then definition cannot be specified.

  • definition object

    The inference definition for the model. If definition is specified, then compressed_definition cannot be specified.

    Hide definition attributes Show definition attributes object
    • preprocessors array[object]

      Collection of preprocessors

      Hide preprocessors attributes Show preprocessors attributes object
      • frequency_encoding object
        Hide frequency_encoding attributes Show frequency_encoding attributes object
        • field string Required
        • feature_name string Required
        • frequency_map object Required
      • one_hot_encoding object
        Hide one_hot_encoding attributes Show one_hot_encoding attributes object
        • field string Required
        • hot_map object Required
      • target_mean_encoding object
        Hide target_mean_encoding attributes Show target_mean_encoding attributes object
        • field string Required
        • feature_name string Required
        • target_map object Required
        • default_value number Required
    • trained_model object Required

      The definition of the trained model.

      Hide trained_model attributes Show trained_model attributes object
      • tree object

        The definition for a binary decision tree.

        Hide tree attributes Show tree attributes object
        • classification_labels array[string]
        • feature_names array[string] Required
        • target_type string
        • tree_structure array[object] Required
      • tree_node object

        The definition of a node in a tree. There are two major types of nodes: leaf nodes and not-leaf nodes.

        • Leaf nodes only need node_index and leaf_value defined.
        • All other nodes need split_feature, left_child, right_child, threshold, decision_type, and default_left defined.
        Hide tree_node attributes Show tree_node attributes object
        • decision_type string
        • default_left boolean
        • leaf_value number
        • left_child number
        • node_index number Required
        • right_child number
        • split_feature number
        • split_gain number
        • threshold number
      • ensemble object

        The definition for an ensemble model

        Hide ensemble attributes Show ensemble attributes object
        • classification_labels array[string]
        • feature_names array[string]
        • target_type string
        • trained_models array[object] Required
  • description string

    A human-readable description of the inference trained model.

  • inference_config object

    The default configuration for inference. This can be either a regression or classification configuration. It must match the underlying definition.trained_model's target_type. For pre-packaged models such as ELSER the config is not required.

    Hide inference_config attributes Show inference_config attributes object
    • regression object

      Regression configuration for inference.

      Hide regression attributes Show regression attributes object
      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • num_top_feature_importance_values number

        Specifies the maximum number of feature importance values per document.

        Default value is 0.

    • classification object

      Classification configuration for inference.

      Hide classification attributes Show classification attributes object
      • num_top_classes number

        Specifies the number of top class predictions to return. Defaults to 0.

      • num_top_feature_importance_values number

        Specifies the maximum number of feature importance values per document.

        Default value is 0.

      • prediction_field_type string

        Specifies the type of the predicted field to write. Acceptable values are: string, number, boolean. When boolean is provided 1.0 is transformed to true and 0.0 to false.

      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • top_classes_results_field string

        Specifies the field to which the top classes are written. Defaults to top_classes.

    • text_classification object

      Text classification configuration for inference.

      Hide text_classification attributes Show text_classification attributes object
      • num_top_classes number

        Specifies the number of top class predictions to return. Defaults to 0.

      • tokenization object

        The tokenization options

      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • classification_labels array[string]

        Classification labels to apply other than the stored labels. Must have the same deminsions as the default configured labels

      • vocabulary object
    • zero_shot_classification object

      Zeroshot classification configuration for inference.

      Hide zero_shot_classification attributes Show zero_shot_classification attributes object
      • tokenization object

        The tokenization options to update when inferring

      • hypothesis_template string

        Hypothesis template used when tokenizing labels for prediction

        Default value is "This example is {}.".

      • classification_labels array[string] Required

        The zero shot classification labels indicating entailment, neutral, and contradiction Must contain exactly and only entailment, neutral, and contradiction

      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • multi_label boolean

        Indicates if more than one true label exists.

        Default value is false.

      • labels array[string]

        The labels to predict.

    • fill_mask object

      Fill mask configuration for inference.

      Hide fill_mask attributes Show fill_mask attributes object
      • mask_token string

        The string/token which will be removed from incoming documents and replaced with the inference prediction(s). In a response, this field contains the mask token for the specified model/tokenizer. Each model and tokenizer has a predefined mask token which cannot be changed. Thus, it is recommended not to set this value in requests. However, if this field is present in a request, its value must match the predefined value for that model/tokenizer, otherwise the request will fail.

      • num_top_classes number

        Specifies the number of top class predictions to return. Defaults to 0.

      • tokenization object

        The tokenization options to update when inferring

      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • vocabulary object Required
    • learning_to_rank object
      Hide learning_to_rank attributes Show learning_to_rank attributes object
      • default_params object
        Hide default_params attribute Show default_params attribute object
        • * object Additional properties
      • feature_extractors array[object]
      • num_top_feature_importance_values number Required
    • ner object

      Named entity recognition configuration for inference.

      Hide ner attributes Show ner attributes object
      • tokenization object

        The tokenization options

      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • classification_labels array[string]

        The token classification labels. Must be IOB formatted tags

      • vocabulary object
    • pass_through object

      Pass through configuration for inference.

      Hide pass_through attributes Show pass_through attributes object
      • tokenization object

        The tokenization options

      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • vocabulary object
    • text_embedding object

      Text embedding configuration for inference.

      Hide text_embedding attributes Show text_embedding attributes object
      • embedding_size number

        The number of dimensions in the embedding output

      • tokenization object

        The tokenization options

      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • vocabulary object Required
    • text_expansion object

      Text expansion configuration for inference.

      Hide text_expansion attributes Show text_expansion attributes object
      • tokenization object

        The tokenization options

      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • vocabulary object Required
    • question_answering object

      Question answering configuration for inference.

      Hide question_answering attributes Show question_answering attributes object
      • num_top_classes number

        Specifies the number of top class predictions to return. Defaults to 0.

      • tokenization object

        The tokenization options to update when inferring

      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • max_answer_length number

        The maximum answer length to consider

  • input object

    The input field names for the model definition.

    Hide input attribute Show input attribute object
    • field_names string | array[string] Required
  • metadata object

    An object map that contains metadata about the model.

  • model_type string

    The model type.

    Supported values include:

    • tree_ensemble: The model definition is an ensemble model of decision trees.
    • lang_ident: A special type reserved for language identification models.
    • pytorch: The stored definition is a PyTorch (specifically a TorchScript) model. Currently only NLP models are supported.

    Values are tree_ensemble, lang_ident, or pytorch.

  • model_size_bytes number

    The estimated memory usage in bytes to keep the trained model in memory. This property is supported only if defer_definition_decompression is true or the model definition is not supplied.

  • platform_architecture string

    The platform architecture (if applicable) of the trained mode. If the model only works on one platform, because it is heavily optimized for a particular processor architecture and OS combination, then this field specifies which. The format of the string must match the platform identifiers used by Elasticsearch, so one of, linux-x86_64, linux-aarch64, darwin-x86_64, darwin-aarch64, or windows-x86_64. For portable models (those that work independent of processor architecture or OS features), leave this field unset.

  • tags array[string]

    An array of tags to organize the model.

  • prefix_strings object

    Optional prefix strings applied at inference

    Hide prefix_strings attributes Show prefix_strings attributes object
    • ingest string

      String prepended to input at ingest

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • model_id string Required

      Identifier for the trained model.

    • model_type string

      The model type

      Supported values include:

      • tree_ensemble: The model definition is an ensemble model of decision trees.
      • lang_ident: A special type reserved for language identification models.
      • pytorch: The stored definition is a PyTorch (specifically a TorchScript) model. Currently only NLP models are supported.

      Values are tree_ensemble, lang_ident, or pytorch.

    • tags array[string] Required

      A comma delimited string of tags. A trained model can have many tags, or none.

    • version string

      The Elasticsearch version number in which the trained model was created.

    • compressed_definition string
    • created_by string

      Information on the creator of the trained model.

    • create_time string | number

      The time when the trained model was created.

      One of:

      The time when the trained model was created.

    • default_field_map object

      Any field map described in the inference configuration takes precedence.

      Hide default_field_map attribute Show default_field_map attribute object
      • * string Additional properties
    • description string

      The free-text description of the trained model.

    • estimated_heap_memory_usage_bytes number

      The estimated heap usage in bytes to keep the trained model in memory.

    • estimated_operations number

      The estimated number of operations to use the trained model.

    • fully_defined boolean

      True if the full model definition is present.

    • inference_config object

      The default configuration for inference. This can be either a regression, classification, or one of the many NLP focused configurations. It must match the underlying definition.trained_model's target_type. For pre-packaged models such as ELSER the config is not required.

      Hide inference_config attributes Show inference_config attributes object
      • regression object

        Regression configuration for inference.

        Hide regression attributes Show regression attributes object
        • results_field string

          The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

        • num_top_feature_importance_values number

          Specifies the maximum number of feature importance values per document.

          Default value is 0.

      • classification object

        Classification configuration for inference.

        Hide classification attributes Show classification attributes object
        • num_top_classes number

          Specifies the number of top class predictions to return. Defaults to 0.

        • num_top_feature_importance_values number

          Specifies the maximum number of feature importance values per document.

          Default value is 0.

        • prediction_field_type string

          Specifies the type of the predicted field to write. Acceptable values are: string, number, boolean. When boolean is provided 1.0 is transformed to true and 0.0 to false.

        • results_field string

          The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

        • top_classes_results_field string

          Specifies the field to which the top classes are written. Defaults to top_classes.

      • text_classification object

        Text classification configuration for inference.

        Hide text_classification attributes Show text_classification attributes object
        • num_top_classes number

          Specifies the number of top class predictions to return. Defaults to 0.

        • tokenization object

          The tokenization options

        • results_field string

          The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

        • classification_labels array[string]

          Classification labels to apply other than the stored labels. Must have the same deminsions as the default configured labels

        • vocabulary object
      • zero_shot_classification object

        Zeroshot classification configuration for inference.

        Hide zero_shot_classification attributes Show zero_shot_classification attributes object
        • tokenization object

          The tokenization options to update when inferring

        • hypothesis_template string

          Hypothesis template used when tokenizing labels for prediction

          Default value is "This example is {}.".

        • classification_labels array[string] Required

          The zero shot classification labels indicating entailment, neutral, and contradiction Must contain exactly and only entailment, neutral, and contradiction

        • results_field string

          The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

        • multi_label boolean

          Indicates if more than one true label exists.

          Default value is false.

        • labels array[string]

          The labels to predict.

      • fill_mask object

        Fill mask configuration for inference.

        Hide fill_mask attributes Show fill_mask attributes object
        • mask_token string

          The string/token which will be removed from incoming documents and replaced with the inference prediction(s). In a response, this field contains the mask token for the specified model/tokenizer. Each model and tokenizer has a predefined mask token which cannot be changed. Thus, it is recommended not to set this value in requests. However, if this field is present in a request, its value must match the predefined value for that model/tokenizer, otherwise the request will fail.

        • num_top_classes number

          Specifies the number of top class predictions to return. Defaults to 0.

        • tokenization object

          The tokenization options to update when inferring

        • results_field string

          The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

        • vocabulary object Required
      • learning_to_rank object
        Hide learning_to_rank attributes Show learning_to_rank attributes object
        • default_params object
          Hide default_params attribute Show default_params attribute object
          • * object Additional properties
        • feature_extractors array[object]
        • num_top_feature_importance_values number Required
      • ner object

        Named entity recognition configuration for inference.

        Hide ner attributes Show ner attributes object
        • tokenization object

          The tokenization options

        • results_field string

          The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

        • classification_labels array[string]

          The token classification labels. Must be IOB formatted tags

        • vocabulary object
      • pass_through object

        Pass through configuration for inference.

        Hide pass_through attributes Show pass_through attributes object
        • tokenization object

          The tokenization options

        • results_field string

          The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

        • vocabulary object
      • text_embedding object

        Text embedding configuration for inference.

        Hide text_embedding attributes Show text_embedding attributes object
        • embedding_size number

          The number of dimensions in the embedding output

        • tokenization object

          The tokenization options

        • results_field string

          The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

        • vocabulary object Required
      • text_expansion object

        Text expansion configuration for inference.

        Hide text_expansion attributes Show text_expansion attributes object
        • tokenization object

          The tokenization options

        • results_field string

          The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

        • vocabulary object Required
      • question_answering object

        Question answering configuration for inference.

        Hide question_answering attributes Show question_answering attributes object
        • num_top_classes number

          Specifies the number of top class predictions to return. Defaults to 0.

        • tokenization object

          The tokenization options to update when inferring

        • results_field string

          The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

        • max_answer_length number

          The maximum answer length to consider

    • input object Required

      The input field names for the model definition.

      Hide input attribute Show input attribute object
      • field_names array[string] Required

        An array of input field names for the model.

    • license_level string

      The license level of the trained model.

    • metadata object

      An object containing metadata about the trained model. For example, models created by data frame analytics contain analysis_config and input objects.

      Hide metadata attributes Show metadata attributes object
      • model_aliases array[string]
      • feature_importance_baseline object

        An object that contains the baseline for feature importance values. For regression analysis, it is a single value. For classification analysis, there is a value for each class.

        Hide feature_importance_baseline attribute Show feature_importance_baseline attribute object
        • * string Additional properties
      • hyperparameters array[object]

        List of the available hyperparameters optimized during the fine_parameter_tuning phase as well as specified by the user.

        Hide hyperparameters attributes Show hyperparameters attributes object
        • absolute_importance number

          A positive number showing how much the parameter influences the variation of the loss function. For hyperparameters with values that are not specified by the user but tuned during hyperparameter optimization.

        • name string Required

          Name of the hyperparameter.

        • relative_importance number

          A number between 0 and 1 showing the proportion of influence on the variation of the loss function among all tuned hyperparameters. For hyperparameters with values that are not specified by the user but tuned during hyperparameter optimization.

        • supplied boolean Required

          Indicates if the hyperparameter is specified by the user (true) or optimized (false).

        • value number Required

          The value of the hyperparameter, either optimized or specified by the user.

      • total_feature_importance array[object]

        An array of the total feature importance for each feature used from the training data set. This array of objects is returned if data frame analytics trained the model and the request includes total_feature_importance in the include request parameter.

        Hide total_feature_importance attributes Show total_feature_importance attributes object
        • feature_name string Required

          The feature for which this importance was calculated.

        • importance array[object] Required

          A collection of feature importance statistics related to the training data set for this particular feature.

        • classes array[object] Required

          If the trained model is a classification model, feature importance statistics are gathered per target class value.

    • model_size_bytes number | string

    • model_package object
      Hide model_package attributes Show model_package attributes object
      • Time unit for milliseconds

      • description string
      • inference_config object
        Hide inference_config attribute Show inference_config attribute object
        • * object Additional properties
      • metadata object
        Hide metadata attribute Show metadata attribute object
        • * object Additional properties
      • minimum_version string
      • model_repository string
      • model_type string
      • packaged_model_id string Required
      • platform_architecture string
      • prefix_strings object
        Hide prefix_strings attributes Show prefix_strings attributes object
        • ingest string

          String prepended to input at ingest

      • size number | string

      • sha256 string
      • tags array[string]
      • vocabulary_file string
    • location object
      Hide location attribute Show location attribute object
      • index object Required
        Hide index attribute Show index attribute object
        • name string Required
    • platform_architecture string
    • prefix_strings object
      Hide prefix_strings attributes Show prefix_strings attributes object
      • ingest string

        String prepended to input at ingest

PUT /_ml/trained_models/{model_id}
curl \
 --request PUT 'http://api.example.com/_ml/trained_models/{model_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"compressed_definition":"string","definition":{"preprocessors":[{"frequency_encoding":{"field":"string","feature_name":"string","frequency_map":{}},"one_hot_encoding":{"field":"string","hot_map":{}},"target_mean_encoding":{"field":"string","feature_name":"string","target_map":{},"default_value":42.0}}],"trained_model":{"tree":{"classification_labels":["string"],"feature_names":["string"],"target_type":"string","tree_structure":[{}]},"tree_node":{"decision_type":"string","default_left":true,"leaf_value":42.0,"left_child":42.0,"node_index":42.0,"right_child":42.0,"split_feature":42.0,"split_gain":42.0,"threshold":42.0},"ensemble":{"classification_labels":["string"],"feature_names":["string"],"target_type":"string","trained_models":[{}]}}},"description":"string","inference_config":{"regression":{"results_field":"string","num_top_feature_importance_values":0},"classification":{"num_top_classes":42.0,"num_top_feature_importance_values":0,"prediction_field_type":"string","results_field":"string","top_classes_results_field":"string"},"text_classification":{"num_top_classes":42.0,"tokenization":{},"results_field":"string","classification_labels":["string"],"vocabulary":{}},"zero_shot_classification":{"tokenization":{},"hypothesis_template":"\"This example is {}.\"","classification_labels":["string"],"results_field":"string","multi_label":false,"labels":["string"]},"fill_mask":{"mask_token":"string","num_top_classes":42.0,"tokenization":{},"results_field":"string","vocabulary":{}},"learning_to_rank":{"default_params":{"additionalProperty1":{},"additionalProperty2":{}},"feature_extractors":[{}],"num_top_feature_importance_values":42.0},"ner":{"tokenization":{},"results_field":"string","classification_labels":["string"],"vocabulary":{}},"pass_through":{"tokenization":{},"results_field":"string","vocabulary":{}},"text_embedding":{"embedding_size":42.0,"tokenization":{},"results_field":"string","vocabulary":{}},"text_expansion":{"tokenization":{},"results_field":"string","vocabulary":{}},"question_answering":{"num_top_classes":42.0,"tokenization":{},"results_field":"string","max_answer_length":42.0}},"input":{"field_names":"string"},"metadata":{},"model_type":"tree_ensemble","model_size_bytes":42.0,"platform_architecture":"string","tags":["string"],"prefix_strings":{"ingest":"string","search":"string"}}'




















Create part of a trained model definition Generally available; Added in 8.0.0

PUT /_ml/trained_models/{model_id}/definition/{part}

Required authorization

  • Cluster privileges: manage_ml

Path parameters

  • model_id string Required

    The unique identifier of the trained model.

  • part number Required

    The definition part number. When the definition is loaded for inference the definition parts are streamed in the order of their part number. The first part must be 0 and the final part must be total_parts - 1.

application/json

Body Required

  • definition string Required

    The definition part for the model. Must be a base64 encoded string.

  • total_definition_length number Required

    The total uncompressed definition length in bytes. Not base64 encoded.

  • total_parts number Required

    The total number of parts that will be uploaded. Must be greater than 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /_ml/trained_models/{model_id}/definition/{part}
PUT _ml/trained_models/elastic__distilbert-base-uncased-finetuned-conll03-english/definition/0
{
    "definition": "...",
    "total_definition_length": 265632637,
    "total_parts": 64
}
resp = client.ml.put_trained_model_definition_part(
    model_id="elastic__distilbert-base-uncased-finetuned-conll03-english",
    part="0",
    definition="...",
    total_definition_length=265632637,
    total_parts=64,
)
const response = await client.ml.putTrainedModelDefinitionPart({
  model_id: "elastic__distilbert-base-uncased-finetuned-conll03-english",
  part: 0,
  definition: "...",
  total_definition_length: 265632637,
  total_parts: 64,
});
response = client.ml.put_trained_model_definition_part(
  model_id: "elastic__distilbert-base-uncased-finetuned-conll03-english",
  part: "0",
  body: {
    "definition": "...",
    "total_definition_length": 265632637,
    "total_parts": 64
  }
)
$resp = $client->ml()->putTrainedModelDefinitionPart([
    "model_id" => "elastic__distilbert-base-uncased-finetuned-conll03-english",
    "part" => "0",
    "body" => [
        "definition" => "...",
        "total_definition_length" => 265632637,
        "total_parts" => 64,
    ],
]);
curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"definition":"...","total_definition_length":265632637,"total_parts":64}' "$ELASTICSEARCH_URL/_ml/trained_models/elastic__distilbert-base-uncased-finetuned-conll03-english/definition/0"
Request example
An example body for a `PUT _ml/trained_models/elastic__distilbert-base-uncased-finetuned-conll03-english/definition/0` request.
{
    "definition": "...",
    "total_definition_length": 265632637,
    "total_parts": 64
}












Update a trained model deployment Generally available; Added in 8.6.0

POST /_ml/trained_models/{model_id}/deployment/_update

Required authorization

  • Cluster privileges: manage_ml

Path parameters

  • model_id string Required

    The unique identifier of the trained model. Currently, only PyTorch models are supported.

Query parameters

  • number_of_allocations number

    The number of model allocations on each node where the model is deployed. All allocations on a node share the same copy of the model in memory but use a separate set of threads to evaluate the model. Increasing this value generally increases the throughput. If this setting is greater than the number of hardware threads it will automatically be changed to a value less than the number of hardware threads.

application/json

Body

  • number_of_allocations number

    The number of model allocations on each node where the model is deployed. All allocations on a node share the same copy of the model in memory but use a separate set of threads to evaluate the model. Increasing this value generally increases the throughput. If this setting is greater than the number of hardware threads it will automatically be changed to a value less than the number of hardware threads. If adaptive_allocations is enabled, do not set this value, because it’s automatically set.

    Default value is 1.

  • adaptive_allocations object

    Adaptive allocations configuration. When enabled, the number of allocations is set based on the current load. If adaptive_allocations is enabled, do not set the number of allocations manually.

    Hide adaptive_allocations attributes Show adaptive_allocations attributes object
    • enabled boolean Required

      If true, adaptive_allocations is enabled

    • min_number_of_allocations number

      Specifies the minimum number of allocations to scale to. If set, it must be greater than or equal to 0. If not defined, the deployment scales to 0.

    • max_number_of_allocations number

      Specifies the maximum number of allocations to scale to. If set, it must be greater than or equal to min_number_of_allocations.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • assignment object Required
      Hide assignment attributes Show assignment attributes object
      • adaptive_allocations object | string | null

        One of:
        Hide attributes Show attributes
        • enabled boolean Required

          If true, adaptive_allocations is enabled

        • min_number_of_allocations number

          Specifies the minimum number of allocations to scale to. If set, it must be greater than or equal to 0. If not defined, the deployment scales to 0.

        • max_number_of_allocations number

          Specifies the maximum number of allocations to scale to. If set, it must be greater than or equal to min_number_of_allocations.

      • assignment_state string Required

        The overall assignment state.

        Supported values include:

        • started: The deployment is usable; at least one node has the model allocated.
        • starting: The deployment has recently started but is not yet usable; the model is not allocated on any nodes.
        • stopping: The deployment is preparing to stop and deallocate the model from the relevant nodes.
        • failed: The deployment is on a failed state and must be re-deployed.

        Values are started, starting, stopping, or failed.

      • max_assigned_allocations number
      • reason string
      • routing_table object Required

        The allocation state for each node.

        Hide routing_table attribute Show routing_table attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • reason string

            The reason for the current state. It is usually populated only when the routing_state is failed.

          • routing_state string Required

            The current routing state.

            Supported values include:

            • failed: The allocation attempt failed.
            • started: The trained model is allocated and ready to accept inference requests.
            • starting: The trained model is attempting to allocate on this node; inference requests are not yet accepted.
            • stopped: The trained model is fully deallocated from this node.
            • stopping: The trained model is being deallocated from this node.

            Values are failed, started, starting, stopped, or stopping.

          • current_allocations number Required

            Current number of allocations.

          • target_allocations number Required

            Target number of allocations.

      • start_time string | number

        The timestamp when the deployment started.

        One of:

        The timestamp when the deployment started.

      • task_parameters object Required
        Hide task_parameters attributes Show task_parameters attributes object
        • model_bytes
        • model_id string Required

          The unique identifier for the trained model.

        • deployment_id string Required

          The unique identifier for the trained model deployment.

        • cache_size
        • number_of_allocations number Required

          The total number of allocations this model is assigned across ML nodes.

        • priority string Required

          Values are normal or low.

        • per_deployment_memory_bytes
        • per_allocation_memory_bytes
        • queue_capacity number Required

          Number of inference requests are allowed in the queue at a time.

        • threads_per_allocation number Required

          Number of threads per allocation.

POST /_ml/trained_models/{model_id}/deployment/_update
POST _ml/trained_models/elastic__distilbert-base-uncased-finetuned-conll03-english/deployment/_update
{
  "number_of_allocations": 4
}
resp = client.ml.update_trained_model_deployment(
    model_id="elastic__distilbert-base-uncased-finetuned-conll03-english",
    number_of_allocations=4,
)
const response = await client.ml.updateTrainedModelDeployment({
  model_id: "elastic__distilbert-base-uncased-finetuned-conll03-english",
  number_of_allocations: 4,
});
response = client.ml.update_trained_model_deployment(
  model_id: "elastic__distilbert-base-uncased-finetuned-conll03-english",
  body: {
    "number_of_allocations": 4
  }
)
$resp = $client->ml()->updateTrainedModelDeployment([
    "model_id" => "elastic__distilbert-base-uncased-finetuned-conll03-english",
    "body" => [
        "number_of_allocations" => 4,
    ],
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"number_of_allocations":4}' "$ELASTICSEARCH_URL/_ml/trained_models/elastic__distilbert-base-uncased-finetuned-conll03-english/deployment/_update"
client.ml().updateTrainedModelDeployment(u -> u
    .modelId("elastic__distilbert-base-uncased-finetuned-conll03-english")
    .numberOfAllocations(4)
);
Request example
An example body for a `POST _ml/trained_models/elastic__distilbert-base-uncased-finetuned-conll03-english/deployment/_update` request.
{
  "number_of_allocations": 4
}





Create an index from a source index Technical preview; Added in 8.18.0

POST /_create_from/{source}/{dest}

All methods and paths for this operation:

PUT /_create_from/{source}/{dest}

POST /_create_from/{source}/{dest}

Copy the mappings and settings from the source index to a destination index while allowing request settings and mappings to override the source values.

Path parameters

  • source string Required

    The source index or data stream name

  • dest string Required

    The destination index or data stream name

application/json

Body

  • mappings_override object

    Mappings overrides to be applied to the destination index (optional)

    Hide mappings_override attributes Show mappings_override attributes object
    • all_field object
      Hide all_field attributes Show all_field attributes object
      • analyzer string Required
      • enabled boolean Required
      • omit_norms boolean Required
      • search_analyzer string Required
      • similarity string Required
      • store boolean Required
      • store_term_vector_offsets boolean Required
      • store_term_vector_payloads boolean Required
      • store_term_vector_positions boolean Required
      • store_term_vectors boolean Required
    • date_detection boolean
    • dynamic string

      Values are strict, runtime, true, or false.

    • dynamic_date_formats array[string]
    • dynamic_templates array[object]
    • _field_names object
      Hide _field_names attribute Show _field_names attribute object
      • enabled boolean Required
    • index_field object
      Hide index_field attribute Show index_field attribute object
      • enabled boolean Required
    • _meta object
      Hide _meta attribute Show _meta attribute object
      • * object Additional properties
    • numeric_detection boolean
    • properties object
    • _routing object
      Hide _routing attribute Show _routing attribute object
      • required boolean Required
    • _size object
      Hide _size attribute Show _size attribute object
      • enabled boolean Required
    • _source object
      Hide _source attributes Show _source attributes object
      • compress boolean
      • compress_threshold string
      • enabled boolean
      • excludes array[string]
      • includes array[string]
      • mode string

        Supported values include:

        • disabled
        • stored
        • synthetic: Instead of storing source documents on disk exactly as you send them, Elasticsearch can reconstruct source content on the fly upon retrieval.

        Values are disabled, stored, or synthetic.

    • runtime object
      Hide runtime attribute Show runtime attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • fields object

          For type composite

          Hide fields attribute Show fields attribute object
          • * object Additional properties
        • fetch_fields array[object]

          For type lookup

          Hide fetch_fields attributes Show fetch_fields attributes object
          • field
          • format string
        • format string

          A custom format for date type runtime fields.

        • input_field string

          For type lookup

        • target_field string

          For type lookup

        • target_index string

          For type lookup

        • script object

          Painless script executed at query time.

          Hide script attributes Show script attributes object
          • source string

            The script source.

          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • type string Required

          Field type, which can be: boolean, composite, date, double, geo_point, ip,keyword, long, or lookup.

          Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

    • enabled boolean
    • subobjects string

      Values are true or false.

    • _data_stream_timestamp object
      Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
      • enabled boolean Required
  • settings_override object

    Settings overrides to be applied to the destination index (optional)

    Index settings
  • remove_index_blocks boolean

    If index blocks should be removed when creating destination index (optional)

    Default value is true.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • acknowledged boolean Required
    • index string Required
    • shards_acknowledged boolean Required
POST /_create_from/{source}/{dest}
POST _create_from/my-index/my-new-index
resp = client.perform_request(
    "POST",
    "/_create_from/my-index/my-new-index",
)
const response = await client.transport.request({
  method: "POST",
  path: "/_create_from/my-index/my-new-index",
});
response = client.perform_request(
  "POST",
  "/_create_from/my-index/my-new-index",
  {},
)
$requestFactory = Psr17FactoryDiscovery::findRequestFactory();
$request = $requestFactory->createRequest(
    "POST",
    "/_create_from/my-index/my-new-index",
);
$resp = $client->sendRequest($request);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_create_from/my-index/my-new-index"
client.indices().createFrom(c -> c
    .dest("my-new-index")
    .source("my-index")
    .createFrom(cr -> cr)
);
















Start the feature migration Generally available; Added in 7.16.0

POST /_migration/system_features

Version upgrades sometimes require changes to how features store configuration information and data in system indices. This API starts the automatic migration process.

Some functionality might be temporarily unavailable during the migration process.

TIP: The API is designed for indirect use by the Upgrade Assistant. We strongly recommend you use the Upgrade Assistant.

Required authorization

  • Index privileges: manage
  • Cluster privileges: manage

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • accepted boolean Required
    • features array[object]
      Hide features attribute Show features attribute object
      • feature_name string Required
    • reason string
POST /_migration/system_features
POST /_migration/system_features
resp = client.migration.post_feature_upgrade()
const response = await client.migration.postFeatureUpgrade();
response = client.migration.post_feature_upgrade
$resp = $client->migration()->postFeatureUpgrade();
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_migration/system_features"
client.migration().postFeatureUpgrade();
Response examples (200)
When you run `POST /_migration/system_features` to start the migration process, the response lists the features that will be migrated.
{
  "accepted" : true,
  "features" : [
    {
      "feature_name" : "security"
    }
  ]
}














































Get the rollup job capabilities Deprecated Technical preview; Added in 6.3.0

GET /_rollup/data/{id}

All methods and paths for this operation:

GET /_rollup/data

GET /_rollup/data/{id}

Get the capabilities of any rollup jobs that have been configured for a specific index or index pattern.

This API is useful because a rollup job is often configured to rollup only a subset of fields from the source index. Furthermore, only certain aggregations can be configured for various fields, leading to a limited subset of functionality depending on that configuration. This API enables you to inspect an index and determine:

  1. Does this index have associated rollup data somewhere in the cluster?
  2. If yes to the first question, what fields were rolled up, what aggregations can be performed, and where does the data live?

Required authorization

  • Cluster privileges: monitor_rollup

Path parameters

  • id string Required

    Index, indices or index-pattern to return rollup capabilities for. _all may be used to fetch rollup capabilities from all jobs.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attribute Show * attribute object
      • rollup_jobs array[object] Required

        There can be multiple, independent jobs configured for a single index or index pattern. Each of these jobs may have different configurations, so the API returns a list of all the various configurations available.

        Hide rollup_jobs attributes Show rollup_jobs attributes object
        • fields object Required
          Hide fields attribute Show fields attribute object
          • * array[object] Additional properties
        • index_pattern string Required
        • job_id string Required
        • rollup_index string Required
GET /_rollup/data/{id}
GET _rollup/data/sensor-*
resp = client.rollup.get_rollup_caps(
    id="sensor-*",
)
const response = await client.rollup.getRollupCaps({
  id: "sensor-*",
});
response = client.rollup.get_rollup_caps(
  id: "sensor-*"
)
$resp = $client->rollup()->getRollupCaps([
    "id" => "sensor-*",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_rollup/data/sensor-*"
client.rollup().getRollupCaps(g -> g
    .id("sensor-*")
);
Response examples (200)
A successful response from `GET _rollup/data/sensor-*` for a rollup job that targets the index pattern `sensor-*`. The response contains the rollup job ID, the index that holds the rolled data, and the index pattern that the job was targeting. It also shows a list of fields that contain data eligible for rollup searches. For example, you can use a `min`, `max`, or `sum` aggregation on the `temperature` field, but only a `date_histogram` on `timestamp`.
{
  "sensor-*" : {
    "rollup_jobs" : [
      {
        "job_id" : "sensor",
        "rollup_index" : "sensor_rollup",
        "index_pattern" : "sensor-*",
        "fields" : {
          "node" : [
            {
              "agg" : "terms"
            }
          ],
          "temperature" : [
            {
              "agg" : "min"
            },
            {
              "agg" : "max"
            },
            {
              "agg" : "sum"
            }
          ],
          "timestamp" : [
            {
              "agg" : "date_histogram",
              "time_zone" : "UTC",
              "fixed_interval" : "1h",
              "delay": "7d"
            }
          ],
          "voltage" : [
            {
              "agg" : "avg"
            }
          ]
        }
      }
    ]
  }
}





















Delete a script or search template Generally available

DELETE /_scripts/{id}

Deletes a stored script or search template.

Required authorization

  • Cluster privileges: manage

Path parameters

  • id string Required

    The identifier for the stored script or search template.

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

    Values are -1 or 0.

  • timeout string

    The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

DELETE /_scripts/{id}
DELETE _scripts/my-search-template
resp = client.delete_script(
    id="my-search-template",
)
const response = await client.deleteScript({
  id: "my-search-template",
});
response = client.delete_script(
  id: "my-search-template"
)
$resp = $client->deleteScript([
    "id" => "my-search-template",
]);
curl -X DELETE -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_scripts/my-search-template"
client.deleteScript(d -> d
    .id("my-search-template")
);

Get script contexts Generally available

GET /_script_context

Get a list of supported script contexts and their methods.

Required authorization

  • Cluster privileges: manage

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • contexts array[object] Required
      Hide contexts attributes Show contexts attributes object
      • methods array[object] Required
        Hide methods attributes Show methods attributes object
        • name string Required
        • return_type string Required
        • params array[object] Required
      • name string Required
GET /_script_context
GET _script_context
resp = client.get_script_context()
const response = await client.getScriptContext();
response = client.get_script_context
$resp = $client->getScriptContext();
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_script_context"
client.getScriptContext();












Search

























Close a point in time Generally available; Added in 7.10.0

DELETE /_pit

A point in time must be opened explicitly before being used in search requests. The keep_alive parameter tells Elasticsearch how long it should persist. A point in time is automatically closed when the keep_alive period has elapsed. However, keeping points in time has a cost; close them as soon as they are no longer required for search requests.

application/json

Body

  • id string Required

    The ID of the point-in-time.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • succeeded boolean Required

      If true, all search contexts associated with the point-in-time ID were successfully closed.

    • num_freed number Required

      The number of search contexts that were successfully closed.

DELETE /_pit
DELETE /_pit
{
  "id": "46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA=="
}
resp = client.close_point_in_time(
    id="46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA==",
)
const response = await client.closePointInTime({
  id: "46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA==",
});
response = client.close_point_in_time(
  body: {
    "id": "46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA=="
  }
)
$resp = $client->closePointInTime([
    "body" => [
        "id" => "46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA==",
    ],
]);
curl -X DELETE -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"id":"46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA=="}' "$ELASTICSEARCH_URL/_pit"
client.closePointInTime(c -> c
    .id("46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA==")
);
Request example
Run `DELETE /_pit` to close a point-in-time.
{
  "id": "46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA=="
}
Response examples (200)
A successful response from `DELETE /_pit`.
{
  "succeeded": true, 
  "num_freed": 3     
}

















































































Searchable snapshots






































































Invalidate API keys Generally available; Added in 6.7.0

DELETE /_security/api_key

This API invalidates API keys created by the create API key or grant API key APIs. Invalidated API keys fail authentication, but they can still be viewed using the get API key information and query API key information APIs, for at least the configured retention period, until they are automatically deleted.

To use this API, you must have at least the manage_security, manage_api_key, or manage_own_api_key cluster privileges. The manage_security privilege allows deleting any API key, including both REST and cross cluster API keys. The manage_api_key privilege allows deleting any REST API key, but not cross cluster API keys. The manage_own_api_key only allows deleting REST API keys that are owned by the user. In addition, with the manage_own_api_key privilege, an invalidation request must be issued in one of the three formats:

  • Set the parameter owner=true.
  • Or, set both username and realm_name to match the user's identity.
  • Or, if the request is issued by an API key, that is to say an API key invalidates itself, specify its ID in the ids field.

Required authorization

  • Cluster privileges: manage_api_key,manage_own_api_key
application/json

Body Required

  • id string
  • ids array[string]

    A list of API key ids. This parameter cannot be used with any of name, realm_name, or username.

  • name string

    An API key name. This parameter cannot be used with any of ids, realm_name or username.

  • owner boolean

    Query API keys owned by the currently authenticated user. The realm_name or username parameters cannot be specified when this parameter is set to true as they are assumed to be the currently authenticated ones.

    NOTE: At least one of ids, name, username, and realm_name must be specified if owner is false.

    Default value is false.

  • realm_name string

    The name of an authentication realm. This parameter cannot be used with either ids or name, or when owner flag is set to true.

  • username string

    The username of a user. This parameter cannot be used with either ids or name or when owner flag is set to true.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • error_count number Required

      The number of errors that were encountered when invalidating the API keys.

    • error_details array[object]

      Details about the errors. This field is not present in the response when error_count is 0.

      Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      Hide error_details attributes Show error_details attributes object
      • type string Required

        The type of error

      • reason string | null

        A human-readable explanation of the error, in English.

      • stack_trace string

        The server stack trace. Present only if the error_trace=true parameter was sent with the request.

      • caused_by object

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • root_cause array[object]

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • suppressed array[object]

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

    • invalidated_api_keys array[string] Required

      The IDs of the API keys that were invalidated as part of this request.

    • previously_invalidated_api_keys array[string] Required

      The IDs of the API keys that were already invalidated.

DELETE /_security/api_key
DELETE /_security/api_key
{
  "ids" : [ "VuaCfGcBCdbkQm-e5aOx" ]
}
resp = client.security.invalidate_api_key(
    ids=[
        "VuaCfGcBCdbkQm-e5aOx"
    ],
)
const response = await client.security.invalidateApiKey({
  ids: ["VuaCfGcBCdbkQm-e5aOx"],
});
response = client.security.invalidate_api_key(
  body: {
    "ids": [
      "VuaCfGcBCdbkQm-e5aOx"
    ]
  }
)
$resp = $client->security()->invalidateApiKey([
    "body" => [
        "ids" => array(
            "VuaCfGcBCdbkQm-e5aOx",
        ),
    ],
]);
curl -X DELETE -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"ids":["VuaCfGcBCdbkQm-e5aOx"]}' "$ELASTICSEARCH_URL/_security/api_key"
client.security().invalidateApiKey(i -> i
    .ids("VuaCfGcBCdbkQm-e5aOx")
);
Request examples
Run `DELETE /_security/api_key` to invalidate the API keys identified by ID.
{
  "ids" : [ "VuaCfGcBCdbkQm-e5aOx" ]
}
Run `DELETE /_security/api_key` to invalidate the API keys identified by name.
{
  "name" : "my-api-key"
}
Run `DELETE /_security/api_key` to invalidate all API keys for the `native1` realm.
{
  "realm_name" : "native1"
}
Run `DELETE /_security/api_key` to invalidate all API keys for the user `myuser` in all realms.
{
  "username" : "myuser"
}
Run `DELETE /_security/api_key` to invalidate the API keys identified by ID if they are owned by the currently authenticated user.
{
  "ids" : ["VuaCfGcBCdbkQm-e5aOx"],
  "owner" : "true"
}
Run `DELETE /_security/api_key` to invalidate all API keys for the user `myuser` in the `native1` realm .
{
  "username" : "myuser",
  "realm_name" : "native1"
}
Response examples (200)
A successful response from `DELETE /_security/api_key`.
{
  "invalidated_api_keys": [ 
    "api-key-id-1"
  ],
  "previously_invalidated_api_keys": [ 
    "api-key-id-2",
    "api-key-id-3"
  ],
  "error_count": 2, 
  "error_details": [ 
    {
      "type": "exception",
      "reason": "error occurred while invalidating api keys",
      "caused_by": {
        "type": "illegal_argument_exception",
        "reason": "invalid api key id"
      }
    },
    {
      "type": "exception",
      "reason": "error occurred while invalidating api keys",
      "caused_by": {
        "type": "illegal_argument_exception",
        "reason": "invalid api key id"
      }
    }
  ]
}




Create a service account token Generally available

POST /_security/service/{namespace}/{service}/credential/token/{name}

All methods and paths for this operation:

POST /_security/service/{namespace}/{service}/credential/token

PUT /_security/service/{namespace}/{service}/credential/token/{name}
POST /_security/service/{namespace}/{service}/credential/token/{name}

Create a service accounts token for access without requiring basic authentication.

NOTE: Service account tokens never expire. You must actively delete them if they are no longer needed.

Required authorization

  • Cluster privileges: manage_service_account
External documentation

Path parameters

  • namespace string Required

    The name of the namespace, which is a top-level grouping of service accounts.

  • service string Required

    The name of the service.

  • name string Required

    The name for the service account token. If omitted, a random name will be generated.

    Token names must be at least one and no more than 256 characters. They can contain alphanumeric characters (a-z, A-Z, 0-9), dashes (-), and underscores (_), but cannot begin with an underscore.

    NOTE: Token names must be unique in the context of the associated service account. They must also be globally unique with their fully qualified names, which are comprised of the service account principal and token name, such as <namespace>/<service>/<token-name>.

Query parameters

  • refresh string

    If true then refresh the affected shards to make this operation visible to search, if wait_for (the default) then wait for a refresh to make this operation visible to search, if false then do nothing with refreshes.

    Values are true, false, or wait_for.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • created boolean Required
    • token object Required
      Hide token attributes Show token attributes object
      • name string Required
      • value string Required
POST /_security/service/{namespace}/{service}/credential/token/{name}
POST /_security/service/elastic/fleet-server/credential/token/token1
resp = client.security.create_service_token(
    namespace="elastic",
    service="fleet-server",
    name="token1",
)
const response = await client.security.createServiceToken({
  namespace: "elastic",
  service: "fleet-server",
  name: "token1",
});
response = client.security.create_service_token(
  namespace: "elastic",
  service: "fleet-server",
  name: "token1"
)
$resp = $client->security()->createServiceToken([
    "namespace" => "elastic",
    "service" => "fleet-server",
    "name" => "token1",
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_security/service/elastic/fleet-server/credential/token/token1"
client.security().createServiceToken(c -> c
    .name("token1")
    .namespace("elastic")
    .service("fleet-server")
);
Response examples (200)
A successful response from `POST /_security/service/elastic/fleet-server/credential/token/token1`. The response includes the service account token, its name, and its secret value as a bearer token.
{
  "created": true,
  "token": {
    "name": "token1",
    "value": "AAEAAWVsYXN0aWM...vZmxlZXQtc2VydmVyL3Rva2VuMTo3TFdaSDZ" 
  }
}












Delete application privileges Generally available; Added in 6.4.0

DELETE /_security/privilege/{application}/{name}

To use this API, you must have one of the following privileges:

  • The manage_security cluster privilege (or a greater privilege such as all).
  • The "Manage Application Privileges" global privilege for the application being referenced in the request.

Required authorization

  • Cluster privileges: manage_security
External documentation

Path parameters

  • application string Required

    The name of the application. Application privileges are always associated with exactly one application.

  • name string | array[string] Required

    The name of the privilege.

Query parameters

  • refresh string

    If true (the default) then refresh the affected shards to make this operation visible to search, if wait_for then wait for a refresh to make this operation visible to search, if false then do nothing with refreshes.

    Values are true, false, or wait_for.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attribute Show * attribute object
      • * object Additional properties
        Hide * attribute Show * attribute object
        • found boolean Required
DELETE /_security/privilege/{application}/{name}
DELETE /_security/privilege/myapp/read
resp = client.security.delete_privileges(
    application="myapp",
    name="read",
)
const response = await client.security.deletePrivileges({
  application: "myapp",
  name: "read",
});
response = client.security.delete_privileges(
  application: "myapp",
  name: "read"
)
$resp = $client->security()->deletePrivileges([
    "application" => "myapp",
    "name" => "read",
]);
curl -X DELETE -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_security/privilege/myapp/read"
client.security().deletePrivileges(d -> d
    .application("myapp")
    .name("read")
);
Response examples (200)
A successful response from `DELETE /_security/privilege/myapp/read`. If the privilege is successfully deleted, `found` is set to `true`.
{
  "myapp": {
    "read": {
      "found" : true
    }
  }
}




Create or update roles Generally available

POST /_security/role/{name}

All methods and paths for this operation:

PUT /_security/role/{name}

POST /_security/role/{name}

The role management APIs are generally the preferred way to manage roles in the native realm, rather than using file-based role management. The create or update roles API cannot update roles that are defined in roles files. File-based role management is not available in Elastic Serverless.

Required authorization

  • Cluster privileges: manage_security
External documentation

Path parameters

  • name string Required

    The name of the role.

Query parameters

  • refresh string

    If true (the default) then refresh the affected shards to make this operation visible to search, if wait_for then wait for a refresh to make this operation visible to search, if false then do nothing with refreshes.

    Values are true, false, or wait_for.

application/json

Body Required

  • applications array[object]

    A list of application privilege entries.

    Hide applications attributes Show applications attributes object
    • application string Required

      The name of the application to which this entry applies.

    • privileges array[string] Required

      A list of strings, where each element is the name of an application privilege or action.

    • resources array[string] Required

      A list resources to which the privileges are applied.

  • cluster array[string]

    A list of cluster privileges. These privileges define the cluster-level actions for users with this role.

  • global object Generally available

    An object defining global privileges. A global privilege is a form of cluster privilege that is request-aware. Support for global privileges is currently limited to the management of application privileges.

    Hide global attribute Show global attribute object
    • * object Additional properties
  • indices array[object]

    A list of indices permissions entries.

    Hide indices attributes Show indices attributes object
    • field_security object

      The document fields that the owners of the role have read access to.

      Hide field_security attributes Show field_security attributes object
      • except string | array[string]
      • grant string | array[string]
    • names array[string] Required

      A list of indices (or index name patterns) to which the permissions in this entry apply.

    • privileges array[string] Required

      The index level privileges that owners of the role have on the specified indices.

    • query string | object

      A search query that defines the documents the owners of the role have access to. A document within the specified indices must match this query for it to be accessible by the owners of the role.

      One of:

      A search query that defines the documents the owners of the role have access to. A document within the specified indices must match this query for it to be accessible by the owners of the role.

    • allow_restricted_indices boolean Generally available

      Set to true if using wildcard or regular expressions for patterns that cover restricted indices. Implicitly, restricted indices have limited privileges that can cause pattern tests to fail. If restricted indices are explicitly included in the names list, Elasticsearch checks privileges against these indices regardless of the value set for allow_restricted_indices.

      Default value is false.

  • remote_indices array[object] Generally available; Added in 8.14.0

    A list of remote indices permissions entries.

    NOTE: Remote indices are effective for remote clusters configured with the API key based model. They have no effect for remote clusters configured with the certificate based model.

    The subset of index level privileges that can be defined for remote clusters.

    Hide remote_indices attributes Show remote_indices attributes object
    • clusters string | array[string] Required

      A list of cluster aliases to which the permissions in this entry apply.

    • field_security object

      The document fields that the owners of the role have read access to.

      Hide field_security attributes Show field_security attributes object
      • except string | array[string]
      • grant string | array[string]
    • names array[string] Required

      A list of indices (or index name patterns) to which the permissions in this entry apply.

    • privileges array[string] Required

      The index level privileges that owners of the role have on the specified indices.

    • query string | object

      A search query that defines the documents the owners of the role have access to. A document within the specified indices must match this query for it to be accessible by the owners of the role.

      One of:

      A search query that defines the documents the owners of the role have access to. A document within the specified indices must match this query for it to be accessible by the owners of the role.

    • allow_restricted_indices boolean Generally available

      Set to true if using wildcard or regular expressions for patterns that cover restricted indices. Implicitly, restricted indices have limited privileges that can cause pattern tests to fail. If restricted indices are explicitly included in the names list, Elasticsearch checks privileges against these indices regardless of the value set for allow_restricted_indices.

      Default value is false.

  • remote_cluster array[object] Generally available; Added in 8.15.0

    A list of remote cluster permissions entries.

    The subset of cluster level privileges that can be defined for remote clusters.

    Hide remote_cluster attributes Show remote_cluster attributes object
    • clusters string | array[string] Required

      A list of cluster aliases to which the permissions in this entry apply.

    • privileges array[string] Required

      The cluster level privileges that owners of the role have on the remote cluster.

      Values are monitor_enrich or monitor_stats.

  • metadata object

    Optional metadata. Within the metadata object, keys that begin with an underscore (_) are reserved for system use.

    Hide metadata attribute Show metadata attribute object
    • * object Additional properties
  • run_as array[string]

    A list of users that the owners of this role can impersonate. Note: in Serverless, the run-as feature is disabled. For API compatibility, you can still specify an empty run_as field, but a non-empty list will be rejected.

  • description string

    Optional description of the role descriptor

  • transient_metadata object

    Indicates roles that might be incompatible with the current cluster license, specifically roles with document and field level security. When the cluster license doesn’t allow certain features for a given role, this parameter is updated dynamically to list the incompatible features. If enabled is false, the role is ignored, but is still listed in the response from the authenticate API.

    Hide transient_metadata attribute Show transient_metadata attribute object
    • * object Additional properties

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • role object Required

      When an existing role is updated, created is set to false.

      Hide role attribute Show role attribute object
      • created boolean Required
POST /_security/role/{name}
POST /_security/role/my_admin_role
{
  "description": "Grants full access to all management features within the cluster.",
  "cluster": ["all"],
  "indices": [
    {
      "names": [ "index1", "index2" ],
      "privileges": ["all"],
      "field_security" : { // optional
        "grant" : [ "title", "body" ]
      },
      "query": "{\"match\": {\"title\": \"foo\"}}" // optional
    }
  ],
  "applications": [
    {
      "application": "myapp",
      "privileges": [ "admin", "read" ],
      "resources": [ "*" ]
    }
  ],
  "run_as": [ "other_user" ], // optional
  "metadata" : { // optional
    "version" : 1
  }
}
resp = client.security.put_role(
    name="my_admin_role",
    description="Grants full access to all management features within the cluster.",
    cluster=[
        "all"
    ],
    indices=[
        {
            "names": [
                "index1",
                "index2"
            ],
            "privileges": [
                "all"
            ],
            "field_security": {
                "grant": [
                    "title",
                    "body"
                ]
            },
            "query": "{\"match\": {\"title\": \"foo\"}}"
        }
    ],
    applications=[
        {
            "application": "myapp",
            "privileges": [
                "admin",
                "read"
            ],
            "resources": [
                "*"
            ]
        }
    ],
    run_as=[
        "other_user"
    ],
    metadata={
        "version": 1
    },
)
const response = await client.security.putRole({
  name: "my_admin_role",
  description:
    "Grants full access to all management features within the cluster.",
  cluster: ["all"],
  indices: [
    {
      names: ["index1", "index2"],
      privileges: ["all"],
      field_security: {
        grant: ["title", "body"],
      },
      query: '{"match": {"title": "foo"}}',
    },
  ],
  applications: [
    {
      application: "myapp",
      privileges: ["admin", "read"],
      resources: ["*"],
    },
  ],
  run_as: ["other_user"],
  metadata: {
    version: 1,
  },
});
response = client.security.put_role(
  name: "my_admin_role",
  body: {
    "description": "Grants full access to all management features within the cluster.",
    "cluster": [
      "all"
    ],
    "indices": [
      {
        "names": [
          "index1",
          "index2"
        ],
        "privileges": [
          "all"
        ],
        "field_security": {
          "grant": [
            "title",
            "body"
          ]
        },
        "query": "{\"match\": {\"title\": \"foo\"}}"
      }
    ],
    "applications": [
      {
        "application": "myapp",
        "privileges": [
          "admin",
          "read"
        ],
        "resources": [
          "*"
        ]
      }
    ],
    "run_as": [
      "other_user"
    ],
    "metadata": {
      "version": 1
    }
  }
)
$resp = $client->security()->putRole([
    "name" => "my_admin_role",
    "body" => [
        "description" => "Grants full access to all management features within the cluster.",
        "cluster" => array(
            "all",
        ),
        "indices" => array(
            [
                "names" => array(
                    "index1",
                    "index2",
                ),
                "privileges" => array(
                    "all",
                ),
                "field_security" => [
                    "grant" => array(
                        "title",
                        "body",
                    ),
                ],
                "query" => "{\"match\": {\"title\": \"foo\"}}",
            ],
        ),
        "applications" => array(
            [
                "application" => "myapp",
                "privileges" => array(
                    "admin",
                    "read",
                ),
                "resources" => array(
                    "*",
                ),
            ],
        ),
        "run_as" => array(
            "other_user",
        ),
        "metadata" => [
            "version" => 1,
        ],
    ],
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"description":"Grants full access to all management features within the cluster.","cluster":["all"],"indices":[{"names":["index1","index2"],"privileges":["all"],"field_security":{"grant":["title","body"]},"query":"{\"match\": {\"title\": \"foo\"}}"}],"applications":[{"application":"myapp","privileges":["admin","read"],"resources":["*"]}],"run_as":["other_user"],"metadata":{"version":1}}' "$ELASTICSEARCH_URL/_security/role/my_admin_role"
client.security().putRole(p -> p
    .applications(a -> a
        .application("myapp")
        .privileges(List.of("admin","read"))
        .resources("*")
    )
    .cluster("all")
    .description("Grants full access to all management features within the cluster.")
    .indices(i -> i
        .fieldSecurity(f -> f
            .grant(List.of("title","body"))
        )
        .names(List.of("index1","index2"))
        .privileges("all")
        .query(q -> q
            .match(m -> m
                .field("title")
                .query(FieldValue.of("foo"))
            )
        )
    )
    .metadata("version", JsonData.fromJson("1"))
    .name("my_admin_role")
    .runAs("other_user")
);
Request examples
Run `POST /_security/role/my_admin_role` to create a role.
{
  "description": "Grants full access to all management features within the cluster.",
  "cluster": ["all"],
  "indices": [
    {
      "names": [ "index1", "index2" ],
      "privileges": ["all"],
      "field_security" : { // optional
        "grant" : [ "title", "body" ]
      },
      "query": "{\"match\": {\"title\": \"foo\"}}" // optional
    }
  ],
  "applications": [
    {
      "application": "myapp",
      "privileges": [ "admin", "read" ],
      "resources": [ "*" ]
    }
  ],
  "run_as": [ "other_user" ], // optional
  "metadata" : { // optional
    "version" : 1
  }
}
Run `POST /_security/role/cli_or_drivers_minimal` to configure a role that can run SQL in JDBC.
{
  "cluster": ["cluster:monitor/main"],
  "indices": [
    {
      "names": ["test"],
      "privileges": ["read", "indices:admin/get"]
    }
  ]
}
Run `POST /_security/role/only_remote_access_role` to configure a role with remote indices and remote cluster privileges for a remote cluster.
{
  "remote_indices": [
    {
      "clusters": ["my_remote"], 
      "names": ["logs*"], 
      "privileges": ["read", "read_cross_cluster", "view_index_metadata"] 
    }
  ],
  "remote_cluster": [
    {
      "clusters": ["my_remote"], 
      "privileges": ["monitor_stats"]  
    }
  ]
}
Response examples (200)
A successful response from `POST /_security/role/my_admin_role`.
{
  "role": {
    "created": true 
  }
}












Delete role mappings Generally available; Added in 5.5.0

DELETE /_security/role_mapping/{name}

Role mappings define which roles are assigned to each user. The role mapping APIs are generally the preferred way to manage role mappings rather than using role mapping files. The delete role mappings API cannot remove role mappings that are defined in role mapping files.

Required authorization

  • Cluster privileges: manage_security
External documentation

Path parameters

  • name string Required

    The distinct name that identifies the role mapping. The name is used solely as an identifier to facilitate interaction via the API; it does not affect the behavior of the mapping in any way.

Query parameters

  • refresh string

    If true (the default) then refresh the affected shards to make this operation visible to search, if wait_for then wait for a refresh to make this operation visible to search, if false then do nothing with refreshes.

    Values are true, false, or wait_for.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • found boolean Required

      If the mapping is successfully deleted, found is true. Otherwise, found is false.

DELETE /_security/role_mapping/{name}
DELETE /_security/role_mapping/mapping1
resp = client.security.delete_role_mapping(
    name="mapping1",
)
const response = await client.security.deleteRoleMapping({
  name: "mapping1",
});
response = client.security.delete_role_mapping(
  name: "mapping1"
)
$resp = $client->security()->deleteRoleMapping([
    "name" => "mapping1",
]);
curl -X DELETE -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_security/role_mapping/mapping1"
client.security().deleteRoleMapping(d -> d
    .name("mapping1")
);
Response examples (200)
A successful response from `DELETE /_security/role_mapping/mapping1`. If the mapping is successfully deleted, `found` is `true`.
{
  "found" : true
}
























Enable a user profile Generally available; Added in 8.2.0

POST /_security/profile/{uid}/_enable

All methods and paths for this operation:

PUT /_security/profile/{uid}/_enable

POST /_security/profile/{uid}/_enable

Enable user profiles to make them visible in user profile searches.

NOTE: The user profile feature is designed only for use by Kibana and Elastic's Observability, Enterprise Search, and Elastic Security solutions. Individual users and external applications should not call this API directly. Elastic reserves the right to change or remove this feature in future releases without prior notice.

When you activate a user profile, it's automatically enabled and visible in user profile searches. If you later disable the user profile, you can use the enable user profile API to make the profile visible in these searches again.

Required authorization

  • Cluster privileges: manage_user_profile

Path parameters

  • uid string Required

    A unique identifier for the user profile.

Query parameters

  • refresh string

    If 'true', Elasticsearch refreshes the affected shards to make this operation visible to search. If 'wait_for', it waits for a refresh to make this operation visible to search. If 'false', nothing is done with refreshes.

    Values are true, false, or wait_for.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

POST /_security/profile/{uid}/_enable
POST /_security/profile/u_79HkWkwmnBH5gqFKwoxggWPjEBOur1zLPXQPEl1VBW0_0/_enable
resp = client.security.enable_user_profile(
    uid="u_79HkWkwmnBH5gqFKwoxggWPjEBOur1zLPXQPEl1VBW0_0",
)
const response = await client.security.enableUserProfile({
  uid: "u_79HkWkwmnBH5gqFKwoxggWPjEBOur1zLPXQPEl1VBW0_0",
});
response = client.security.enable_user_profile(
  uid: "u_79HkWkwmnBH5gqFKwoxggWPjEBOur1zLPXQPEl1VBW0_0"
)
$resp = $client->security()->enableUserProfile([
    "uid" => "u_79HkWkwmnBH5gqFKwoxggWPjEBOur1zLPXQPEl1VBW0_0",
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_security/profile/u_79HkWkwmnBH5gqFKwoxggWPjEBOur1zLPXQPEl1VBW0_0/_enable"
client.security().enableUserProfile(e -> e
    .uid("u_79HkWkwmnBH5gqFKwoxggWPjEBOur1zLPXQPEl1VBW0_0")
);

Enroll Kibana Generally available; Added in 8.0.0

GET /_security/enroll/kibana

Enable a Kibana instance to configure itself for communication with a secured Elasticsearch cluster.

NOTE: This API is currently intended for internal use only by Kibana. Kibana uses this API internally to configure itself for communications with an Elasticsearch cluster that already has security features enabled.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • token object Required
      Hide token attributes Show token attributes object
      • name string Required

        The name of the bearer token for the elastic/kibana service account.

      • value string Required

        The value of the bearer token for the elastic/kibana service account. Use this value to authenticate the service account with Elasticsearch.

    • http_ca string Required

      The CA certificate used to sign the node certificates that Elasticsearch uses for TLS on the HTTP layer. The certificate is returned as a Base64 encoded string of the ASN.1 DER encoding of the certificate.

GET /_security/enroll/kibana
GET /_security/enroll/kibana
resp = client.security.enroll_kibana()
const response = await client.security.enrollKibana();
response = client.security.enroll_kibana
$resp = $client->security()->enrollKibana();
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_security/enroll/kibana"
client.security().enrollKibana();
Response examples (200)
A successful response from `GET /_security/enroll/kibana`.
{
  "token" : {
    "name" : "enroll-process-token-1629123923000", 
    "value": "AAEAAWVsYXN0aWM...vZmxlZXQtc2VydmVyL3Rva2VuMTo3TFdaSDZ" 
  },
  "http_ca" : "MIIJlAIBAzVoGCSqGSIb3...vsDfsA3UZBAjEPfhubpQysAICAA=", 
}




















































Authenticate OpenID Connect Generally available

POST /_security/oidc/authenticate

Exchange an OpenID Connect authentication response message for an Elasticsearch internal access token and refresh token that can be subsequently used for authentication.

Elasticsearch exposes all the necessary OpenID Connect related functionality with the OpenID Connect APIs. These APIs are used internally by Kibana in order to provide OpenID Connect based authentication, but can also be used by other, custom web applications or other clients.

application/json

Body Required

  • nonce string Required

    Associate a client session with an ID token and mitigate replay attacks. This value needs to be the same as the one that was provided to the /_security/oidc/prepare API or the one that was generated by Elasticsearch and included in the response to that call.

  • realm string

    The name of the OpenID Connect realm. This property is useful in cases where multiple realms are defined.

  • redirect_uri string Required

    The URL to which the OpenID Connect Provider redirected the User Agent in response to an authentication request after a successful authentication. This URL must be provided as-is (URL encoded), taken from the body of the response or as the value of a location header in the response from the OpenID Connect Provider.

  • state string Required

    Maintain state between the authentication request and the response. This value needs to be the same as the one that was provided to the /_security/oidc/prepare API or the one that was generated by Elasticsearch and included in the response to that call.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • access_token string Required

      The Elasticsearch access token.

    • expires_in number Required

      The duration (in seconds) of the tokens.

    • refresh_token string Required

      The Elasticsearch refresh token.

    • type string Required

      The type of token.

POST /_security/oidc/authenticate
POST /_security/oidc/authenticate
{
  "redirect_uri" : "https://oidc-kibana.elastic.co:5603/api/security/oidc/callback?code=jtI3Ntt8v3_XvcLzCFGq&state=4dbrihtIAt3wBTwo6DxK-vdk-sSyDBV8Yf0AjdkdT5I",
  "state" : "4dbrihtIAt3wBTwo6DxK-vdk-sSyDBV8Yf0AjdkdT5I",
  "nonce" : "WaBPH0KqPVdG5HHdSxPRjfoZbXMCicm5v1OiAj0DUFM",
  "realm" : "oidc1"
}
resp = client.security.oidc_authenticate(
    redirect_uri="https://oidc-kibana.elastic.co:5603/api/security/oidc/callback?code=jtI3Ntt8v3_XvcLzCFGq&state=4dbrihtIAt3wBTwo6DxK-vdk-sSyDBV8Yf0AjdkdT5I",
    state="4dbrihtIAt3wBTwo6DxK-vdk-sSyDBV8Yf0AjdkdT5I",
    nonce="WaBPH0KqPVdG5HHdSxPRjfoZbXMCicm5v1OiAj0DUFM",
    realm="oidc1",
)
const response = await client.security.oidcAuthenticate({
  redirect_uri:
    "https://oidc-kibana.elastic.co:5603/api/security/oidc/callback?code=jtI3Ntt8v3_XvcLzCFGq&state=4dbrihtIAt3wBTwo6DxK-vdk-sSyDBV8Yf0AjdkdT5I",
  state: "4dbrihtIAt3wBTwo6DxK-vdk-sSyDBV8Yf0AjdkdT5I",
  nonce: "WaBPH0KqPVdG5HHdSxPRjfoZbXMCicm5v1OiAj0DUFM",
  realm: "oidc1",
});
response = client.security.oidc_authenticate(
  body: {
    "redirect_uri": "https://oidc-kibana.elastic.co:5603/api/security/oidc/callback?code=jtI3Ntt8v3_XvcLzCFGq&state=4dbrihtIAt3wBTwo6DxK-vdk-sSyDBV8Yf0AjdkdT5I",
    "state": "4dbrihtIAt3wBTwo6DxK-vdk-sSyDBV8Yf0AjdkdT5I",
    "nonce": "WaBPH0KqPVdG5HHdSxPRjfoZbXMCicm5v1OiAj0DUFM",
    "realm": "oidc1"
  }
)
$resp = $client->security()->oidcAuthenticate([
    "body" => [
        "redirect_uri" => "https://oidc-kibana.elastic.co:5603/api/security/oidc/callback?code=jtI3Ntt8v3_XvcLzCFGq&state=4dbrihtIAt3wBTwo6DxK-vdk-sSyDBV8Yf0AjdkdT5I",
        "state" => "4dbrihtIAt3wBTwo6DxK-vdk-sSyDBV8Yf0AjdkdT5I",
        "nonce" => "WaBPH0KqPVdG5HHdSxPRjfoZbXMCicm5v1OiAj0DUFM",
        "realm" => "oidc1",
    ],
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"redirect_uri":"https://oidc-kibana.elastic.co:5603/api/security/oidc/callback?code=jtI3Ntt8v3_XvcLzCFGq&state=4dbrihtIAt3wBTwo6DxK-vdk-sSyDBV8Yf0AjdkdT5I","state":"4dbrihtIAt3wBTwo6DxK-vdk-sSyDBV8Yf0AjdkdT5I","nonce":"WaBPH0KqPVdG5HHdSxPRjfoZbXMCicm5v1OiAj0DUFM","realm":"oidc1"}' "$ELASTICSEARCH_URL/_security/oidc/authenticate"
client.security().oidcAuthenticate(o -> o
    .nonce("WaBPH0KqPVdG5HHdSxPRjfoZbXMCicm5v1OiAj0DUFM")
    .realm("oidc1")
    .redirectUri("https://oidc-kibana.elastic.co:5603/api/security/oidc/callback?code=jtI3Ntt8v3_XvcLzCFGq&state=4dbrihtIAt3wBTwo6DxK-vdk-sSyDBV8Yf0AjdkdT5I")
    .state("4dbrihtIAt3wBTwo6DxK-vdk-sSyDBV8Yf0AjdkdT5I")
);
Request example
Run `POST /_security/oidc/authenticate` to exchange the response that was returned from the OpenID Connect Provider after a successful authentication for an Elasticsearch access token and refresh token. This example is from an authentication that uses the authorization code grant flow.
{
  "redirect_uri" : "https://oidc-kibana.elastic.co:5603/api/security/oidc/callback?code=jtI3Ntt8v3_XvcLzCFGq&state=4dbrihtIAt3wBTwo6DxK-vdk-sSyDBV8Yf0AjdkdT5I",
  "state" : "4dbrihtIAt3wBTwo6DxK-vdk-sSyDBV8Yf0AjdkdT5I",
  "nonce" : "WaBPH0KqPVdG5HHdSxPRjfoZbXMCicm5v1OiAj0DUFM",
  "realm" : "oidc1"
}
Response examples (200)
A successful response from `POST /_security/oidc/authenticate`. It contains the access and refresh tokens that were generated, the token duration (in seconds), and the type.
{
  "access_token" : "dGhpcyBpcyBub3QgYSByZWFsIHRva2VuIGJ1dCBpdCBpcyBvbmx5IHRlc3QgZGF0YS4gZG8gbm90IHRyeSB0byByZWFkIHRva2VuIQ==",
  "type" : "Bearer",
  "expires_in" : 1200,
  "refresh_token": "vLBPvmAB6KvwvJZr27cS"
}

























































































Get snapshot repository information Generally available; Added in 0.0.0

GET /_snapshot/{repository}

All methods and paths for this operation:

GET /_snapshot

GET /_snapshot/{repository}

Required authorization

  • Cluster privileges: monitor_snapshot

Path parameters

  • repository string | array[string] Required

    A comma-separated list of snapshot repository names used to limit the request. Wildcard (*) expressions are supported including combining wildcards with exclude patterns starting with -.

    To get information about all snapshot repositories registered in the cluster, omit this parameter or use * or _all.

Query parameters

  • local boolean

    If true, the request gets information from the local node only. If false, the request gets information from the master node.

  • master_timeout string

    The period to wait for the master node. If the master node is not available before the timeout expires, the request fails and returns an error. To indicate that the request should never timeout, set it to -1.

    Values are -1 or 0.

Responses

  • 200 application/json
GET /_snapshot/{repository}
GET /_snapshot/my_repository
resp = client.snapshot.get_repository(
    name="my_repository",
)
const response = await client.snapshot.getRepository({
  name: "my_repository",
});
response = client.snapshot.get_repository(
  repository: "my_repository"
)
$resp = $client->snapshot()->getRepository([
    "repository" => "my_repository",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_snapshot/my_repository"
client.snapshot().getRepository(g -> g
    .name("my_repository")
);
Response examples (200)
A successful response from `GET /_snapshot/my_repository`.
{
  "my_repository" : {
    "type" : "fs",
    "uuid" : "0JLknrXbSUiVPuLakHjBrQ",
    "settings" : {
      "location" : "my_backup_location"
    }
  }
}




Delete snapshot repositories Generally available; Added in 0.0.0

DELETE /_snapshot/{repository}

When a repository is unregistered, Elasticsearch removes only the reference to the location where the repository is storing the snapshots. The snapshots themselves are left untouched and in place.

Required authorization

  • Cluster privileges: manage

Path parameters

  • repository string | array[string] Required

    The ame of the snapshot repositories to unregister. Wildcard (*) patterns are supported.

Query parameters

  • master_timeout string

    The period to wait for the master node. If the master node is not available before the timeout expires, the request fails and returns an error. To indicate that the request should never timeout, set it to -1.

    Values are -1 or 0.

  • timeout string

    The period to wait for a response from all relevant nodes in the cluster after updating the cluster metadata. If no response is received before the timeout expires, the cluster metadata update still applies but the response will indicate that it was not completely acknowledged. To indicate that the request should never timeout, set it to -1.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

DELETE /_snapshot/{repository}
DELETE /_snapshot/my_repository
resp = client.snapshot.delete_repository(
    name="my_repository",
)
const response = await client.snapshot.deleteRepository({
  name: "my_repository",
});
response = client.snapshot.delete_repository(
  repository: "my_repository"
)
$resp = $client->snapshot()->deleteRepository([
    "repository" => "my_repository",
]);
curl -X DELETE -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_snapshot/my_repository"
client.snapshot().deleteRepository(d -> d
    .name("my_repository")
);

























Delete a policy Generally available; Added in 7.4.0

DELETE /_slm/policy/{policy_id}

Delete a snapshot lifecycle policy definition. This operation prevents any future snapshots from being taken but does not cancel in-progress snapshots or remove previously-taken snapshots.

Required authorization

  • Cluster privileges: manage_slm

Path parameters

  • policy_id string Required

    The id of the snapshot lifecycle policy to remove

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

DELETE /_slm/policy/{policy_id}
DELETE /_slm/policy/daily-snapshots
resp = client.slm.delete_lifecycle(
    policy_id="daily-snapshots",
)
const response = await client.slm.deleteLifecycle({
  policy_id: "daily-snapshots",
});
response = client.slm.delete_lifecycle(
  policy_id: "daily-snapshots"
)
$resp = $client->slm()->deleteLifecycle([
    "policy_id" => "daily-snapshots",
]);
curl -X DELETE -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_slm/policy/daily-snapshots"
client.slm().deleteLifecycle(d -> d
    .policyId("daily-snapshots")
);






























































Get a synonym rule Generally available; Added in 8.10.0

GET /_synonyms/{set_id}/{rule_id}

Get a synonym rule from a synonym set.

Required authorization

  • Cluster privileges: manage_search_synonyms

Path parameters

  • set_id string Required

    The ID of the synonym set to retrieve the synonym rule from.

  • rule_id string Required

    The ID of the synonym rule to retrieve.

Responses

GET /_synonyms/{set_id}/{rule_id}
GET _synonyms/my-synonyms-set/test-1
resp = client.synonyms.get_synonym_rule(
    set_id="my-synonyms-set",
    rule_id="test-1",
)
const response = await client.synonyms.getSynonymRule({
  set_id: "my-synonyms-set",
  rule_id: "test-1",
});
response = client.synonyms.get_synonym_rule(
  set_id: "my-synonyms-set",
  rule_id: "test-1"
)
$resp = $client->synonyms()->getSynonymRule([
    "set_id" => "my-synonyms-set",
    "rule_id" => "test-1",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_synonyms/my-synonyms-set/test-1"
client.synonyms().getSynonymRule(g -> g
    .ruleId("test-1")
    .setId("my-synonyms-set")
);
Response examples (200)
A successful response from `GET _synonyms/my-synonyms-set/test-1`.
{
  "id": "test-1",
  "synonyms": "hello, hi"
}




Delete a synonym rule Generally available; Added in 8.10.0

DELETE /_synonyms/{set_id}/{rule_id}

Delete a synonym rule from a synonym set.

Required authorization

  • Cluster privileges: manage_search_synonyms

Path parameters

  • set_id string Required

    The ID of the synonym set to update.

  • rule_id string Required

    The ID of the synonym rule to delete.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • result string Required

      The update operation result.

      Values are created, updated, deleted, not_found, or noop.

    • reload_analyzers_details object Required

      Updating synonyms in a synonym set reloads the associated analyzers. This information is the analyzers reloading result.

      Hide reload_analyzers_details attributes Show reload_analyzers_details attributes object
      • reload_details array[object] Required
        Hide reload_details attributes Show reload_details attributes object
        • index string Required
        • reloaded_analyzers array[string] Required
        • reloaded_node_ids array[string] Required
      • _shards object Required
        Hide _shards attributes Show _shards attributes object
        • failed number Required

          The number of shards the operation or search attempted to run on but failed.

        • successful number Required

          The number of shards the operation or search succeeded on.

        • total number Required

          The number of shards the operation or search will run on overall.

        • failures array[object]
        • skipped number
DELETE /_synonyms/{set_id}/{rule_id}
DELETE _synonyms/my-synonyms-set/test-1
resp = client.synonyms.delete_synonym_rule(
    set_id="my-synonyms-set",
    rule_id="test-1",
)
const response = await client.synonyms.deleteSynonymRule({
  set_id: "my-synonyms-set",
  rule_id: "test-1",
});
response = client.synonyms.delete_synonym_rule(
  set_id: "my-synonyms-set",
  rule_id: "test-1"
)
$resp = $client->synonyms()->deleteSynonymRule([
    "set_id" => "my-synonyms-set",
    "rule_id" => "test-1",
]);
curl -X DELETE -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_synonyms/my-synonyms-set/test-1"
client.synonyms().deleteSynonymRule(d -> d
    .ruleId("test-1")
    .setId("my-synonyms-set")
);
Response examples (200)
A successful response from `DELETE _synonyms/my-synonyms-set/test-1`. All analyzers using this synonyms set will be reloaded automatically to reflect the rule being deleted.
{
  "result": "deleted",
  "reload_analyzers_details": {
    "_shards": {
      "total": 2,
      "successful": 1,
      "failed": 0
    },
    "reload_details": [
      {
        "index": "test-index",
        "reloaded_analyzers": [
          "my_search_analyzer"
        ],
        "reloaded_node_ids": [
          "1wYFZzq8Sxeu_Jvt9mlbkg"
        ]
      }
    ]
  }
}











































Delete a transform Generally available; Added in 7.5.0

DELETE /_transform/{transform_id}

Required authorization

  • Cluster privileges: manage_transform

Path parameters

  • transform_id string Required

    Identifier for the transform.

Query parameters

  • force boolean

    If this value is false, the transform must be stopped before it can be deleted. If true, the transform is deleted regardless of its current state.

  • delete_dest_index boolean

    If this value is true, the destination index is deleted together with the transform. If false, the destination index will not be deleted

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

DELETE /_transform/{transform_id}
DELETE _transform/ecommerce_transform
resp = client.transform.delete_transform(
    transform_id="ecommerce_transform",
)
const response = await client.transform.deleteTransform({
  transform_id: "ecommerce_transform",
});
response = client.transform.delete_transform(
  transform_id: "ecommerce_transform"
)
$resp = $client->transform()->deleteTransform([
    "transform_id" => "ecommerce_transform",
]);
curl -X DELETE -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_transform/ecommerce_transform"
client.transform().deleteTransform(d -> d
    .transformId("ecommerce_transform")
);
Response examples (200)
A successful response when the transform is deleted.
{
  "acknowledged": true
}