Get shard recovery information

GET /_cat/recovery/{index}
Api key auth Basic auth Bearer auth

Get information about ongoing and completed shard recoveries. Shard recovery is the process of initializing a shard copy, such as restoring a primary shard from a snapshot or syncing a replica shard from a primary shard. When a shard recovery completes, the recovered shard is available for search and indexing. For data streams, the API returns information about the stream’s backing indices. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the index recovery API.

Path parameters

  • index string | array[string] Required

    A comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.

Query parameters

  • active_only boolean

    If true, the response only includes ongoing shard recoveries.

  • bytes string

    The unit used to display byte values.

    Values are b, kb, mb, gb, tb, or pb.

  • detailed boolean

    If true, the response includes detailed information about shard recoveries.

  • index string | array[string]

    Comma-separated list or wildcard expression of index names to limit the returned information

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

  • time string

    Unit used to display time values.

    Values are nanos, micros, ms, s, m, h, or d.

Responses

GET /_cat/recovery/{index} 
curl \
 --request GET 'http://api.example.com/_cat/recovery/{index}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _cat/recovery?v=true&format=json`. In this example, the source and target nodes are the same because the recovery type is `store`, meaning they were read from local storage on node start. 
[
  {
    "index": "my-index-000001 ",
    "shard": "0",
    "time": "13ms",
    "type": "store",
    "stage": "done",
    "source_host": "n/a",
    "source_node": "n/a",
    "target_host": "127.0.0.1",
    "target_node": "node-0",
    "repository": "n/a",
    "snapshot": "n/a",
    "files": "0",
    "files_recovered": "0",
    "files_percent": "100.0%",
    "files_total": "13",
    "bytes": "0b",
    "bytes_recovered": "0b",
    "bytes_percent": "100.0%",
    "bytes_total": "9928b",
    "translog_ops": "0",
    "translog_ops_recovered": "0",
    "translog_ops_percent": "100.0%"
  }
]
A successful response from `GET _cat/recovery?v=true&h=i,s,t,ty,st,shost,thost,f,fp,b,bp&format=json`. You can retrieve information about an ongoing recovery for example when you increase the replica count of an index and bring another node online to host the replicas. In this example, the recovery type is `peer`, meaning the shard recovered from another node. The `files` and `bytes` are real-time measurements. 
[
  {
    "i": "my-index-000001",
    "s": "0",
    "t": "1252ms",
    "ty": "peer",
    "st": "done",
    "shost": "192.168.1.1",
    "thost": "192.168.1.1",
    "f": "0",
    "fp": "100.0%",
    "b": "0b",
    "bp": "100.0%",
  }
]
A successful response from `GET _cat/recovery?v=true&h=i,s,t,ty,st,rep,snap,f,fp,b,bp&format=json`. You can restore backups of an index using the snapshot and restore API. You can use the cat recovery API to get information about a snapshot recovery. 
[
  {
    "i": "my-index-000001",
    "s": "0",
    "t": "1978ms",
    "ty": "snapshot",
    "st": "done",
    "rep": "my-repo",
    "snap": "snap-1",
    "f": "79",
    "fp": "8.0%",
    "b": "12086",
    "bp": "9.0%"
  }
]

Get segment information

GET /_cat/segments
Api key auth Basic auth Bearer auth

Get low-level information about the Lucene segments in index shards. For data streams, the API returns information about the backing indices. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the index segments API.

Query parameters

  • bytes string

    The unit used to display byte values.

    Values are b, kb, mb, gb, tb, or pb.

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

  • local boolean

    If true, the request computes the list of selected nodes from the local cluster state. If false the list of selected nodes are computed from the cluster state of the master node. In both cases the coordinating node will send requests for further information to each selected node.

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • index string
    • shard string

      The shard name.

    • prirep string

      The shard type: primary or replica.

    • ip string

      The IP address of the node where it lives.

    • id string
    • segment string

      The segment name, which is derived from the segment generation and used internally to create file names in the directory of the shard.

    • generation string

      The segment generation number. Elasticsearch increments this generation number for each segment written then uses this number to derive the segment name.

    • docs.count string

      The number of documents in the segment. This excludes deleted documents and counts any nested documents separately from their parents. It also excludes documents which were indexed recently and do not yet belong to a segment.

    • docs.deleted string

      The number of deleted documents in the segment, which might be higher or lower than the number of delete operations you have performed. This number excludes deletes that were performed recently and do not yet belong to a segment. Deleted documents are cleaned up by the automatic merge process if it makes sense to do so. Also, Elasticsearch creates extra deleted documents to internally track the recent history of operations on a shard.

    • size number | string

      One of:
      ByteSize number ByteSize string

    • size.memory number | string

      One of:
      ByteSize number ByteSize string
    • committed string

      If true, the segment is synced to disk. Segments that are synced can survive a hard reboot. If false, the data from uncommitted segments is also stored in the transaction log so that Elasticsearch is able to replay changes on the next start.

    • searchable string

      If true, the segment is searchable. If false, the segment has most likely been written to disk but needs a refresh to be searchable.

    • version string
    • compound string

      If true, the segment is stored in a compound file. This means Lucene merged all files from the segment in a single file to save file descriptors.
GET /_cat/segments 
curl \
 --request GET 'http://api.example.com/_cat/segments' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_cat/segments?v=true&format=json`. 
[
  {
    "index": "test",
    "shard": "0",
    "prirep": "p",
    "ip": "127.0.0.1",
    "segment": "_0",
    "generation": "0",
    "docs.count": "1",
    "docs.deleted": "0",
    "size": "3kb",
    "size.memory": "0",
    "committed": "false",
    "searchable": "true",
    "version": "9.12.0",
    "compound": "true"
  },
  {
    "index": "test1",
    "shard": "0",
    "prirep": "p",
    "ip": "127.0.0.1",
    "segment": "_0",
    "generation": "0",
    "docs.count": "1",
    "docs.deleted": "0",
    "size": "3kb",
    "size.memory": "0",
    "committed": "false",
    "searchable": "true",
    "version": "9.12.0",
    "compound": "true"
  }
]

Get index template information Added in 5.2.0

GET /_cat/templates
Api key auth Basic auth Bearer auth

Get information about the index templates in a cluster. You can use index templates to apply index settings and field mappings to new indices at creation. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the get index template API.

Query parameters

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

  • local boolean

    If true, the request computes the list of selected nodes from the local cluster state. If false the list of selected nodes are computed from the cluster state of the master node. In both cases the coordinating node will send requests for further information to each selected node.

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

Responses

GET /_cat/templates 
curl \
 --request GET 'http://api.example.com/_cat/templates' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _cat/templates/my-template-*?v=true&s=name&format=json`. 
[
  {
    "name": "my-template-0",
    "index_patterns": "[te*]",
    "order": "500",
    "version": null,
    "composed_of": "[]"
  },
  {
    "name": "my-template-1",
    "index_patterns": "[tea*]",
    "order": "501",
    "version": null,
    "composed_of": "[]"
  },
  {
    "name": "my-template-2",
    "index_patterns": "[teak*]",
    "order": "502",
    "version": "7",
    "composed_of": "[]"
  }
]

Get transform information Added in 7.7.0

GET /_cat/transforms
Api key auth Basic auth Bearer auth

Get configuration and usage information about transforms.

CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get transform statistics API.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request: contains wildcard expressions and there are no transforms that match; contains the _all string or no identifiers and there are no matches; contains wildcard expressions and there are only partial matches. If true, it returns an empty transforms array when there are no matches and the subset of results when there are partial matches. If false, the request returns a 404 status code when there are no matches or only partial matches.

  • from number

    Skips the specified number of transforms.

  • h string | array[string]

    Comma-separated list of column names to display.

    Supported values include:

    • changes_last_detection_time (or cldt): The timestamp when changes were last detected in the source indices.
    • checkpoint (or cp): The sequence number for the checkpoint.
    • checkpoint_duration_time_exp_avg (or cdtea, checkpointTimeExpAvg): Exponential moving average of the duration of the checkpoint, in milliseconds.
    • checkpoint_progress (or c, checkpointProgress): The progress of the next checkpoint that is currently in progress.
    • create_time (or ct, createTime): The time the transform was created.
    • delete_time (or dtime): The amount of time spent deleting, in milliseconds.
    • description (or d): The description of the transform.
    • dest_index (or di, destIndex): The destination index for the transform. The mappings of the destination index are deduced based on the source fields when possible. If alternate mappings are required, use the Create index API prior to starting the transform.
    • documents_deleted (or docd): The number of documents that have been deleted from the destination index due to the retention policy for this transform.
    • documents_indexed (or doci): The number of documents that have been indexed into the destination index for the transform.
    • docs_per_second (or dps): Specifies a limit on the number of input documents per second. This setting throttles the transform by adding a wait time between search requests. The default value is null, which disables throttling.
    • documents_processed (or docp): The number of documents that have been processed from the source index of the transform.
    • frequency (or f): The interval between checks for changes in the source indices when the transform is running continuously. Also determines the retry interval in the event of transient failures while the transform is searching or indexing. The minimum value is 1s and the maximum is 1h. The default value is 1m.
    • id: Identifier for the transform.
    • index_failure (or if): The number of indexing failures.
    • index_time (or itime): The amount of time spent indexing, in milliseconds.
    • index_total (or it): The number of index operations.
    • indexed_documents_exp_avg (or idea): Exponential moving average of the number of new documents that have been indexed.
    • last_search_time (or lst, lastSearchTime): The timestamp of the last search in the source indices. This field is only shown if the transform is running.
    • max_page_search_size (or mpsz): Defines the initial page size to use for the composite aggregation for each checkpoint. If circuit breaker exceptions occur, the page size is dynamically adjusted to a lower value. The minimum value is 10 and the maximum is 65,536. The default value is 500.
    • pages_processed (or pp): The number of search or bulk index operations processed. Documents are processed in batches instead of individually.
    • pipeline (or p): The unique identifier for an ingest pipeline.
    • processed_documents_exp_avg (or pdea): Exponential moving average of the number of documents that have been processed.
    • processing_time (or pt): The amount of time spent processing results, in milliseconds.
    • reason (or r): If a transform has a failed state, this property provides details about the reason for the failure.
    • search_failure (or sf): The number of search failures.
    • search_time (or stime): The amount of time spent searching, in milliseconds.
    • search_total (or st): The number of search operations on the source index for the transform.
    • source_index (or si, sourceIndex): The source indices for the transform. It can be a single index, an index pattern (for example, "my-index-*"), an array of indices (for example, ["my-index-000001", "my-index-000002"]), or an array of index patterns (for example, ["my-index-*", "my-other-index-*"]. For remote indices use the syntax "remote_name:index_name". If any indices are in remote clusters then the master node and at least one transform node must have the remote_cluster_client node role.

    • state (or s): The status of the transform, which can be one of the following values:

      • aborting: The transform is aborting.
      • failed: The transform failed. For more information about the failure, check the reason field.
      • indexing: The transform is actively processing data and creating new documents.
      • started: The transform is running but not actively indexing data.
      • stopped: The transform is stopped.
      • stopping: The transform is stopping.

    • transform_type (or tt): Indicates the type of transform: batch or continuous.

    • trigger_count (or tc): The number of times the transform has been triggered by the scheduler. For example, the scheduler triggers the transform indexer to check for updates or ingest new data at an interval specified in the frequency property.

    • version (or v): The version of Elasticsearch that existed on the node when the transform was created.

    Values are changes_last_detection_time, cldt, checkpoint, cp, checkpoint_duration_time_exp_avg, cdtea, checkpointTimeExpAvg, checkpoint_progress, c, checkpointProgress, create_time, ct, createTime, delete_time, dtime, description, d, dest_index, di, destIndex, documents_deleted, docd, documents_indexed, doci, docs_per_second, dps, documents_processed, docp, frequency, f, id, index_failure, if, index_time, itime, index_total, it, indexed_documents_exp_avg, idea, last_search_time, lst, lastSearchTime, max_page_search_size, mpsz, pages_processed, pp, pipeline, p, processed_documents_exp_avg, pdea, processing_time, pt, reason, r, search_failure, sf, search_time, stime, search_total, st, source_index, si, sourceIndex, state, s, transform_type, tt, trigger_count, tc, version, or v.

  • s string | array[string]

    Comma-separated list of column names or column aliases used to sort the response.

    Supported values include:

    • changes_last_detection_time (or cldt): The timestamp when changes were last detected in the source indices.
    • checkpoint (or cp): The sequence number for the checkpoint.
    • checkpoint_duration_time_exp_avg (or cdtea, checkpointTimeExpAvg): Exponential moving average of the duration of the checkpoint, in milliseconds.
    • checkpoint_progress (or c, checkpointProgress): The progress of the next checkpoint that is currently in progress.
    • create_time (or ct, createTime): The time the transform was created.
    • delete_time (or dtime): The amount of time spent deleting, in milliseconds.
    • description (or d): The description of the transform.
    • dest_index (or di, destIndex): The destination index for the transform. The mappings of the destination index are deduced based on the source fields when possible. If alternate mappings are required, use the Create index API prior to starting the transform.
    • documents_deleted (or docd): The number of documents that have been deleted from the destination index due to the retention policy for this transform.
    • documents_indexed (or doci): The number of documents that have been indexed into the destination index for the transform.
    • docs_per_second (or dps): Specifies a limit on the number of input documents per second. This setting throttles the transform by adding a wait time between search requests. The default value is null, which disables throttling.
    • documents_processed (or docp): The number of documents that have been processed from the source index of the transform.
    • frequency (or f): The interval between checks for changes in the source indices when the transform is running continuously. Also determines the retry interval in the event of transient failures while the transform is searching or indexing. The minimum value is 1s and the maximum is 1h. The default value is 1m.
    • id: Identifier for the transform.
    • index_failure (or if): The number of indexing failures.
    • index_time (or itime): The amount of time spent indexing, in milliseconds.
    • index_total (or it): The number of index operations.
    • indexed_documents_exp_avg (or idea): Exponential moving average of the number of new documents that have been indexed.
    • last_search_time (or lst, lastSearchTime): The timestamp of the last search in the source indices. This field is only shown if the transform is running.
    • max_page_search_size (or mpsz): Defines the initial page size to use for the composite aggregation for each checkpoint. If circuit breaker exceptions occur, the page size is dynamically adjusted to a lower value. The minimum value is 10 and the maximum is 65,536. The default value is 500.
    • pages_processed (or pp): The number of search or bulk index operations processed. Documents are processed in batches instead of individually.
    • pipeline (or p): The unique identifier for an ingest pipeline.
    • processed_documents_exp_avg (or pdea): Exponential moving average of the number of documents that have been processed.
    • processing_time (or pt): The amount of time spent processing results, in milliseconds.
    • reason (or r): If a transform has a failed state, this property provides details about the reason for the failure.
    • search_failure (or sf): The number of search failures.
    • search_time (or stime): The amount of time spent searching, in milliseconds.
    • search_total (or st): The number of search operations on the source index for the transform.
    • source_index (or si, sourceIndex): The source indices for the transform. It can be a single index, an index pattern (for example, "my-index-*"), an array of indices (for example, ["my-index-000001", "my-index-000002"]), or an array of index patterns (for example, ["my-index-*", "my-other-index-*"]. For remote indices use the syntax "remote_name:index_name". If any indices are in remote clusters then the master node and at least one transform node must have the remote_cluster_client node role.

    • state (or s): The status of the transform, which can be one of the following values:

      • aborting: The transform is aborting.
      • failed: The transform failed. For more information about the failure, check the reason field.
      • indexing: The transform is actively processing data and creating new documents.
      • started: The transform is running but not actively indexing data.
      • stopped: The transform is stopped.
      • stopping: The transform is stopping.

    • transform_type (or tt): Indicates the type of transform: batch or continuous.

    • trigger_count (or tc): The number of times the transform has been triggered by the scheduler. For example, the scheduler triggers the transform indexer to check for updates or ingest new data at an interval specified in the frequency property.

    • version (or v): The version of Elasticsearch that existed on the node when the transform was created.

    Values are changes_last_detection_time, cldt, checkpoint, cp, checkpoint_duration_time_exp_avg, cdtea, checkpointTimeExpAvg, checkpoint_progress, c, checkpointProgress, create_time, ct, createTime, delete_time, dtime, description, d, dest_index, di, destIndex, documents_deleted, docd, documents_indexed, doci, docs_per_second, dps, documents_processed, docp, frequency, f, id, index_failure, if, index_time, itime, index_total, it, indexed_documents_exp_avg, idea, last_search_time, lst, lastSearchTime, max_page_search_size, mpsz, pages_processed, pp, pipeline, p, processed_documents_exp_avg, pdea, processing_time, pt, reason, r, search_failure, sf, search_time, stime, search_total, st, source_index, si, sourceIndex, state, s, transform_type, tt, trigger_count, tc, version, or v.

  • time string

    The unit used to display time values.

    Values are nanos, micros, ms, s, m, h, or d.

  • size number

    The maximum number of transforms to obtain.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • id string
    • state string

      The status of the transform. Returned values include: aborting: The transform is aborting. failed: The transform failed. For more information about the failure, check thereasonfield. indexing: The transform is actively processing data and creating new documents. started: The transform is running but not actively indexing data. stopped: The transform is stopped. stopping`: The transform is stopping.

    • checkpoint string

      The sequence number for the checkpoint.

    • documents_processed string

      The number of documents that have been processed from the source index of the transform.

    • checkpoint_progress string | null

      The progress of the next checkpoint that is currently in progress.

      One of:
      string-1 string string-2 string | null

    • last_search_time string | null

      The timestamp of the last search in the source indices. This field is shown only if the transform is running.

      One of:
      string-1 string string-2 string | null

    • changes_last_detection_time string | null

      The timestamp when changes were last detected in the source indices.

      One of:
      string-1 string string-2 string | null
    • create_time string

      The time the transform was created.

    • version string
    • source_index string

      The source indices for the transform.

    • dest_index string

      The destination index for the transform.

    • pipeline string

      The unique identifier for the ingest pipeline.

    • description string

      The description of the transform.

    • transform_type string

      The type of transform: batch or continuous.

    • frequency string

      The interval between checks for changes in the source indices when the transform is running continuously.

    • max_page_search_size string

      The initial page size that is used for the composite aggregation for each checkpoint.

    • docs_per_second string

      The number of input documents per second.

    • reason string

      If a transform has a failed state, these details describe the reason for failure.

    • search_total string

      The total number of search operations on the source index for the transform.

    • search_failure string

      The total number of search failures.

    • search_time string

      The total amount of search time, in milliseconds.

    • index_total string

      The total number of index operations done by the transform.

    • index_failure string

      The total number of indexing failures.

    • index_time string

      The total time spent indexing documents, in milliseconds.

    • documents_indexed string

      The number of documents that have been indexed into the destination index for the transform.

    • delete_time string

      The total time spent deleting documents, in milliseconds.

    • documents_deleted string

      The number of documents deleted from the destination index due to the retention policy for the transform.

    • trigger_count string

      The number of times the transform has been triggered by the scheduler. For example, the scheduler triggers the transform indexer to check for updates or ingest new data at an interval specified in the frequency property.

    • pages_processed string

      The number of search or bulk index operations processed. Documents are processed in batches instead of individually.

    • processing_time string

      The total time spent processing results, in milliseconds.

    • checkpoint_duration_time_exp_avg string

      The exponential moving average of the duration of the checkpoint, in milliseconds.

    • indexed_documents_exp_avg string

      The exponential moving average of the number of new documents that have been indexed.

    • processed_documents_exp_avg string

      The exponential moving average of the number of documents that have been processed.
GET /_cat/transforms 
curl \
 --request GET 'http://api.example.com/_cat/transforms' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_cat/transforms?v=true&format=json`. 
[
  {
    "id" : "ecommerce_transform",
    "state" : "started",
    "checkpoint" : "1",
    "documents_processed" : "705",
    "checkpoint_progress" : "100.00",
    "changes_last_detection_time" : null
  }
]

Get the cluster state Added in 1.3.0

GET /_cluster/state/{metric}/{index}
Api key auth Basic auth Bearer auth

Get comprehensive information about the state of the cluster.

The cluster state is an internal data structure which keeps track of a variety of information needed by every node, including the identity and attributes of the other nodes in the cluster; cluster-wide settings; index metadata, including the mapping and settings for each index; the location and status of every shard copy in the cluster.

The elected master node ensures that every node in the cluster has a copy of the same cluster state. This API lets you retrieve a representation of this internal state for debugging or diagnostic purposes. You may need to consult the Elasticsearch source code to determine the precise meaning of the response.

By default the API will route requests to the elected master node since this node is the authoritative source of cluster states. You can also retrieve the cluster state held on the node handling the API request by adding the ?local=true query parameter.

Elasticsearch may need to expend significant effort to compute a response to this API in larger clusters, and the response may comprise a very large quantity of data. If you use this API repeatedly, your cluster may become unstable.

WARNING: The response is a representation of an internal data structure. Its format is not subject to the same compatibility guarantees as other more stable APIs and may change from version to version. Do not query this API using external monitoring tools. Instead, obtain the information you require using other more stable cluster APIs.

Path parameters

  • metric string | array[string] Required

    Limit the information returned to the specified metrics

  • index string | array[string] Required

    A comma-separated list of index names; use _all or empty string to perform the operation on all indices

Query parameters

  • allow_no_indices boolean

    Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes _all string or when no indices have been specified)

  • expand_wildcards string | array[string]

    Whether to expand wildcard expression to concrete indices that are open, closed or both.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.

    Values are all, open, closed, hidden, or none.

  • flat_settings boolean

    Return settings in flat format (default: false)

  • ignore_unavailable boolean

    Whether specified concrete indices should be ignored when unavailable (missing or closed)

  • local boolean

    Return local information, do not retrieve the state from master node (default: false)

  • master_timeout string

    Specify timeout for connection to master

    Values are -1 or 0.

  • wait_for_metadata_version number

    Wait for the metadata version to be equal or greater than the specified metadata version

  • wait_for_timeout string

    The maximum time to wait for wait_for_metadata_version before timing out

    Values are -1 or 0.

Responses

GET /_cluster/state/{metric}/{index} 
curl \
 --request GET 'http://api.example.com/_cluster/state/{metric}/{index}' \
 --header "Authorization: $API_KEY"

Clear the archived repositories metering Technical preview

DELETE /_nodes/{node_id}/_repositories_metering/{max_archive_version}
Api key auth Basic auth Bearer auth

Clear the archived repositories metering information in the cluster.

Path parameters

  • node_id string | array[string] Required

    Comma-separated list of node IDs or names used to limit returned information.

  • max_archive_version number Required

    Specifies the maximum archive_version to be cleared from the archive.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object
      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]
        Hide failures attributes Show failures attributes object
      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string Required
    • nodes object Required

      Contains repositories metering information for the nodes selected by the request.

      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • repository_name string Required
        • repository_type string Required

          Repository type.

        • repository_location object Required
          Hide repository_location attributes Show repository_location attributes object
        • repository_ephemeral_id string Required

        • Time unit for milliseconds

        • Time unit for milliseconds

        • archived boolean Required

          A flag that tells whether or not this object has been archived. When a repository is closed or updated the repository metering information is archived and kept for a certain period of time. This allows retrieving the repository metering information of previous repository instantiations.

        • cluster_version number
        • request_counts object Required
          Hide request_counts attributes Show request_counts attributes object
          • GetBlobProperties number

            Number of Get Blob Properties requests (Azure)

          • GetBlob number

            Number of Get Blob requests (Azure)

          • ListBlobs number

            Number of List Blobs requests (Azure)

          • PutBlob number

            Number of Put Blob requests (Azure)

          • PutBlock number

            Number of Put Block (Azure)

          • PutBlockList number

            Number of Put Block List requests

          • GetObject number

            Number of get object requests (GCP, S3)

          • ListObjects number

            Number of list objects requests (GCP, S3)

          • InsertObject number

            Number of insert object requests, including simple, multipart and resumable uploads. Resumable uploads can perform multiple http requests to insert a single object but they are considered as a single request since they are billed as an individual operation. (GCP)

          • PutObject number

            Number of PutObject requests (S3)

          • PutMultipartObject number

            Number of Multipart requests, including CreateMultipartUpload, UploadPart and CompleteMultipartUpload requests (S3)
DELETE /_nodes/{node_id}/_repositories_metering/{max_archive_version} 
curl \
 --request DELETE 'http://api.example.com/_nodes/{node_id}/_repositories_metering/{max_archive_version}' \
 --header "Authorization: $API_KEY"

Set the connector sync job stats Technical preview

PUT /_connector/_sync_job/{connector_sync_job_id}/_stats
Api key auth Basic auth Bearer auth

Stats include: deleted_document_count, indexed_document_count, indexed_document_volume, and total_document_count. You can also update last_seen. This API is mainly used by the connector service for updating sync job information.

To sync data using self-managed connectors, you need to deploy the Elastic connector service on your own infrastructure. This service runs automatically on Elastic Cloud for Elastic managed connectors.

Path parameters

application/json

Body Required

  • deleted_document_count number Required

    The number of documents the sync job deleted.

  • indexed_document_count number Required

    The number of documents the sync job indexed.

  • indexed_document_volume number Required

    The total size of the data (in MiB) the sync job indexed.

  • last_seen string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • metadata object
    Hide metadata attribute Show metadata attribute object
    • * object Additional properties
  • total_document_count number

    The total number of documents in the target index after the sync job finished.

Responses

PUT /_connector/_sync_job/{connector_sync_job_id}/_stats 
curl \
 --request PUT 'http://api.example.com/_connector/_sync_job/{connector_sync_job_id}/_stats' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"deleted_document_count":42.0,"indexed_document_count":42.0,"indexed_document_volume":42.0,"last_seen":"string","metadata":{"additionalProperty1":{},"additionalProperty2":{}},"total_document_count":42.0}'

Get auto-follow patterns Added in 6.5.0

GET /_ccr/auto_follow
Api key auth Basic auth Bearer auth

Get cross-cluster replication auto-follow patterns.

External documentation

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

    Values are -1 or 0.

Responses

GET /_ccr/auto_follow 
curl \
 --request GET 'http://api.example.com/_ccr/auto_follow' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_ccr/auto_follow/my_auto_follow_pattern`, which gets auto-follow patterns. 
{
  "patterns": [
    {
      "name": "my_auto_follow_pattern",
      "pattern": {
        "active": true,
        "remote_cluster" : "remote_cluster",
        "leader_index_patterns" :
        [
          "leader_index*"
        ],
        "leader_index_exclusion_patterns":
        [
          "leader_index_001"
        ],
        "follow_index_pattern" : "{{leader_index}}-follower"
      }
    }
  ]
}

Get data stream stats Added in 7.9.0

GET /_data_stream/{name}/_stats
Api key auth Basic auth Bearer auth

Get statistics for one or more data streams.

Path parameters

  • name string Required

    Comma-separated list of data streams used to limit the request. Wildcard expressions (*) are supported. To target all data streams in a cluster, omit this parameter or use *.

Query parameters

  • expand_wildcards string | array[string]

    Type of data stream that wildcard patterns can match. Supports comma-separated values, such as open,hidden.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.

    Values are all, open, closed, hidden, or none.

Responses

GET /_data_stream/{name}/_stats 
curl \
 --request GET 'http://api.example.com/_data_stream/{name}/_stats' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response for retrieving statistics for a data stream. 
{
  "_shards": {
    "total": 10,
    "successful": 5,
    "failed": 0
  },
  "data_stream_count": 2,
  "backing_indices": 5,
  "total_store_size": "7kb",
  "total_store_size_bytes": 7268,
  "data_streams": [
    {
      "data_stream": "my-data-stream",
      "backing_indices": 3,
      "store_size": "3.7kb",
      "store_size_bytes": 3772,
      "maximum_timestamp": 1607512028000
    },
    {
      "data_stream": "my-data-stream-two",
      "backing_indices": 2,
      "store_size": "3.4kb",
      "store_size_bytes": 3496,
      "maximum_timestamp": 1607425567000
    }
  ]
}

Downsample an index Technical preview

POST /{index}/_downsample/{target_index}
Api key auth Basic auth Bearer auth

Aggregate a time series (TSDS) index and store pre-computed statistical summaries (min, max, sum, value_count and avg) for each metric field grouped by a configured time interval. For example, a TSDS index that contains metrics sampled every 10 seconds can be downsampled to an hourly index. All documents within an hour interval are summarized and stored as a single document in the downsample index.

NOTE: Only indices in a time series data stream are supported. Neither field nor document level security can be defined on the source index. The source index must be read only (index.blocks.write: true).

Path parameters

  • index string Required

    Name of the time series index to downsample.

  • target_index string Required

    Name of the index to create.

application/json

Body Required

  • fixed_interval string Required

    A date histogram interval. Similar to Duration with additional units: w (week), M (month), q (quarter) and y (year)

Responses

POST /{index}/_downsample/{target_index} 
curl \
 --request POST 'http://api.example.com/{index}/_downsample/{target_index}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"fixed_interval\": \"1d\"\n}"'
Request example 
{
  "fixed_interval": "1d"
}

Create or update a document in an index

POST /{index}/_doc
Api key auth Basic auth Bearer auth

Add a JSON document to the specified data stream or index and make it searchable. If the target is an index and the document already exists, the request updates the document and increments its version.

NOTE: You cannot use this API to send update requests for existing documents in a data stream.

If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:

  • To add or overwrite a document using the PUT /<target>/_doc/<_id> request format, you must have the create, index, or write index privilege.
  • To add a document using the POST /<target>/_doc/ request format, you must have the create_doc, create, index, or write index privilege.
  • To automatically create a data stream or index with this API request, you must have the auto_configure, create_index, or manage index privilege.

Automatic data stream creation requires a matching index template with data stream enabled.

NOTE: Replica shards might not all be started when an indexing operation returns successfully. By default, only the primary is required. Set wait_for_active_shards to change this default behavior.

Automatically create data streams and indices

If the request's target doesn't exist and matches an index template with a data_stream definition, the index operation automatically creates the data stream.

If the target doesn't exist and doesn't match a data stream template, the operation automatically creates the index and applies any matching index templates.

NOTE: Elasticsearch includes several built-in index templates. To avoid naming collisions with these templates, refer to index pattern documentation.

If no mapping exists, the index operation creates a dynamic mapping. By default, new fields and objects are automatically added to the mapping if needed.

Automatic index creation is controlled by the action.auto_create_index setting. If it is true, any index can be created automatically. You can modify this setting to explicitly allow or block automatic creation of indices that match specified patterns or set it to false to turn off automatic index creation entirely. Specify a comma-separated list of patterns you want to allow or prefix each pattern with + or - to indicate whether it should be allowed or blocked. When a list is specified, the default behaviour is to disallow.

NOTE: The action.auto_create_index setting affects the automatic creation of indices only. It does not affect the creation of data streams.

Optimistic concurrency control

Index operations can be made conditional and only be performed if the last modification to the document was assigned the sequence number and primary term specified by the if_seq_no and if_primary_term parameters. If a mismatch is detected, the operation will result in a VersionConflictException and a status code of 409.

Routing

By default, shard placement — or routing — is controlled by using a hash of the document's ID value. For more explicit control, the value fed into the hash function used by the router can be directly specified on a per-operation basis using the routing parameter.

When setting up explicit mapping, you can also use the _routing field to direct the index operation to extract the routing value from the document itself. This does come at the (very minimal) cost of an additional document parsing pass. If the _routing mapping is defined and set to be required, the index operation will fail if no routing value is provided or extracted.

NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.

Distributed

The index operation is directed to the primary shard based on its route and performed on the actual node containing this shard. After the primary shard completes the operation, if needed, the update is distributed to applicable replicas.

Active shards

To improve the resiliency of writes to the system, indexing operations can be configured to wait for a certain number of active shard copies before proceeding with the operation. If the requisite number of active shard copies are not available, then the write operation must wait and retry, until either the requisite shard copies have started or a timeout occurs. By default, write operations only wait for the primary shards to be active before proceeding (that is to say wait_for_active_shards is 1). This default can be overridden in the index settings dynamically by setting index.write.wait_for_active_shards. To alter this behavior per operation, use the wait_for_active_shards request parameter.

Valid values are all or any positive integer up to the total number of configured copies per shard in the index (which is number_of_replicas+1). Specifying a negative value or a number greater than the number of shard copies will throw an error.

For example, suppose you have a cluster of three nodes, A, B, and C and you create an index index with the number of replicas set to 3 (resulting in 4 shard copies, one more copy than there are nodes). If you attempt an indexing operation, by default the operation will only ensure the primary copy of each shard is available before proceeding. This means that even if B and C went down and A hosted the primary shard copies, the indexing operation would still proceed with only one copy of the data. If wait_for_active_shards is set on the request to 3 (and all three nodes are up), the indexing operation will require 3 active shard copies before proceeding. This requirement should be met because there are 3 active nodes in the cluster, each one holding a copy of the shard. However, if you set wait_for_active_shards to all (or to 4, which is the same in this situation), the indexing operation will not proceed as you do not have all 4 copies of each shard active in the index. The operation will timeout unless a new node is brought up in the cluster to host the fourth copy of the shard.

It is important to note that this setting greatly reduces the chances of the write operation not writing to the requisite number of shard copies, but it does not completely eliminate the possibility, because this check occurs before the write operation starts. After the write operation is underway, it is still possible for replication to fail on any number of shard copies but still succeed on the primary. The _shards section of the API response reveals the number of shard copies on which replication succeeded and failed.

No operation (noop) updates

When updating a document by using this API, a new version of the document is always created even if the document hasn't changed. If this isn't acceptable use the _update API with detect_noop set to true. The detect_noop option isn't available on this API because it doesn’t fetch the old source and isn't able to compare it against the new source.

There isn't a definitive rule for when noop updates aren't acceptable. It's a combination of lots of factors like how frequently your data source sends updates that are actually noops and how many queries per second Elasticsearch runs on the shard receiving the updates.

Versioning

Each indexed document is given a version number. By default, internal versioning is used that starts at 1 and increments with each update, deletes included. Optionally, the version number can be set to an external value (for example, if maintained in a database). To enable this functionality, version_type should be set to external. The value provided must be a numeric, long value greater than or equal to 0, and less than around 9.2e+18.

NOTE: Versioning is completely real time, and is not affected by the near real time aspects of search operations. If no version is provided, the operation runs without any version checks.

When using the external version type, the system checks to see if the version number passed to the index request is greater than the version of the currently stored document. If true, the document will be indexed and the new version number used. If the value provided is less than or equal to the stored document's version number, a version conflict will occur and the index operation will fail. For example:

PUT my-index-000001/_doc/1?version=2&version_type=external
{
  "user": {
    "id": "elkbee"
  }
}

In this example, the operation will succeed since the supplied version of 2 is higher than the current document version of 1.
If the document was already updated and its version was set to 2 or higher, the indexing command will fail and result in a conflict (409 HTTP status code).

A nice side effect is that there is no need to maintain strict ordering of async indexing operations run as a result of changes to a source database, as long as version numbers from the source database are used.
Even the simple case of updating the Elasticsearch index using data from a database is simplified if external versioning is used, as only the latest version will be used if the index operations arrive out of order.
External documentation

Path parameters

  • index string Required

    The name of the data stream or index to target. If the target doesn't exist and matches the name or wildcard (*) pattern of an index template with a data_stream definition, this request creates the data stream. If the target doesn't exist and doesn't match a data stream template, this request creates the index. You can check for existing targets with the resolve index API.

Query parameters

  • if_primary_term number

    Only perform the operation if the document has this primary term.

  • if_seq_no number

    Only perform the operation if the document has this sequence number.

  • include_source_on_error boolean

    True or false if to include the document source in the error message in case of parsing errors.

  • op_type string

    Set to create to only index the document if it does not already exist (put if absent). If a document with the specified _id already exists, the indexing operation will fail. The behavior is the same as using the <index>/_create endpoint. If a document ID is specified, this paramater defaults to index. Otherwise, it defaults to create. If the request targets a data stream, an op_type of create is required.

    Supported values include:

    • index: Overwrite any documents that already exist.
    • create: Only index documents that do not already exist.

    Values are index or create.

  • pipeline string

    The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, then setting the value to _none disables the default ingest pipeline for this request. If a final pipeline is configured it will always run, regardless of the value of this parameter.

  • refresh string

    If true, Elasticsearch refreshes the affected shards to make this operation visible to search. If wait_for, it waits for a refresh to make this operation visible to search. If false, it does nothing with refreshes.

    Values are true, false, or wait_for.

  • routing string

    A custom value that is used to route operations to a specific shard.

  • timeout string

    The period the request waits for the following operations: automatic index creation, dynamic mapping updates, waiting for active shards.

    This parameter is useful for situations where the primary shard assigned to perform the operation might not be available when the operation runs. Some reasons for this might be that the primary shard is currently recovering from a gateway or undergoing relocation. By default, the operation will wait on the primary shard to become available for at least 1 minute before failing and responding with an error. The actual wait time could be longer, particularly when multiple waits occur.

    Values are -1 or 0.

  • version number

    An explicit version number for concurrency control. It must be a non-negative long number.

  • version_type string

    The version type.

    Supported values include:

    • internal: Use internal versioning that starts at 1 and increments with each update or delete.
    • external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.
    • external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document. NOTE: The external_gte version type is meant for special use cases and should be used with care. If used incorrectly, it can result in loss of data.
    • force: This option is deprecated because it can cause primary and replica shards to diverge.

    Values are internal, external, external_gte, or force.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. You can set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). The default value of 1 means it waits for each primary shard to be active.

    Values are all or index-setting.

  • require_alias boolean

    If true, the destination must be an index alias.

application/json

Body Required

object object

Responses

POST /{index}/_doc 
curl \
 --request POST 'http://api.example.com/{index}/_doc' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"@timestamp\": \"2099-11-15T13:12:00\",\n  \"message\": \"GET /search HTTP/1.1 200 1070000\",\n  \"user\": {\n    \"id\": \"kimchy\"\n  }\n}"'
Request examples
Run `POST my-index-000001/_doc/` to index a document. When you use the `POST /<target>/_doc/` request format, the `op_type` is automatically set to `create` and the index operation generates a unique ID for the document. 
{
  "@timestamp": "2099-11-15T13:12:00",
  "message": "GET /search HTTP/1.1 200 1070000",
  "user": {
    "id": "kimchy"
  }
}
Run `PUT my-index-000001/_doc/1` to insert a JSON document into the `my-index-000001` index with an `_id` of 1. 
{
  "@timestamp": "2099-11-15T13:12:00",
  "message": "GET /search HTTP/1.1 200 1070000",
  "user": {
    "id": "kimchy"
  }
}
Response examples (200)
A successful response from `POST my-index-000001/_doc/`, which contains an automated document ID. 
{
  "_shards": {
    "total": 2,
    "failed": 0,
    "successful": 2
  },
  "_index": "my-index-000001",
  "_id": "W0tpsmIBdwcYyG50zbta",
  "_version": 1,
  "_seq_no": 0,
  "_primary_term": 1,
  "result": "created"
}
A successful response from `PUT my-index-000001/_doc/1`. 
{
  "_shards": {
    "total": 2,
    "failed": 0,
    "successful": 2
  },
  "_index": "my-index-000001",
  "_id": "1",
  "_version": 1,
  "_seq_no": 0,
  "_primary_term": 1,
  "result": "created"
}

Get term vector information

POST /{index}/_termvectors/{id}
Api key auth Basic auth Bearer auth

Get information and statistics about terms in the fields of a particular document.

You can retrieve term vectors for documents stored in the index or for artificial documents passed in the body of the request. You can specify the fields you are interested in through the fields parameter or by adding the fields to the request body. For example:

GET /my-index-000001/_termvectors/1?fields=message

Fields can be specified using wildcards, similar to the multi match query.

Term vectors are real-time by default, not near real-time. This can be changed by setting realtime parameter to false.

You can request three types of values: term information, term statistics, and field statistics. By default, all term information and field statistics are returned for all fields but term statistics are excluded.

Term information

  • term frequency in the field (always returned)
  • term positions (positions: true)
  • start and end offsets (offsets: true)
  • term payloads (payloads: true), as base64 encoded bytes

If the requested information wasn't stored in the index, it will be computed on the fly if possible. Additionally, term vectors could be computed for documents not even existing in the index, but instead provided by the user.


Start and end offsets assume UTF-16 encoding is being used. If you want to use these offsets in order to get the original text that produced this token, you should make sure that the string you are taking a sub-string of is also encoded using UTF-16.

Behaviour

The term and field statistics are not accurate. Deleted documents are not taken into account. The information is only retrieved for the shard the requested document resides in. The term and field statistics are therefore only useful as relative measures whereas the absolute numbers have no meaning in this context. By default, when requesting term vectors of artificial documents, a shard to get the statistics from is randomly selected. Use routing only to hit a particular shard.

Path parameters

  • index string Required

    The name of the index that contains the document.

  • id string Required

    A unique identifier for the document.

Query parameters

  • fields string | array[string]

    A comma-separated list or wildcard expressions of fields to include in the statistics. It is used as the default list unless a specific field list is provided in the completion_fields or fielddata_fields parameters.

  • field_statistics boolean

    If true, the response includes:

    • The document count (how many documents contain this field).
    • The sum of document frequencies (the sum of document frequencies for all terms in this field).
    • The sum of total term frequencies (the sum of total term frequencies of each term in this field).
  • offsets boolean

    If true, the response includes term offsets.

  • payloads boolean

    If true, the response includes term payloads.

  • positions boolean

    If true, the response includes term positions.

  • preference string

    The node or shard the operation should be performed on. It is random by default.

  • realtime boolean

    If true, the request is real-time as opposed to near-real-time.

  • routing string

    A custom value that is used to route operations to a specific shard.

  • term_statistics boolean

    If true, the response includes:

    • The total term frequency (how often a term occurs in all documents).
    • The document frequency (the number of documents containing the current term).

    By default these values are not returned since term statistics can have a serious performance impact.

  • version number

    If true, returns the document version as part of a hit.

  • version_type string

    The version type.

    Supported values include:

    • internal: Use internal versioning that starts at 1 and increments with each update or delete.
    • external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.
    • external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document. NOTE: The external_gte version type is meant for special use cases and should be used with care. If used incorrectly, it can result in loss of data.
    • force: This option is deprecated because it can cause primary and replica shards to diverge.

    Values are internal, external, external_gte, or force.

application/json

Body

  • doc object

    An artificial document (a document not present in the index) for which you want to retrieve term vectors.

  • filter object
    Hide filter attributes Show filter attributes object
    • max_doc_freq number

      Ignore words which occur in more than this many docs. Defaults to unbounded.

    • max_num_terms number

      The maximum number of terms that must be returned per field.

    • max_term_freq number

      Ignore words with more than this frequency in the source doc. It defaults to unbounded.

    • max_word_length number

      The maximum word length above which words will be ignored. Defaults to unbounded.

    • min_doc_freq number

      Ignore terms which do not occur in at least this many docs.

    • min_term_freq number

      Ignore words with less than this frequency in the source doc.

    • min_word_length number

      The minimum word length below which words will be ignored.

  • per_field_analyzer object

    Override the default per-field analyzer. This is useful in order to generate term vectors in any fashion, especially when using artificial documents. When providing an analyzer for a field that already stores term vectors, the term vectors will be regenerated.

    Hide per_field_analyzer attribute Show per_field_analyzer attribute object
    • * string Additional properties
  • fields string | array[string]
  • field_statistics boolean

    If true, the response includes:

    • The document count (how many documents contain this field).
    • The sum of document frequencies (the sum of document frequencies for all terms in this field).
    • The sum of total term frequencies (the sum of total term frequencies of each term in this field).
  • offsets boolean

    If true, the response includes term offsets.

  • payloads boolean

    If true, the response includes term payloads.

  • positions boolean

    If true, the response includes term positions.

  • term_statistics boolean

    If true, the response includes:

    • The total term frequency (how often a term occurs in all documents).
    • The document frequency (the number of documents containing the current term).

    By default these values are not returned since term statistics can have a serious performance impact.

  • routing string
  • version number
  • version_type string

    Values are internal, external, external_gte, or force.

Responses

POST /{index}/_termvectors/{id} 
curl \
 --request POST 'http://api.example.com/{index}/_termvectors/{id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"fields\" : [\"text\"],\n  \"offsets\" : true,\n  \"payloads\" : true,\n  \"positions\" : true,\n  \"term_statistics\" : true,\n  \"field_statistics\" : true\n}"'
Request examples
Run `GET /my-index-000001/_termvectors/1` to return all information and statistics for field `text` in document 1. 
{
  "fields" : ["text"],
  "offsets" : true,
  "payloads" : true,
  "positions" : true,
  "term_statistics" : true,
  "field_statistics" : true
}
Run `GET /my-index-000001/_termvectors/1` to set per-field analyzers. A different analyzer than the one at the field may be provided by using the `per_field_analyzer` parameter. 
{
  "doc" : {
    "fullname" : "John Doe",
    "text" : "test test test"
  },
  "fields": ["fullname"],
  "per_field_analyzer" : {
    "fullname": "keyword"
  }
}
Run `GET /imdb/_termvectors` to filter the terms returned based on their tf-idf scores. It returns the three most "interesting" keywords from the artificial document having the given "plot" field value. Notice that the keyword "Tony" or any stop words are not part of the response, as their tf-idf must be too low. 
{
  "doc": {
    "plot": "When wealthy industrialist Tony Stark is forced to build an armored suit after a life-threatening incident, he ultimately decides to use its technology to fight against evil."
  },
  "term_statistics": true,
  "field_statistics": true,
  "positions": false,
  "offsets": false,
  "filter": {
    "max_num_terms": 3,
    "min_term_freq": 1,
    "min_doc_freq": 1
  }
}
Run `GET /my-index-000001/_termvectors/1`. Term vectors which are not explicitly stored in the index are automatically computed on the fly. This request returns all information and statistics for the fields in document 1, even though the terms haven't been explicitly stored in the index. Note that for the field text, the terms are not regenerated. 
{
  "fields" : ["text", "some_field_without_term_vectors"],
  "offsets" : true,
  "positions" : true,
  "term_statistics" : true,
  "field_statistics" : true
}
Run `GET /my-index-000001/_termvectors`. Term vectors can be generated for artificial documents, that is for documents not present in the index. If dynamic mapping is turned on (default), the document fields not in the original mapping will be dynamically created. 
{
  "doc" : {
    "fullname" : "John Doe",
    "text" : "test test test"
  }
}
Response examples (200)
A successful response from `GET /my-index-000001/_termvectors/1`. 
{
  "_index": "my-index-000001",
  "_id": "1",
  "_version": 1,
  "found": true,
  "took": 6,
  "term_vectors": {
    "text": {
      "field_statistics": {
        "sum_doc_freq": 4,
        "doc_count": 2,
        "sum_ttf": 6
      },
      "terms": {
        "test": {
          "doc_freq": 2,
          "ttf": 4,
          "term_freq": 3,
          "tokens": [
            {
              "position": 0,
              "start_offset": 0,
              "end_offset": 4,
              "payload": "d29yZA=="
            },
            {
              "position": 1,
              "start_offset": 5,
              "end_offset": 9,
              "payload": "d29yZA=="
            },
            {
              "position": 2,
              "start_offset": 10,
              "end_offset": 14,
              "payload": "d29yZA=="
            }
          ]
        }
      }
    }
  }
}
A successful response from `GET /my-index-000001/_termvectors` with `per_field_analyzer` in the request body. 
{
  "_index": "my-index-000001",
  "_version": 0,
  "found": true,
  "took": 6,
  "term_vectors": {
    "fullname": {
      "field_statistics": {
          "sum_doc_freq": 2,
          "doc_count": 4,
          "sum_ttf": 4
      },
      "terms": {
          "John Doe": {
            "term_freq": 1,
            "tokens": [
                {
                  "position": 0,
                  "start_offset": 0,
                  "end_offset": 8
                }
            ]
          }
      }
    }
  }
}
A successful response from `GET /my-index-000001/_termvectors` with a `filter` in the request body. 
{
  "_index": "imdb",
  "_version": 0,
  "found": true,
  "term_vectors": {
      "plot": {
        "field_statistics": {
            "sum_doc_freq": 3384269,
            "doc_count": 176214,
            "sum_ttf": 3753460
        },
        "terms": {
            "armored": {
              "doc_freq": 27,
              "ttf": 27,
              "term_freq": 1,
              "score": 9.74725
            },
            "industrialist": {
              "doc_freq": 88,
              "ttf": 88,
              "term_freq": 1,
              "score": 8.590818
            },
            "stark": {
              "doc_freq": 44,
              "ttf": 47,
              "term_freq": 1,
              "score": 9.272792
            }
        }
      }
  }
}

Add an index block Added in 7.9.0

PUT /{index}/_block/{block}
Api key auth Basic auth Bearer auth

Add an index block to an index. Index blocks limit the operations allowed on an index by blocking specific operation types.

Path parameters

  • index string Required

    A comma-separated list or wildcard expression of index names used to limit the request. By default, you must explicitly name the indices you are adding blocks to. To allow the adding of blocks to indices with _all, *, or other wildcard expressions, change the action.destructive_requires_name setting to false. You can update this setting in the elasticsearch.yml file or by using the cluster update settings API.

  • block string Required

    The block type to add to the index.

    Supported values include:

    • metadata: Disable metadata changes, such as closing the index.
    • read: Disable read operations.
    • read_only: Disable write operations and metadata changes.
    • write: Disable write operations. However, metadata changes are still allowed.

    Values are metadata, read, read_only, or write.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • expand_wildcards string | array[string]

    The type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. It supports comma-separated values, such as open,hidden.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.

    Values are all, open, closed, hidden, or none.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • master_timeout string

    The period to wait for the master node. If the master node is not available before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

    Values are -1 or 0.

  • timeout string

    The period to wait for a response from all relevant nodes in the cluster after updating the cluster metadata. If no response is received before the timeout expires, the cluster metadata update still applies but the response will indicate that it was not completely acknowledged. It can also be set to -1 to indicate that the request should never timeout.

    Values are -1 or 0.

Responses

PUT /{index}/_block/{block} 
curl \
 --request PUT 'http://api.example.com/{index}/_block/{block}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `PUT /my-index-000001/_block/write`, which adds an index block to an index.' 
{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "indices" : [ {
    "name" : "my-index-000001",
    "blocked" : true
  } ]
}

Check aliases

HEAD /_alias/{name}
Api key auth Basic auth Bearer auth

Check if one or more data stream or index aliases exist.

Path parameters

  • name string | array[string] Required

    Comma-separated list of aliases to check. Supports wildcards (*).

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.

    Values are all, open, closed, hidden, or none.

  • ignore_unavailable boolean

    If false, requests that include a missing data stream or index in the target indices or data streams return an error.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

HEAD /_alias/{name} 
curl \
 --request HEAD 'http://api.example.com/_alias/{name}' \
 --header "Authorization: $API_KEY"

Update field mappings

PUT /{index}/_mapping
Api key auth Basic auth Bearer auth

Add new fields to an existing data stream or index. You can also use this API to change the search settings of existing fields and add new properties to existing object fields. For data streams, these changes are applied to all backing indices by default.

Add multi-fields to an existing field

Multi-fields let you index the same field in different ways. You can use this API to update the fields mapping parameter and enable multi-fields for an existing field. WARNING: If an index (or data stream) contains documents when you add a multi-field, those documents will not have values for the new multi-field. You can populate the new multi-field with the update by query API.

Change supported mapping parameters for an existing field

The documentation for each mapping parameter indicates whether you can update it for an existing field using this API. For example, you can use the update mapping API to update the ignore_above parameter.

Change the mapping of an existing field

Except for supported mapping parameters, you can't change the mapping or field type of an existing field. Changing an existing field could invalidate data that's already indexed.

If you need to change the mapping of a field in a data stream's backing indices, refer to documentation about modifying data streams. If you need to change the mapping of a field in other indices, create a new index with the correct mapping and reindex your data into that index.

Rename a field

Renaming a field would invalidate data already indexed under the old field name. Instead, add an alias field to create an alternate field name.

External documentation

Path parameters

  • index string | array[string] Required

    A comma-separated list of index names the mapping should be added to (supports wildcards); use _all or omit to add the mapping on all indices.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.

    Values are all, open, closed, hidden, or none.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • write_index_only boolean

    If true, the mappings are applied only to the current write index for the target.

application/json

Body Required

  • date_detection boolean

    Controls whether dynamic date detection is enabled.

  • dynamic string

    Values are strict, runtime, true, or false.

  • dynamic_date_formats array[string]

    If date detection is enabled then new string fields are checked against 'dynamic_date_formats' and if the value matches then a new date field is added instead of string.

  • dynamic_templates array[object]

    Specify dynamic templates for the mapping.

  • _field_names object
    Hide _field_names attribute Show _field_names attribute object
  • _meta object
    Hide _meta attribute Show _meta attribute object
    • * object Additional properties
  • numeric_detection boolean

    Automatically map strings into numeric data types for all fields.

  • properties object

    Mapping for a field. For new fields, this mapping can include:

    • Field name
    • Field data type
    • Mapping parameters
  • _routing object
    Hide _routing attribute Show _routing attribute object
  • _source object
    Hide _source attributes Show _source attributes object
  • runtime object
    Hide runtime attribute Show runtime attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • fields object

        For type composite

        Hide fields attribute Show fields attribute object
        • * object Additional properties
          Hide * attribute Show * attribute object
          • type string Required

            Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

      • fetch_fields array[object]

        For type lookup

        Hide fetch_fields attributes Show fetch_fields attributes object
        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • format string
      • format string

        A custom format for date type runtime fields.

      • input_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_index string
      • script object
        Hide script attributes Show script attributes object

        • source string | object

          One of:
          ScriptSource string SearchRequestBody object
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          Hide params attribute Show params attribute object
          • * object Additional properties

        • lang string

          Any of:
          ScriptLanguage string ScriptLanguage string

          Values are painless, expression, mustache, or java.

        • options object
          Hide options attribute Show options attribute object
          • * string Additional properties
      • type string Required

        Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
PUT /{index}/_mapping 
curl \
 --request PUT 'http://api.example.com/{index}/_mapping' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"properties\": {\n    \"user\": {\n      \"properties\": {\n        \"name\": {\n          \"type\": \"keyword\"\n        }\n      }\n    }\n  }\n}"'
Request example
The update mapping API can be applied to multiple data streams or indices with a single request. For example, run `PUT /my-index-000001,my-index-000002/_mapping` to update mappings for the `my-index-000001` and `my-index-000002` indices at the same time. 
{
  "properties": {
    "user": {
      "properties": {
        "name": {
          "type": "keyword"
        }
      }
    }
  }
}

Validate a query Added in 1.3.0

POST /_validate/query
Api key auth Basic auth Bearer auth

Validates a query without running it.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • all_shards boolean

    If true, the validation is executed on all shards instead of one random shard per index.

  • analyzer string

    Analyzer to use for the query string. This parameter can only be used when the q query string parameter is specified.

  • analyze_wildcard boolean

    If true, wildcard and prefix queries are analyzed.

  • default_operator string

    The default operator for query string query: AND or OR.

    Values are and, AND, or, or OR.

  • df string

    Field to use as default where no field prefix is given in the query string. This parameter can only be used when the q query string parameter is specified.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.

    Values are all, open, closed, hidden, or none.

  • explain boolean

    If true, the response returns detailed information if an error has occurred.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • lenient boolean

    If true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored.

  • rewrite boolean

    If true, returns a more detailed explanation showing the actual Lucene query that will be executed.

  • q string

    Query in the Lucene query string syntax.

application/json

Body

Responses

POST /_validate/query 
curl \
 --request POST 'http://api.example.com/_validate/query' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"query":{}}'

Perform chat completion inference Added in 8.18.0

POST /_inference/chat_completion/{inference_id}/_stream
Api key auth Basic auth Bearer auth

The chat completion inference API enables real-time responses for chat completion tasks by delivering answers incrementally, reducing response times during computation. It only works with the chat_completion task type for openai and elastic inference services.

IMPORTANT: The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.

NOTE: The chat_completion task type is only available within the _stream API and only supports streaming. The Chat completion inference API and the Stream inference API differ in their response structure and capabilities. The Chat completion inference API provides more comprehensive customization options through more fields and function calling support. If you use the openai service or the elastic service, use the Chat completion inference API.

Path parameters

Query parameters

  • timeout string

    Specifies the amount of time to wait for the inference request to complete.

    Values are -1 or 0.

application/json

Body Required

  • messages array[object] Required

    A list of objects representing the conversation. Requests should generally only add new messages from the user (role user). The other message roles (assistant, system, or tool) should generally only be copied from the response to a previous completion request, such that the messages array is built up throughout a conversation.

    Hide messages attributes Show messages attributes object
  • model string

    The ID of the model to use.

  • max_completion_tokens number

    The upper bound limit for the number of tokens that can be generated for a completion request.

  • stop array[string]

    A sequence of strings to control when the model should stop generating additional tokens.

  • temperature number

    The sampling temperature to use.

  • tool_choice string | object

    One of:
    CompletionToolType string CompletionToolChoice object
  • tools array[object]

    A list of tools that the model can call.

    Hide tools attributes Show tools attributes object
    • type string Required

      The type of tool.

    • function object Required
      Hide function attributes Show function attributes object
      • description string

        A description of what the function does. This is used by the model to choose when and how to call the function.

      • name string Required

        The name of the function.

      • parameters object

        The parameters the functional accepts. This should be formatted as a JSON object.

      • strict boolean

        Whether to enable schema adherence when generating the function call.

  • top_p number

    Nucleus sampling, an alternative to sampling with temperature.

Responses

POST /_inference/chat_completion/{inference_id}/_stream 
curl \
 --request POST 'http://api.example.com/_inference/chat_completion/{inference_id}/_stream' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"model\": \"gpt-4o\",\n  \"messages\": [\n      {\n          \"role\": \"user\",\n          \"content\": \"What is Elastic?\"\n      }\n  ]\n}"'
Request examples
Run `POST _inference/chat_completion/openai-completion/_stream` to perform a chat completion on the example question with streaming. 
{
  "model": "gpt-4o",
  "messages": [
      {
          "role": "user",
          "content": "What is Elastic?"
      }
  ]
}
Run `POST POST _inference/chat_completion/openai-completion/_stream` to perform a chat completion using an Assistant message with `tool_calls`. 
{
  "messages": [
      {
          "role": "assistant",
          "content": "Let's find out what the weather is",
          "tool_calls": [ 
              {
                  "id": "call_KcAjWtAww20AihPHphUh46Gd",
                  "type": "function",
                  "function": {
                      "name": "get_current_weather",
                      "arguments": "{\"location\":\"Boston, MA\"}"
                  }
              }
          ]
      },
      { 
          "role": "tool",
          "content": "The weather is cold",
          "tool_call_id": "call_KcAjWtAww20AihPHphUh46Gd"
      }
  ]
}
Run `POST POST _inference/chat_completion/openai-completion/_stream` to perform a chat completion using a User message with `tools` and `tool_choice`. 
{
  "messages": [
      {
          "role": "user",
          "content": [
              {
                  "type": "text",
                  "text": "What's the price of a scarf?"
              }
          ]
      }
  ],
  "tools": [
      {
          "type": "function",
          "function": {
              "name": "get_current_price",
              "description": "Get the current price of a item",
              "parameters": {
                  "type": "object",
                  "properties": {
                      "item": {
                          "id": "123"
                      }
                  }
              }
          }
      }
  ],
  "tool_choice": {
      "type": "function",
      "function": {
          "name": "get_current_price"
      }
  }
}
Response examples (200)
A successful response when performing a chat completion task using a User message with `tools` and `tool_choice`. 
event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[{"delta":{"content":"","role":"assistant"},"index":0}],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk"}}

event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[{"delta":{"content":Elastic"},"index":0}],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk"}}

event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[{"delta":{"content":" is"},"index":0}],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk"}}

(...)

event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk","usage":{"completion_tokens":28,"prompt_tokens":16,"total_tokens":44}}} 

event: message
data: [DONE]

Perform inference on the service Added in 8.11.0

POST /_inference/{inference_id}
Api key auth Basic auth Bearer auth

This API enables you to use machine learning models to perform specific tasks on data that you provide as an input. It returns a response with the results of the tasks. The inference endpoint you use can perform one specific task that has been defined when the endpoint was created with the create inference API.

For details about using this API with a service, such as Amazon Bedrock, Anthropic, or HuggingFace, refer to the service-specific documentation.


The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.

Path parameters

  • inference_id string Required

    The unique identifier for the inference endpoint.

Query parameters

  • timeout string

    The amount of time to wait for the inference request to complete.

    Values are -1 or 0.

application/json

Body

  • query string

    The query input, which is required only for the rerank task. It is not required for other tasks.

  • input string | array[string] Required

    The text on which you want to perform the inference task. It can be a single string or an array.


    Inference endpoints for the completion task type currently only support a single string as input.

    One of:
    string-1 string array-2 array[string]
  • task_settings object

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • text_embedding_bytes array[object]
      Hide text_embedding_bytes attribute Show text_embedding_bytes attribute object
      • embedding array[number] Required

        Text Embedding results containing bytes are represented as Dense Vectors of bytes.

    • text_embedding_bits array[object]
      Hide text_embedding_bits attribute Show text_embedding_bits attribute object
      • embedding array[number] Required

        Text Embedding results containing bytes are represented as Dense Vectors of bytes.

    • text_embedding array[object]
      Hide text_embedding attribute Show text_embedding attribute object
      • embedding array[number] Required

        Text Embedding results are represented as Dense Vectors of floats.

    • sparse_embedding array[object]
      Hide sparse_embedding attribute Show sparse_embedding attribute object
      • embedding object Required

        Sparse Embedding tokens are represented as a dictionary of string to double.

        Hide embedding attribute Show embedding attribute object
        • * number Additional properties
    • completion array[object]
      Hide completion attribute Show completion attribute object
    • rerank array[object]
      Hide rerank attributes Show rerank attributes object
POST /_inference/{inference_id} 
curl \
 --request POST 'http://api.example.com/_inference/{inference_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"query":"string","input":"string","task_settings":{}}'

Simulate data ingestion Technical preview

GET /_ingest/{index}/_simulate
Api key auth Basic auth Bearer auth

Run ingest pipelines against a set of provided documents, optionally with substitute pipeline definitions, to simulate ingesting data into an index.

This API is meant to be used for troubleshooting or pipeline development, as it does not actually index any data into Elasticsearch.

The API runs the default and final pipeline for that index against a set of documents provided in the body of the request. If a pipeline contains a reroute processor, it follows that reroute processor to the new index, running that index's pipelines as well the same way that a non-simulated ingest would. No data is indexed into Elasticsearch. Instead, the transformed document is returned, along with the list of pipelines that have been run and the name of the index where the document would have been indexed if this were not a simulation. The transformed document is validated against the mappings that would apply to this index, and any validation error is reported in the result.

This API differs from the simulate pipeline API in that you specify a single pipeline for that API, and it runs only that one pipeline. The simulate pipeline API is more useful for developing a single pipeline, while the simulate ingest API is more useful for troubleshooting the interaction of the various pipelines that get applied when ingesting into an index.

By default, the pipeline definitions that are currently in the system are used. However, you can supply substitute pipeline definitions in the body of the request. These will be used in place of the pipeline definitions that are already in the system. This can be used to replace existing pipeline definitions or to create new ones. The pipeline substitutions are used only within this request.

Path parameters

  • index string Required

    The index to simulate ingesting into. This value can be overridden by specifying an index on each document. If you specify this parameter in the request path, it is used for any documents that do not explicitly specify an index argument.

Query parameters

  • pipeline string

    The pipeline to use as the default pipeline. This value can be used to override the default pipeline of the index.

application/json

Body Required

  • docs array[object] Required

    Sample documents to test in the pipeline.

    Hide docs attributes Show docs attributes object
  • component_template_substitutions object

    A map of component template names to substitute component template definition objects.

    Hide component_template_substitutions attribute Show component_template_substitutions attribute object
  • index_template_substitutions object

    A map of index template names to substitute index template definition objects.

    Hide index_template_substitutions attribute Show index_template_substitutions attribute object
  • mapping_addition object
    Hide mapping_addition attributes Show mapping_addition attributes object
    • all_field object
      Hide all_field attributes Show all_field attributes object
    • date_detection boolean
    • dynamic string

      Values are strict, runtime, true, or false.

    • dynamic_date_formats array[string]
    • dynamic_templates array[object]
    • _field_names object
      Hide _field_names attribute Show _field_names attribute object
    • index_field object
      Hide index_field attribute Show index_field attribute object
    • _meta object
      Hide _meta attribute Show _meta attribute object
      • * object Additional properties
    • numeric_detection boolean
    • properties object
    • _routing object
      Hide _routing attribute Show _routing attribute object
    • _size object
      Hide _size attribute Show _size attribute object
    • _source object
      Hide _source attributes Show _source attributes object
    • runtime object
      Hide runtime attribute Show runtime attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • fields object

          For type composite

          Hide fields attribute Show fields attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
            • type string Required

              Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

        • fetch_fields array[object]

          For type lookup

          Hide fetch_fields attributes Show fetch_fields attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • format string
        • format string

          A custom format for date type runtime fields.

        • input_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_index string
        • script object
          Hide script attributes Show script attributes object

          • source string | object

            One of:
            ScriptSource string SearchRequestBody object
          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            Hide params attribute Show params attribute object
            • * object Additional properties

          • lang string

            Any of:
            ScriptLanguage string ScriptLanguage string

            Values are painless, expression, mustache, or java.

          • options object
            Hide options attribute Show options attribute object
            • * string Additional properties
        • type string Required

          Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

    • enabled boolean
    • subobjects string

      Values are true or false.

    • _data_stream_timestamp object
      Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
  • pipeline_substitutions object

    Pipelines to test. If you don’t specify the pipeline request path parameter, this parameter is required. If you specify both this and the request path parameter, the API only uses the request path parameter.

    Hide pipeline_substitutions attribute Show pipeline_substitutions attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • description string

        Description of the ingest pipeline.

      • on_failure array[object]

        Processors to run immediately after a processor failure.

        Hide on_failure attributes Show on_failure attributes object
        • append object
          Hide append attributes Show append attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • value
          • allow_duplicates boolean

            If false, the processor does not append values already present in the field.

        • attachment object
          Hide attachment attributes Show attachment attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • indexed_chars number

            The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

          • indexed_chars_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • remove_binary boolean

            If true, the binary field will be removed from the document

          • resource_name string

            Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

        • bytes object
          Hide bytes attributes Show bytes attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • circle object
          Hide circle attributes Show circle attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • error_distance number Required

            The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • shape_type string Required

            Values are geo_shape or shape.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • community_id object
          Hide community_id attributes Show community_id attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • source_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • source_port string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_port string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • iana_number string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • icmp_type string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • icmp_code string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • transport string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • seed number

            Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • convert object
          Hide convert attributes Show convert attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • type string Required

            Values are integer, long, double, float, boolean, ip, string, or auto.

        • csv object
          Hide csv attributes Show csv attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • empty_value object

            Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • quote string

            Quote used in CSV, has to be single character string.

          • separator string

            Separator used in CSV, has to be single character string.

          • target_fields string | array[string] Required
          • trim boolean

            Trim whitespaces in unquoted fields.

        • date object
          Hide date attributes Show date attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • formats array[string] Required

            An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

          • locale string

            The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • timezone string

            The timezone to use when parsing the date. Supports template snippets.

          • output_format string

            The format to use when writing the date to target_field. Must be a valid java time pattern.

        • date_index_name object
          Hide date_index_name attributes Show date_index_name attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • date_formats array[string]

            An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

          • date_rounding string Required

            How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • index_name_format string

            The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

          • index_name_prefix string

            A prefix of the index name to be prepended before the printed date. Supports template snippets.

          • locale string

            The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

          • timezone string

            The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

        • dissect object
          Hide dissect attributes Show dissect attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • append_separator string

            The character(s) that separate the appended fields.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern string Required

            The pattern to apply to the field.

        • dot_expander object
          Hide dot_expander attributes Show dot_expander attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • override boolean

            Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

          • path string

            The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

        • drop object
          Hide drop attributes Show drop attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

        • enrich object
          Hide enrich attributes Show enrich attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • max_matches number

            The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

          • override boolean

            If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

          • policy_name string Required

            The name of the enrich policy to use.

          • shape_relation string

            Values are intersects, disjoint, within, or contains.

          • target_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • fail object
          Hide fail attributes Show fail attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • message string Required

            The error message thrown by the processor. Supports template snippets.

        • fingerprint object
          Hide fingerprint attributes Show fingerprint attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • fields string | array[string] Required
          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • salt string

            Salt value for the hash function.

          • method string

            Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

          • ignore_missing boolean

            If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

        • foreach object
          Hide foreach attributes Show foreach attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true, the processor silently exits without changing the document if the field is null or missing.

          • processor object Required
        • ip_location object
          Hide ip_location attributes Show ip_location attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • database_file string

            The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • first_only boolean

            If true, only the first found IP location data will be returned, even if the field contains an array.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • properties array[string]

            Controls what properties are added to the target_field based on the IP location lookup.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • download_database_on_pipeline_creation boolean

            If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

        • geo_grid object
          Hide geo_grid attributes Show geo_grid attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            The field to interpret as a geo-tile.= The field format is determined by the tile_type.

          • tile_type string Required

            Values are geotile, geohex, or geohash.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • parent_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • children_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • non_children_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • precision_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_format string

            Values are geojson or wkt.

        • geoip object
          Hide geoip attributes Show geoip attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • database_file string

            The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • first_only boolean

            If true, only the first found geoip data will be returned, even if the field contains an array.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • properties array[string]

            Controls what properties are added to the target_field based on the geoip lookup.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • download_database_on_pipeline_creation boolean

            If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

        • grok object
          Hide grok attributes Show grok attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • ecs_compatibility string

            Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern_definitions object

            A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

          • patterns array[string] Required

            An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

          • trace_match boolean

            When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

        • gsub object
          Hide gsub attributes Show gsub attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern string Required

            The pattern to be replaced.

          • replacement string Required

            The string to replace the matching patterns with.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • html_strip object
          Hide html_strip attributes Show html_strip attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document,

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • inference object
          Hide inference attributes Show inference attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • model_id string Required
          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field_map object

            Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

          • inference_config object
          • input_output
          • ignore_missing boolean

            If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.

        • join object
          Hide join attributes Show join attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • separator string Required

            The separator character.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • json object
          Hide json attributes Show json attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • add_to_root boolean

            Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

          • add_to_root_conflict_strategy string

            Values are replace or merge.

          • allow_duplicate_keys boolean

            When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • kv object
          Hide kv attributes Show kv attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • exclude_keys array[string]

            List of keys to exclude from document.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field_split string Required

            Regex pattern to use for splitting key-value pairs.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • include_keys array[string]

            List of keys to filter and insert into document. Defaults to including all keys.

          • prefix string

            Prefix to be added to extracted keys.

          • strip_brackets boolean

            If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • trim_key string

            String of characters to trim from extracted keys.

          • trim_value string

            String of characters to trim from extracted values.

          • value_split string Required

            Regex pattern to use for splitting the key from the value within a key-value pair.

        • lowercase object
          Hide lowercase attributes Show lowercase attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • network_direction object
          Hide network_direction attributes Show network_direction attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • source_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • internal_networks array[string]

            List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

          • internal_networks_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • pipeline object
          Hide pipeline attributes Show pipeline attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • name string Required
          • ignore_missing_pipeline boolean

            Whether to ignore missing pipelines instead of failing.

        • redact object
          Hide redact attributes Show redact attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • patterns array[string] Required

            A list of grok expressions to match and redact named captures with

          • pattern_definitions object
          • prefix string

            Start a redacted section with this token

          • suffix string

            End a redacted section with this token

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • skip_if_unlicensed boolean

            If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

          • trace_redact boolean

            If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

        • registered_domain object
          Hide registered_domain attributes Show registered_domain attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • remove object
          Hide remove attributes Show remove attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string | array[string] Required
          • keep string | array[string]
          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • rename object
          Hide rename attributes Show rename attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • reroute object
          Hide reroute attributes Show reroute attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • destination string

            A static value for the target. Can’t be set when the dataset or namespace option is set.

          • dataset
          • namespace
        • script object
          Hide script attributes Show script attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • id string
          • lang
          • params object

            Object containing parameters for the script.

          • source
        • set object
          Hide set attributes Show set attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • copy_from string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_empty_value boolean

            If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

          • media_type string

            The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

          • override boolean

            If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

          • value object

            The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

        • set_security_user object
          Hide set_security_user attributes Show set_security_user attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Controls what user related properties are added to the field.

        • sort object
          Hide sort attributes Show sort attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • order string

            Values are asc or desc.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • split object
          Hide split attributes Show split attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • preserve_trailing boolean

            Preserves empty trailing fields, if any.

          • separator string Required

            A regex which matches the separator, for example, , or \s+.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • terminate object
          Hide terminate attributes Show terminate attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

        • trim object
          Hide trim attributes Show trim attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • uppercase object
          Hide uppercase attributes Show uppercase attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • urldecode object
          Hide urldecode attributes Show urldecode attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • uri_parts object
          Hide uri_parts attributes Show uri_parts attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • keep_original boolean

            If true, the processor copies the unparsed URI to <target_field>.original.

          • remove_if_successful boolean

            If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • user_agent object
          Hide user_agent attributes Show user_agent attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • regex_file string

            The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Controls what properties are added to target_field.

            Values are name, os, device, original, or version.

          • extract_device_type boolean Beta

            Extracts device type from the user agent string on a best-effort basis.

      • processors array[object]

        Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.

        Hide processors attributes Show processors attributes object
        • append object
          Hide append attributes Show append attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • value
          • allow_duplicates boolean

            If false, the processor does not append values already present in the field.

        • attachment object
          Hide attachment attributes Show attachment attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • indexed_chars number

            The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

          • indexed_chars_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • remove_binary boolean

            If true, the binary field will be removed from the document

          • resource_name string

            Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

        • bytes object
          Hide bytes attributes Show bytes attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • circle object
          Hide circle attributes Show circle attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • error_distance number Required

            The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • shape_type string Required

            Values are geo_shape or shape.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • community_id object
          Hide community_id attributes Show community_id attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • source_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • source_port string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_port string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • iana_number string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • icmp_type string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • icmp_code string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • transport string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • seed number

            Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • convert object
          Hide convert attributes Show convert attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • type string Required

            Values are integer, long, double, float, boolean, ip, string, or auto.

        • csv object
          Hide csv attributes Show csv attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • empty_value object

            Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • quote string

            Quote used in CSV, has to be single character string.

          • separator string

            Separator used in CSV, has to be single character string.

          • target_fields string | array[string] Required
          • trim boolean

            Trim whitespaces in unquoted fields.

        • date object
          Hide date attributes Show date attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • formats array[string] Required

            An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

          • locale string

            The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • timezone string

            The timezone to use when parsing the date. Supports template snippets.

          • output_format string

            The format to use when writing the date to target_field. Must be a valid java time pattern.

        • date_index_name object
          Hide date_index_name attributes Show date_index_name attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • date_formats array[string]

            An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

          • date_rounding string Required

            How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • index_name_format string

            The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

          • index_name_prefix string

            A prefix of the index name to be prepended before the printed date. Supports template snippets.

          • locale string

            The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

          • timezone string

            The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

        • dissect object
          Hide dissect attributes Show dissect attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • append_separator string

            The character(s) that separate the appended fields.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern string Required

            The pattern to apply to the field.

        • dot_expander object
          Hide dot_expander attributes Show dot_expander attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • override boolean

            Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

          • path string

            The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

        • drop object
          Hide drop attributes Show drop attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

        • enrich object
          Hide enrich attributes Show enrich attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • max_matches number

            The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

          • override boolean

            If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

          • policy_name string Required

            The name of the enrich policy to use.

          • shape_relation string

            Values are intersects, disjoint, within, or contains.

          • target_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • fail object
          Hide fail attributes Show fail attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • message string Required

            The error message thrown by the processor. Supports template snippets.

        • fingerprint object
          Hide fingerprint attributes Show fingerprint attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • fields string | array[string] Required
          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • salt string

            Salt value for the hash function.

          • method string

            Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

          • ignore_missing boolean

            If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

        • foreach object
          Hide foreach attributes Show foreach attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true, the processor silently exits without changing the document if the field is null or missing.

          • processor object Required
        • ip_location object
          Hide ip_location attributes Show ip_location attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • database_file string

            The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • first_only boolean

            If true, only the first found IP location data will be returned, even if the field contains an array.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • properties array[string]

            Controls what properties are added to the target_field based on the IP location lookup.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • download_database_on_pipeline_creation boolean

            If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

        • geo_grid object
          Hide geo_grid attributes Show geo_grid attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            The field to interpret as a geo-tile.= The field format is determined by the tile_type.

          • tile_type string Required

            Values are geotile, geohex, or geohash.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • parent_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • children_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • non_children_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • precision_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_format string

            Values are geojson or wkt.

        • geoip object
          Hide geoip attributes Show geoip attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • database_file string

            The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • first_only boolean

            If true, only the first found geoip data will be returned, even if the field contains an array.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • properties array[string]

            Controls what properties are added to the target_field based on the geoip lookup.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • download_database_on_pipeline_creation boolean

            If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

        • grok object
          Hide grok attributes Show grok attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • ecs_compatibility string

            Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern_definitions object

            A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

          • patterns array[string] Required

            An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

          • trace_match boolean

            When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

        • gsub object
          Hide gsub attributes Show gsub attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern string Required

            The pattern to be replaced.

          • replacement string Required

            The string to replace the matching patterns with.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • html_strip object
          Hide html_strip attributes Show html_strip attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document,

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • inference object
          Hide inference attributes Show inference attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • model_id string Required
          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field_map object

            Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

          • inference_config object
          • input_output
          • ignore_missing boolean

            If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.

        • join object
          Hide join attributes Show join attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • separator string Required

            The separator character.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • json object
          Hide json attributes Show json attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • add_to_root boolean

            Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

          • add_to_root_conflict_strategy string

            Values are replace or merge.

          • allow_duplicate_keys boolean

            When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • kv object
          Hide kv attributes Show kv attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • exclude_keys array[string]

            List of keys to exclude from document.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field_split string Required

            Regex pattern to use for splitting key-value pairs.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • include_keys array[string]

            List of keys to filter and insert into document. Defaults to including all keys.

          • prefix string

            Prefix to be added to extracted keys.

          • strip_brackets boolean

            If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • trim_key string

            String of characters to trim from extracted keys.

          • trim_value string

            String of characters to trim from extracted values.

          • value_split string Required

            Regex pattern to use for splitting the key from the value within a key-value pair.

        • lowercase object
          Hide lowercase attributes Show lowercase attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • network_direction object
          Hide network_direction attributes Show network_direction attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • source_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • internal_networks array[string]

            List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

          • internal_networks_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • pipeline object
          Hide pipeline attributes Show pipeline attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • name string Required
          • ignore_missing_pipeline boolean

            Whether to ignore missing pipelines instead of failing.

        • redact object
          Hide redact attributes Show redact attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • patterns array[string] Required

            A list of grok expressions to match and redact named captures with

          • pattern_definitions object
          • prefix string

            Start a redacted section with this token

          • suffix string

            End a redacted section with this token

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • skip_if_unlicensed boolean

            If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

          • trace_redact boolean

            If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

        • registered_domain object
          Hide registered_domain attributes Show registered_domain attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • remove object
          Hide remove attributes Show remove attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string | array[string] Required
          • keep string | array[string]
          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • rename object
          Hide rename attributes Show rename attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • reroute object
          Hide reroute attributes Show reroute attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • destination string

            A static value for the target. Can’t be set when the dataset or namespace option is set.

          • dataset
          • namespace
        • script object
          Hide script attributes Show script attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • id string
          • lang
          • params object

            Object containing parameters for the script.

          • source
        • set object
          Hide set attributes Show set attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • copy_from string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_empty_value boolean

            If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

          • media_type string

            The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

          • override boolean

            If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

          • value object

            The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

        • set_security_user object
          Hide set_security_user attributes Show set_security_user attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Controls what user related properties are added to the field.

        • sort object
          Hide sort attributes Show sort attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • order string

            Values are asc or desc.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • split object
          Hide split attributes Show split attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • preserve_trailing boolean

            Preserves empty trailing fields, if any.

          • separator string Required

            A regex which matches the separator, for example, , or \s+.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • terminate object
          Hide terminate attributes Show terminate attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

        • trim object
          Hide trim attributes Show trim attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • uppercase object
          Hide uppercase attributes Show uppercase attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • urldecode object
          Hide urldecode attributes Show urldecode attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • uri_parts object
          Hide uri_parts attributes Show uri_parts attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • keep_original boolean

            If true, the processor copies the unparsed URI to <target_field>.original.

          • remove_if_successful boolean

            If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • user_agent object
          Hide user_agent attributes Show user_agent attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • regex_file string

            The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Controls what properties are added to target_field.

            Values are name, os, device, original, or version.

          • extract_device_type boolean Beta

            Extracts device type from the user agent string on a best-effort basis.

      • version number
      • deprecated boolean

        Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.

      • _meta object
        Hide _meta attribute Show _meta attribute object
        • * object Additional properties

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • docs array[object] Required
      Hide docs attribute Show docs attribute object
      • doc object
        Hide doc attributes Show doc attributes object
        • _id string Required
        • _index string Required
        • _source object Required

          JSON body for the document.

          Hide _source attribute Show _source attribute object
          • * object Additional properties

        • _version number | string Required

          Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

          Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

          One of:
          VersionNumber number StringifiedVersionNumber string
        • executed_pipelines array[string] Required

          A list of the names of the pipelines executed on this document.

        • ignored_fields array[object]

          A list of the fields that would be ignored at the indexing step. For example, a field whose value is larger than the allowed limit would make it through all of the pipelines, but would not be indexed into Elasticsearch.

          Hide ignored_fields attribute Show ignored_fields attribute object
          • * string Additional properties
        • error object
          Hide error attributes Show error attributes object
GET /_ingest/{index}/_simulate 
curl \
 --request GET 'http://api.example.com/_ingest/{index}/_simulate' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"docs\": [\n    {\n      \"_id\": 123,\n      \"_index\": \"my-index\",\n      \"_source\": {\n        \"foo\": \"bar\"\n      }\n    },\n    {\n      \"_id\": 456,\n      \"_index\": \"my-index\",\n      \"_source\": {\n        \"foo\": \"rab\"\n      }\n    }\n  ]\n}"'
Request examples
In this example the index `my-index` has a default pipeline called `my-pipeline` and a final pipeline called `my-final-pipeline`. Since both documents are being ingested into `my-index`, both pipelines are run using the pipeline definitions that are already in the system. 
{
  "docs": [
    {
      "_id": 123,
      "_index": "my-index",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_id": 456,
      "_index": "my-index",
      "_source": {
        "foo": "rab"
      }
    }
  ]
}
In this example the index `my-index` has a default pipeline called `my-pipeline` and a final pipeline called `my-final-pipeline`. But a substitute definition of `my-pipeline` is provided in `pipeline_substitutions`. The substitute `my-pipeline` will be used in place of the `my-pipeline` that is in the system, and then the `my-final-pipeline` that is already defined in the system will run. 
{
  "docs": [
    {
      "_index": "my-index",
      "_id": 123,
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_index": "my-index",
      "_id": 456,
      "_source": {
        "foo": "rab"
      }
    }
  ],
  "pipeline_substitutions": {
    "my-pipeline": {
      "processors": [
        {
          "uppercase": {
            "field": "foo"
          }
        }
      ]
    }
  }
}
In this example, imagine that the index `my-index` has a strict mapping with only the `foo` keyword field defined. Say that field mapping came from a component template named `my-mappings-template`. You want to test adding a new field, `bar`. So a substitute definition of `my-mappings-template` is provided in `component_template_substitutions`. The substitute `my-mappings-template` will be used in place of the existing mapping for `my-index` and in place of the `my-mappings-template` that is in the system. 
{
  "docs": [
    {
      "_index": "my-index",
      "_id": "123",
      "_source": {
        "foo": "foo"
      }
    },
    {
      "_index": "my-index",
      "_id": "456",
      "_source": {
        "bar": "rab"
      }
    }
  ],
  "component_template_substitutions": {
    "my-mappings_template": {
      "template": {
        "mappings": {
          "dynamic": "strict",
          "properties": {
            "foo": {
              "type": "keyword"
            },
            "bar": {
              "type": "keyword"
            }
          }
        }
      }
    }
  }
}
The pipeline, component template, and index template substitutions replace the existing pipeline details for the duration of this request. 
{
  "docs": [
    {
      "_id": "id",
      "_index": "my-index",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_id": "id",
      "_index": "my-index",
      "_source": {
        "foo": "rab"
      }
    }
  ],
  "pipeline_substitutions": {
    "my-pipeline": {
      "processors": [
        {
          "set": {
            "field": "field3",
            "value": "value3"
          }
        }
      ]
    }
  },
  "component_template_substitutions": {
    "my-component-template": {
      "template": {
        "mappings": {
          "dynamic": true,
          "properties": {
            "field3": {
              "type": "keyword"
            }
          }
        },
        "settings": {
          "index": {
            "default_pipeline": "my-pipeline"
          }
        }
      }
    }
  },
  "index_template_substitutions": {
    "my-index-template": {
      "index_patterns": [
        "my-index-*"
      ],
      "composed_of": [
        "component_template_1",
        "component_template_2"
      ]
    }
  },
  "mapping_addition": {
    "dynamic": "strict",
    "properties": {
      "foo": {
        "type": "keyword"
      }
    }
  }
}
Response examples (200)
A successful response when the simulation uses pipeline definitions that are already in the system. 
{
  "docs": [
    {
      "doc": null,
      "_id": 123,
      "_index": "my-index",
      "_version": -3,
      "_source": {
        "field1": "value1",
        "field2": "value2",
        "foo": "bar"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    },
    {
      "doc": null,
      "_id": 456,
      "_index": "my-index",
      "_version": "-3,",
      "_source": {
        "field1": "value1",
        "field2": "value2",
        "foo": "rab"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    }
  ]
}
A successful response when the simulation uses pipeline substitutions. 
{
  "docs": [
    {
      "doc": null,
      "_id": 123,
      "_index": "my-index",
      "_version": -3,
      "_source": {
        "field2": "value2",
        "foo": "BAR"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    },
    {
      "doc": null,
      "_id": 456,
      "_index": "my-index",
      "_version": -3,
      "_source": {
        "field2": "value2",
        "foo": "RAB"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    }
  ]
}
A successful response when the simulation uses pipeline substitutions. 
{
  "docs": [
    {
      "doc": {
        "_id": "123",
        "_index": "my-index",
        "_version": -3,
        "_source": {
          "foo": "foo"
        },
        "executed_pipelines": []
      }
    },
    {
      "doc": {
        "_id": "456",
        "_index": "my-index",
        "_version": -3,
        "_source": {
          "bar": "rab"
        },
      "executed_pipelines": []
      }
    }
  ]
}

Licensing

Licensing APIs enable you to manage your licenses.

For more information about the different types of licenses, refer to Elastic subscriptions

Get Logstash pipelines Added in 7.12.0

GET /_logstash/pipeline/{id}
Api key auth Basic auth Bearer auth

Get pipelines that are used for Logstash Central Management.

External documentation

Path parameters

  • id string | array[string] Required

    A comma-separated list of pipeline identifiers.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • description string Required

        A description of the pipeline. This description is not used by Elasticsearch or Logstash.

      • last_modified string | number Required

        A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

        One of:
        DateTime string UnitMillis number
      • pipeline string Required

        The configuration for the pipeline.

        External documentation
      • pipeline_metadata object Required
        Hide pipeline_metadata attributes Show pipeline_metadata attributes object
      • pipeline_settings object Required
        Hide pipeline_settings attributes Show pipeline_settings attributes object
        • pipeline.workers number Required

          The number of workers that will, in parallel, execute the filter and output stages of the pipeline.

        • pipeline.batch.size number Required

          The maximum number of events an individual worker thread will collect from inputs before attempting to execute its filters and outputs.

        • pipeline.batch.delay number Required

          When creating pipeline event batches, how long in milliseconds to wait for each event before dispatching an undersized batch to pipeline workers.

        • queue.type string Required

          The internal queuing model to use for event buffering.

        • queue.max_bytes string Required

          The total capacity of the queue (queue.type: persisted) in number of bytes.

        • queue.checkpoint.writes number Required

          The maximum number of written events before forcing a checkpoint when persistent queues are enabled (queue.type: persisted).

      • username string Required

        The user who last updated the pipeline.
GET /_logstash/pipeline/{id} 
curl \
 --request GET 'http://api.example.com/_logstash/pipeline/{id}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _logstash/pipeline/my_pipeline`. 
{
  "my_pipeline": {
    "description": "Sample pipeline for illustration purposes",
    "last_modified": "2021-01-02T02:50:51.250Z",
    "pipeline_metadata": {
      "type": "logstash_pipeline",
      "version": "1"
    },
    "username": "elastic",
    "pipeline": "input {}\\n filter { grok {} }\\n output {}",
    "pipeline_settings": {
      "pipeline.workers": 1,
      "pipeline.batch.size": 125,
      "pipeline.batch.delay": 50,
      "queue.type": "memory",
      "queue.max_bytes": "1gb",
      "queue.checkpoint.writes": 1024
    }
  }
}

Close anomaly detection jobs Added in 5.4.0

POST /_ml/anomaly_detectors/{job_id}/_close
Api key auth Basic auth Bearer auth

A job can be opened and closed multiple times throughout its lifecycle. A closed job cannot receive data or perform analysis operations, but you can still explore and navigate results. When you close a job, it runs housekeeping tasks such as pruning the model history, flushing buffers, calculating final results and persisting the model snapshots. Depending upon the size of the job, it could take several minutes to close and the equivalent time to re-open. After it is closed, the job has a minimal overhead on the cluster except for maintaining its meta data. Therefore it is a best practice to close jobs that are no longer required to process data. If you close an anomaly detection job whose datafeed is running, the request first tries to stop the datafeed. This behavior is equivalent to calling stop datafeed API with the same timeout and force parameters as the close job request. When a datafeed that has a specified end date stops, it automatically closes its associated job.

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job. It can be a job identifier, a group name, or a wildcard expression. You can close multiple anomaly detection jobs in a single API request by using a group name, a comma-separated list of jobs, or a wildcard expression. You can close all jobs by using _all or by specifying * as the job identifier.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request: contains wildcard expressions and there are no jobs that match; contains the _all string or no identifiers and there are no matches; or contains wildcard expressions and there are only partial matches. By default, it returns an empty jobs array when there are no matches and the subset of results when there are partial matches. If false, the request returns a 404 status code when there are no matches or only partial matches.

  • force boolean

    Use to close a failed job, or to forcefully close a job which has not responded to its initial close request; the request returns without performing the associated actions such as flushing buffers and persisting the model snapshots. If you want the job to be in a consistent state after the close job API returns, do not set to true. This parameter should be used only in situations where the job has already failed or where you are not interested in results the job might have recently produced or might produce in the future.

  • timeout string

    Controls the time to wait until a job has closed.

    Values are -1 or 0.

application/json

Body

  • allow_no_match boolean

    Refer to the description for the allow_no_match query parameter.

  • force boolean

    Refer to the descriptiion for the force query parameter.

  • timeout string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
POST /_ml/anomaly_detectors/{job_id}/_close 
curl \
 --request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/_close' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"allow_no_match":true,"force":true,"timeout":"string"}'
Response examples (200)
A successful response when closing anomaly detection jobs. 
{
  "closed": true
}

Get calendar configuration info Added in 6.2.0

GET /_ml/calendars
Api key auth Basic auth Bearer auth

Query parameters

  • from number

    Skips the specified number of calendars. This parameter is supported only when you omit the calendar identifier.

  • size number

    Specifies the maximum number of calendars to obtain. This parameter is supported only when you omit the calendar identifier.

application/json

Body

  • page object
    Hide page attributes Show page attributes object
    • from number

      Skips the specified number of items.

    • size number

      Specifies the maximum number of items to obtain.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • calendars array[object] Required
      Hide calendars attributes Show calendars attributes object
    • count number Required
GET /_ml/calendars 
curl \
 --request GET 'http://api.example.com/_ml/calendars' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"page":{"from":42.0,"size":42.0}}'

Get anomaly detection job model snapshot upgrade usage info Added in 7.16.0

GET /_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}/_upgrade/_stats
Api key auth Basic auth Bearer auth

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job.

  • snapshot_id string Required

    A numerical character string that uniquely identifies the model snapshot. You can get information for multiple snapshots by using a comma-separated list or a wildcard expression. You can get all snapshots by using _all, by specifying * as the snapshot ID, or by omitting the snapshot ID.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    • Contains wildcard expressions and there are no jobs that match.
    • Contains the _all string or no identifiers and there are no matches.
    • Contains wildcard expressions and there are only partial matches.

    The default value is true, which returns an empty jobs array when there are no matches and the subset of results when there are partial matches. If this parameter is false, the request returns a 404 status code when there are no matches or only partial matches.

Responses

GET /_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}/_upgrade/_stats 
curl \
 --request GET 'http://api.example.com/_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}/_upgrade/_stats' \
 --header "Authorization: $API_KEY"

Open anomaly detection jobs Added in 5.4.0

POST /_ml/anomaly_detectors/{job_id}/_open
Api key auth Basic auth Bearer auth

An anomaly detection job must be opened to be ready to receive and analyze data. It can be opened and closed multiple times throughout its lifecycle. When you open a new job, it starts with an empty model. When you open an existing job, the most recent model state is automatically loaded. The job is ready to resume its analysis from where it left off, once new data is received.

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job.

Query parameters

  • timeout string

    Controls the time to wait until a job has opened.

    Values are -1 or 0.

application/json

Body

  • timeout string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
POST /_ml/anomaly_detectors/{job_id}/_open 
curl \
 --request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/_open' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"timeout\": \"35m\"\n}"'
Request example
A request to open anomaly detection jobs. The timeout specifies to wait 35 minutes for the job to open. 
{
  "timeout": "35m"
}
Response examples (200)
A successful response when opening an anomaly detection job. 
{
  "opened": true,
  "node": "node-1"
}

Send data to an anomaly detection job for analysis Deprecated Added in 5.4.0

POST /_ml/anomaly_detectors/{job_id}/_data
Api key auth Basic auth Bearer auth

IMPORTANT: For each job, data can be accepted from only a single connection at a time. It is not currently possible to post data to multiple jobs using wildcards or a comma-separated list.

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job. The job must have a state of open to receive and process the data.

Query parameters

  • reset_end string | number

    Specifies the end of the bucket resetting range.

  • reset_start string | number

    Specifies the start of the bucket resetting range.

application/json

Body Required

object object

Responses

POST /_ml/anomaly_detectors/{job_id}/_data 
curl \
 --request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/_data' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '[{}]'

Machine learning data frame analytics

Learn more about data frame analytics

Preview features used by data frame analytics Added in 7.13.0

GET /_ml/data_frame/analytics/_preview
Api key auth Basic auth Bearer auth

Preview the extracted features used by a data frame analytics config.

application/json

Body

  • config object
    Hide config attributes Show config attributes object
    • source object Required
      Hide source attributes Show source attributes object
      • index string | array[string] Required
      • runtime_mappings object
        Hide runtime_mappings attribute Show runtime_mappings attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • fields object

            For type composite

            Hide fields attribute Show fields attribute object
            • * object Additional properties
              Hide * attribute Show * attribute object
              • type string Required

                Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

          • fetch_fields array[object]

            For type lookup

            Hide fetch_fields attributes Show fetch_fields attributes object
            • field string Required

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • format string
          • format string

            A custom format for date type runtime fields.

          • input_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_index string
          • script object
            Hide script attributes Show script attributes object
          • type string Required

            Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

      • _source object
        Hide _source attributes Show _source attributes object
        • includes array[string]

          An array of strings that defines the fields that will be excluded from the analysis. You do not need to add fields with unsupported data types to excludes, these fields are excluded from the analysis automatically.

        • excludes array[string]

          An array of strings that defines the fields that will be included in the analysis.

      • query object

        The Elasticsearch query domain-specific language (DSL). This value corresponds to the query object in an Elasticsearch search POST body. All the options that are supported by Elasticsearch can be used, as this object is passed verbatim to Elasticsearch. By default, this property has the following value: {"match_all": {}}.

        Query DSL
    • analysis object Required
      Hide analysis attributes Show analysis attributes object
      • classification object
        Hide classification attributes Show classification attributes object
        • alpha number

          Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This parameter affects loss calculations by acting as a multiplier of the tree depth. Higher alpha values result in shallower trees and faster training times. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to zero.

        • dependent_variable string Required

          Defines which field of the document is to be predicted. It must match one of the fields in the index being used to train. If this field is missing from a document, then that document will not be used for training, but a prediction with the trained model will be generated for it. It is also known as continuous target variable. For classification analysis, the data type of the field must be numeric (integer, short, long, byte), categorical (ip or keyword), or boolean. There must be no more than 30 different values in this field. For regression analysis, the data type of the field must be numeric.

        • downsample_factor number

          Advanced configuration option. Controls the fraction of data that is used to compute the derivatives of the loss function for tree training. A small value results in the use of a small fraction of the data. If this value is set to be less than 1, accuracy typically improves. However, too small a value may result in poor convergence for the ensemble and so require more trees. By default, this value is calculated during hyperparameter optimization. It must be greater than zero and less than or equal to 1.

        • early_stopping_enabled boolean

          Advanced configuration option. Specifies whether the training process should finish if it is not finding any better performing models. If disabled, the training process can take significantly longer and the chance of finding a better performing model is unremarkable.

        • eta number

          Advanced configuration option. The shrinkage applied to the weights. Smaller values result in larger forests which have a better generalization error. However, larger forests cause slower training. By default, this value is calculated during hyperparameter optimization. It must be a value between 0.001 and 1.

        • eta_growth_rate_per_tree number

          Advanced configuration option. Specifies the rate at which eta increases for each new tree that is added to the forest. For example, a rate of 1.05 increases eta by 5% for each extra tree. By default, this value is calculated during hyperparameter optimization. It must be between 0.5 and 2.

        • feature_bag_fraction number

          Advanced configuration option. Defines the fraction of features that will be used when selecting a random bag for each candidate split. By default, this value is calculated during hyperparameter optimization.

        • feature_processors array[object]

          Advanced configuration option. A collection of feature preprocessors that modify one or more included fields. The analysis uses the resulting one or more features instead of the original document field. However, these features are ephemeral; they are not stored in the destination index. Multiple feature_processors entries can refer to the same document fields. Automatic categorical feature encoding still occurs for the fields that are unprocessed by a custom processor or that have categorical values. Use this property only if you want to override the automatic feature encoding of the specified fields.

          Hide feature_processors attributes Show feature_processors attributes object
        • gamma number

          Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies a linear penalty associated with the size of individual trees in the forest. A high gamma value causes training to prefer small trees. A small gamma value results in larger individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

        • lambda number

          Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies an L2 regularization term which applies to leaf weights of the individual trees in the forest. A high lambda value causes training to favor small leaf weights. This behavior makes the prediction function smoother at the expense of potentially not being able to capture relevant relationships between the features and the dependent variable. A small lambda value results in large individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

        • max_optimization_rounds_per_hyperparameter number

          Advanced configuration option. A multiplier responsible for determining the maximum number of hyperparameter optimization steps in the Bayesian optimization procedure. The maximum number of steps is determined based on the number of undefined hyperparameters times the maximum optimization rounds per hyperparameter. By default, this value is calculated during hyperparameter optimization.

        • max_trees number

          Advanced configuration option. Defines the maximum number of decision trees in the forest. The maximum value is 2000. By default, this value is calculated during hyperparameter optimization.

        • num_top_feature_importance_values number

          Advanced configuration option. Specifies the maximum number of feature importance values per document to return. By default, no feature importance calculation occurs.

        • prediction_field_name string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • randomize_seed number

          Defines the seed for the random generator that is used to pick training data. By default, it is randomly generated. Set it to a specific value to use the same training data each time you start a job (assuming other related parameters such as source and analyzed_fields are the same).

        • soft_tree_depth_limit number

          Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This soft limit combines with the soft_tree_depth_tolerance to penalize trees that exceed the specified depth; the regularized loss increases quickly beyond this depth. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.

        • soft_tree_depth_tolerance number

          Advanced configuration option. This option controls how quickly the regularized loss increases when the tree depth exceeds soft_tree_depth_limit. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.01.

        • training_percent string | number

          One of:
          Percentage string Percentage number
        • class_assignment_objective string
        • num_top_classes number

          Defines the number of categories for which the predicted probabilities are reported. It must be non-negative or -1. If it is -1 or greater than the total number of categories, probabilities are reported for all categories; if you have a large number of categories, there could be a significant effect on the size of your destination index. NOTE: To use the AUC ROC evaluation method, num_top_classes must be set to -1 or a value greater than or equal to the total number of categories.

      • outlier_detection object
        Hide outlier_detection attributes Show outlier_detection attributes object
        • compute_feature_influence boolean

          Specifies whether the feature influence calculation is enabled.

        • feature_influence_threshold number

          The minimum outlier score that a document needs to have in order to calculate its feature influence score. Value range: 0-1.

        • method string

          The method that outlier detection uses. Available methods are lof, ldof, distance_kth_nn, distance_knn, and ensemble. The default value is ensemble, which means that outlier detection uses an ensemble of different methods and normalises and combines their individual outlier scores to obtain the overall outlier score.

        • n_neighbors number

          Defines the value for how many nearest neighbors each method of outlier detection uses to calculate its outlier score. When the value is not set, different values are used for different ensemble members. This default behavior helps improve the diversity in the ensemble; only override it if you are confident that the value you choose is appropriate for the data set.

        • outlier_fraction number

          The proportion of the data set that is assumed to be outlying prior to outlier detection. For example, 0.05 means it is assumed that 5% of values are real outliers and 95% are inliers.

        • standardization_enabled boolean

          If true, the following operation is performed on the columns before computing outlier scores: (x_i - mean(x_i)) / sd(x_i).

      • regression object
        Hide regression attributes Show regression attributes object
        • alpha number

          Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This parameter affects loss calculations by acting as a multiplier of the tree depth. Higher alpha values result in shallower trees and faster training times. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to zero.

        • dependent_variable string Required

          Defines which field of the document is to be predicted. It must match one of the fields in the index being used to train. If this field is missing from a document, then that document will not be used for training, but a prediction with the trained model will be generated for it. It is also known as continuous target variable. For classification analysis, the data type of the field must be numeric (integer, short, long, byte), categorical (ip or keyword), or boolean. There must be no more than 30 different values in this field. For regression analysis, the data type of the field must be numeric.

        • downsample_factor number

          Advanced configuration option. Controls the fraction of data that is used to compute the derivatives of the loss function for tree training. A small value results in the use of a small fraction of the data. If this value is set to be less than 1, accuracy typically improves. However, too small a value may result in poor convergence for the ensemble and so require more trees. By default, this value is calculated during hyperparameter optimization. It must be greater than zero and less than or equal to 1.

        • early_stopping_enabled boolean

          Advanced configuration option. Specifies whether the training process should finish if it is not finding any better performing models. If disabled, the training process can take significantly longer and the chance of finding a better performing model is unremarkable.

        • eta number

          Advanced configuration option. The shrinkage applied to the weights. Smaller values result in larger forests which have a better generalization error. However, larger forests cause slower training. By default, this value is calculated during hyperparameter optimization. It must be a value between 0.001 and 1.

        • eta_growth_rate_per_tree number

          Advanced configuration option. Specifies the rate at which eta increases for each new tree that is added to the forest. For example, a rate of 1.05 increases eta by 5% for each extra tree. By default, this value is calculated during hyperparameter optimization. It must be between 0.5 and 2.

        • feature_bag_fraction number

          Advanced configuration option. Defines the fraction of features that will be used when selecting a random bag for each candidate split. By default, this value is calculated during hyperparameter optimization.

        • feature_processors array[object]

          Advanced configuration option. A collection of feature preprocessors that modify one or more included fields. The analysis uses the resulting one or more features instead of the original document field. However, these features are ephemeral; they are not stored in the destination index. Multiple feature_processors entries can refer to the same document fields. Automatic categorical feature encoding still occurs for the fields that are unprocessed by a custom processor or that have categorical values. Use this property only if you want to override the automatic feature encoding of the specified fields.

          Hide feature_processors attributes Show feature_processors attributes object
        • gamma number

          Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies a linear penalty associated with the size of individual trees in the forest. A high gamma value causes training to prefer small trees. A small gamma value results in larger individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

        • lambda number

          Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies an L2 regularization term which applies to leaf weights of the individual trees in the forest. A high lambda value causes training to favor small leaf weights. This behavior makes the prediction function smoother at the expense of potentially not being able to capture relevant relationships between the features and the dependent variable. A small lambda value results in large individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

        • max_optimization_rounds_per_hyperparameter number

          Advanced configuration option. A multiplier responsible for determining the maximum number of hyperparameter optimization steps in the Bayesian optimization procedure. The maximum number of steps is determined based on the number of undefined hyperparameters times the maximum optimization rounds per hyperparameter. By default, this value is calculated during hyperparameter optimization.

        • max_trees number

          Advanced configuration option. Defines the maximum number of decision trees in the forest. The maximum value is 2000. By default, this value is calculated during hyperparameter optimization.

        • num_top_feature_importance_values number

          Advanced configuration option. Specifies the maximum number of feature importance values per document to return. By default, no feature importance calculation occurs.

        • prediction_field_name string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • randomize_seed number

          Defines the seed for the random generator that is used to pick training data. By default, it is randomly generated. Set it to a specific value to use the same training data each time you start a job (assuming other related parameters such as source and analyzed_fields are the same).

        • soft_tree_depth_limit number

          Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This soft limit combines with the soft_tree_depth_tolerance to penalize trees that exceed the specified depth; the regularized loss increases quickly beyond this depth. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.

        • soft_tree_depth_tolerance number

          Advanced configuration option. This option controls how quickly the regularized loss increases when the tree depth exceeds soft_tree_depth_limit. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.01.

        • training_percent string | number

          One of:
          Percentage string Percentage number
        • loss_function string

          The loss function used during regression. Available options are mse (mean squared error), msle (mean squared logarithmic error), huber (Pseudo-Huber loss).

        • loss_function_parameter number

          A positive number that is used as a parameter to the loss_function.

    • model_memory_limit string
    • max_num_threads number
    • analyzed_fields object
      Hide analyzed_fields attributes Show analyzed_fields attributes object
      • includes array[string]

        An array of strings that defines the fields that will be excluded from the analysis. You do not need to add fields with unsupported data types to excludes, these fields are excluded from the analysis automatically.

      • excludes array[string]

        An array of strings that defines the fields that will be included in the analysis.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • feature_values array[object] Required

      An array of objects that contain feature name and value pairs. The features have been processed and indicate what will be sent to the model for training.

      Hide feature_values attribute Show feature_values attribute object
      • * string Additional properties
GET /_ml/data_frame/analytics/_preview 
curl \
 --request GET 'http://api.example.com/_ml/data_frame/analytics/_preview' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"config":{"source":{"index":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"_source":{"includes":["string"],"excludes":["string"]},"query":{}},"analysis":{"classification":{"alpha":42.0,"dependent_variable":"string","downsample_factor":42.0,"early_stopping_enabled":true,"eta":42.0,"eta_growth_rate_per_tree":42.0,"feature_bag_fraction":42.0,"feature_processors":[{"frequency_encoding":{},"multi_encoding":{},"n_gram_encoding":{},"one_hot_encoding":{},"target_mean_encoding":{}}],"gamma":42.0,"lambda":42.0,"max_optimization_rounds_per_hyperparameter":42.0,"max_trees":42.0,"num_top_feature_importance_values":42.0,"prediction_field_name":"string","randomize_seed":42.0,"soft_tree_depth_limit":42.0,"soft_tree_depth_tolerance":42.0,"":"string","class_assignment_objective":"string","num_top_classes":42.0},"outlier_detection":{"compute_feature_influence":true,"feature_influence_threshold":42.0,"method":"string","n_neighbors":42.0,"outlier_fraction":42.0,"standardization_enabled":true},"regression":{"alpha":42.0,"dependent_variable":"string","downsample_factor":42.0,"early_stopping_enabled":true,"eta":42.0,"eta_growth_rate_per_tree":42.0,"feature_bag_fraction":42.0,"feature_processors":[{"frequency_encoding":{},"multi_encoding":{},"n_gram_encoding":{},"one_hot_encoding":{},"target_mean_encoding":{}}],"gamma":42.0,"lambda":42.0,"max_optimization_rounds_per_hyperparameter":42.0,"max_trees":42.0,"num_top_feature_importance_values":42.0,"prediction_field_name":"string","randomize_seed":42.0,"soft_tree_depth_limit":42.0,"soft_tree_depth_tolerance":42.0,"":"string","loss_function":"string","loss_function_parameter":42.0}},"model_memory_limit":"string","max_num_threads":42.0,"analyzed_fields":{"includes":["string"],"excludes":["string"]}}}'

Stop data frame analytics jobs Added in 7.3.0

POST /_ml/data_frame/analytics/{id}/_stop
Api key auth Basic auth Bearer auth

A data frame analytics job can be started and stopped multiple times throughout its lifecycle.

Path parameters

  • id string Required

    Identifier for the data frame analytics job. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    1. Contains wildcard expressions and there are no data frame analytics jobs that match.
    2. Contains the _all string or no identifiers and there are no matches.
    3. Contains wildcard expressions and there are only partial matches.

    The default value is true, which returns an empty data_frame_analytics array when there are no matches and the subset of results when there are partial matches. If this parameter is false, the request returns a 404 status code when there are no matches or only partial matches.

  • force boolean

    If true, the data frame analytics job is stopped forcefully.

  • timeout string

    Controls the amount of time to wait until the data frame analytics job stops. Defaults to 20 seconds.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
POST /_ml/data_frame/analytics/{id}/_stop 
curl \
 --request POST 'http://api.example.com/_ml/data_frame/analytics/{id}/_stop' \
 --header "Authorization: $API_KEY"

Prepare a node to be shut down Added in 7.13.0

PUT /_nodes/{node_id}/shutdown
Api key auth Basic auth Bearer auth

NOTE: This feature is designed for indirect use by Elastic Cloud, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.

If you specify a node that is offline, it will be prepared for shut down when it rejoins the cluster.

If the operator privileges feature is enabled, you must be an operator to use this API.

The API migrates ongoing tasks and index shards to other nodes as needed to prepare a node to be restarted or shut down and removed from the cluster. This ensures that Elasticsearch can be stopped safely with minimal disruption to the cluster.

You must specify the type of shutdown: restart, remove, or replace. If a node is already being prepared for shutdown, you can use this API to change the shutdown type.

IMPORTANT: This API does NOT terminate the Elasticsearch process. Monitor the node shutdown status to determine when it is safe to stop Elasticsearch.

Path parameters

  • node_id string Required

    The node identifier. This parameter is not validated against the cluster's active nodes. This enables you to register a node for shut down while it is offline. No error is thrown if you specify an invalid node ID.

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are nanos, micros, ms, s, m, h, or d.

  • timeout string

    The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are nanos, micros, ms, s, m, h, or d.

application/json

Body Required

  • type string Required

    Values are restart, remove, or replace.

  • reason string Required

    A human-readable reason that the node is being shut down. This field provides information for other cluster operators; it does not affect the shut down process.

  • allocation_delay string

    Only valid if type is restart. Controls how long Elasticsearch will wait for the node to restart and join the cluster before reassigning its shards to other nodes. This works the same as delaying allocation with the index.unassigned.node_left.delayed_timeout setting. If you specify both a restart allocation delay and an index-level allocation delay, the longer of the two is used.

  • target_node_name string

    Only valid if type is replace. Specifies the name of the node that is replacing the node being shut down. Shards from the shut down node are only allowed to be allocated to the target node, and no other data will be allocated to the target node. During relocation of data certain allocation rules are ignored, such as disk watermarks or user attribute filtering rules.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
PUT /_nodes/{node_id}/shutdown 
curl \
 --request PUT 'http://api.example.com/_nodes/{node_id}/shutdown' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"type\": \"restart\",\n  \"reason\": \"Demonstrating how the node shutdown API works\",\n  \"allocation_delay\": \"20m\"\n}"'
Request example
Register a node for shutdown with `PUT /_nodes/USpTGYaBSIKbgSUJR2Z9lg/shutdown`. The `restart` type prepares the node to be restarted. 
{
  "type": "restart",
  "reason": "Demonstrating how the node shutdown API works",
  "allocation_delay": "20m"
}

Script

Use the script support APIs to get a list of supported script contexts and languages. Use the stored script APIs to manage stored scripts and search templates.

External documentation

Search a vector tile Added in 7.15.0

POST /{index}/_mvt/{field}/{zoom}/{x}/{y}
Api key auth Basic auth Bearer auth

Search a vector tile for geospatial values. Before using this API, you should be familiar with the Mapbox vector tile specification. The API returns results as a binary mapbox vector tile.

Internally, Elasticsearch translates a vector tile search API request into a search containing:

  • A geo_bounding_box query on the <field>. The query uses the <zoom>/<x>/<y> tile as a bounding box.
  • A geotile_grid or geohex_grid aggregation on the <field>. The grid_agg parameter determines the aggregation type. The aggregation uses the <zoom>/<x>/<y> tile as a bounding box.
  • Optionally, a geo_bounds aggregation on the <field>. The search only includes this aggregation if the exact_bounds parameter is true.
  • If the optional parameter with_labels is true, the internal search will include a dynamic runtime field that calls the getLabelPosition function of the geometry doc value. This enables the generation of new point features containing suggested geometry labels, so that, for example, multi-polygons will have only one label.

For example, Elasticsearch may translate a vector tile search API request with a grid_agg argument of geotile and an exact_bounds argument of true into the following search

GET my-index/_search
{
  "size": 10000,
  "query": {
    "geo_bounding_box": {
      "my-geo-field": {
        "top_left": {
          "lat": -40.979898069620134,
          "lon": -45
        },
        "bottom_right": {
          "lat": -66.51326044311186,
          "lon": 0
        }
      }
    }
  },
  "aggregations": {
    "grid": {
      "geotile_grid": {
        "field": "my-geo-field",
        "precision": 11,
        "size": 65536,
        "bounds": {
          "top_left": {
            "lat": -40.979898069620134,
            "lon": -45
          },
          "bottom_right": {
            "lat": -66.51326044311186,
            "lon": 0
          }
        }
      }
    },
    "bounds": {
      "geo_bounds": {
        "field": "my-geo-field",
        "wrap_longitude": false
      }
    }
  }
}

The API returns results as a binary Mapbox vector tile. Mapbox vector tiles are encoded as Google Protobufs (PBF). By default, the tile contains three layers:

  • A hits layer containing a feature for each <field> value matching the geo_bounding_box query.
  • An aggs layer containing a feature for each cell of the geotile_grid or geohex_grid. The layer only contains features for cells with matching data.
  • A meta layer containing:
    • A feature containing a bounding box. By default, this is the bounding box of the tile.
    • Value ranges for any sub-aggregations on the geotile_grid or geohex_grid.
    • Metadata for the search.

The API only returns features that can display at its zoom level. For example, if a polygon feature has no area at its zoom level, the API omits it. The API returns errors as UTF-8 encoded JSON.

IMPORTANT: You can specify several options for this API as either a query parameter or request body parameter. If you specify both parameters, the query parameter takes precedence.

Grid precision for geotile

For a grid_agg of geotile, you can use cells in the aggs layer as tiles for lower zoom levels. grid_precision represents the additional zoom levels available through these cells. The final precision is computed by as follows: <zoom> + grid_precision. For example, if <zoom> is 7 and grid_precision is 8, then the geotile_grid aggregation will use a precision of 15. The maximum final precision is 29. The grid_precision also determines the number of cells for the grid as follows: (2^grid_precision) x (2^grid_precision). For example, a value of 8 divides the tile into a grid of 256 x 256 cells. The aggs layer only contains features for cells with matching data.

Grid precision for geohex

For a grid_agg of geohex, Elasticsearch uses <zoom> and grid_precision to calculate a final precision as follows: <zoom> + grid_precision.

This precision determines the H3 resolution of the hexagonal cells produced by the geohex aggregation. The following table maps the H3 resolution for each precision. For example, if <zoom> is 3 and grid_precision is 3, the precision is 6. At a precision of 6, hexagonal cells have an H3 resolution of 2. If <zoom> is 3 and grid_precision is 4, the precision is 7. At a precision of 7, hexagonal cells have an H3 resolution of 3.

Precision Unique tile bins H3 resolution Unique hex bins Ratio
1 4 0 122 30.5
2 16 0 122 7.625
3 64 1 842 13.15625
4 256 1 842 3.2890625
5 1024 2 5882 5.744140625
6 4096 2 5882 1.436035156
7 16384 3 41162 2.512329102
8 65536 3 41162 0.6280822754
9 262144 4 288122 1.099098206
10 1048576 4 288122 0.2747745514
11 4194304 5 2016842 0.4808526039
12 16777216 6 14117882 0.8414913416
13 67108864 6 14117882 0.2103728354
14 268435456 7 98825162 0.3681524172
15 1073741824 8 691776122 0.644266719
16 4294967296 8 691776122 0.1610666797
17 17179869184 9 4842432842 0.2818666889
18 68719476736 10 33897029882 0.4932667053
19 274877906944 11 237279209162 0.8632167343
20 1099511627776 11 237279209162 0.2158041836
21 4398046511104 12 1660954464122 0.3776573213
22 17592186044416 13 11626681248842 0.6609003122
23 70368744177664 13 11626681248842 0.165225078
24 281474976710656 14 81386768741882 0.2891438866
25 1125899906842620 15 569707381193162 0.5060018015
26 4503599627370500 15 569707381193162 0.1265004504
27 18014398509482000 15 569707381193162 0.03162511259
28 72057594037927900 15 569707381193162 0.007906278149
29 288230376151712000 15 569707381193162 0.001976569537

Hexagonal cells don't align perfectly on a vector tile. Some cells may intersect more than one vector tile. To compute the H3 resolution for each precision, Elasticsearch compares the average density of hexagonal bins at each resolution with the average density of tile bins at each zoom level. Elasticsearch uses the H3 resolution that is closest to the corresponding geotile density.

External documentation

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams, indices, or aliases to search

  • field string Required

    Field containing geospatial data to return

  • zoom number Required

    Zoom level for the vector tile to search

  • x number Required

    X coordinate for the vector tile to search

  • y number Required

    Y coordinate for the vector tile to search

Query parameters

  • exact_bounds boolean

    If false, the meta layer's feature is the bounding box of the tile. If true, the meta layer's feature is a bounding box resulting from a geo_bounds aggregation. The aggregation runs on values that intersect the // tile with wrap_longitude set to false. The resulting bounding box may be larger than the vector tile.

  • extent number

    The size, in pixels, of a side of the tile. Vector tiles are square with equal sides.

  • grid_agg string

    Aggregation used to create a grid for field.

    Values are geotile or geohex.

  • grid_precision number

    Additional zoom levels available through the aggs layer. For example, if is 7 and grid_precision is 8, you can zoom in up to level 15. Accepts 0-8. If 0, results don't include the aggs layer.

  • grid_type string

    Determines the geometry type for features in the aggs layer. In the aggs layer, each feature represents a geotile_grid cell. If 'grid' each feature is a Polygon of the cells bounding box. If 'point' each feature is a Point that is the centroid of the cell.

    Values are grid, point, or centroid.

  • size number

    Maximum number of features to return in the hits layer. Accepts 0-10000. If 0, results don't include the hits layer.

  • with_labels boolean

    If true, the hits and aggs layers will contain additional point features representing suggested label positions for the original features.

    • Point and MultiPoint features will have one of the points selected.
    • Polygon and MultiPolygon features will have a single point generated, either the centroid, if it is within the polygon, or another point within the polygon selected from the sorted triangle-tree.
    • LineString features will likewise provide a roughly central point selected from the triangle-tree.
    • The aggregation results will provide one central point for each aggregation bucket.

    All attributes from the original features will also be copied to the new label features. In addition, the new features will be distinguishable using the tag _mvt_label_position.

application/json

Body

  • aggs object

    Sub-aggregations for the geotile_grid.

    It supports the following aggregation types:

    • avg
    • boxplot
    • cardinality
    • extended stats
    • max
    • median absolute deviation
    • min
    • percentile
    • percentile-rank
    • stats
    • sum
    • value count

    The aggregation names can't start with _mvt_. The _mvt_ prefix is reserved for internal aggregations.

  • buffer number

    The size, in pixels, of a clipping buffer outside the tile. This allows renderers to avoid outline artifacts from geometries that extend past the extent of the tile.

  • exact_bounds boolean

    If false, the meta layer's feature is the bounding box of the tile. If true, the meta layer's feature is a bounding box resulting from a geo_bounds aggregation. The aggregation runs on values that intersect the <zoom>/<x>/<y> tile with wrap_longitude set to false. The resulting bounding box may be larger than the vector tile.

  • extent number

    The size, in pixels, of a side of the tile. Vector tiles are square with equal sides.

  • fields string | array[string]
  • grid_agg string

    Values are geotile or geohex.

  • grid_precision number

    Additional zoom levels available through the aggs layer. For example, if <zoom> is 7 and grid_precision is 8, you can zoom in up to level 15. Accepts 0-8. If 0, results don't include the aggs layer.

  • grid_type string

    Values are grid, point, or centroid.

  • query object

    An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

    External documentation
  • runtime_mappings object
    Hide runtime_mappings attribute Show runtime_mappings attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • fields object

        For type composite

        Hide fields attribute Show fields attribute object
        • * object Additional properties
          Hide * attribute Show * attribute object
          • type string Required

            Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

      • fetch_fields array[object]

        For type lookup

        Hide fetch_fields attributes Show fetch_fields attributes object
        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • format string
      • format string

        A custom format for date type runtime fields.

      • input_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_index string
      • script object
        Hide script attributes Show script attributes object

        • source string | object

          One of:
          ScriptSource string SearchRequestBody object
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          Hide params attribute Show params attribute object
          • * object Additional properties

        • lang string

          Any of:
          ScriptLanguage string ScriptLanguage string

          Values are painless, expression, mustache, or java.

        • options object
          Hide options attribute Show options attribute object
          • * string Additional properties
      • type string Required

        Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

  • size number

    The maximum number of features to return in the hits layer. Accepts 0-10000. If 0, results don't include the hits layer.

  • sort string | object | array[string | object]

    One of:
    Field string SortOptions object Sort array[string | object]

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • track_total_hits boolean | number

    Number of hits matching the query to count accurately. If true, the exact number of hits is returned at the cost of some performance. If false, the response does not include the total number of hits matching the query. Defaults to 10,000 hits.

  • with_labels boolean

    If true, the hits and aggs layers will contain additional point features representing suggested label positions for the original features.

    • Point and MultiPoint features will have one of the points selected.
    • Polygon and MultiPolygon features will have a single point generated, either the centroid, if it is within the polygon, or another point within the polygon selected from the sorted triangle-tree.
    • LineString features will likewise provide a roughly central point selected from the triangle-tree.
    • The aggregation results will provide one central point for each aggregation bucket.

    All attributes from the original features will also be copied to the new label features. In addition, the new features will be distinguishable using the tag _mvt_label_position.

Responses

POST /{index}/_mvt/{field}/{zoom}/{x}/{y} 
curl \
 --request POST 'http://api.example.com/{index}/_mvt/{field}/{zoom}/{x}/{y}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"grid_agg\": \"geotile\",\n  \"grid_precision\": 2,\n  \"fields\": [\n    \"name\",\n    \"price\"\n  ],\n  \"query\": {\n    \"term\": {\n      \"included\": true\n    }\n  },\n  \"aggs\": {\n    \"min_price\": {\n      \"min\": {\n        \"field\": \"price\"\n      }\n    },\n    \"max_price\": {\n      \"max\": {\n        \"field\": \"price\"\n      }\n    },\n    \"avg_price\": {\n      \"avg\": {\n        \"field\": \"price\"\n      }\n    }\n  }\n}"'
Request example
Run `GET museums/_mvt/location/13/4207/2692` to search an index for `location` values that intersect the `13/4207/2692` vector tile. 
{
  "grid_agg": "geotile",
  "grid_precision": 2,
  "fields": [
    "name",
    "price"
  ],
  "query": {
    "term": {
      "included": true
    }
  },
  "aggs": {
    "min_price": {
      "min": {
        "field": "price"
      }
    },
    "max_price": {
      "max": {
        "field": "price"
      }
    },
    "avg_price": {
      "avg": {
        "field": "price"
      }
    }
  }
}
Response examples (200)
A successful response from `GET museums/_mvt/location/13/4207/2692`. It returns results as a binary vector tile. When decoded into JSON, the tile contains the following data. 
{
  "hits": {
    "extent": 4096,
    "version": 2,
    "features": [
      {
        "geometry": {
          "type": "Point",
          "coordinates": [
            3208,
            3864
          ]
        },
        "properties": {
          "_id": "1",
          "_index": "museums",
          "name": "NEMO Science Museum",
          "price": 1750
        },
        "type": 1
      },
      {
        "geometry": {
          "type": "Point",
          "coordinates": [
            3429,
            3496
          ]
        },
        "properties": {
          "_id": "3",
          "_index": "museums",
          "name": "Nederlands Scheepvaartmuseum",
          "price": 1650
        },
        "type": 1
      },
      {
        "geometry": {
          "type": "Point",
          "coordinates": [
            3429,
            3496
          ]
        },
        "properties": {
          "_id": "4",
          "_index": "museums",
          "name": "Amsterdam Centre for Architecture",
          "price": 0
        },
        "type": 1
      }
    ]
  },
  "aggs": {
    "extent": 4096,
    "version": 2,
    "features": [
      {
        "geometry": {
          "type": "Polygon",
          "coordinates": [
            [
              [
                3072,
                3072
              ],
              [
                4096,
                3072
              ],
              [
                4096,
                4096
              ],
              [
                3072,
                4096
              ],
              [
                3072,
                3072
              ]
            ]
          ]
        },
        "properties": {
          "_count": 3,
          "max_price.value": 1750.0,
          "min_price.value": 0.0,
          "avg_price.value": 1133.3333333333333
        },
        "type": 3
      }
    ]
  },
  "meta": {
    "extent": 4096,
    "version": 2,
    "features": [
      {
        "geometry": {
          "type": "Polygon",
          "coordinates": [
            [
              [
                0,
                0
              ],
              [
                4096,
                0
              ],
              [
                4096,
                4096
              ],
              [
                0,
                4096
              ],
              [
                0,
                0
              ]
            ]
          ]
        },
        "properties": {
          "_shards.failed": 0,
          "_shards.skipped": 0,
          "_shards.successful": 1,
          "_shards.total": 1,
          "aggregations._count.avg": 3.0,
          "aggregations._count.count": 1,
          "aggregations._count.max": 3.0,
          "aggregations._count.min": 3.0,
          "aggregations._count.sum": 3.0,
          "aggregations.avg_price.avg": 1133.3333333333333,
          "aggregations.avg_price.count": 1,
          "aggregations.avg_price.max": 1133.3333333333333,
          "aggregations.avg_price.min": 1133.3333333333333,
          "aggregations.avg_price.sum": 1133.3333333333333,
          "aggregations.max_price.avg": 1750.0,
          "aggregations.max_price.count": 1,
          "aggregations.max_price.max": 1750.0,
          "aggregations.max_price.min": 1750.0,
          "aggregations.max_price.sum": 1750.0,
          "aggregations.min_price.avg": 0.0,
          "aggregations.min_price.count": 1,
          "aggregations.min_price.max": 0.0,
          "aggregations.min_price.min": 0.0,
          "aggregations.min_price.sum": 0.0,
          "hits.max_score": 0.0,
          "hits.total.relation": "eq",
          "hits.total.value": 3,
          "timed_out": false,
          "took": 2
        },
        "type": 3
      }
    ]
  }
}

Create or update role mappings Added in 5.5.0

PUT /_security/role_mapping/{name}
Api key auth Basic auth Bearer auth

Role mappings define which roles are assigned to each user. Each mapping has rules that identify users and a list of roles that are granted to those users. The role mapping APIs are generally the preferred way to manage role mappings rather than using role mapping files. The create or update role mappings API cannot update role mappings that are defined in role mapping files.

NOTE: This API does not create roles. Rather, it maps users to existing roles. Roles can be created by using the create or update roles API or roles files.

Role templates

The most common use for role mappings is to create a mapping from a known value on the user to a fixed role name. For example, all users in the cn=admin,dc=example,dc=com LDAP group should be given the superuser role in Elasticsearch. The roles field is used for this purpose.

For more complex needs, it is possible to use Mustache templates to dynamically determine the names of the roles that should be granted to the user. The role_templates field is used for this purpose.

NOTE: To use role templates successfully, the relevant scripting feature must be enabled. Otherwise, all attempts to create a role mapping with role templates fail.

All of the user fields that are available in the role mapping rules are also available in the role templates. Thus it is possible to assign a user to a role that reflects their username, their groups, or the name of the realm to which they authenticated.

By default a template is evaluated to produce a single string that is the name of the role which should be assigned to the user. If the format of the template is set to "json" then the template is expected to produce a JSON string or an array of JSON strings for the role names.

External documentation

Path parameters

  • name string Required

    The distinct name that identifies the role mapping. The name is used solely as an identifier to facilitate interaction via the API; it does not affect the behavior of the mapping in any way.

Query parameters

  • refresh string

    If true (the default) then refresh the affected shards to make this operation visible to search, if wait_for then wait for a refresh to make this operation visible to search, if false then do nothing with refreshes.

    Values are true, false, or wait_for.

application/json

Body Required

  • enabled boolean

    Mappings that have enabled set to false are ignored when role mapping is performed.

  • metadata object
    Hide metadata attribute Show metadata attribute object
    • * object Additional properties
  • roles array[string]

    A list of role names that are granted to the users that match the role mapping rules. Exactly one of roles or role_templates must be specified.

  • role_templates array[object]

    A list of Mustache templates that will be evaluated to determine the roles names that should granted to the users that match the role mapping rules. Exactly one of roles or role_templates must be specified.

    Hide role_templates attributes Show role_templates attributes object
    • format string

      Values are string or json.

    • template object Required
      Hide template attributes Show template attributes object

      • source string | object

        One of:
        ScriptSource string SearchRequestBody object
      • id string
      • params object

        Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

        Hide params attribute Show params attribute object
        • * object Additional properties

      • lang string

        Any of:
        ScriptLanguage string ScriptLanguage string

        Values are painless, expression, mustache, or java.

      • options object
        Hide options attribute Show options attribute object
        • * string Additional properties
  • rules object
    Hide rules attributes Show rules attributes object
  • run_as array[string]

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
PUT /_security/role_mapping/{name} 
curl \
 --request PUT 'http://api.example.com/_security/role_mapping/{name}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"roles\": [ \"user\"],\n  \"enabled\": true, \n  \"rules\": {\n    \"field\" : { \"username\" : \"*\" }\n  },\n  \"metadata\" : { \n    \"version\" : 1\n  }\n}"'
Request examples
Run `POST /_security/role_mapping/mapping1` to assign the `user` role to all users. 
{
  "roles": [ "user"],
  "enabled": true, 
  "rules": {
    "field" : { "username" : "*" }
  },
  "metadata" : { 
    "version" : 1
  }
}
Run `POST /_security/role_mapping/mapping2` to assign the "user" and "admin" roles to specific users. 
{
  "roles": [ "user", "admin" ],
  "enabled": true,
  "rules": {
    "field" : { "username" : [ "esadmin01", "esadmin02" ] }
  }
}
Run `POST /_security/role_mapping/mapping3` to match users who authenticated against a specific realm. 
{
  "roles": [ "ldap-user" ],
  "enabled": true,
  "rules": {
    "field" : { "realm.name" : "ldap1" }
  }
}
Run `POST /_security/role_mapping/mapping4` to match any user where either the username is `esadmin` or the user is in the `cn=admin,dc=example,dc=com group`. This example is useful when the group names in your identity management system (such as Active Directory, or a SAML Identity Provider) do not have a one-to-one correspondence with the names of roles in Elasticsearch. The role mapping is the means by which you link a group name with a role name. 
{
  "roles": [ "superuser" ],
  "enabled": true,
  "rules": {
    "any": [
      {
        "field": {
          "username": "esadmin"
        }
      },
      {
        "field": {
          "groups": "cn=admins,dc=example,dc=com"
        }
      }
    ]
  }
}
Run `POST /_security/role_mapping/mapping5` to use an array syntax for the groups field when there are multiple groups. This pattern matches any of the groups (rather than all of the groups). 
{
  "role_templates": [
    {
      "template": { "source": "{{#tojson}}groups{{/tojson}}" }, 
      "format" : "json" 
    }
  ],
  "rules": {
    "field" : { "realm.name" : "saml1" }
  },
  "enabled": true
}
Run `POST /_security/role_mapping/mapping6` for rare cases when the names of your groups may be an exact match for the names of your Elasticsearch roles. This can be the case when your SAML Identity Provider includes its own "group mapping" feature and can be configured to release Elasticsearch role names in the user's SAML attributes. In these cases it is possible to use a template that treats the group names as role names. NOTE: This should only be done if you intend to define roles for all of the provided groups. Mapping a user to a large number of unnecessary or undefined roles is inefficient and can have a negative effect on system performance. If you only need to map a subset of the groups, you should do it by using explicit mappings. The `tojson` mustache function is used to convert the list of group names into a valid JSON array. Because the template produces a JSON array, the `format` must be set to `json`. 
{
  "role_templates": [
    {
      "template": { "source": "{{#tojson}}groups{{/tojson}}" }, 
      "format" : "json" 
    }
  ],
  "rules": {
    "field" : { "realm.name" : "saml1" }
  },
  "enabled": true
}
Run `POST /_security/role_mapping/mapping7` to match users within a particular LDAP sub-tree in a specific realm. 
{
  "roles": [ "ldap-example-user" ],
  "enabled": true,
  "rules": {
    "all": [
      { "field" : { "dn" : "*,ou=subtree,dc=example,dc=com" } },
      { "field" : { "realm.name" : "ldap1" } }
    ]
  }
}
Run `POST /_security/role_mapping/mapping8` to assign rules that are complex and include wildcard matching. For example, this mapping matches any user where all of these conditions are met: the Distinguished Name matches the pattern `*,ou=admin,dc=example,dc=com`, or the `username` is `es-admin`, or the `username` is `es-system`; the user is in the `cn=people,dc=example,dc=com` group; the user does not have a `terminated_date`. 
{
  "roles": [ "superuser" ],
  "enabled": true,
  "rules": {
    "all": [
      {
        "any": [
          {
            "field": {
              "dn": "*,ou=admin,dc=example,dc=com"
            }
          },
          {
            "field": {
              "username": [ "es-admin", "es-system" ]
            }
          }
        ]
      },
      {
        "field": {
          "groups": "cn=people,dc=example,dc=com"
        }
      },
      {
        "except": {
          "field": {
            "metadata.terminated_date": null
          }
        }
      }
    ]
  }
}
Run `POST /_security/role_mapping/mapping9` to use templated roles to automatically map every user to their own custom role. In this example every user who authenticates using the `cloud-saml` realm will be automatically mapped to two roles: the `saml_user` role and a role that is their username prefixed with `_user_`. For example, the user `nwong` would be assigned the `saml_user` and `_user_nwong` roles. 
{
  "rules": { "field": { "realm.name": "cloud-saml" } },
  "role_templates": [
    { "template": { "source" : "saml_user" } }, 
    { "template": { "source" : "_user_{{username}}" } }
  ],
  "enabled": true
}
Response examples (200)
A successful response from `POST /_security/role_mapping/mapping1`. 
{
  "role_mapping" : {
    "created" : true 
  }
}

Enroll Kibana Added in 8.0.0

GET /_security/enroll/kibana
Api key auth Basic auth Bearer auth

Enable a Kibana instance to configure itself for communication with a secured Elasticsearch cluster.

NOTE: This API is currently intended for internal use only by Kibana. Kibana uses this API internally to configure itself for communications with an Elasticsearch cluster that already has security features enabled.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • token object Required
      Hide token attributes Show token attributes object
      • name string Required

        The name of the bearer token for the elastic/kibana service account.

      • value string Required

        The value of the bearer token for the elastic/kibana service account. Use this value to authenticate the service account with Elasticsearch.

    • http_ca string Required

      The CA certificate used to sign the node certificates that Elasticsearch uses for TLS on the HTTP layer. The certificate is returned as a Base64 encoded string of the ASN.1 DER encoding of the certificate.
GET /_security/enroll/kibana 
curl \
 --request GET 'http://api.example.com/_security/enroll/kibana' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_security/enroll/kibana`. 
{
  "token" : {
    "name" : "enroll-process-token-1629123923000", 
    "value": "AAEAAWVsYXN0aWM...vZmxlZXQtc2VydmVyL3Rva2VuMTo3TFdaSDZ" 
  },
  "http_ca" : "MIIJlAIBAzVoGCSqGSIb3...vsDfsA3UZBAjEPfhubpQysAICAA=", 
}

Get a token Added in 5.5.0

POST /_security/oauth2/token
Api key auth Basic auth Bearer auth

Create a bearer token for access without requiring basic authentication. The tokens are created by the Elasticsearch Token Service, which is automatically enabled when you configure TLS on the HTTP interface. Alternatively, you can explicitly enable the xpack.security.authc.token.enabled setting. When you are running in production mode, a bootstrap check prevents you from enabling the token service unless you also enable TLS on the HTTP interface.

The get token API takes the same parameters as a typical OAuth 2.0 token API except for the use of a JSON request body.

A successful get token API call returns a JSON structure that contains the access token, the amount of time (seconds) that the token expires in, the type, and the scope if available.

The tokens returned by the get token API have a finite period of time for which they are valid and after that time period, they can no longer be used. That time period is defined by the xpack.security.authc.token.timeout setting. If you want to invalidate a token immediately, you can do so by using the invalidate token API.

External documentation
application/json

Body Required

  • grant_type string

    Values are password, client_credentials, _kerberos, or refresh_token.

  • scope string

    The scope of the token. Currently tokens are only issued for a scope of FULL regardless of the value sent with the request.

  • password string
  • kerberos_ticket string

    The base64 encoded kerberos ticket. If you specify the _kerberos grant type, this parameter is required. This parameter is not valid with any other supported grant type.

  • refresh_token string

    The string that was returned when you created the token, which enables you to extend its life. If you specify the refresh_token grant type, this parameter is required. This parameter is not valid with any other supported grant type.

  • username string

Responses

POST /_security/oauth2/token 
curl \
 --request POST 'http://api.example.com/_security/oauth2/token' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"grant_type\" : \"client_credentials\"\n}"'
Request examples
Run `POST /_security/oauth2/token` to obtain a token using the `client_credentials` grant type, which simply creates a token as the authenticated user. 
{
  "grant_type" : "client_credentials"
}
Run `POST /_security/oauth2/token` to obtain a token for the `test_admin` user using the password grant type. This request needs to be made by an authenticated user with sufficient privileges that may or may not be the same as the one whose username is passed in the `username` parameter. 
{
  "grant_type" : "password",
  "username" : "test_admin",
  "password" : "x-pack-test-password"
}
Response examples (200)
A successful response from `POST /_security/oauth2/token`. 
{
  "access_token" : "dGhpcyBpcyBub3QgYSByZWFsIHRva2VuIGJ1dCBpdCBpcyBvbmx5IHRlc3QgZGF0YS4gZG8gbm90IHRyeSB0byByZWFkIHRva2VuIQ==",
  "type" : "Bearer",
  "expires_in" : 1200,
  "authentication" : {
    "username" : "test_admin",
    "roles" : [
      "superuser"
    ],
    "full_name" : null,
    "email" : null,
    "metadata" : { },
    "enabled" : true,
    "authentication_realm" : {
      "name" : "file",
      "type" : "file"
    },
    "lookup_realm" : {
      "name" : "file",
      "type" : "file"
    },
    "authentication_type" : "realm"
  }
}
A successful response from `POST /_security/oauth2/token`. 
{
  "access_token" : "dGhpcyBpcyBub3QgYSByZWFsIHRva2VuIGJ1dCBpdCBpcyBvbmx5IHRlc3QgZGF0YS4gZG8gbm90IHRyeSB0byByZWFkIHRva2VuIQ==",
  "type" : "Bearer",
  "expires_in" : 1200,
  "authentication" : {
    "username" : "test_admin",
    "roles" : [
      "superuser"
    ],
    "full_name" : null,
    "email" : null,
    "metadata" : { },
    "enabled" : true,
    "authentication_realm" : {
      "name" : "file",
      "type" : "file"
    },
    "lookup_realm" : {
      "name" : "file",
      "type" : "file"
    },
    "authentication_type" : "realm"
  }
}

Get SSL certificates Added in 6.2.0

GET /_ssl/certificates
Api key auth Basic auth Bearer auth

Get information about the X.509 certificates that are used to encrypt communications in the cluster. The API returns a list that includes certificates from all TLS contexts including:

  • Settings for transport and HTTP interfaces
  • TLS settings that are used within authentication realms
  • TLS settings for remote monitoring exporters

The list includes certificates that are used for configuring trust, such as those configured in the xpack.security.transport.ssl.truststore and xpack.security.transport.ssl.certificate_authorities settings. It also includes certificates that are used for configuring server identity, such as xpack.security.http.ssl.keystore and xpack.security.http.ssl.certificate settings.

The list does not include certificates that are sourced from the default SSL context of the Java Runtime Environment (JRE), even if those certificates are in use within Elasticsearch.

NOTE: When a PKCS#11 token is configured as the truststore of the JRE, the API returns all the certificates that are included in the PKCS#11 token irrespective of whether these are used in the Elasticsearch TLS configuration.

If Elasticsearch is configured to use a keystore or truststore, the API output includes all certificates in that store, even though some of the certificates might not be in active use within the cluster.

External documentation

Responses

  • 200 application/json
    Hide response attributes Show response attributes object

    • alias string | null Required

      If the path refers to a container file (a jks keystore, or a PKCS#12 file), it is the alias of the certificate. Otherwise, it is null.

      One of:
      string-1 string string-2 string | null

    • expiry string | number Required

      A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

      One of:
      DateTime string UnitMillis number
    • format string Required

      The format of the file. Valid values include jks, PKCS12, and PEM.

    • has_private_key boolean Required

      Indicates whether Elasticsearch has access to the private key for this certificate.

    • issuer string

      The Distinguished Name of the certificate's issuer.

    • path string Required

      The path to the certificate, as configured in the elasticsearch.yml file.

    • serial_number string Required

      The hexadecimal representation of the certificate's serial number.

    • subject_dn string Required

      The Distinguished Name of the certificate's subject.
GET /_ssl/certificates 
curl \
 --request GET 'http://api.example.com/_ssl/certificates' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_ssl/certificates`, which provides information about the certificates on a single node of Elasticsearch. 
[
  {
    "path": "certs/elastic-certificates.p12",
    "format": "PKCS12",
    "alias": "instance",
    "subject_dn": "CN=Elastic Certificate Tool Autogenerated CA",
    "serial_number": "a20f0ee901e8f69dc633ff633e5cd5437cdb4137",
    "has_private_key": false,
    "expiry": "2021-01-15T20:42:49.000Z"
  },
  {
    "path": "certs/elastic-certificates.p12",
    "format": "PKCS12",
    "alias": "ca",
    "subject_dn": "CN=Elastic Certificate Tool Autogenerated CA",
    "serial_number": "a20f0ee901e8f69dc633ff633e5cd5437cdb4137",
    "has_private_key": false,
    "expiry": "2021-01-15T20:42:49.000Z"
  },
  {
    "path": "certs/elastic-certificates.p12",
    "format": "PKCS12",
    "alias": "instance",
    "subject_dn": "CN=instance",
    "serial_number": "fc1905e1494dc5230218d079c47a617088f84ce0",
    "has_private_key": true,
    "expiry": "2021-01-15T20:44:32.000Z"
  }
]

Clone a snapshot Added in 7.10.0

PUT /_snapshot/{repository}/{snapshot}/_clone/{target_snapshot}
Api key auth Basic auth Bearer auth

Clone part of all of a snapshot into another snapshot in the same repository.

Path parameters

  • repository string Required

    The name of the snapshot repository that both source and target snapshot belong to.

  • snapshot string Required

    The source snapshot name.

  • target_snapshot string Required

    The target snapshot name.

Query parameters

  • master_timeout string

    The period to wait for the master node. If the master node is not available before the timeout expires, the request fails and returns an error. To indicate that the request should never timeout, set it to -1.

    Values are -1 or 0.

application/json

Body Required

  • indices string Required

    A comma-separated list of indices to include in the snapshot. Multi-target syntax is supported.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
PUT /_snapshot/{repository}/{snapshot}/_clone/{target_snapshot} 
curl \
 --request PUT 'http://api.example.com/_snapshot/{repository}/{snapshot}/_clone/{target_snapshot}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"indices\": \"index_a,index_b\"\n}"'
Request example
Run `PUT /_snapshot/my_repository/source_snapshot/_clone/target_snapshot` to clone the `source_snapshot` into a new `target_snapshot`. 
{
  "indices": "index_a,index_b"
}

Get the snapshot status Added in 7.8.0

GET /_snapshot/{repository}/_status
Api key auth Basic auth Bearer auth

Get a detailed description of the current state for each shard participating in the snapshot.

Note that this API should be used only to obtain detailed shard-level information for ongoing snapshots. If this detail is not needed or you want to obtain information about one or more existing snapshots, use the get snapshot API.

If you omit the <snapshot> request path parameter, the request retrieves information only for currently running snapshots. This usage is preferred. If needed, you can specify <repository> and <snapshot> to retrieve information for specific snapshots, even if they're not currently running.

WARNING: Using the API to return the status of any snapshots other than currently running snapshots can be expensive. The API requires a read from the repository for each shard in each snapshot. For example, if you have 100 snapshots with 1,000 shards each, an API request that includes all snapshots will require 100,000 reads (100 snapshots x 1,000 shards).

Depending on the latency of your storage, such requests can take an extremely long time to return results. These requests can also tax machine resources and, when using cloud storage, incur high processing costs.

Path parameters

  • repository string Required

    The snapshot repository name used to limit the request. It supports wildcards (*) if <snapshot> isn't specified.

Query parameters

  • ignore_unavailable boolean

    If false, the request returns an error for any snapshots that are unavailable. If true, the request ignores snapshots that are unavailable, such as those that are corrupted or temporarily cannot be returned.

  • master_timeout string

    The period to wait for the master node. If the master node is not available before the timeout expires, the request fails and returns an error. To indicate that the request should never timeout, set it to -1.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • snapshots array[object] Required
      Hide snapshots attributes Show snapshots attributes object
      • include_global_state boolean Required

        Indicates whether the current cluster state is included in the snapshot.

      • indices object Required
        Hide indices attribute Show indices attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • shards object Required
            Hide shards attribute Show shards attribute object
            • * object Additional properties
          • shards_stats object Required
            Hide shards_stats attributes Show shards_stats attributes object
            • done number Required

              The number of shards that initialized, started, and finalized successfully.

            • failed number Required

              The number of shards that failed to be included in the snapshot.

            • finalizing number Required

              The number of shards that are finalizing but are not done.

            • initializing number Required

              The number of shards that are still initializing.

            • started number Required

              The number of shards that have started but are not finalized.

            • total number Required

              The total number of shards included in the snapshot.

          • stats object Required
            Hide stats attributes Show stats attributes object
      • repository string Required

        The name of the repository that includes the snapshot.

      • shards_stats object Required
        Hide shards_stats attributes Show shards_stats attributes object
        • done number Required

          The number of shards that initialized, started, and finalized successfully.

        • failed number Required

          The number of shards that failed to be included in the snapshot.

        • finalizing number Required

          The number of shards that are finalizing but are not done.

        • initializing number Required

          The number of shards that are still initializing.

        • started number Required

          The number of shards that have started but are not finalized.

        • total number Required

          The total number of shards included in the snapshot.

      • snapshot string Required

        The name of the snapshot.

      • state string Required

        The current snapshot state:

        • FAILED: The snapshot finished with an error and failed to store any data.
        • STARTED: The snapshot is currently running.
        • SUCCESS: The snapshot completed.
      • stats object Required
        Hide stats attributes Show stats attributes object
      • uuid string Required
GET /_snapshot/{repository}/_status 
curl \
 --request GET 'http://api.example.com/_snapshot/{repository}/_status' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _snapshot/my_repository/snapshot_2/_status`. The response contains detailed status information for `snapshot_2` in the `my_repository` repository. 
{
  "snapshots" : [
    {
      "snapshot" : "snapshot_2",
      "repository" : "my_repository",
      "uuid" : "lNeQD1SvTQCqqJUMQSwmGg",
      "state" : "SUCCESS",
      "include_global_state" : false,
      "shards_stats" : {
        "initializing" : 0,
        "started" : 0,
        "finalizing" : 0,
        "done" : 1,
        "failed" : 0,
        "total" : 1
      },
      "stats" : {
        "incremental" : {
          "file_count" : 3,
          "size_in_bytes" : 5969
        },
        "total" : {
          "file_count" : 4,
          "size_in_bytes" : 6024
        },
        "start_time_in_millis" : 1594829326691,
        "time_in_millis" : 205
      },
      "indices" : {
        "index_1" : {
          "shards_stats" : {
            "initializing" : 0,
            "started" : 0,
            "finalizing" : 0,
            "done" : 1,
            "failed" : 0,
            "total" : 1
          },
          "stats" : {
            "incremental" : {
              "file_count" : 3,
              "size_in_bytes" : 5969
            },
            "total" : {
              "file_count" : 4,
              "size_in_bytes" : 6024
            },
            "start_time_in_millis" : 1594829326896,
            "time_in_millis" : 0
          },
          "shards" : {
            "0" : {
              "stage" : "DONE",
              "stats" : {
                "incremental" : {
                  "file_count" : 3,
                  "size_in_bytes" : 5969
                },
                "total" : {
                  "file_count" : 4,
                  "size_in_bytes" : 6024
                },
                "start_time_in_millis" : 1594829326896,
                "time_in_millis" : 0
              }
            }
          }
        }
      }
    }
  ]
}

Run a watch

PUT /_watcher/watch/_execute
Api key auth Basic auth Bearer auth

This API can be used to force execution of the watch outside of its triggering logic or to simulate the watch execution for debugging purposes.

For testing and debugging purposes, you also have fine-grained control on how the watch runs. You can run the watch without running all of its actions or alternatively by simulating them. You can also force execution by ignoring the watch condition and control whether a watch record would be written to the watch history after it runs.

You can use the run watch API to run watches that are not yet registered by specifying the watch definition inline. This serves as great tool for testing and debugging your watches prior to adding them to Watcher.

When Elasticsearch security features are enabled on your cluster, watches are run with the privileges of the user that stored the watches. If your user is allowed to read index a, but not index b, then the exact same set of rules will apply during execution of a watch.

When using the run watch API, the authorization data of the user that called the API will be used as a base, instead of the information who stored the watch.

Query parameters

  • debug boolean

    Defines whether the watch runs in debug mode.

application/json

Body

  • action_modes object

    Determines how to handle the watch actions as part of the watch execution.

    Hide action_modes attribute Show action_modes attribute object
    • * string Additional properties

      Values are simulate, force_simulate, execute, force_execute, or skip.

  • alternative_input object

    When present, the watch uses this object as a payload instead of executing its own input.

    Hide alternative_input attribute Show alternative_input attribute object
    • * object Additional properties
  • ignore_condition boolean

    When set to true, the watch execution uses the always condition. This can also be specified as an HTTP parameter.

  • record_execution boolean

    When set to true, the watch record representing the watch execution result is persisted to the .watcher-history index for the current time. In addition, the status of the watch is updated, possibly throttling subsequent runs. This can also be specified as an HTTP parameter.

  • simulated_actions object
    Hide simulated_actions attributes Show simulated_actions attributes object
  • trigger_data object
    Hide trigger_data attributes Show trigger_data attributes object

    • scheduled_time string | number Required

      A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

      One of:
      DateTime string UnitMillis number

    • triggered_time string | number

      A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

      One of:
      DateTime string UnitMillis number
  • watch object
    Hide watch attributes Show watch attributes object
    • actions object Required
      Hide actions attribute Show actions attribute object
    • condition object Required
      Hide condition attributes Show condition attributes object
      • always object
      • array_compare object
        Hide array_compare attribute Show array_compare attribute object
        • * object Additional properties
          Hide * attribute Show * attribute object
      • compare object
        Hide compare attribute Show compare attribute object
        • * object Additional properties
      • never object
      • script object
        Hide script attributes Show script attributes object

        • lang string

          Any of:
          ScriptLanguage string ScriptLanguage string

          Values are painless, expression, mustache, or java.

        • params object
          Hide params attribute Show params attribute object
          • * object Additional properties

        • source string | object

          One of:
          ScriptSource string SearchRequestBody object
        • id string
    • input object Required
      Hide input attributes Show input attributes object
      • chain object
        Hide chain attribute Show chain attribute object
        • inputs array[object] Required
          Hide inputs attribute Show inputs attribute object
      • http object
        Hide http attributes Show http attributes object
        • extract array[string]
        • request object
          Hide request attributes Show request attributes object
          • auth object
            Hide auth attribute Show auth attribute object
          • body string
          • connection_timeout string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • headers object
            Hide headers attribute Show headers attribute object
            • * string Additional properties
          • host string
          • method string

            Values are head, get, post, put, or delete.

          • params object
            Hide params attribute Show params attribute object
            • * string Additional properties
          • path string
          • port number
          • proxy object
            Hide proxy attributes Show proxy attributes object
          • read_timeout string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • scheme string

            Values are http or https.

          • url string
        • response_content_type string

          Values are json, yaml, or text.

      • search object
        Hide search attributes Show search attributes object
        • extract array[string]
        • request object Required
          Hide request attributes Show request attributes object
          • body object
            Hide body attribute Show body attribute object
          • indices array[string]
          • indices_options object
            Hide indices_options attributes Show indices_options attributes object
            • allow_no_indices boolean

              If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

            • expand_wildcards string | array[string]
            • ignore_unavailable boolean

              If true, missing or closed indices are not included in the response.

            • ignore_throttled boolean

              If true, concrete, expanded or aliased indices are ignored when frozen.

          • search_type string

            Values are query_then_fetch or dfs_query_then_fetch.

          • template object
            Hide template attributes Show template attributes object
            • explain boolean
            • id string
            • params object
              Hide params attribute Show params attribute object
              • * object Additional properties
            • profile boolean
            • source string

              An inline search template. Supports the same parameters as the search API's request body. Also supports Mustache variables. If no id is specified, this parameter is required.

          • rest_total_hits_as_int boolean
        • timeout string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • simple object
        Hide simple attribute Show simple attribute object
        • * object Additional properties
    • metadata object
      Hide metadata attribute Show metadata attribute object
      • * object Additional properties
    • status object
      Hide status attributes Show status attributes object
    • throttle_period string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • throttle_period_in_millis number

      Time unit for milliseconds

    • transform object
      Hide transform attributes Show transform attributes object
      • chain array[object]
      • script object
        Hide script attributes Show script attributes object
        • lang string
        • params object
          Hide params attribute Show params attribute object
          • * object Additional properties

        • source string | object

          One of:
          ScriptSource string SearchRequestBody object
        • id string
      • search object
        Hide search attributes Show search attributes object
        • request object Required
          Hide request attributes Show request attributes object
          • body object
            Hide body attribute Show body attribute object
          • indices array[string]
          • indices_options object
            Hide indices_options attributes Show indices_options attributes object
            • allow_no_indices boolean

              If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

            • expand_wildcards string | array[string]
            • ignore_unavailable boolean

              If true, missing or closed indices are not included in the response.

            • ignore_throttled boolean

              If true, concrete, expanded or aliased indices are ignored when frozen.

          • search_type string

            Values are query_then_fetch or dfs_query_then_fetch.

          • template object
            Hide template attributes Show template attributes object
            • explain boolean
            • id string
            • params object
              Hide params attribute Show params attribute object
              • * object Additional properties
            • profile boolean
            • source string

              An inline search template. Supports the same parameters as the search API's request body. Also supports Mustache variables. If no id is specified, this parameter is required.

          • rest_total_hits_as_int boolean
        • timeout string Required

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • trigger object Required
      Hide trigger attribute Show trigger attribute object
      • schedule object
        Hide schedule attributes Show schedule attributes object
        • timezone string
        • cron string
        • daily object
          Hide daily attribute Show daily attribute object
        • hourly object
          Hide hourly attribute Show hourly attribute object
        • interval string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • monthly object | array[object]

          One of:
          TimeOfMonth object array-2 array[object]
          Hide attributes Show attributes
          • at array[string] Required
          • on array[number] Required

        • weekly object | array[object]

          One of:
          TimeOfWeek object array-2 array[object]
          Hide attributes Show attributes
          • at array[string] Required
          • on array[string] Required

            Values are sunday, monday, tuesday, wednesday, thursday, friday, or saturday.

        • yearly object | array[object]

          One of:
          TimeOfYear object array-2 array[object]
          Hide attributes Show attributes
          • at array[string] Required
          • int array[string] Required

            Values are january, february, march, april, may, june, july, august, september, october, november, or december.

          • on array[number] Required

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _id string Required
    • watch_record object Required
      Hide watch_record attributes Show watch_record attributes object
      • condition object Required
        Hide condition attributes Show condition attributes object
        • always object
        • array_compare object
          Hide array_compare attribute Show array_compare attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
        • compare object
          Hide compare attribute Show compare attribute object
          • * object Additional properties
        • never object
        • script object
          Hide script attributes Show script attributes object

          • lang string

            Any of:
            ScriptLanguage string ScriptLanguage string

            Values are painless, expression, mustache, or java.

          • params object
            Hide params attribute Show params attribute object
            • * object Additional properties

          • source string | object

            One of:
            ScriptSource string SearchRequestBody object
          • id string
      • input object Required
        Hide input attributes Show input attributes object
        • chain object
          Hide chain attribute Show chain attribute object
          • inputs array[object] Required
            Hide inputs attribute Show inputs attribute object
        • http object
          Hide http attributes Show http attributes object
          • extract array[string]
          • request object
            Hide request attributes Show request attributes object
            • auth object
              Hide auth attribute Show auth attribute object
            • body string
            • connection_timeout string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • headers object
              Hide headers attribute Show headers attribute object
              • * string Additional properties
            • host string
            • method string

              Values are head, get, post, put, or delete.

            • params object
              Hide params attribute Show params attribute object
              • * string Additional properties
            • path string
            • port number
            • proxy object
              Hide proxy attributes Show proxy attributes object
            • read_timeout string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • scheme string

              Values are http or https.

            • url string
          • response_content_type string

            Values are json, yaml, or text.

        • search object
          Hide search attributes Show search attributes object
          • extract array[string]
          • request object Required
            Hide request attributes Show request attributes object
            • body object
              Hide body attribute Show body attribute object
              • query object Required

                An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

            • indices array[string]
            • indices_options object
              Hide indices_options attributes Show indices_options attributes object
              • allow_no_indices boolean

                If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

              • expand_wildcards string | array[string]
              • ignore_unavailable boolean

                If true, missing or closed indices are not included in the response.

              • ignore_throttled boolean

                If true, concrete, expanded or aliased indices are ignored when frozen.

            • search_type string

              Values are query_then_fetch or dfs_query_then_fetch.

            • template object
              Hide template attributes Show template attributes object
              • explain boolean
              • id string
              • params object
              • profile boolean
              • source string

                An inline search template. Supports the same parameters as the search API's request body. Also supports Mustache variables. If no id is specified, this parameter is required.

            • rest_total_hits_as_int boolean
          • timeout string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • simple object
          Hide simple attribute Show simple attribute object
          • * object Additional properties
      • messages array[string] Required
      • metadata object
        Hide metadata attribute Show metadata attribute object
        • * object Additional properties
      • node string Required
      • result object Required
        Hide result attributes Show result attributes object
        • actions array[object] Required
          Hide actions attributes Show actions attributes object
        • condition object Required
          Hide condition attributes Show condition attributes object
          • met boolean Required
          • status string Required

            Values are success, failure, simulated, or throttled.

          • type string Required

            Values are always, never, script, compare, or array_compare.

        • execution_duration number

          Time unit for milliseconds

        • execution_time string | number Required

          A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

          One of:
          DateTime string UnitMillis number
        • input object Required
          Hide input attributes Show input attributes object
          • payload object Required
            Hide payload attribute Show payload attribute object
            • * object Additional properties
          • status string Required

            Values are success, failure, simulated, or throttled.

          • type string Required

            Values are http, search, or simple.

      • state string Required

        Values are awaits_execution, checking, execution_not_needed, throttled, executed, failed, deleted_while_queued, or not_executed_already_queued.

      • trigger_event object Required
        Hide trigger_event attributes Show trigger_event attributes object
      • user string Required
      • watch_id string Required
      • status object
        Hide status attributes Show status attributes object
PUT /_watcher/watch/_execute 
curl \
 --request PUT 'http://api.example.com/_watcher/watch/_execute' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"trigger_data\" : { \n    \"triggered_time\" : \"now\",\n    \"scheduled_time\" : \"now\"\n  },\n  \"alternative_input\" : { \n    \"foo\" : \"bar\"\n  },\n  \"ignore_condition\" : true, \n  \"action_modes\" : {\n    \"my-action\" : \"force_simulate\" \n  },\n  \"record_execution\" : true \n}"'
Request examples
Run `POST _watcher/watch/my_watch/_execute` to run a watch. The input defined in the watch is ignored and the `alternative_input` is used as the payload. The condition as defined by the watch is ignored and is assumed to evaluate to true. The `force_simulate` action forces the simulation of `my-action`. Forcing the simulation means that throttling is ignored and the watch is simulated by Watcher instead of being run normally. 
{
  "trigger_data" : { 
    "triggered_time" : "now",
    "scheduled_time" : "now"
  },
  "alternative_input" : { 
    "foo" : "bar"
  },
  "ignore_condition" : true, 
  "action_modes" : {
    "my-action" : "force_simulate" 
  },
  "record_execution" : true 
}
Run `POST _watcher/watch/my_watch/_execute` and set a different mode for each action. 
{
  "action_modes" : {
    "action1" : "force_simulate",
    "action2" : "skip"
  }
}
Run `POST _watcher/watch/_execute` to run a watch inline. All other settings for this API still apply when inlining a watch. In this example, while the inline watch defines a compare condition, during the execution this condition will be ignored. 
{
  "watch" : {
    "trigger" : { "schedule" : { "interval" : "10s" } },
    "input" : {
      "search" : {
        "request" : {
          "indices" : [ "logs" ],
          "body" : {
            "query" : {
              "match" : { "message": "error" }
            }
          }
        }
      }
    },
    "condition" : {
      "compare" : { "ctx.payload.hits.total" : { "gt" : 0 }}
    },
    "actions" : {
      "log_error" : {
        "logging" : {
          "text" : "Found {{ctx.payload.hits.total}} errors in the logs"
        }
      }
    }
  }
}
Response examples (200)
A successful response from `POST _watcher/watch/my_watch/_execute`. 
{
  "_id": "my_watch_0-2015-06-02T23:17:55.124Z", 
  "watch_record": { 
    "@timestamp": "2015-06-02T23:17:55.124Z",
    "watch_id": "my_watch",
    "node": "my_node",
    "messages": [],
    "trigger_event": {
      "type": "manual",
      "triggered_time": "2015-06-02T23:17:55.124Z",
      "manual": {
        "schedule": {
          "scheduled_time": "2015-06-02T23:17:55.124Z"
        }
      }
    },
    "state": "executed",
    "status": {
      "version": 1,
      "execution_state": "executed",
      "state": {
        "active": true,
        "timestamp": "2015-06-02T23:17:55.111Z"
      },
      "last_checked": "2015-06-02T23:17:55.124Z",
      "last_met_condition": "2015-06-02T23:17:55.124Z",
      "actions": {
        "test_index": {
          "ack": {
            "timestamp": "2015-06-02T23:17:55.124Z",
            "state": "ackable"
          },
          "last_execution": {
            "timestamp": "2015-06-02T23:17:55.124Z",
            "successful": true
          },
          "last_successful_execution": {
            "timestamp": "2015-06-02T23:17:55.124Z",
            "successful": true
          }
        }
      }
    },
    "input": {
      "simple": {
        "payload": {
          "send": "yes"
        }
      }
    },
    "condition": {
      "always": {}
    },
    "result": { 
      "execution_time": "2015-06-02T23:17:55.124Z",
      "execution_duration": 12608,
      "input": {
        "type": "simple",
        "payload": {
          "foo": "bar"
        },
        "status": "success"
      },
      "condition": {
        "type": "always",
        "met": true,
        "status": "success"
      },
      "actions": [
        {
          "id": "test_index",
          "index": {
            "response": {
              "index": "test",
              "version": 1,
              "created": true,
              "result": "created",
              "id": "AVSHKzPa9zx62AzUzFXY"
            }
          },
          "status": "success",
          "type": "index"
        }
      ]
    },
    "user": "test_admin" 
  }
}

Stop the watch service

POST /_watcher/_stop
Api key auth Basic auth Bearer auth

Stop the Watcher service if it is running.

Query parameters

  • master_timeout string

    The period to wait for the master node. If the master node is not available before the timeout expires, the request fails and returns an error. To indicate that the request should never timeout, set it to -1.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
POST /_watcher/_stop 
curl \
 --request POST 'http://api.example.com/_watcher/_stop' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `POST _watcher/_stop`. 
{
  "acknowledged": true
}