Get index template information Added in 5.2.0

GET /_cat/templates/{name}

Get information about the index templates in a cluster. You can use index templates to apply index settings and field mappings to new indices at creation. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the get index template API.

Path parameters

  • name string Required

    The name of the template to return. Accepts wildcard expressions. If omitted, all templates are returned.

Query parameters

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

  • local boolean

    If true, the request computes the list of selected nodes from the local cluster state. If false the list of selected nodes are computed from the cluster state of the master node. In both cases the coordinating node will send requests for further information to each selected node.

  • Period to wait for a connection to the master node.

Responses

GET /_cat/templates/{name}
curl \
 --request GET 'http://api.example.com/_cat/templates/{name}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _cat/templates/my-template-*?v=true&s=name&format=json`.
[
  {
    "name": "my-template-0",
    "index_patterns": "[te*]",
    "order": "500",
    "version": null,
    "composed_of": "[]"
  },
  {
    "name": "my-template-1",
    "index_patterns": "[tea*]",
    "order": "501",
    "version": null,
    "composed_of": "[]"
  },
  {
    "name": "my-template-2",
    "index_patterns": "[teak*]",
    "order": "502",
    "version": "7",
    "composed_of": "[]"
  }
]













































































































Get node information Added in 1.3.0

GET /_nodes/{node_id}

By default, the API returns all attributes and core settings for cluster nodes.

Path parameters

  • node_id string | array[string] Required

    Comma-separated list of node IDs or names used to limit returned information.

Query parameters

  • If true, returns settings in flat format.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

Responses

GET /_nodes/{node_id}
curl \
 --request GET 'http://api.example.com/_nodes/{node_id}' \
 --header "Authorization: $API_KEY"
Response examples (200)
An abbreviated response when requesting cluster nodes information.
{
    "_nodes": {},
    "cluster_name": "elasticsearch",
    "nodes": {
      "USpTGYaBSIKbgSUJR2Z9lg": {
        "name": "node-0",
        "transport_address": "192.168.17:9300",
        "host": "node-0.elastic.co",
        "ip": "192.168.17",
        "version": "{version}",
        "transport_version": 100000298,
        "index_version": 100000074,
        "component_versions": {
          "ml_config_version": 100000162,
          "transform_config_version": 100000096
        },
        "build_flavor": "default",
        "build_type": "{build_type}",
        "build_hash": "587409e",
        "roles": [
          "master",
          "data",
          "ingest"
        ],
        "attributes": {},
        "plugins": [
          {
            "name": "analysis-icu",
            "version": "{version}",
            "description": "The ICU Analysis plugin integrates Lucene ICU
  module into elasticsearch, adding ICU relates analysis components.",
            "classname":
  "org.elasticsearch.plugin.analysis.icu.AnalysisICUPlugin",
            "has_native_controller": false
          }
        ],
        "modules": [
          {
            "name": "lang-painless",
            "version": "{version}",
            "description": "An easy, safe and fast scripting language for
  Elasticsearch",
            "classname": "org.elasticsearch.painless.PainlessPlugin",
            "has_native_controller": false
          }
        ]
      }
    }
}




























Get node statistics

GET /_nodes/{node_id}/stats/{metric}

Get statistics for nodes in a cluster. By default, all stats are returned. You can limit the returned information by using metrics.

Path parameters

  • node_id string | array[string] Required

    Comma-separated list of node IDs or names used to limit returned information.

  • metric string | array[string] Required

    Limit the information returned to the specified metrics

Query parameters

  • completion_fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in fielddata and suggest statistics.

  • fielddata_fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in fielddata statistics.

  • fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in the statistics.

  • groups boolean

    Comma-separated list of search groups to include in the search statistics.

  • If true, the call reports the aggregated disk usage of each one of the Lucene index files (only applies if segment stats are requested).

  • level string

    Indicates whether statistics are aggregated at the cluster, index, or shard level.

    Values are cluster, indices, or shards.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

  • types array[string]

    A comma-separated list of document types for the indexing index metric.

  • If true, the response includes information from segments that are not loaded into memory.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object
      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]
        Hide failures attributes Show failures attributes object
      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • nodes object Required
      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • Statistics about adaptive replica selection.

          Hide adaptive_selection attribute Show adaptive_selection attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • The exponentially weighted moving average queue size of search requests on the keyed node.

            • A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • The exponentially weighted moving average response time, in nanoseconds, of search requests on the keyed node.

            • A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • The exponentially weighted moving average service time, in nanoseconds, of search requests on the keyed node.

            • The number of outstanding search requests to the keyed node from the node these stats are for.

            • rank string

              The rank of this node; used for shard selection when routing search requests.

        • breakers object

          Statistics about the field data circuit breaker.

          Hide breakers attribute Show breakers attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • Estimated memory used for the operation.

            • Estimated memory used, in bytes, for the operation.

            • Memory limit for the circuit breaker.

            • Memory limit, in bytes, for the circuit breaker.

            • overhead number

              A constant that all estimates for the circuit breaker are multiplied with to calculate a final estimate.

            • tripped number

              Total number of times the circuit breaker has been triggered and prevented an out of memory error.

        • fs object
          Hide fs attributes Show fs attributes object
          • data array[object]

            List of all file stores.

          • Last time the file stores statistics were refreshed. Recorded in milliseconds since the Unix Epoch.

          • total object
            Hide total attributes Show total attributes object
            • Total disk space available to this Java virtual machine on all file stores. Depending on OS or process level restrictions, this might appear less than free. This is the actual amount of free disk space the Elasticsearch node can utilise.

            • Total number of bytes available to this Java virtual machine on all file stores. Depending on OS or process level restrictions, this might appear less than free_in_bytes. This is the actual amount of free disk space the Elasticsearch node can utilise.

            • free string

              Total unallocated disk space in all file stores.

            • Total number of unallocated bytes in all file stores.

            • total string

              Total size of all file stores.

            • Total size of all file stores in bytes.

          • io_stats object
            Hide io_stats attributes Show io_stats attributes object
            • devices array[object]

              Array of disk metrics for each device that is backing an Elasticsearch data path. These disk metrics are probed periodically and averages between the last probe and the current probe are computed.

            • total object
        • host string
        • http object
          Hide http attributes Show http attributes object
          • Current number of open HTTP connections for the node.

          • Total number of HTTP connections opened for the node.

          • clients array[object]

            Information on current and recently-closed HTTP client connections. Clients that have been closed longer than the http.client_stats.closed_channels.max_age setting will not be represented here.

          • routes object Required Added in 8.12.0

            Detailed HTTP stats broken down by route

            Hide routes attribute Show routes attribute object
            • * object Additional properties
        • ingest object
          Hide ingest attributes Show ingest attributes object
          • Contains statistics about ingest pipelines for the node.

            Hide pipelines attribute Show pipelines attribute object
            • * object Additional properties
          • total object
            Hide total attributes Show total attributes object
            • count number Required

              Total number of documents ingested during the lifetime of this node.

            • current number Required

              Total number of documents currently being ingested.

            • failed number Required

              Total number of failed ingest operations during the lifetime of this node.

        • ip string | array[string]

          IP address and port for the node.

        • jvm object
          Hide jvm attributes Show jvm attributes object
          • Contains statistics about JVM buffer pools for the node.

            Hide buffer_pools attribute Show buffer_pools attribute object
            • * object Additional properties
          • classes object
            Hide classes attributes Show classes attributes object
          • gc object
            Hide gc attribute Show gc attribute object
            • Contains statistics about JVM garbage collectors for the node.

          • mem object
            Hide mem attributes Show mem attributes object
          • threads object
            Hide threads attributes Show threads attributes object
            • count number

              Number of active threads in use by JVM.

            • Highest number of threads used by JVM.

          • Last time JVM statistics were refreshed.

          • uptime string

            Human-readable JVM uptime. Only returned if the human query parameter is true.

          • JVM uptime in milliseconds.

        • name string
        • os object
          Hide os attributes Show os attributes object
          • cpu object
            Hide cpu attributes Show cpu attributes object
            • percent number
            • sys string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • user string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • swap object
            Hide swap attributes Show swap attributes object
          • cgroup object
            Hide cgroup attributes Show cgroup attributes object
        • process object
          Hide process attributes Show process attributes object
          • cpu object
            Hide cpu attributes Show cpu attributes object
            • percent number
            • sys string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • user string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • mem object
            Hide mem attributes Show mem attributes object
          • Number of opened file descriptors associated with the current or -1 if not supported.

          • Maximum number of file descriptors allowed on the system, or -1 if not supported.

          • Last time the statistics were refreshed. Recorded in milliseconds since the Unix Epoch.

        • roles array[string]
          • @doc_id node-roles

          Values are master, data, data_cold, data_content, data_frozen, data_hot, data_warm, client, ingest, ml, voting_only, transform, remote_cluster_client, or coordinating_only.

        • script object
          Hide script attributes Show script attributes object
          • Total number of times the script cache has evicted old data.

          • Total number of inline script compilations performed by the node.

          • Contains this recent history of script compilations.

            Hide compilations_history attribute Show compilations_history attribute object
            • * number Additional properties
          • Total number of times the script compilation circuit breaker has limited inline script compilations.

          • contexts array[object]
        • Statistics about each thread pool, including current size, queue and rejected tasks.

          Hide thread_pool attribute Show thread_pool attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • active number

              Number of active threads in the thread pool.

            • Number of tasks completed by the thread pool executor.

            • largest number

              Highest number of active threads in the thread pool.

            • queue number

              Number of tasks in queue for the thread pool.

            • rejected number

              Number of tasks rejected by the thread pool executor.

            • threads number

              Number of threads in the thread pool.

        • Hide transport attributes Show transport attributes object
          • The distribution of the time spent handling each inbound message on a transport thread, represented as a histogram.

          • The distribution of the time spent sending each outbound transport message on a transport thread, represented as a histogram.

          • rx_count number

            Total number of RX (receive) packets received by the node during internal cluster communication.

          • rx_size string

            Size of RX packets received by the node during internal cluster communication.

          • Size, in bytes, of RX packets received by the node during internal cluster communication.

          • Current number of inbound TCP connections used for internal communication between nodes.

          • tx_count number

            Total number of TX (transmit) packets sent by the node during internal cluster communication.

          • tx_size string

            Size of TX packets sent by the node during internal cluster communication.

          • Size, in bytes, of TX packets sent by the node during internal cluster communication.

          • The cumulative number of outbound transport connections that this node has opened since it started. Each transport connection may comprise multiple TCP connections but is only counted once in this statistic. Transport connections are typically long-lived so this statistic should remain constant in a stable cluster.

        • Contains a list of attributes for the node.

          Hide attributes attribute Show attributes attribute object
          • * string Additional properties
        • Hide discovery attributes Show discovery attributes object
          • Hide cluster_state_queue attributes Show cluster_state_queue attributes object
            • total number

              Total number of cluster states in queue.

            • pending number

              Number of pending cluster states in queue.

            • Number of committed cluster states in queue.

          • Hide published_cluster_states attributes Show published_cluster_states attributes object
          • Contains low-level statistics about how long various activities took during cluster state updates while the node was the elected master. Omitted if the node is not master-eligible. Every field whose name ends in _time within this object is also represented as a raw number of milliseconds in a field whose name ends in _time_millis. The human-readable fields with a _time suffix are only returned if requested with the ?human=true query parameter.

            Hide cluster_state_update attribute Show cluster_state_update attribute object
            • * object Additional properties
          • Hide serialized_cluster_states attributes Show serialized_cluster_states attributes object
          • Hide cluster_applier_stats attribute Show cluster_applier_stats attribute object
        • Hide indexing_pressure attribute Show indexing_pressure attribute object
          • memory object
            Hide memory attributes Show memory attributes object
        • indices object
          Hide indices attributes Show indices attributes object
GET /_nodes/{node_id}/stats/{metric}
curl \
 --request GET 'http://api.example.com/_nodes/{node_id}/stats/{metric}' \
 --header "Authorization: $API_KEY"




Get node statistics

GET /_nodes/{node_id}/stats/{metric}/{index_metric}

Get statistics for nodes in a cluster. By default, all stats are returned. You can limit the returned information by using metrics.

Path parameters

  • node_id string | array[string] Required

    Comma-separated list of node IDs or names used to limit returned information.

  • metric string | array[string] Required

    Limit the information returned to the specified metrics

  • index_metric string | array[string] Required

    Limit the information returned for indices metric to the specific index metrics. It can be used only if indices (or all) metric is specified.

Query parameters

  • completion_fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in fielddata and suggest statistics.

  • fielddata_fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in fielddata statistics.

  • fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in the statistics.

  • groups boolean

    Comma-separated list of search groups to include in the search statistics.

  • If true, the call reports the aggregated disk usage of each one of the Lucene index files (only applies if segment stats are requested).

  • level string

    Indicates whether statistics are aggregated at the cluster, index, or shard level.

    Values are cluster, indices, or shards.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

  • types array[string]

    A comma-separated list of document types for the indexing index metric.

  • If true, the response includes information from segments that are not loaded into memory.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object
      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]
        Hide failures attributes Show failures attributes object
      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • nodes object Required
      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • Statistics about adaptive replica selection.

          Hide adaptive_selection attribute Show adaptive_selection attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • The exponentially weighted moving average queue size of search requests on the keyed node.

            • A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • The exponentially weighted moving average response time, in nanoseconds, of search requests on the keyed node.

            • A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • The exponentially weighted moving average service time, in nanoseconds, of search requests on the keyed node.

            • The number of outstanding search requests to the keyed node from the node these stats are for.

            • rank string

              The rank of this node; used for shard selection when routing search requests.

        • breakers object

          Statistics about the field data circuit breaker.

          Hide breakers attribute Show breakers attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • Estimated memory used for the operation.

            • Estimated memory used, in bytes, for the operation.

            • Memory limit for the circuit breaker.

            • Memory limit, in bytes, for the circuit breaker.

            • overhead number

              A constant that all estimates for the circuit breaker are multiplied with to calculate a final estimate.

            • tripped number

              Total number of times the circuit breaker has been triggered and prevented an out of memory error.

        • fs object
          Hide fs attributes Show fs attributes object
          • data array[object]

            List of all file stores.

          • Last time the file stores statistics were refreshed. Recorded in milliseconds since the Unix Epoch.

          • total object
            Hide total attributes Show total attributes object
            • Total disk space available to this Java virtual machine on all file stores. Depending on OS or process level restrictions, this might appear less than free. This is the actual amount of free disk space the Elasticsearch node can utilise.

            • Total number of bytes available to this Java virtual machine on all file stores. Depending on OS or process level restrictions, this might appear less than free_in_bytes. This is the actual amount of free disk space the Elasticsearch node can utilise.

            • free string

              Total unallocated disk space in all file stores.

            • Total number of unallocated bytes in all file stores.

            • total string

              Total size of all file stores.

            • Total size of all file stores in bytes.

          • io_stats object
            Hide io_stats attributes Show io_stats attributes object
            • devices array[object]

              Array of disk metrics for each device that is backing an Elasticsearch data path. These disk metrics are probed periodically and averages between the last probe and the current probe are computed.

            • total object
        • host string
        • http object
          Hide http attributes Show http attributes object
          • Current number of open HTTP connections for the node.

          • Total number of HTTP connections opened for the node.

          • clients array[object]

            Information on current and recently-closed HTTP client connections. Clients that have been closed longer than the http.client_stats.closed_channels.max_age setting will not be represented here.

          • routes object Required Added in 8.12.0

            Detailed HTTP stats broken down by route

            Hide routes attribute Show routes attribute object
            • * object Additional properties
        • ingest object
          Hide ingest attributes Show ingest attributes object
          • Contains statistics about ingest pipelines for the node.

            Hide pipelines attribute Show pipelines attribute object
            • * object Additional properties
          • total object
            Hide total attributes Show total attributes object
            • count number Required

              Total number of documents ingested during the lifetime of this node.

            • current number Required

              Total number of documents currently being ingested.

            • failed number Required

              Total number of failed ingest operations during the lifetime of this node.

        • ip string | array[string]

          IP address and port for the node.

        • jvm object
          Hide jvm attributes Show jvm attributes object
          • Contains statistics about JVM buffer pools for the node.

            Hide buffer_pools attribute Show buffer_pools attribute object
            • * object Additional properties
          • classes object
            Hide classes attributes Show classes attributes object
          • gc object
            Hide gc attribute Show gc attribute object
            • Contains statistics about JVM garbage collectors for the node.

          • mem object
            Hide mem attributes Show mem attributes object
          • threads object
            Hide threads attributes Show threads attributes object
            • count number

              Number of active threads in use by JVM.

            • Highest number of threads used by JVM.

          • Last time JVM statistics were refreshed.

          • uptime string

            Human-readable JVM uptime. Only returned if the human query parameter is true.

          • JVM uptime in milliseconds.

        • name string
        • os object
          Hide os attributes Show os attributes object
          • cpu object
            Hide cpu attributes Show cpu attributes object
            • percent number
            • sys string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • user string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • swap object
            Hide swap attributes Show swap attributes object
          • cgroup object
            Hide cgroup attributes Show cgroup attributes object
        • process object
          Hide process attributes Show process attributes object
          • cpu object
            Hide cpu attributes Show cpu attributes object
            • percent number
            • sys string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • user string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • mem object
            Hide mem attributes Show mem attributes object
          • Number of opened file descriptors associated with the current or -1 if not supported.

          • Maximum number of file descriptors allowed on the system, or -1 if not supported.

          • Last time the statistics were refreshed. Recorded in milliseconds since the Unix Epoch.

        • roles array[string]
          • @doc_id node-roles

          Values are master, data, data_cold, data_content, data_frozen, data_hot, data_warm, client, ingest, ml, voting_only, transform, remote_cluster_client, or coordinating_only.

        • script object
          Hide script attributes Show script attributes object
          • Total number of times the script cache has evicted old data.

          • Total number of inline script compilations performed by the node.

          • Contains this recent history of script compilations.

            Hide compilations_history attribute Show compilations_history attribute object
            • * number Additional properties
          • Total number of times the script compilation circuit breaker has limited inline script compilations.

          • contexts array[object]
        • Statistics about each thread pool, including current size, queue and rejected tasks.

          Hide thread_pool attribute Show thread_pool attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • active number

              Number of active threads in the thread pool.

            • Number of tasks completed by the thread pool executor.

            • largest number

              Highest number of active threads in the thread pool.

            • queue number

              Number of tasks in queue for the thread pool.

            • rejected number

              Number of tasks rejected by the thread pool executor.

            • threads number

              Number of threads in the thread pool.

        • Hide transport attributes Show transport attributes object
          • The distribution of the time spent handling each inbound message on a transport thread, represented as a histogram.

          • The distribution of the time spent sending each outbound transport message on a transport thread, represented as a histogram.

          • rx_count number

            Total number of RX (receive) packets received by the node during internal cluster communication.

          • rx_size string

            Size of RX packets received by the node during internal cluster communication.

          • Size, in bytes, of RX packets received by the node during internal cluster communication.

          • Current number of inbound TCP connections used for internal communication between nodes.

          • tx_count number

            Total number of TX (transmit) packets sent by the node during internal cluster communication.

          • tx_size string

            Size of TX packets sent by the node during internal cluster communication.

          • Size, in bytes, of TX packets sent by the node during internal cluster communication.

          • The cumulative number of outbound transport connections that this node has opened since it started. Each transport connection may comprise multiple TCP connections but is only counted once in this statistic. Transport connections are typically long-lived so this statistic should remain constant in a stable cluster.

        • Contains a list of attributes for the node.

          Hide attributes attribute Show attributes attribute object
          • * string Additional properties
        • Hide discovery attributes Show discovery attributes object
          • Hide cluster_state_queue attributes Show cluster_state_queue attributes object
            • total number

              Total number of cluster states in queue.

            • pending number

              Number of pending cluster states in queue.

            • Number of committed cluster states in queue.

          • Hide published_cluster_states attributes Show published_cluster_states attributes object
          • Contains low-level statistics about how long various activities took during cluster state updates while the node was the elected master. Omitted if the node is not master-eligible. Every field whose name ends in _time within this object is also represented as a raw number of milliseconds in a field whose name ends in _time_millis. The human-readable fields with a _time suffix are only returned if requested with the ?human=true query parameter.

            Hide cluster_state_update attribute Show cluster_state_update attribute object
            • * object Additional properties
          • Hide serialized_cluster_states attributes Show serialized_cluster_states attributes object
          • Hide cluster_applier_stats attribute Show cluster_applier_stats attribute object
        • Hide indexing_pressure attribute Show indexing_pressure attribute object
          • memory object
            Hide memory attributes Show memory attributes object
        • indices object
          Hide indices attributes Show indices attributes object
GET /_nodes/{node_id}/stats/{metric}/{index_metric}
curl \
 --request GET 'http://api.example.com/_nodes/{node_id}/stats/{metric}/{index_metric}' \
 --header "Authorization: $API_KEY"



































































































































































































Get cross-cluster replication stats Added in 6.5.0

GET /_ccr/stats

This API returns stats about auto-following and the same shard-level stats as the get follower stats API.

Query parameters

  • The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

  • timeout string

    The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

Responses

GET /_ccr/stats
curl \
 --request GET 'http://api.example.com/_ccr/stats' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_ccr/stats` that returns cross-cluster replication stats.
{
  "auto_follow_stats" : {
    "number_of_failed_follow_indices" : 0,
    "number_of_failed_remote_cluster_state_requests" : 0,
    "number_of_successful_follow_indices" : 1,
    "recent_auto_follow_errors" : [],
    "auto_followed_clusters" : []
  },
  "follow_stats" : {
    "indices" : [
      {
        "index" : "follower_index",
        "total_global_checkpoint_lag" : 256,
        "shards" : [
          {
            "remote_cluster" : "remote_cluster",
            "leader_index" : "leader_index",
            "follower_index" : "follower_index",
            "shard_id" : 0,
            "leader_global_checkpoint" : 1024,
            "leader_max_seq_no" : 1536,
            "follower_global_checkpoint" : 768,
            "follower_max_seq_no" : 896,
            "last_requested_seq_no" : 897,
            "outstanding_read_requests" : 8,
            "outstanding_write_requests" : 2,
            "write_buffer_operation_count" : 64,
            "follower_mapping_version" : 4,
            "follower_settings_version" : 2,
            "follower_aliases_version" : 8,
            "total_read_time_millis" : 32768,
            "total_read_remote_exec_time_millis" : 16384,
            "successful_read_requests" : 32,
            "failed_read_requests" : 0,
            "operations_read" : 896,
            "bytes_read" : 32768,
            "total_write_time_millis" : 16384,
            "write_buffer_size_in_bytes" : 1536,
            "successful_write_requests" : 16,
            "failed_write_requests" : 0,
            "operations_written" : 832,
            "read_exceptions" : [ ],
            "time_since_last_read_millis" : 8
          }
        ]
      }
    ]
  }
}










































































































Delete documents Added in 5.0.0

POST /{index}/_delete_by_query

Deletes documents that match the specified query.

If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or alias:

  • read
  • delete or write

You can specify the query criteria in the request URI or the request body using the same syntax as the search API. When you submit a delete by query request, Elasticsearch gets a snapshot of the data stream or index when it begins processing the request and deletes matching documents using internal versioning. If a document changes between the time that the snapshot is taken and the delete operation is processed, it results in a version conflict and the delete operation fails.

NOTE: Documents with a version equal to 0 cannot be deleted using delete by query because internal versioning does not support 0 as a valid version number.

While processing a delete by query request, Elasticsearch performs multiple search requests sequentially to find all of the matching documents to delete. A bulk delete request is performed for each batch of matching documents. If a search or bulk request is rejected, the requests are retried up to 10 times, with exponential back off. If the maximum retry limit is reached, processing halts and all failed requests are returned in the response. Any delete requests that completed successfully still stick, they are not rolled back.

You can opt to count version conflicts instead of halting and returning by setting conflicts to proceed. Note that if you opt to count version conflicts the operation could attempt to delete more documents from the source than max_docs until it has successfully deleted max_docs documents, or it has gone through every document in the source query.

Throttling delete requests

To control the rate at which delete by query issues batches of delete operations, you can set requests_per_second to any positive decimal number. This pads each batch with a wait time to throttle the rate. Set requests_per_second to -1 to disable throttling.

Throttling uses a wait time between batches so that the internal scroll requests can be given a timeout that takes the request padding into account. The padding time is the difference between the batch size divided by the requests_per_second and the time spent writing. By default the batch size is 1000, so if requests_per_second is set to 500:

target_time = 1000 / 500 per second = 2 seconds
wait_time = target_time - write_time = 2 seconds - .5 seconds = 1.5 seconds

Since the batch is issued as a single _bulk request, large batch sizes cause Elasticsearch to create many requests and wait before starting the next set. This is "bursty" instead of "smooth".

Slicing

Delete by query supports sliced scroll to parallelize the delete process. This can improve efficiency and provide a convenient way to break the request down into smaller parts.

Setting slices to auto lets Elasticsearch choose the number of slices to use. This setting will use one slice per shard, up to a certain limit. If there are multiple source data streams or indices, it will choose the number of slices based on the index or backing index with the smallest number of shards. Adding slices to the delete by query operation creates sub-requests which means it has some quirks:

  • You can see these requests in the tasks APIs. These sub-requests are "child" tasks of the task for the request with slices.
  • Fetching the status of the task for the request with slices only contains the status of completed slices.
  • These sub-requests are individually addressable for things like cancellation and rethrottling.
  • Rethrottling the request with slices will rethrottle the unfinished sub-request proportionally.
  • Canceling the request with slices will cancel each sub-request.
  • Due to the nature of slices each sub-request won't get a perfectly even portion of the documents. All documents will be addressed, but some slices may be larger than others. Expect larger slices to have a more even distribution.
  • Parameters like requests_per_second and max_docs on a request with slices are distributed proportionally to each sub-request. Combine that with the earlier point about distribution being uneven and you should conclude that using max_docs with slices might not result in exactly max_docs documents being deleted.
  • Each sub-request gets a slightly different snapshot of the source data stream or index though these are all taken at approximately the same time.

If you're slicing manually or otherwise tuning automatic slicing, keep in mind that:

  • Query performance is most efficient when the number of slices is equal to the number of shards in the index or backing index. If that number is large (for example, 500), choose a lower number as too many slices hurts performance. Setting slices higher than the number of shards generally does not improve efficiency and adds overhead.
  • Delete performance scales linearly across available resources with the number of slices.

Whether query or delete performance dominates the runtime depends on the documents being reindexed and cluster resources.

Cancel a delete by query operation

Any delete by query can be canceled using the task cancel API. For example:

POST _tasks/r1A2WoRbTwKZ516z6NEs5A:36619/_cancel

The task ID can be found by using the get tasks API.

Cancellation should happen quickly but might take a few seconds. The get task status API will continue to list the delete by query task until this task checks that it has been cancelled and terminates itself.

Path parameters

  • index string | array[string] Required

    A comma-separated list of data streams, indices, and aliases to search. It supports wildcards (*). To search all data streams or indices, omit this parameter or use * or _all.

Query parameters

  • If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • analyzer string

    Analyzer to use for the query string. This parameter can be used only when the q query string parameter is specified.

  • If true, wildcard and prefix queries are analyzed. This parameter can be used only when the q query string parameter is specified.

  • What to do if delete by query hits version conflicts: abort or proceed.

    Supported values include:

    • abort: Stop reindexing if there are conflicts.
    • proceed: Continue reindexing even if there are conflicts.

    Values are abort or proceed.

  • The default operator for query string query: AND or OR. This parameter can be used only when the q query string parameter is specified.

    Values are and, AND, or, or OR.

  • df string

    The field to use as default where no field prefix is given in the query string. This parameter can be used only when the q query string parameter is specified.

  • expand_wildcards string | array[string]

    The type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. It supports comma-separated values, such as open,hidden.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.
  • from number

    Skips the specified number of documents.

  • If false, the request returns an error if it targets a missing or closed index.

  • lenient boolean

    If true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored. This parameter can be used only when the q query string parameter is specified.

  • max_docs number

    The maximum number of documents to process. Defaults to all documents. When set to a value less then or equal to scroll_size, a scroll will not be used to retrieve the results for the operation.

  • The node or shard the operation should be performed on. It is random by default.

  • refresh boolean

    If true, Elasticsearch refreshes all shards involved in the delete by query after the request completes. This is different than the delete API's refresh parameter, which causes just the shard that received the delete request to be refreshed. Unlike the delete API, it does not support wait_for.

  • If true, the request cache is used for this request. Defaults to the index-level setting.

  • The throttle for this request in sub-requests per second.

  • routing string

    A custom value used to route operations to a specific shard.

  • q string

    A query in the Lucene query string syntax.

  • scroll string

    The period to retain the search context for scrolling.

  • The size of the scroll request that powers the operation.

  • The explicit timeout for each search request. It defaults to no timeout.

  • The type of the search operation. Available options include query_then_fetch and dfs_query_then_fetch.

    Supported values include:

    • query_then_fetch: Documents are scored using local term and document frequencies for the shard. This is usually faster but less accurate.
    • dfs_query_then_fetch: Documents are scored using global term and document frequencies across all shards. This is usually slower but more accurate.

    Values are query_then_fetch or dfs_query_then_fetch.

  • slices number | string

    The number of slices this task should be divided into.

  • sort array[string]

    A comma-separated list of <field>:<direction> pairs.

  • stats array[string]

    The specific tag of the request for logging and statistical purposes.

  • The maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting.

    Use with caution. Elasticsearch applies this parameter to each shard handling the request. When possible, let Elasticsearch perform early termination automatically. Avoid specifying this parameter for requests that target data streams with backing indices across multiple data tiers.

  • timeout string

    The period each deletion request waits for active shards.

  • version boolean

    If true, returns the document version as part of a hit.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). The timeout value controls how long each write request waits for unavailable shards to become available.

  • If true, the request blocks until the operation is complete. If false, Elasticsearch performs some preflight checks, launches the request, and returns a task you can use to cancel or get the status of the task. Elasticsearch creates a record of this task as a document at .tasks/task/${taskId}. When you are done with a task, you should delete the task document so Elasticsearch can reclaim the space.

application/json

Body Required

  • max_docs number

    The maximum number of documents to delete.

  • query object

    An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

    External documentation
  • slice object
    Hide slice attributes Show slice attributes object
    • field string

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • id string Required
    • max number Required

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • batches number

      The number of scroll responses pulled back by the delete by query.

    • deleted number

      The number of documents that were successfully deleted.

    • failures array[object]

      An array of failures if there were any unrecoverable errors during the process. If this array is not empty, the request ended abnormally because of those failures. Delete by query is implemented using batches and any failures cause the entire process to end but all failures in the current batch are collected into the array. You can use the conflicts option to prevent reindex from ending on version conflicts.

      Hide failures attributes Show failures attributes object
    • noops number

      This field is always equal to zero for delete by query. It exists only so that delete by query, update by query, and reindex APIs return responses with the same structure.

    • The number of requests per second effectively run during the delete by query.

    • retries object
      Hide retries attributes Show retries attributes object
      • bulk number Required

        The number of bulk actions retried.

    • slice_id number
    • A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • Time unit for milliseconds

    • A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • Time unit for milliseconds

    • timed_out boolean

      If true, some requests run during the delete by query operation timed out.

    • took number

      Time unit for milliseconds

    • total number

      The number of documents that were successfully processed.

    • The number of version conflicts that the delete by query hit.

POST /{index}/_delete_by_query
curl \
 --request POST 'http://api.example.com/{index}/_delete_by_query' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"query\": {\n    \"match_all\": {}\n  }\n}"'
Run `POST /my-index-000001,my-index-000002/_delete_by_query` to delete all documents from multiple data streams or indices.
{
  "query": {
    "match_all": {}
  }
}
Run `POST my-index-000001/_delete_by_query` to delete a document by using a unique attribute.
{
  "query": {
    "term": {
      "user.id": "kimchy"
    }
  },
  "max_docs": 1
}
Run `POST my-index-000001/_delete_by_query` to slice a delete by query manually. Provide a slice ID and total number of slices.
{
  "slice": {
    "id": 0,
    "max": 2
  },
  "query": {
    "range": {
      "http.response.bytes": {
        "lt": 2000000
      }
    }
  }
}
Run `POST my-index-000001/_delete_by_query?refresh&slices=5` to let delete by query automatically parallelize using sliced scroll to slice on `_id`. The `slices` query parameter value specifies the number of slices to use.
{
  "query": {
    "range": {
      "http.response.bytes": {
        "lt": 2000000
      }
    }
  }
}
Response examples (200)
A successful response from `POST /my-index-000001/_delete_by_query`.
{
  "took" : 147,
  "timed_out": false,
  "total": 119,
  "deleted": 119,
  "batches": 1,
  "version_conflicts": 0,
  "noops": 0,
  "retries": {
    "bulk": 0,
    "search": 0
  },
  "throttled_millis": 0,
  "requests_per_second": -1.0,
  "throttled_until_millis": 0,
  "failures" : [ ]
}








Check for a document source Added in 5.4.0

HEAD /{index}/_source/{id}

Check whether a document source exists in an index. For example:

HEAD my-index-000001/_source/1

A document's source is not available if it is disabled in the mapping.

External documentation

Path parameters

  • index string Required

    A comma-separated list of data streams, indices, and aliases. It supports wildcards (*).

  • id string Required

    A unique identifier for the document.

Query parameters

  • The node or shard the operation should be performed on. By default, the operation is randomized between the shard replicas.

  • realtime boolean

    If true, the request is real-time as opposed to near-real-time.

  • refresh boolean

    If true, the request refreshes the relevant shards before retrieving the document. Setting it to true should be done after careful thought and verification that this does not cause a heavy load on the system (and slow down indexing).

  • routing string

    A custom value used to route operations to a specific shard.

  • _source boolean | string | array[string]

    Indicates whether to return the _source field (true or false) or lists the fields to return.

  • _source_excludes string | array[string]

    A comma-separated list of source fields to exclude in the response.

  • _source_includes string | array[string]

    A comma-separated list of source fields to include in the response.

  • version number

    The version number for concurrency control. It must match the current version of the document for the request to succeed.

  • The version type.

    Supported values include:

    • internal: Use internal versioning that starts at 1 and increments with each update or delete.
    • external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.
    • external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document. NOTE: The external_gte version type is meant for special use cases and should be used with care. If used incorrectly, it can result in loss of data.
    • force: This option is deprecated because it can cause primary and replica shards to diverge.

    Values are internal, external, external_gte, or force.

Responses

HEAD /{index}/_source/{id}
curl \
 --request HEAD 'http://api.example.com/{index}/_source/{id}' \
 --header "Authorization: $API_KEY"









































































































































































Run multiple Fleet searches Technical preview

GET /{index}/_fleet/_fleet_msearch

Run several Fleet searches with a single API request. The API follows the same structure as the multi search API. However, similar to the Fleet search API, it supports the wait_for_checkpoints parameter.

Path parameters

  • index string Required

    A single target to search. If the target is an index alias, it must resolve to a single index.

Query parameters

  • If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • If true, network roundtrips between the coordinating node and remote clusters are minimized for cross-cluster search requests.

  • expand_wildcards string | array[string]

    Type of index that wildcard expressions can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.
  • If true, concrete, expanded or aliased indices are ignored when frozen.

  • If true, missing or closed indices are not included in the response.

  • Maximum number of concurrent searches the multi search API can execute.

  • Maximum number of concurrent shard requests that each sub-search request executes per node.

  • Defines a threshold that enforces a pre-filter roundtrip to prefilter search shards based on query rewriting if the number of shards the search request expands to exceeds the threshold. This filter roundtrip can limit the number of shards significantly if for instance a shard can not match any documents based on its rewrite method i.e., if date filters are mandatory to match but the shard bounds and the query are disjoint.

  • Indicates whether global term and document frequencies should be used when scoring returned documents.

    Supported values include:

    • query_then_fetch: Documents are scored using local term and document frequencies for the shard. This is usually faster but less accurate.
    • dfs_query_then_fetch: Documents are scored using global term and document frequencies across all shards. This is usually slower but more accurate.

    Values are query_then_fetch or dfs_query_then_fetch.

  • If true, hits.total are returned as an integer in the response. Defaults to false, which returns an object.

  • typed_keys boolean

    Specifies whether aggregation and suggester names should be prefixed by their respective types in the response.

  • A comma separated list of checkpoints. When configured, the search API will only be executed on a shard after the relevant checkpoint has become visible for search. Defaults to an empty list which will cause Elasticsearch to immediately execute the search.

  • If true, returns partial results if there are shard request timeouts or shard failures. If false, returns an error with no partial results. Defaults to the configured cluster setting search.default_allow_partial_results, which is true by default.

application/json

Body object Required

One of:

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
GET /{index}/_fleet/_fleet_msearch
curl \
 --request GET 'http://api.example.com/{index}/_fleet/_fleet_msearch' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '[{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"index":"string","preference":"string","request_cache":true,"routing":"string","search_type":"query_then_fetch","ccs_minimize_roundtrips":true,"allow_partial_search_results":true,"ignore_throttled":true}]'














































Import a dangling index Added in 7.9.0

POST /_dangling/{index_uuid}

If Elasticsearch encounters index data that is absent from the current cluster state, those indices are considered to be dangling. For example, this can happen if you delete more than cluster.indices.tombstones.size indices while an Elasticsearch node is offline.

Path parameters

  • index_uuid string Required

    The UUID of the index to import. Use the get dangling indices API to locate the UUID.

Query parameters

  • accept_data_loss boolean Required

    This parameter must be set to true to import a dangling index. Because Elasticsearch cannot know where the dangling index data came from or determine which shard copies are fresh and which are stale, it cannot guarantee that the imported data represents the latest state of the index when it was last in the cluster.

  • Specify timeout for connection to master

  • timeout string

    Explicit operation timeout

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

POST /_dangling/{index_uuid}
curl \
 --request POST 'http://api.example.com/_dangling/{index_uuid}?accept_data_loss=true' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `POST /_dangling/zmM4e0JtBkeUjiHD-MihPQ?accept_data_loss=true`.
{
  "acknowledged": true
}












































































































































































































Get mapping definitions

GET /{index}/_mapping

For data streams, the API retrieves mappings for the stream’s backing indices.

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.

Query parameters

  • If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.
  • If false, the request returns an error if it targets a missing or closed index.

  • local boolean Deprecated

    If true, the request retrieves information from the local node only.

  • Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

Responses

GET /{index}/_mapping
curl \
 --request GET 'http://api.example.com/{index}/_mapping' \
 --header "Authorization: $API_KEY"








































































Resolve the cluster Added in 8.13.0

GET /_resolve/cluster

Resolve the specified index expressions to return information about each cluster, including the local "querying" cluster, if included. If no index expression is provided, the API will return information about all the remote clusters that are configured on the querying cluster.

This endpoint is useful before doing a cross-cluster search in order to determine which remote clusters should be included in a search.

You use the same index expression with this endpoint as you would for cross-cluster search. Index and cluster exclusions are also supported with this endpoint.

For each cluster in the index expression, information is returned about:

  • Whether the querying ("local") cluster is currently connected to each remote cluster specified in the index expression. Note that this endpoint actively attempts to contact the remote clusters, unlike the remote/info endpoint.
  • Whether each remote cluster is configured with skip_unavailable as true or false.
  • Whether there are any indices, aliases, or data streams on that cluster that match the index expression.
  • Whether the search is likely to have errors returned when you do the cross-cluster search (including any authorization errors if you do not have permission to query the index).
  • Cluster version information, including the Elasticsearch server version.

For example, GET /_resolve/cluster/my-index-*,cluster*:my-index-* returns information about the local cluster and all remotely configured clusters that start with the alias cluster*. Each cluster returns information about whether it has any indices, aliases or data streams that match my-index-*.

Note on backwards compatibility

The ability to query without an index expression was added in version 8.18, so when querying remote clusters older than that, the local cluster will send the index expression dummy* to those remote clusters. Thus, if an errors occur, you may see a reference to that index expression even though you didn't request it. If it causes a problem, you can instead include an index expression like *:* to bypass the issue.

You may want to exclude a cluster or index from a search when:

  • A remote cluster is not currently connected and is configured with skip_unavailable=false. Running a cross-cluster search under those conditions will cause the entire search to fail.
  • A cluster has no matching indices, aliases or data streams for the index expression (or your user does not have permissions to search them). For example, suppose your index expression is logs*,remote1:logs* and the remote1 cluster has no indices, aliases or data streams that match logs*. In that case, that cluster will return no results from that cluster if you include it in a cross-cluster search.
  • The index expression (combined with any query parameters you specify) will likely cause an exception to be thrown when you do the search. In these cases, the "error" field in the _resolve/cluster response will be present. (This is also where security/permission errors will be shown.)
  • A remote cluster is an older version that does not support the feature you want to use in your search.

Test availability of remote clusters

The remote/info endpoint is commonly used to test whether the "local" cluster (the cluster being queried) is connected to its remote clusters, but it does not necessarily reflect whether the remote cluster is available or not. The remote cluster may be available, while the local cluster is not currently connected to it.

You can use the _resolve/cluster API to attempt to reconnect to remote clusters. For example with GET _resolve/cluster or GET _resolve/cluster/*:*. The connected field in the response will indicate whether it was successful. If a connection was (re-)established, this will also cause the remote/info endpoint to now indicate a connected status.

Query parameters

  • If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar. NOTE: This option is only supported when specifying an index expression. You will get an error if you specify index options to the _resolve/cluster API endpoint that takes no index expression.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none. NOTE: This option is only supported when specifying an index expression. You will get an error if you specify index options to the _resolve/cluster API endpoint that takes no index expression.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.
  • ignore_throttled boolean Deprecated

    If true, concrete, expanded, or aliased indices are ignored when frozen. NOTE: This option is only supported when specifying an index expression. You will get an error if you specify index options to the _resolve/cluster API endpoint that takes no index expression.

  • If false, the request returns an error if it targets a missing or closed index. NOTE: This option is only supported when specifying an index expression. You will get an error if you specify index options to the _resolve/cluster API endpoint that takes no index expression.

  • timeout string

    The maximum time to wait for remote clusters to respond. If a remote cluster does not respond within this timeout period, the API response will show the cluster as not connected and include an error message that the request timed out.

    The default timeout is unset and the query can take as long as the networking layer is configured to wait for remote clusters that are not responding (typically 30 seconds).

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
GET /_resolve/cluster
curl \
 --request GET 'http://api.example.com/_resolve/cluster' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_resolve/cluster/my-index*,clust*:my-index*`. Each cluster has its own response section. The cluster you sent the request to is labelled as "(local)".
{
  "(local)": {
    "connected": true,
    "skip_unavailable": false,
    "matching_indices": true,
    "version": {
      "number": "8.13.0",
      "build_flavor": "default",
      "minimum_wire_compatibility_version": "7.17.0",
      "minimum_index_compatibility_version": "7.0.0"
    }
  },
  "cluster_one": {
    "connected": true,
    "skip_unavailable": true,
    "matching_indices": true,
    "version": {
      "number": "8.13.0",
      "build_flavor": "default",
      "minimum_wire_compatibility_version": "7.17.0",
      "minimum_index_compatibility_version": "7.0.0"
    }
  },
  "cluster_two": {
    "connected": true,
    "skip_unavailable": false,
    "matching_indices": true,
    "version": {
      "number": "8.13.0",
      "build_flavor": "default",
      "minimum_wire_compatibility_version": "7.17.0",
      "minimum_index_compatibility_version": "7.0.0"
    }
  }
}
A successful response from `GET /_resolve/cluster/not-present,clust*:my-index*,oldcluster:*?ignore_unavailable=false&timeout=5s`. This type of request can be used to identify potential problems with your cross-cluster search. Note also that a `timeout` of 5 seconds is sent, which sets the maximum time the query will wait for remote clusters to respond. The local cluster has no index called `not_present`. Searching with `ignore_unavailable=false` would return a "no such index" error. The `cluster_one` remote cluster has no indices that match the pattern `my-index*`. There may be no indices that match the pattern or the index could be closed. The `cluster_two` remote cluster is not connected (the attempt to connect failed). Since this cluster is marked as `skip_unavailable=false`, you should probably exclude this cluster from the search by adding `-cluster_two:*` to the search index expression. For `cluster_three`, the error message indicates that this remote cluster did not respond within the 5-second timeout window specified, so it is also marked as not connected. The `oldcluster` remote cluster shows that it has matching indices, but no version information is included. This indicates that the cluster version predates the introduction of the `_resolve/cluster` API, so you may want to exclude it from your cross-cluster search.
{
  "(local)": {
    "connected": true,
    "skip_unavailable": false,
    "error": "no such index [not_present]"
  },
  "cluster_one": {
    "connected": true,
    "skip_unavailable": true,
    "matching_indices": false,
    "version": {
      "number": "8.13.0",
      "build_flavor": "default",
      "minimum_wire_compatibility_version": "7.17.0",
      "minimum_index_compatibility_version": "7.0.0"
    }
  },
  "cluster_two": {
    "connected": false,
    "skip_unavailable": false
  },
  "cluster_three": {
    "connected": false,
    "skip_unavailable": false,
    "error": "Request timed out before receiving a response from the remote cluster"
  },
  "oldcluster": {
    "connected": true,
    "skip_unavailable": false,
    "matching_indices": true
  }
}













































































































Explain the lifecycle state Added in 6.6.0

GET /{index}/_ilm/explain

Get the current lifecycle status for one or more indices. For data streams, the API retrieves the current lifecycle status for the stream's backing indices.

The response indicates when the index entered each lifecycle state, provides the definition of the running phase, and information about any failures.

Path parameters

  • index string Required

    Comma-separated list of data streams, indices, and aliases to target. Supports wildcards (*). To target all data streams and indices, use * or _all.

Query parameters

  • Filters the returned indices to only indices that are managed by ILM and are in an error state, either due to an encountering an error while executing the policy, or attempting to use a policy that does not exist.

  • Filters the returned indices to only indices that are managed by ILM.

  • Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
GET /{index}/_ilm/explain
curl \
 --request GET 'http://api.example.com/{index}/_ilm/explain' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response when retrieving the current ILM status for an index.
{
  "indices": {
    "my-index-000001": {
      "index": "my-index-000001",
      "index_creation_date_millis": 1538475653281,
      "index_creation_date": "2018-10-15T13:45:21.981Z",
      "time_since_index_creation": "15s",
      "managed": true,
      "policy": "my_policy",
      "lifecycle_date_millis": 1538475653281,
      "lifecycle_date": "2018-10-15T13:45:21.981Z",
      "age": "15s",
      "phase": "new",
      "phase_time_millis": 1538475653317,
      "phase_time": "2018-10-15T13:45:22.577Z",
      "action": "complete"
      "action_time_millis": 1538475653317,
      "action_time": "2018-10-15T13:45:22.577Z",
      "step": "complete",
      "step_time_millis": 1538475653317,
      "step_time": "2018-10-15T13:45:22.577Z"
    }
  }
}

Query parameters

  • Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

Responses

GET /_ilm/policy
curl \
 --request GET 'http://api.example.com/_ilm/policy' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response when retrieving a lifecycle policy.
{
  "my_policy": {
    "version": 1,
    "modified_date": 82392349,
    "policy": {
      "phases": {
        "warm": {
          "min_age": "10d",
          "actions": {
            "forcemerge": {
              "max_num_segments": 1
            }
          }
        },
        "delete": {
          "min_age": "30d",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        }
      }
    },
    "in_use_by" : {
      "indices" : [],
      "data_streams" : [],
      "composable_templates" : []
    }
  }
}




























Inference

Inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.













































































Create an Google AI Studio inference endpoint Added in 8.15.0

PUT /_inference/{task_type}/{googleaistudio_inference_id}

Create an inference endpoint to perform an inference task with the googleaistudio service.

When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running. After creating the endpoint, wait for the model deployment to complete before using it. To verify the deployment status, use the get trained model statistics API. Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count". Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.

Path parameters

  • task_type string Required

    The type of the inference task that the model will perform.

    Values are completion or text_embedding.

  • The unique identifier of the inference endpoint.

application/json

Body

  • Hide chunking_settings attributes Show chunking_settings attributes object
    • The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

    • overlap number

      The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

    • The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

    • strategy string

      The chunking strategy: sentence or word.

  • service string Required

    Value is googleaistudio.

  • service_settings object Required
    Hide service_settings attributes Show service_settings attributes object
    • api_key string Required

      A valid API key of your Google Gemini account.

    • model_id string Required

      The name of the model to use for the inference task. Refer to the Google documentation for the list of supported models.

      External documentation
    • Hide rate_limit attribute Show rate_limit attribute object

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • Hide chunking_settings attributes Show chunking_settings attributes object
      • The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

      • overlap number

        The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

      • The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

      • strategy string

        The chunking strategy: sentence or word.

    • service string Required

      The service type

    • service_settings object Required
    • inference_id string Required

      The inference Id

    • task_type string Required

      Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.

PUT /_inference/{task_type}/{googleaistudio_inference_id}
curl \
 --request PUT 'http://api.example.com/_inference/{task_type}/{googleaistudio_inference_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"service\": \"googleaistudio\",\n    \"service_settings\": {\n        \"api_key\": \"api-key\",\n        \"model_id\": \"model-id\"\n    }\n}"'
Request example
Run `PUT _inference/completion/google_ai_studio_completion` to create an inference endpoint to perform a `completion` task type.
{
    "service": "googleaistudio",
    "service_settings": {
        "api_key": "api-key",
        "model_id": "model-id"
    }
}






































































































































Simulate data ingestion Technical preview

POST /_ingest/_simulate

Run ingest pipelines against a set of provided documents, optionally with substitute pipeline definitions, to simulate ingesting data into an index.

This API is meant to be used for troubleshooting or pipeline development, as it does not actually index any data into Elasticsearch.

The API runs the default and final pipeline for that index against a set of documents provided in the body of the request. If a pipeline contains a reroute processor, it follows that reroute processor to the new index, running that index's pipelines as well the same way that a non-simulated ingest would. No data is indexed into Elasticsearch. Instead, the transformed document is returned, along with the list of pipelines that have been run and the name of the index where the document would have been indexed if this were not a simulation. The transformed document is validated against the mappings that would apply to this index, and any validation error is reported in the result.

This API differs from the simulate pipeline API in that you specify a single pipeline for that API, and it runs only that one pipeline. The simulate pipeline API is more useful for developing a single pipeline, while the simulate ingest API is more useful for troubleshooting the interaction of the various pipelines that get applied when ingesting into an index.

By default, the pipeline definitions that are currently in the system are used. However, you can supply substitute pipeline definitions in the body of the request. These will be used in place of the pipeline definitions that are already in the system. This can be used to replace existing pipeline definitions or to create new ones. The pipeline substitutions are used only within this request.

Query parameters

  • pipeline string

    The pipeline to use as the default pipeline. This value can be used to override the default pipeline of the index.

application/json

Body Required

  • docs array[object] Required

    Sample documents to test in the pipeline.

    Hide docs attributes Show docs attributes object
  • A map of component template names to substitute component template definition objects.

    Hide component_template_substitutions attribute Show component_template_substitutions attribute object
  • A map of index template names to substitute index template definition objects.

    Hide index_template_substitutions attribute Show index_template_substitutions attribute object
  • Hide mapping_addition attributes Show mapping_addition attributes object
    • Hide all_field attributes Show all_field attributes object
    • dynamic string

      Values are strict, runtime, true, or false.

    • dynamic_templates array[object]
    • Hide _field_names attribute Show _field_names attribute object
    • Hide index_field attribute Show index_field attribute object
    • _meta object
      Hide _meta attribute Show _meta attribute object
      • * object Additional properties
    • _routing object
      Hide _routing attribute Show _routing attribute object
    • _size object
      Hide _size attribute Show _size attribute object
    • _source object
      Hide _source attributes Show _source attributes object
    • runtime object
      Hide runtime attribute Show runtime attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • fields object

          For type composite

          Hide fields attribute Show fields attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
            • type string Required

              Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

        • fetch_fields array[object]

          For type lookup

          Hide fetch_fields attributes Show fetch_fields attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • format string
        • format string

          A custom format for date type runtime fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • script object
          Hide script attributes Show script attributes object
          • source string | object

            One of:
          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            Hide params attribute Show params attribute object
            • * object Additional properties
          • lang string

            Any of:

            Values are painless, expression, mustache, or java.

          • options object
            Hide options attribute Show options attribute object
            • * string Additional properties
        • type string Required

          Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

    • enabled boolean
    • Values are true or false.

    • Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
  • Pipelines to test. If you don’t specify the pipeline request path parameter, this parameter is required. If you specify both this and the request path parameter, the API only uses the request path parameter.

    Hide pipeline_substitutions attribute Show pipeline_substitutions attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • Description of the ingest pipeline.

      • on_failure array[object]

        Processors to run immediately after a processor failure.

        Hide on_failure attributes Show on_failure attributes object
        • append object
          Hide append attributes Show append attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If false, the processor does not append values already present in the field.

        • Hide attachment attributes Show attachment attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true, the binary field will be removed from the document

          • Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

        • bytes object
          Hide bytes attributes Show bytes attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • circle object
          Hide circle attributes Show circle attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • error_distance number Required

            The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • shape_type string Required

            Values are geo_shape or shape.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Hide community_id attributes Show community_id attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • seed number

            Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

          • If true and any required fields are missing, the processor quietly exits without modifying the document.

        • convert object
          Hide convert attributes Show convert attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • type string Required

            Values are integer, long, double, float, boolean, ip, string, or auto.

        • csv object
          Hide csv attributes Show csv attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • quote string

            Quote used in CSV, has to be single character string.

          • Separator used in CSV, has to be single character string.

          • target_fields string | array[string] Required
          • trim boolean

            Trim whitespaces in unquoted fields.

        • date object
          Hide date attributes Show date attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • formats array[string] Required

            An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

          • locale string

            The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • timezone string

            The timezone to use when parsing the date. Supports template snippets.

          • The format to use when writing the date to target_field. Must be a valid java time pattern.

        • Hide date_index_name attributes Show date_index_name attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • date_formats array[string]

            An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

          • date_rounding string Required

            How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

          • A prefix of the index name to be prepended before the printed date. Supports template snippets.

          • locale string

            The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

          • timezone string

            The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

        • dissect object
          Hide dissect attributes Show dissect attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • The character(s) that separate the appended fields.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern string Required

            The pattern to apply to the field.

        • Hide dot_expander attributes Show dot_expander attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • override boolean

            Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

          • path string

            The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

        • drop object
          Hide drop attributes Show drop attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

        • enrich object
          Hide enrich attributes Show enrich attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

          • override boolean

            If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

          • policy_name string Required

            The name of the enrich policy to use.

          • Values are intersects, disjoint, within, or contains.

          • target_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • fail object
          Hide fail attributes Show fail attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • message string Required

            The error message thrown by the processor. Supports template snippets.

        • Hide fingerprint attributes Show fingerprint attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • fields string | array[string] Required
          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • salt string

            Salt value for the hash function.

          • method string

            Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

          • If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

        • foreach object
          Hide foreach attributes Show foreach attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true, the processor silently exits without changing the document if the field is null or missing.

          • processor object Required
        • Hide ip_location attributes Show ip_location attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • first_only boolean

            If true, only the first found IP location data will be returned, even if the field contains an array.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • properties array[string]

            Controls what properties are added to the target_field based on the IP location lookup.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

        • geo_grid object
          Hide geo_grid attributes Show geo_grid attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            The field to interpret as a geo-tile.= The field format is determined by the tile_type.

          • tile_type string Required

            Values are geotile, geohex, or geohash.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • Values are geojson or wkt.

        • geoip object
          Hide geoip attributes Show geoip attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • first_only boolean

            If true, only the first found geoip data will be returned, even if the field contains an array.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • properties array[string]

            Controls what properties are added to the target_field based on the geoip lookup.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

        • grok object
          Hide grok attributes Show grok attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

          • patterns array[string] Required

            An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

          • When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

        • gsub object
          Hide gsub attributes Show gsub attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern string Required

            The pattern to be replaced.

          • replacement string Required

            The string to replace the matching patterns with.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Hide html_strip attributes Show html_strip attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist or is null, the processor quietly exits without modifying the document,

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Hide inference attributes Show inference attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • model_id string Required
          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

          • If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.

        • join object
          Hide join attributes Show join attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • separator string Required

            The separator character.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • json object
          Hide json attributes Show json attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

          • Values are replace or merge.

          • When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • kv object
          Hide kv attributes Show kv attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • exclude_keys array[string]

            List of keys to exclude from document.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field_split string Required

            Regex pattern to use for splitting key-value pairs.

          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • include_keys array[string]

            List of keys to filter and insert into document. Defaults to including all keys.

          • prefix string

            Prefix to be added to extracted keys.

          • If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • trim_key string

            String of characters to trim from extracted keys.

          • String of characters to trim from extracted values.

          • value_split string Required

            Regex pattern to use for splitting the key from the value within a key-value pair.

        • Hide lowercase attributes Show lowercase attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Hide network_direction attributes Show network_direction attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • internal_networks array[string]

            List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and any required fields are missing, the processor quietly exits without modifying the document.

        • pipeline object
          Hide pipeline attributes Show pipeline attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • name string Required
          • Whether to ignore missing pipelines instead of failing.

        • redact object
          Hide redact attributes Show redact attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • patterns array[string] Required

            A list of grok expressions to match and redact named captures with

          • prefix string

            Start a redacted section with this token

          • suffix string

            End a redacted section with this token

          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

          • If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

        • Hide registered_domain attributes Show registered_domain attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and any required fields are missing, the processor quietly exits without modifying the document.

        • remove object
          Hide remove attributes Show remove attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string | array[string] Required
          • keep string | array[string]
          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • rename object
          Hide rename attributes Show rename attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • target_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • reroute object
          Hide reroute attributes Show reroute attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • A static value for the target. Can’t be set when the dataset or namespace option is set.

        • script object
          Hide script attributes Show script attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • id string
          • params object

            Object containing parameters for the script.

        • set object
          Hide set attributes Show set attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

          • The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

          • override boolean

            If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

          • value object

            The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

        • Hide set_security_user attributes Show set_security_user attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Controls what user related properties are added to the field.

        • sort object
          Hide sort attributes Show sort attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • order string

            Values are asc or desc.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • split object
          Hide split attributes Show split attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • Preserves empty trailing fields, if any.

          • separator string Required

            A regex which matches the separator, for example, , or \s+.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Hide terminate attributes Show terminate attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

        • trim object
          Hide trim attributes Show trim attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Hide uppercase attributes Show uppercase attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Hide urldecode attributes Show urldecode attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Hide uri_parts attributes Show uri_parts attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • If true, the processor copies the unparsed URI to <target_field>.original.

          • If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Hide user_agent attributes Show user_agent attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Controls what properties are added to target_field.

            Values are name, os, device, original, or version.

          • Extracts device type from the user agent string on a best-effort basis.

      • processors array[object]

        Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.

        Hide processors attributes Show processors attributes object
        • append object
          Hide append attributes Show append attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If false, the processor does not append values already present in the field.

        • Hide attachment attributes Show attachment attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true, the binary field will be removed from the document

          • Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

        • bytes object
          Hide bytes attributes Show bytes attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • circle object
          Hide circle attributes Show circle attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • error_distance number Required

            The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • shape_type string Required

            Values are geo_shape or shape.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Hide community_id attributes Show community_id attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • seed number

            Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

          • If true and any required fields are missing, the processor quietly exits without modifying the document.

        • convert object
          Hide convert attributes Show convert attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • type string Required

            Values are integer, long, double, float, boolean, ip, string, or auto.

        • csv object
          Hide csv attributes Show csv attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • quote string

            Quote used in CSV, has to be single character string.

          • Separator used in CSV, has to be single character string.

          • target_fields string | array[string] Required
          • trim boolean

            Trim whitespaces in unquoted fields.

        • date object
          Hide date attributes Show date attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • formats array[string] Required

            An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

          • locale string

            The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • timezone string

            The timezone to use when parsing the date. Supports template snippets.

          • The format to use when writing the date to target_field. Must be a valid java time pattern.

        • Hide date_index_name attributes Show date_index_name attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • date_formats array[string]

            An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

          • date_rounding string Required

            How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

          • A prefix of the index name to be prepended before the printed date. Supports template snippets.

          • locale string

            The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

          • timezone string

            The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

        • dissect object
          Hide dissect attributes Show dissect attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • The character(s) that separate the appended fields.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern string Required

            The pattern to apply to the field.

        • Hide dot_expander attributes Show dot_expander attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • override boolean

            Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

          • path string

            The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

        • drop object
          Hide drop attributes Show drop attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

        • enrich object
          Hide enrich attributes Show enrich attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

          • override boolean

            If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

          • policy_name string Required

            The name of the enrich policy to use.

          • Values are intersects, disjoint, within, or contains.

          • target_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • fail object
          Hide fail attributes Show fail attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • message string Required

            The error message thrown by the processor. Supports template snippets.

        • Hide fingerprint attributes Show fingerprint attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • fields string | array[string] Required
          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • salt string

            Salt value for the hash function.

          • method string

            Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

          • If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

        • foreach object
          Hide foreach attributes Show foreach attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true, the processor silently exits without changing the document if the field is null or missing.

          • processor object Required
        • Hide ip_location attributes Show ip_location attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • first_only boolean

            If true, only the first found IP location data will be returned, even if the field contains an array.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • properties array[string]

            Controls what properties are added to the target_field based on the IP location lookup.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

        • geo_grid object
          Hide geo_grid attributes Show geo_grid attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            The field to interpret as a geo-tile.= The field format is determined by the tile_type.

          • tile_type string Required

            Values are geotile, geohex, or geohash.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • Values are geojson or wkt.

        • geoip object
          Hide geoip attributes Show geoip attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • first_only boolean

            If true, only the first found geoip data will be returned, even if the field contains an array.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • properties array[string]

            Controls what properties are added to the target_field based on the geoip lookup.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

        • grok object
          Hide grok attributes Show grok attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

          • patterns array[string] Required

            An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

          • When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

        • gsub object
          Hide gsub attributes Show gsub attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern string Required

            The pattern to be replaced.

          • replacement string Required

            The string to replace the matching patterns with.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Hide html_strip attributes Show html_strip attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist or is null, the processor quietly exits without modifying the document,

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Hide inference attributes Show inference attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • model_id string Required
          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

          • If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.

        • join object
          Hide join attributes Show join attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • separator string Required

            The separator character.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • json object
          Hide json attributes Show json attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

          • Values are replace or merge.

          • When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • kv object
          Hide kv attributes Show kv attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • exclude_keys array[string]

            List of keys to exclude from document.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field_split string Required

            Regex pattern to use for splitting key-value pairs.

          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • include_keys array[string]

            List of keys to filter and insert into document. Defaults to including all keys.

          • prefix string

            Prefix to be added to extracted keys.

          • If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • trim_key string

            String of characters to trim from extracted keys.

          • String of characters to trim from extracted values.

          • value_split string Required

            Regex pattern to use for splitting the key from the value within a key-value pair.

        • Hide lowercase attributes Show lowercase attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Hide network_direction attributes Show network_direction attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • internal_networks array[string]

            List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and any required fields are missing, the processor quietly exits without modifying the document.

        • pipeline object
          Hide pipeline attributes Show pipeline attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • name string Required
          • Whether to ignore missing pipelines instead of failing.

        • redact object
          Hide redact attributes Show redact attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • patterns array[string] Required

            A list of grok expressions to match and redact named captures with

          • prefix string

            Start a redacted section with this token

          • suffix string

            End a redacted section with this token

          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

          • If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

        • Hide registered_domain attributes Show registered_domain attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and any required fields are missing, the processor quietly exits without modifying the document.

        • remove object
          Hide remove attributes Show remove attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string | array[string] Required
          • keep string | array[string]
          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • rename object
          Hide rename attributes Show rename attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • target_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • reroute object
          Hide reroute attributes Show reroute attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • A static value for the target. Can’t be set when the dataset or namespace option is set.

        • script object
          Hide script attributes Show script attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • id string
          • params object

            Object containing parameters for the script.

        • set object
          Hide set attributes Show set attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

          • The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

          • override boolean

            If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

          • value object

            The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

        • Hide set_security_user attributes Show set_security_user attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Controls what user related properties are added to the field.

        • sort object
          Hide sort attributes Show sort attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • order string

            Values are asc or desc.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • split object
          Hide split attributes Show split attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • Preserves empty trailing fields, if any.

          • separator string Required

            A regex which matches the separator, for example, , or \s+.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Hide terminate attributes Show terminate attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

        • trim object
          Hide trim attributes Show trim attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Hide uppercase attributes Show uppercase attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Hide urldecode attributes Show urldecode attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Hide uri_parts attributes Show uri_parts attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • If true, the processor copies the unparsed URI to <target_field>.original.

          • If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Hide user_agent attributes Show user_agent attributes object
          • Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if object
          • Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • If true and field does not exist, the processor quietly exits without modifying the document.

          • The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Controls what properties are added to target_field.

            Values are name, os, device, original, or version.

          • Extracts device type from the user agent string on a best-effort basis.

      • version number
      • deprecated boolean

        Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.

      • _meta object
        Hide _meta attribute Show _meta attribute object
        • * object Additional properties

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • docs array[object] Required
      Hide docs attribute Show docs attribute object
      • doc object
        Hide doc attributes Show doc attributes object
        • _id string Required
        • _index string Required
        • _source object Required

          JSON body for the document.

          Hide _source attribute Show _source attribute object
          • * object Additional properties
        • _version number | string Required

          Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

          Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

        • executed_pipelines array[string] Required

          A list of the names of the pipelines executed on this document.

        • ignored_fields array[object]

          A list of the fields that would be ignored at the indexing step. For example, a field whose value is larger than the allowed limit would make it through all of the pipelines, but would not be indexed into Elasticsearch.

          Hide ignored_fields attribute Show ignored_fields attribute object
          • * string Additional properties
        • error object
          Hide error attributes Show error attributes object
POST /_ingest/_simulate
curl \
 --request POST 'http://api.example.com/_ingest/_simulate' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"docs\": [\n    {\n      \"_id\": 123,\n      \"_index\": \"my-index\",\n      \"_source\": {\n        \"foo\": \"bar\"\n      }\n    },\n    {\n      \"_id\": 456,\n      \"_index\": \"my-index\",\n      \"_source\": {\n        \"foo\": \"rab\"\n      }\n    }\n  ]\n}"'
In this example the index `my-index` has a default pipeline called `my-pipeline` and a final pipeline called `my-final-pipeline`. Since both documents are being ingested into `my-index`, both pipelines are run using the pipeline definitions that are already in the system.
{
  "docs": [
    {
      "_id": 123,
      "_index": "my-index",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_id": 456,
      "_index": "my-index",
      "_source": {
        "foo": "rab"
      }
    }
  ]
}
In this example the index `my-index` has a default pipeline called `my-pipeline` and a final pipeline called `my-final-pipeline`. But a substitute definition of `my-pipeline` is provided in `pipeline_substitutions`. The substitute `my-pipeline` will be used in place of the `my-pipeline` that is in the system, and then the `my-final-pipeline` that is already defined in the system will run.
{
  "docs": [
    {
      "_index": "my-index",
      "_id": 123,
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_index": "my-index",
      "_id": 456,
      "_source": {
        "foo": "rab"
      }
    }
  ],
  "pipeline_substitutions": {
    "my-pipeline": {
      "processors": [
        {
          "uppercase": {
            "field": "foo"
          }
        }
      ]
    }
  }
}
In this example, imagine that the index `my-index` has a strict mapping with only the `foo` keyword field defined. Say that field mapping came from a component template named `my-mappings-template`. You want to test adding a new field, `bar`. So a substitute definition of `my-mappings-template` is provided in `component_template_substitutions`. The substitute `my-mappings-template` will be used in place of the existing mapping for `my-index` and in place of the `my-mappings-template` that is in the system.
{
  "docs": [
    {
      "_index": "my-index",
      "_id": "123",
      "_source": {
        "foo": "foo"
      }
    },
    {
      "_index": "my-index",
      "_id": "456",
      "_source": {
        "bar": "rab"
      }
    }
  ],
  "component_template_substitutions": {
    "my-mappings_template": {
      "template": {
        "mappings": {
          "dynamic": "strict",
          "properties": {
            "foo": {
              "type": "keyword"
            },
            "bar": {
              "type": "keyword"
            }
          }
        }
      }
    }
  }
}
The pipeline, component template, and index template substitutions replace the existing pipeline details for the duration of this request.
{
  "docs": [
    {
      "_id": "id",
      "_index": "my-index",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_id": "id",
      "_index": "my-index",
      "_source": {
        "foo": "rab"
      }
    }
  ],
  "pipeline_substitutions": {
    "my-pipeline": {
      "processors": [
        {
          "set": {
            "field": "field3",
            "value": "value3"
          }
        }
      ]
    }
  },
  "component_template_substitutions": {
    "my-component-template": {
      "template": {
        "mappings": {
          "dynamic": true,
          "properties": {
            "field3": {
              "type": "keyword"
            }
          }
        },
        "settings": {
          "index": {
            "default_pipeline": "my-pipeline"
          }
        }
      }
    }
  },
  "index_template_substitutions": {
    "my-index-template": {
      "index_patterns": [
        "my-index-*"
      ],
      "composed_of": [
        "component_template_1",
        "component_template_2"
      ]
    }
  },
  "mapping_addition": {
    "dynamic": "strict",
    "properties": {
      "foo": {
        "type": "keyword"
      }
    }
  }
}
A successful response when the simulation uses pipeline definitions that are already in the system.
{
  "docs": [
    {
      "doc": null,
      "_id": 123,
      "_index": "my-index",
      "_version": -3,
      "_source": {
        "field1": "value1",
        "field2": "value2",
        "foo": "bar"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    },
    {
      "doc": null,
      "_id": 456,
      "_index": "my-index",
      "_version": "-3,",
      "_source": {
        "field1": "value1",
        "field2": "value2",
        "foo": "rab"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    }
  ]
}
A successful response when the simulation uses pipeline substitutions.
{
  "docs": [
    {
      "doc": null,
      "_id": 123,
      "_index": "my-index",
      "_version": -3,
      "_source": {
        "field2": "value2",
        "foo": "BAR"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    },
    {
      "doc": null,
      "_id": 456,
      "_index": "my-index",
      "_version": -3,
      "_source": {
        "field2": "value2",
        "foo": "RAB"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    }
  ]
}
A successful response when the simulation uses pipeline substitutions.
{
  "docs": [
    {
      "doc": {
        "_id": "123",
        "_index": "my-index",
        "_version": -3,
        "_source": {
          "foo": "foo"
        },
        "executed_pipelines": []
      }
    },
    {
      "doc": {
        "_id": "456",
        "_index": "my-index",
        "_version": -3,
        "_source": {
          "bar": "rab"
        },
      "executed_pipelines": []
      }
    }
  ]
}

































Start a basic license Added in 6.3.0

POST /_license/start_basic

Start an indefinite basic license, which gives access to all the basic features.

NOTE: In order to start a basic license, you must not currently have a basic license.

If the basic license does not support all of the features that are available with your current license, however, you are notified in the response. You must then re-submit the API request with the acknowledge parameter set to true.

To check the status of your basic license, use the get basic license API.

Query parameters

  • whether the user has acknowledged acknowledge messages (default: false)

  • Period to wait for a connection to the master node.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

Responses

POST /_license/start_basic
curl \
 --request POST 'http://api.example.com/_license/start_basic' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `POST /_license/start_basic?acknowledge=true`. If you currently have a license with more features than a basic license and you start a basic license, you must pass the acknowledge parameter.
{
  "basic_was_started": true,
  "acknowledged": true
}











































































































































































































































Get model snapshots info Added in 5.4.0

GET /_ml/anomaly_detectors/{job_id}/model_snapshots

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job.

Query parameters

  • desc boolean

    If true, the results are sorted in descending order.

  • end string | number

    Returns snapshots with timestamps earlier than this time.

  • from number

    Skips the specified number of snapshots.

  • size number

    Specifies the maximum number of snapshots to obtain.

  • sort string

    Specifies the sort field for the requested snapshots. By default, the snapshots are sorted by their timestamp.

  • start string | number

    Returns snapshots with timestamps after this time.

application/json

Body

  • desc boolean

    Refer to the description for the desc query parameter.

  • end string | number

    A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

  • page object
    Hide page attributes Show page attributes object
    • from number

      Skips the specified number of items.

    • size number

      Specifies the maximum number of items to obtain.

  • sort string

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • start string | number

    A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

Responses

GET /_ml/anomaly_detectors/{job_id}/model_snapshots
curl \
 --request GET 'http://api.example.com/_ml/anomaly_detectors/{job_id}/model_snapshots' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"desc":true,"":"string","page":{"from":42.0,"size":42.0},"sort":"string"}'
































































Update a filter Added in 6.4.0

POST /_ml/filters/{filter_id}/_update

Updates the description of a filter, adds items, or removes items from the list.

Path parameters

  • filter_id string Required

    A string that uniquely identifies a filter.

application/json

Body Required

Responses

POST /_ml/filters/{filter_id}/_update
curl \
 --request POST 'http://api.example.com/_ml/filters/{filter_id}/_update' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"add_items":["string"],"description":"string","remove_items":["string"]}'

































Explain data frame analytics config Added in 7.3.0

POST /_ml/data_frame/analytics/_explain

This API provides explanations for a data frame analytics config that either exists already or one that has not been created yet. The following explanations are provided:

  • which fields are included or not in the analysis and why,
  • how much memory is estimated to be required. The estimate can be used when deciding the appropriate value for model_memory_limit setting later on. If you have object fields or fields that are excluded via source filtering, they are not included in the explanation.
application/json

Body

  • source object
    Hide source attributes Show source attributes object
    • index string | array[string] Required
    • Hide runtime_mappings attribute Show runtime_mappings attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • fields object

          For type composite

          Hide fields attribute Show fields attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
            • type string Required

              Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

        • fetch_fields array[object]

          For type lookup

          Hide fetch_fields attributes Show fetch_fields attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • format string
        • format string

          A custom format for date type runtime fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • script object
          Hide script attributes Show script attributes object
          • source string | object

            One of:
          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            Hide params attribute Show params attribute object
            • * object Additional properties
          • lang string

            Any of:

            Values are painless, expression, mustache, or java.

          • options object
            Hide options attribute Show options attribute object
            • * string Additional properties
        • type string Required

          Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

    • _source object
      Hide _source attributes Show _source attributes object
      • includes array[string]

        An array of strings that defines the fields that will be excluded from the analysis. You do not need to add fields with unsupported data types to excludes, these fields are excluded from the analysis automatically.

      • excludes array[string]

        An array of strings that defines the fields that will be included in the analysis.

    • query object

      The Elasticsearch query domain-specific language (DSL). This value corresponds to the query object in an Elasticsearch search POST body. All the options that are supported by Elasticsearch can be used, as this object is passed verbatim to Elasticsearch. By default, this property has the following value: {"match_all": {}}.

      Query DSL
  • dest object
    Hide dest attributes Show dest attributes object
    • index string Required
    • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • analysis object
    Hide analysis attributes Show analysis attributes object
    • Hide classification attributes Show classification attributes object
      • alpha number

        Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This parameter affects loss calculations by acting as a multiplier of the tree depth. Higher alpha values result in shallower trees and faster training times. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to zero.

      • dependent_variable string Required

        Defines which field of the document is to be predicted. It must match one of the fields in the index being used to train. If this field is missing from a document, then that document will not be used for training, but a prediction with the trained model will be generated for it. It is also known as continuous target variable. For classification analysis, the data type of the field must be numeric (integer, short, long, byte), categorical (ip or keyword), or boolean. There must be no more than 30 different values in this field. For regression analysis, the data type of the field must be numeric.

      • Advanced configuration option. Controls the fraction of data that is used to compute the derivatives of the loss function for tree training. A small value results in the use of a small fraction of the data. If this value is set to be less than 1, accuracy typically improves. However, too small a value may result in poor convergence for the ensemble and so require more trees. By default, this value is calculated during hyperparameter optimization. It must be greater than zero and less than or equal to 1.

      • Advanced configuration option. Specifies whether the training process should finish if it is not finding any better performing models. If disabled, the training process can take significantly longer and the chance of finding a better performing model is unremarkable.

      • eta number

        Advanced configuration option. The shrinkage applied to the weights. Smaller values result in larger forests which have a better generalization error. However, larger forests cause slower training. By default, this value is calculated during hyperparameter optimization. It must be a value between 0.001 and 1.

      • Advanced configuration option. Specifies the rate at which eta increases for each new tree that is added to the forest. For example, a rate of 1.05 increases eta by 5% for each extra tree. By default, this value is calculated during hyperparameter optimization. It must be between 0.5 and 2.

      • Advanced configuration option. Defines the fraction of features that will be used when selecting a random bag for each candidate split. By default, this value is calculated during hyperparameter optimization.

      • feature_processors array[object]

        Advanced configuration option. A collection of feature preprocessors that modify one or more included fields. The analysis uses the resulting one or more features instead of the original document field. However, these features are ephemeral; they are not stored in the destination index. Multiple feature_processors entries can refer to the same document fields. Automatic categorical feature encoding still occurs for the fields that are unprocessed by a custom processor or that have categorical values. Use this property only if you want to override the automatic feature encoding of the specified fields.

        Hide feature_processors attributes Show feature_processors attributes object
        • Hide frequency_encoding attributes Show frequency_encoding attributes object
          • feature_name string Required
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • frequency_map object Required

            The resulting frequency map for the field value. If the field value is missing from the frequency_map, the resulting value is 0.

        • Hide multi_encoding attribute Show multi_encoding attribute object
          • processors array[number] Required

            The ordered array of custom processors to execute. Must be more than 1.

        • Hide n_gram_encoding attributes Show n_gram_encoding attributes object
          • The feature name prefix. Defaults to ngram__.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • length number

            Specifies the length of the n-gram substring. Defaults to 50. Must be greater than 0.

          • n_grams array[number] Required

            Specifies which n-grams to gather. It’s an array of integer values where the minimum value is 1, and a maximum value is 5.

          • start number

            Specifies the zero-indexed start of the n-gram substring. Negative values are allowed for encoding n-grams of string suffixes. Defaults to 0.

          • custom boolean
        • Hide one_hot_encoding attributes Show one_hot_encoding attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • hot_map string Required

            The one hot map mapping the field value with the column name.

        • Hide target_mean_encoding attributes Show target_mean_encoding attributes object
          • default_value number Required

            The default value if field value is not found in the target_map.

          • feature_name string Required
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_map object Required

            The field value to target mean transition map.

      • gamma number

        Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies a linear penalty associated with the size of individual trees in the forest. A high gamma value causes training to prefer small trees. A small gamma value results in larger individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

      • lambda number

        Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies an L2 regularization term which applies to leaf weights of the individual trees in the forest. A high lambda value causes training to favor small leaf weights. This behavior makes the prediction function smoother at the expense of potentially not being able to capture relevant relationships between the features and the dependent variable. A small lambda value results in large individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

      • Advanced configuration option. A multiplier responsible for determining the maximum number of hyperparameter optimization steps in the Bayesian optimization procedure. The maximum number of steps is determined based on the number of undefined hyperparameters times the maximum optimization rounds per hyperparameter. By default, this value is calculated during hyperparameter optimization.

      • Advanced configuration option. Defines the maximum number of decision trees in the forest. The maximum value is 2000. By default, this value is calculated during hyperparameter optimization.

      • Advanced configuration option. Specifies the maximum number of feature importance values per document to return. By default, no feature importance calculation occurs.

      • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Defines the seed for the random generator that is used to pick training data. By default, it is randomly generated. Set it to a specific value to use the same training data each time you start a job (assuming other related parameters such as source and analyzed_fields are the same).

      • Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This soft limit combines with the soft_tree_depth_tolerance to penalize trees that exceed the specified depth; the regularized loss increases quickly beyond this depth. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.

      • Advanced configuration option. This option controls how quickly the regularized loss increases when the tree depth exceeds soft_tree_depth_limit. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.01.

      • Defines the number of categories for which the predicted probabilities are reported. It must be non-negative or -1. If it is -1 or greater than the total number of categories, probabilities are reported for all categories; if you have a large number of categories, there could be a significant effect on the size of your destination index. NOTE: To use the AUC ROC evaluation method, num_top_classes must be set to -1 or a value greater than or equal to the total number of categories.

    • Hide outlier_detection attributes Show outlier_detection attributes object
      • Specifies whether the feature influence calculation is enabled.

      • The minimum outlier score that a document needs to have in order to calculate its feature influence score. Value range: 0-1.

      • method string

        The method that outlier detection uses. Available methods are lof, ldof, distance_kth_nn, distance_knn, and ensemble. The default value is ensemble, which means that outlier detection uses an ensemble of different methods and normalises and combines their individual outlier scores to obtain the overall outlier score.

      • Defines the value for how many nearest neighbors each method of outlier detection uses to calculate its outlier score. When the value is not set, different values are used for different ensemble members. This default behavior helps improve the diversity in the ensemble; only override it if you are confident that the value you choose is appropriate for the data set.

      • The proportion of the data set that is assumed to be outlying prior to outlier detection. For example, 0.05 means it is assumed that 5% of values are real outliers and 95% are inliers.

      • If true, the following operation is performed on the columns before computing outlier scores: (x_i - mean(x_i)) / sd(x_i).

    • Hide regression attributes Show regression attributes object
      • alpha number

        Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This parameter affects loss calculations by acting as a multiplier of the tree depth. Higher alpha values result in shallower trees and faster training times. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to zero.

      • dependent_variable string Required

        Defines which field of the document is to be predicted. It must match one of the fields in the index being used to train. If this field is missing from a document, then that document will not be used for training, but a prediction with the trained model will be generated for it. It is also known as continuous target variable. For classification analysis, the data type of the field must be numeric (integer, short, long, byte), categorical (ip or keyword), or boolean. There must be no more than 30 different values in this field. For regression analysis, the data type of the field must be numeric.

      • Advanced configuration option. Controls the fraction of data that is used to compute the derivatives of the loss function for tree training. A small value results in the use of a small fraction of the data. If this value is set to be less than 1, accuracy typically improves. However, too small a value may result in poor convergence for the ensemble and so require more trees. By default, this value is calculated during hyperparameter optimization. It must be greater than zero and less than or equal to 1.

      • Advanced configuration option. Specifies whether the training process should finish if it is not finding any better performing models. If disabled, the training process can take significantly longer and the chance of finding a better performing model is unremarkable.

      • eta number

        Advanced configuration option. The shrinkage applied to the weights. Smaller values result in larger forests which have a better generalization error. However, larger forests cause slower training. By default, this value is calculated during hyperparameter optimization. It must be a value between 0.001 and 1.

      • Advanced configuration option. Specifies the rate at which eta increases for each new tree that is added to the forest. For example, a rate of 1.05 increases eta by 5% for each extra tree. By default, this value is calculated during hyperparameter optimization. It must be between 0.5 and 2.

      • Advanced configuration option. Defines the fraction of features that will be used when selecting a random bag for each candidate split. By default, this value is calculated during hyperparameter optimization.

      • feature_processors array[object]

        Advanced configuration option. A collection of feature preprocessors that modify one or more included fields. The analysis uses the resulting one or more features instead of the original document field. However, these features are ephemeral; they are not stored in the destination index. Multiple feature_processors entries can refer to the same document fields. Automatic categorical feature encoding still occurs for the fields that are unprocessed by a custom processor or that have categorical values. Use this property only if you want to override the automatic feature encoding of the specified fields.

        Hide feature_processors attributes Show feature_processors attributes object
        • Hide frequency_encoding attributes Show frequency_encoding attributes object
          • feature_name string Required
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • frequency_map object Required

            The resulting frequency map for the field value. If the field value is missing from the frequency_map, the resulting value is 0.

        • Hide multi_encoding attribute Show multi_encoding attribute object
          • processors array[number] Required

            The ordered array of custom processors to execute. Must be more than 1.

        • Hide n_gram_encoding attributes Show n_gram_encoding attributes object
          • The feature name prefix. Defaults to ngram__.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • length number

            Specifies the length of the n-gram substring. Defaults to 50. Must be greater than 0.

          • n_grams array[number] Required

            Specifies which n-grams to gather. It’s an array of integer values where the minimum value is 1, and a maximum value is 5.

          • start number

            Specifies the zero-indexed start of the n-gram substring. Negative values are allowed for encoding n-grams of string suffixes. Defaults to 0.

          • custom boolean
        • Hide one_hot_encoding attributes Show one_hot_encoding attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • hot_map string Required

            The one hot map mapping the field value with the column name.

        • Hide target_mean_encoding attributes Show target_mean_encoding attributes object
          • default_value number Required

            The default value if field value is not found in the target_map.

          • feature_name string Required
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_map object Required

            The field value to target mean transition map.

      • gamma number

        Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies a linear penalty associated with the size of individual trees in the forest. A high gamma value causes training to prefer small trees. A small gamma value results in larger individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

      • lambda number

        Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies an L2 regularization term which applies to leaf weights of the individual trees in the forest. A high lambda value causes training to favor small leaf weights. This behavior makes the prediction function smoother at the expense of potentially not being able to capture relevant relationships between the features and the dependent variable. A small lambda value results in large individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

      • Advanced configuration option. A multiplier responsible for determining the maximum number of hyperparameter optimization steps in the Bayesian optimization procedure. The maximum number of steps is determined based on the number of undefined hyperparameters times the maximum optimization rounds per hyperparameter. By default, this value is calculated during hyperparameter optimization.

      • Advanced configuration option. Defines the maximum number of decision trees in the forest. The maximum value is 2000. By default, this value is calculated during hyperparameter optimization.

      • Advanced configuration option. Specifies the maximum number of feature importance values per document to return. By default, no feature importance calculation occurs.

      • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Defines the seed for the random generator that is used to pick training data. By default, it is randomly generated. Set it to a specific value to use the same training data each time you start a job (assuming other related parameters such as source and analyzed_fields are the same).

      • Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This soft limit combines with the soft_tree_depth_tolerance to penalize trees that exceed the specified depth; the regularized loss increases quickly beyond this depth. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.

      • Advanced configuration option. This option controls how quickly the regularized loss increases when the tree depth exceeds soft_tree_depth_limit. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.01.

      • The loss function used during regression. Available options are mse (mean squared error), msle (mean squared logarithmic error), huber (Pseudo-Huber loss).

      • A positive number that is used as a parameter to the loss_function.

  • A description of the job.

  • The approximate maximum amount of memory resources that are permitted for analytical processing. If your elasticsearch.yml file contains an xpack.ml.max_model_memory_limit setting, an error occurs when you try to create data frame analytics jobs that have model_memory_limit values greater than that setting.

  • The maximum number of threads to be used by the analysis. Using more threads may decrease the time necessary to complete the analysis at the cost of using more CPU. Note that the process may use additional threads for operational functionality other than the analysis itself.

  • Hide analyzed_fields attributes Show analyzed_fields attributes object
    • includes array[string]

      An array of strings that defines the fields that will be excluded from the analysis. You do not need to add fields with unsupported data types to excludes, these fields are excluded from the analysis automatically.

    • excludes array[string]

      An array of strings that defines the fields that will be included in the analysis.

  • Specifies whether this job can start when there is insufficient machine learning node capacity for it to be immediately assigned to a node.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • field_selection array[object] Required

      An array of objects that explain selection for each field, sorted by the field names.

      Hide field_selection attributes Show field_selection attributes object
      • is_included boolean Required

        Whether the field is selected to be included in the analysis.

      • is_required boolean Required

        Whether the field is required.

      • The feature type of this field for the analysis. May be categorical or numerical.

      • mapping_types array[string] Required

        The mapping types of the field.

      • name string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • reason string

        The reason a field is not selected to be included in the analysis.

    • memory_estimation object Required
      Hide memory_estimation attributes Show memory_estimation attributes object
      • Estimated memory usage under the assumption that overflowing to disk is allowed during data frame analytics. expected_memory_with_disk is usually smaller than expected_memory_without_disk as using disk allows to limit the main memory needed to perform data frame analytics.

      • Estimated memory usage under the assumption that the whole data frame analytics should happen in memory (i.e. without overflowing to disk).

POST /_ml/data_frame/analytics/_explain
curl \
 --request POST 'http://api.example.com/_ml/data_frame/analytics/_explain' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"source\": {\n    \"index\": \"houses_sold_last_10_yrs\"\n  },\n  \"analysis\": {\n    \"regression\": {\n      \"dependent_variable\": \"price\"\n    }\n  }\n}"'
Request example
Run `POST _ml/data_frame/analytics/_explain` to explain a data frame analytics job configuration.
{
  "source": {
    "index": "houses_sold_last_10_yrs"
  },
  "analysis": {
    "regression": {
      "dependent_variable": "price"
    }
  }
}
Response examples (200)
A succesful response for explaining a data frame analytics job configuration.
{
  "field_selection": [
    {
      "field": "number_of_bedrooms",
      "mappings_types": [
        "integer"
      ],
      "is_included": true,
      "is_required": false,
      "feature_type": "numerical"
    },
    {
      "field": "postcode",
      "mappings_types": [
        "text"
      ],
      "is_included": false,
      "is_required": false,
      "reason": "[postcode.keyword] is preferred because it is aggregatable"
    },
    {
      "field": "postcode.keyword",
      "mappings_types": [
        "keyword"
      ],
      "is_included": true,
      "is_required": false,
      "feature_type": "categorical"
    },
    {
      "field": "price",
      "mappings_types": [
        "float"
      ],
      "is_included": true,
      "is_required": true,
      "feature_type": "numerical"
    }
  ],
  "memory_estimation": {
    "expected_memory_without_disk": "128MB",
    "expected_memory_with_disk": "32MB"
  }
}
















Get data frame analytics job stats Added in 7.3.0

GET /_ml/data_frame/analytics/{id}/_stats

Path parameters

  • id string Required

    Identifier for the data frame analytics job. If you do not specify this option, the API returns information for the first hundred data frame analytics jobs.

Query parameters

  • Specifies what to do when the request:

    1. Contains wildcard expressions and there are no data frame analytics jobs that match.
    2. Contains the _all string or no identifiers and there are no matches.
    3. Contains wildcard expressions and there are only partial matches.

    The default value returns an empty data_frame_analytics array when there are no matches and the subset of results when there are partial matches. If this parameter is false, the request returns a 404 status code when there are no matches or only partial matches.

  • from number

    Skips the specified number of data frame analytics jobs.

  • size number

    Specifies the maximum number of data frame analytics jobs to obtain.

  • verbose boolean

    Defines whether the stats response should be verbose.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • count number Required
    • data_frame_analytics array[object] Required

      An array of objects that contain usage information for data frame analytics jobs, which are sorted by the id value in ascending order.

      Hide data_frame_analytics attributes Show data_frame_analytics attributes object
      • Hide analysis_stats attributes Show analysis_stats attributes object
        • Hide classification_stats attributes Show classification_stats attributes object
          • hyperparameters object Required
            Hide hyperparameters attributes Show hyperparameters attributes object
            • alpha number

              Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This parameter affects loss calculations by acting as a multiplier of the tree depth. Higher alpha values result in shallower trees and faster training times. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to zero.

            • lambda number

              Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies an L2 regularization term which applies to leaf weights of the individual trees in the forest. A high lambda value causes training to favor small leaf weights. This behavior makes the prediction function smoother at the expense of potentially not being able to capture relevant relationships between the features and the dependent variable. A small lambda value results in large individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

            • gamma number

              Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies a linear penalty associated with the size of individual trees in the forest. A high gamma value causes training to prefer small trees. A small gamma value results in larger individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

            • eta number

              Advanced configuration option. The shrinkage applied to the weights. Smaller values result in larger forests which have a better generalization error. However, larger forests cause slower training. By default, this value is calculated during hyperparameter optimization. It must be a value between 0.001 and 1.

            • Advanced configuration option. Specifies the rate at which eta increases for each new tree that is added to the forest. For example, a rate of 1.05 increases eta by 5% for each extra tree. By default, this value is calculated during hyperparameter optimization. It must be between 0.5 and 2.

            • Advanced configuration option. Defines the fraction of features that will be used when selecting a random bag for each candidate split. By default, this value is calculated during hyperparameter optimization.

            • Advanced configuration option. Controls the fraction of data that is used to compute the derivatives of the loss function for tree training. A small value results in the use of a small fraction of the data. If this value is set to be less than 1, accuracy typically improves. However, too small a value may result in poor convergence for the ensemble and so require more trees. By default, this value is calculated during hyperparameter optimization. It must be greater than zero and less than or equal to 1.

            • If the algorithm fails to determine a non-trivial tree (more than a single leaf), this parameter determines how many of such consecutive failures are tolerated. Once the number of attempts exceeds the threshold, the forest training stops.

            • Advanced configuration option. A multiplier responsible for determining the maximum number of hyperparameter optimization steps in the Bayesian optimization procedure. The maximum number of steps is determined based on the number of undefined hyperparameters times the maximum optimization rounds per hyperparameter. By default, this value is calculated during hyperparameter optimization.

            • Advanced configuration option. Defines the maximum number of decision trees in the forest. The maximum value is 2000. By default, this value is calculated during hyperparameter optimization.

            • The maximum number of folds for the cross-validation procedure.

            • Determines the maximum number of splits for every feature that can occur in a decision tree when the tree is trained.

            • Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This soft limit combines with the soft_tree_depth_tolerance to penalize trees that exceed the specified depth; the regularized loss increases quickly beyond this depth. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.

            • Advanced configuration option. This option controls how quickly the regularized loss increases when the tree depth exceeds soft_tree_depth_limit. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.01.

          • iteration number Required

            The number of iterations on the analysis.

          • Time unit for milliseconds

          • timing_stats object Required
            Hide timing_stats attributes Show timing_stats attributes object
          • validation_loss object Required
            Hide validation_loss attributes Show validation_loss attributes object
            • fold_values array[string] Required

              Validation loss values for every added decision tree during the forest growing procedure.

            • loss_type string Required

              The type of the loss metric. For example, binomial_logistic.

        • Hide outlier_detection_stats attributes Show outlier_detection_stats attributes object
          • parameters object Required
            Hide parameters attributes Show parameters attributes object
            • Specifies whether the feature influence calculation is enabled.

            • The minimum outlier score that a document needs to have in order to calculate its feature influence score. Value range: 0-1

            • method string

              The method that outlier detection uses. Available methods are lof, ldof, distance_kth_nn, distance_knn, and ensemble. The default value is ensemble, which means that outlier detection uses an ensemble of different methods and normalises and combines their individual outlier scores to obtain the overall outlier score.

            • Defines the value for how many nearest neighbors each method of outlier detection uses to calculate its outlier score. When the value is not set, different values are used for different ensemble members. This default behavior helps improve the diversity in the ensemble; only override it if you are confident that the value you choose is appropriate for the data set.

            • The proportion of the data set that is assumed to be outlying prior to outlier detection. For example, 0.05 means it is assumed that 5% of values are real outliers and 95% are inliers.

            • If true, the following operation is performed on the columns before computing outlier scores: (x_i - mean(x_i)) / sd(x_i).

          • Time unit for milliseconds

          • timing_stats object Required
            Hide timing_stats attributes Show timing_stats attributes object
        • Hide regression_stats attributes Show regression_stats attributes object
          • hyperparameters object Required
            Hide hyperparameters attributes Show hyperparameters attributes object
            • alpha number

              Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This parameter affects loss calculations by acting as a multiplier of the tree depth. Higher alpha values result in shallower trees and faster training times. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to zero.

            • lambda number

              Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies an L2 regularization term which applies to leaf weights of the individual trees in the forest. A high lambda value causes training to favor small leaf weights. This behavior makes the prediction function smoother at the expense of potentially not being able to capture relevant relationships between the features and the dependent variable. A small lambda value results in large individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

            • gamma number

              Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies a linear penalty associated with the size of individual trees in the forest. A high gamma value causes training to prefer small trees. A small gamma value results in larger individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

            • eta number

              Advanced configuration option. The shrinkage applied to the weights. Smaller values result in larger forests which have a better generalization error. However, larger forests cause slower training. By default, this value is calculated during hyperparameter optimization. It must be a value between 0.001 and 1.

            • Advanced configuration option. Specifies the rate at which eta increases for each new tree that is added to the forest. For example, a rate of 1.05 increases eta by 5% for each extra tree. By default, this value is calculated during hyperparameter optimization. It must be between 0.5 and 2.

            • Advanced configuration option. Defines the fraction of features that will be used when selecting a random bag for each candidate split. By default, this value is calculated during hyperparameter optimization.

            • Advanced configuration option. Controls the fraction of data that is used to compute the derivatives of the loss function for tree training. A small value results in the use of a small fraction of the data. If this value is set to be less than 1, accuracy typically improves. However, too small a value may result in poor convergence for the ensemble and so require more trees. By default, this value is calculated during hyperparameter optimization. It must be greater than zero and less than or equal to 1.

            • If the algorithm fails to determine a non-trivial tree (more than a single leaf), this parameter determines how many of such consecutive failures are tolerated. Once the number of attempts exceeds the threshold, the forest training stops.

            • Advanced configuration option. A multiplier responsible for determining the maximum number of hyperparameter optimization steps in the Bayesian optimization procedure. The maximum number of steps is determined based on the number of undefined hyperparameters times the maximum optimization rounds per hyperparameter. By default, this value is calculated during hyperparameter optimization.

            • Advanced configuration option. Defines the maximum number of decision trees in the forest. The maximum value is 2000. By default, this value is calculated during hyperparameter optimization.

            • The maximum number of folds for the cross-validation procedure.

            • Determines the maximum number of splits for every feature that can occur in a decision tree when the tree is trained.

            • Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This soft limit combines with the soft_tree_depth_tolerance to penalize trees that exceed the specified depth; the regularized loss increases quickly beyond this depth. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.

            • Advanced configuration option. This option controls how quickly the regularized loss increases when the tree depth exceeds soft_tree_depth_limit. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.01.

          • iteration number Required

            The number of iterations on the analysis.

          • Time unit for milliseconds

          • timing_stats object Required
            Hide timing_stats attributes Show timing_stats attributes object
          • validation_loss object Required
            Hide validation_loss attributes Show validation_loss attributes object
            • fold_values array[string] Required

              Validation loss values for every added decision tree during the forest growing procedure.

            • loss_type string Required

              The type of the loss metric. For example, binomial_logistic.

      • For running jobs only, contains messages relating to the selection of a node to run the job.

      • data_counts object Required
        Hide data_counts attributes Show data_counts attributes object
        • skipped_docs_count number Required

          The number of documents that are skipped during the analysis because they contained values that are not supported by the analysis. For example, outlier detection does not support missing fields so it skips documents with missing fields. Likewise, all types of analysis skip documents that contain arrays with more than one element.

        • test_docs_count number Required

          The number of documents that are not used for training the model and can be used for testing.

        • training_docs_count number Required

          The number of documents that are used for training the model.

      • id string Required
      • memory_usage object Required
        Hide memory_usage attributes Show memory_usage attributes object
        • This value is present when the status is hard_limit and it is a new estimate of how much memory the job needs.

        • peak_usage_bytes number Required

          The number of bytes used at the highest peak of memory usage.

        • status string Required

          The memory usage status.

        • Time unit for milliseconds

      • node object
        Hide node attributes Show node attributes object
      • progress array[object] Required

        The progress report of the data frame analytics job by phase.

        Hide progress attributes Show progress attributes object
        • phase string Required

          Defines the phase of the data frame analytics job.

        • progress_percent number Required

          The progress that the data frame analytics job has made expressed in percentage.

      • state string Required

        Values are started, stopped, starting, stopping, or failed.

GET /_ml/data_frame/analytics/{id}/_stats
curl \
 --request GET 'http://api.example.com/_ml/data_frame/analytics/{id}/_stats' \
 --header "Authorization: $API_KEY"


























































































































Start the feature migration Added in 7.16.0

POST /_migration/system_features

Version upgrades sometimes require changes to how features store configuration information and data in system indices. This API starts the automatic migration process.

Some functionality might be temporarily unavailable during the migration process.

TIP: The API is designed for indirect use by the Upgrade Assistant. We strongly recommend you use the Upgrade Assistant.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
POST /_migration/system_features
curl \
 --request POST 'http://api.example.com/_migration/system_features' \
 --header "Authorization: $API_KEY"
Response examples (200)
When you run `POST /_migration/system_features` to start the migration process, the response lists the features that will be migrated.
{
  "accepted" : true,
  "features" : [
    {
      "feature_name" : "security"
    }
  ]
}





Prepare a node to be shut down Added in 7.13.0

PUT /_nodes/{node_id}/shutdown

NOTE: This feature is designed for indirect use by Elastic Cloud, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.

If you specify a node that is offline, it will be prepared for shut down when it rejoins the cluster.

If the operator privileges feature is enabled, you must be an operator to use this API.

The API migrates ongoing tasks and index shards to other nodes as needed to prepare a node to be restarted or shut down and removed from the cluster. This ensures that Elasticsearch can be stopped safely with minimal disruption to the cluster.

You must specify the type of shutdown: restart, remove, or replace. If a node is already being prepared for shutdown, you can use this API to change the shutdown type.

IMPORTANT: This API does NOT terminate the Elasticsearch process. Monitor the node shutdown status to determine when it is safe to stop Elasticsearch.

Path parameters

  • node_id string Required

    The node identifier. This parameter is not validated against the cluster's active nodes. This enables you to register a node for shut down while it is offline. No error is thrown if you specify an invalid node ID.

Query parameters

  • The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are nanos, micros, ms, s, m, h, or d.

  • timeout string

    The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are nanos, micros, ms, s, m, h, or d.

application/json

Body Required

  • type string Required

    Values are restart, remove, or replace.

  • reason string Required

    A human-readable reason that the node is being shut down. This field provides information for other cluster operators; it does not affect the shut down process.

  • Only valid if type is restart. Controls how long Elasticsearch will wait for the node to restart and join the cluster before reassigning its shards to other nodes. This works the same as delaying allocation with the index.unassigned.node_left.delayed_timeout setting. If you specify both a restart allocation delay and an index-level allocation delay, the longer of the two is used.

  • Only valid if type is replace. Specifies the name of the node that is replacing the node being shut down. Shards from the shut down node are only allowed to be allocated to the target node, and no other data will be allocated to the target node. During relocation of data certain allocation rules are ignored, such as disk watermarks or user attribute filtering rules.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /_nodes/{node_id}/shutdown
curl \
 --request PUT 'http://api.example.com/_nodes/{node_id}/shutdown' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"type\": \"restart\",\n  \"reason\": \"Demonstrating how the node shutdown API works\",\n  \"allocation_delay\": \"20m\"\n}"'
Request example
Register a node for shutdown with `PUT /_nodes/USpTGYaBSIKbgSUJR2Z9lg/shutdown`. The `restart` type prepares the node to be restarted.
{
  "type": "restart",
  "reason": "Demonstrating how the node shutdown API works",
  "allocation_delay": "20m"
}






















































Get rollup job information Deprecated Technical preview

GET /_rollup/job

Get the configuration, stats, and status of rollup jobs.

NOTE: This API returns only active (both STARTED and STOPPED) jobs. If a job was created, ran for a while, then was deleted, the API does not return any details about it. For details about a historical rollup job, the rollup capabilities API may be more useful.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • jobs array[object] Required
      Hide jobs attributes Show jobs attributes object
      • config object Required
        Hide config attributes Show config attributes object
        • cron string Required
        • groups object Required
          Hide groups attributes Show groups attributes object
          • Hide date_histogram attributes Show date_histogram attributes object
            • delay string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • field string Required

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • format string
            • interval string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • Hide histogram attributes Show histogram attributes object
            • fields string | array[string] Required
            • interval number Required

              The interval of histogram buckets to be generated when rolling up. For example, a value of 5 creates buckets that are five units wide (0-5, 5-10, etc). Note that only one interval can be specified in the histogram group, meaning that all fields being grouped via the histogram must share the same interval.

          • terms object
            Hide terms attribute Show terms attribute object
            • fields string | array[string] Required
        • id string Required
        • index_pattern string Required
        • metrics array[object] Required
          Hide metrics attributes Show metrics attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • metrics array[string] Required

            An array of metrics to collect for the field. At least one metric must be configured.

            Values are min, max, sum, avg, or value_count.

        • page_size number Required
        • rollup_index string Required
        • timeout string Required

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • stats object Required
        Hide stats attributes Show stats attributes object
      • status object Required
        Hide status attributes Show status attributes object
GET /_rollup/job
curl \
 --request GET 'http://api.example.com/_rollup/job' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _rollup/job/sensor`.
{
  "jobs": [
    {
      "config": {
        "id": "sensor",
        "index_pattern": "sensor-*",
        "rollup_index": "sensor_rollup",
        "cron": "*/30 * * * * ?",
        "groups": {
          "date_histogram": {
            "fixed_interval": "1h",
            "delay": "7d",
            "field": "timestamp",
            "time_zone": "UTC"
          },
          "terms": {
            "fields": [
              "node"
            ]
          }
        },
        "metrics": [
          {
            "field": "temperature",
            "metrics": [
              "min",
              "max",
              "sum"
            ]
          },
          {
            "field": "voltage",
            "metrics": [
              "avg"
            ]
          }
        ],
        "timeout": "20s",
        "page_size": 1000
      },
      "status": {
        "job_state": "stopped"
      },
      "stats": {
        "pages_processed": 0,
        "documents_processed": 0,
        "rollups_indexed": 0,
        "trigger_count": 0,
        "index_failures": 0,
        "index_time_in_ms": 0,
        "index_total": 0,
        "search_failures": 0,
        "search_time_in_ms": 0,
        "search_total": 0,
        "processing_time_in_ms": 0,
        "processing_total": 0
      }
    }
  ]
}

Get the rollup job capabilities Deprecated Technical preview

GET /_rollup/data/{id}

Get the capabilities of any rollup jobs that have been configured for a specific index or index pattern.

This API is useful because a rollup job is often configured to rollup only a subset of fields from the source index. Furthermore, only certain aggregations can be configured for various fields, leading to a limited subset of functionality depending on that configuration. This API enables you to inspect an index and determine:

  1. Does this index have associated rollup data somewhere in the cluster?
  2. If yes to the first question, what fields were rolled up, what aggregations can be performed, and where does the data live?

Path parameters

  • id string Required

    Index, indices or index-pattern to return rollup capabilities for. _all may be used to fetch rollup capabilities from all jobs.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attribute Show * attribute object
      • rollup_jobs array[object] Required

        There can be multiple, independent jobs configured for a single index or index pattern. Each of these jobs may have different configurations, so the API returns a list of all the various configurations available.

        Hide rollup_jobs attributes Show rollup_jobs attributes object
GET /_rollup/data/{id}
curl \
 --request GET 'http://api.example.com/_rollup/data/{id}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _rollup/data/sensor-*` for a rollup job that targets the index pattern `sensor-*`. The response contains the rollup job ID, the index that holds the rolled data, and the index pattern that the job was targeting. It also shows a list of fields that contain data eligible for rollup searches. For example, you can use a `min`, `max`, or `sum` aggregation on the `temperature` field, but only a `date_histogram` on `timestamp`.
{
  "sensor-*" : {
    "rollup_jobs" : [
      {
        "job_id" : "sensor",
        "rollup_index" : "sensor_rollup",
        "index_pattern" : "sensor-*",
        "fields" : {
          "node" : [
            {
              "agg" : "terms"
            }
          ],
          "temperature" : [
            {
              "agg" : "min"
            },
            {
              "agg" : "max"
            },
            {
              "agg" : "sum"
            }
          ],
          "timestamp" : [
            {
              "agg" : "date_histogram",
              "time_zone" : "UTC",
              "fixed_interval" : "1h",
              "delay": "7d"
            }
          ],
          "voltage" : [
            {
              "agg" : "avg"
            }
          ]
        }
      }
    ]
  }
}





























Path parameters

  • id string Required

    The identifier for the stored script or search template. It must be unique within the cluster.

Query parameters

  • context string

    The context in which the script or search template should run. To prevent errors, the API immediately compiles the script or template in this context. If you specify both this and the <context> path parameter, the API uses the request path parameter.

  • The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

  • timeout string

    The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

application/json

Body Required

  • script object Required
    Hide script attributes Show script attributes object
    • lang string Required

      Any of:

      Values are painless, expression, mustache, or java.

    • options object
      Hide options attribute Show options attribute object
      • * string Additional properties
    • source string | object Required

      One of:

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /_scripts/{id}
curl \
 --request PUT 'http://api.example.com/_scripts/{id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"script\": {\n    \"lang\": \"mustache\",\n    \"source\": {\n      \"query\": {\n        \"match\": {\n          \"message\": \"{{query_string}}\"\n        }\n      },\n      \"from\": \"{{from}}\",\n      \"size\": \"{{size}}\"\n    }\n  }\n}"'
Request examples
Run `PUT _scripts/my-search-template` to create a search template.
{
  "script": {
    "lang": "mustache",
    "source": {
      "query": {
        "match": {
          "message": "{{query_string}}"
        }
      },
      "from": "{{from}}",
      "size": "{{size}}"
    }
  }
}
Run `PUT _scripts/my-stored-script` to create a stored script.
{
  "script": {
    "lang": "painless",
    "source": "Math.log(_score * 2) + params['my_modifier']"
  }
}




















Create or update a script or search template

POST /_scripts/{id}/{context}

Creates or updates a stored script or search template.

External documentation

Path parameters

  • id string Required

    The identifier for the stored script or search template. It must be unique within the cluster.

  • context string Required

    The context in which the script or search template should run. To prevent errors, the API immediately compiles the script or template in this context.

Query parameters

  • context string

    The context in which the script or search template should run. To prevent errors, the API immediately compiles the script or template in this context. If you specify both this and the <context> path parameter, the API uses the request path parameter.

  • The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

  • timeout string

    The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

application/json

Body Required

  • script object Required
    Hide script attributes Show script attributes object
    • lang string Required

      Any of:

      Values are painless, expression, mustache, or java.

    • options object
      Hide options attribute Show options attribute object
      • * string Additional properties
    • source string | object Required

      One of:

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

POST /_scripts/{id}/{context}
curl \
 --request POST 'http://api.example.com/_scripts/{id}/{context}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"script\": {\n    \"lang\": \"mustache\",\n    \"source\": {\n      \"query\": {\n        \"match\": {\n          \"message\": \"{{query_string}}\"\n        }\n      },\n      \"from\": \"{{from}}\",\n      \"size\": \"{{size}}\"\n    }\n  }\n}"'
Request examples
Run `PUT _scripts/my-search-template` to create a search template.
{
  "script": {
    "lang": "mustache",
    "source": {
      "query": {
        "match": {
          "message": "{{query_string}}"
        }
      },
      "from": "{{from}}",
      "size": "{{size}}"
    }
  }
}
Run `PUT _scripts/my-stored-script` to create a stored script.
{
  "script": {
    "lang": "painless",
    "source": "Math.log(_score * 2) + params['my_modifier']"
  }
}

Run a script Technical preview

GET /_scripts/painless/_execute

Runs a script and returns a result. Use this API to build and test scripts, such as when defining a script for a runtime field. This API requires very few dependencies and is especially useful if you don't have permissions to write documents on a cluster.

The API uses several contexts, which control how scripts are run, what variables are available at runtime, and what the return type is.

Each context requires a script, but additional parameters depend on the context you're using for that script.

application/json

Body

  • context string

    Values are painless_test, filter, score, boolean_field, date_field, double_field, geo_point_field, ip_field, keyword_field, long_field, or composite_field.

  • Hide context_setup attributes Show context_setup attributes object
    • document object Required

      Document that's temporarily indexed in-memory and accessible from the script.

    • index string Required
    • query object

      An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

      External documentation
  • script object
    Hide script attributes Show script attributes object
    • source string | object

      One of:
    • id string
    • params object

      Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

      Hide params attribute Show params attribute object
      • * object Additional properties
    • lang string

      Any of:

      Values are painless, expression, mustache, or java.

    • options object
      Hide options attribute Show options attribute object
      • * string Additional properties

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
GET /_scripts/painless/_execute
curl \
 --request GET 'http://api.example.com/_scripts/painless/_execute' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"script\": {\n    \"source\": \"params.count / params.total\",\n    \"params\": {\n      \"count\": 100.0,\n      \"total\": 1000.0\n    }\n  }\n}"'
Run `POST /_scripts/painless/_execute`. The `painless_test` context is the default context. It runs scripts without additional parameters. The only variable that is available is `params`, which can be used to access user defined values. The result of the script is always converted to a string.
{
  "script": {
    "source": "params.count / params.total",
    "params": {
      "count": 100.0,
      "total": 1000.0
    }
  }
}
Run `POST /_scripts/painless/_execute` with a `filter` context. It treats scripts as if they were run inside a script query. For testing purposes, a document must be provided so that it will be temporarily indexed in-memory and is accessible from the script. More precisely, the `_source`, stored fields, and doc values of such a document are available to the script being tested.
{
  "script": {
    "source": "doc['field'].value.length() <= params.max_length",
    "params": {
      "max_length": 4
    }
  },
  "context": "filter",
  "context_setup": {
    "index": "my-index-000001",
    "document": {
      "field": "four"
    }
  }
}
Run `POST /_scripts/painless/_execute` with a `score` context. It treats scripts as if they were run inside a `script_score` function in a `function_score` query.
{
  "script": {
    "source": "doc['rank'].value / params.max_rank",
    "params": {
      "max_rank": 5.0
    }
  },
  "context": "score",
  "context_setup": {
    "index": "my-index-000001",
    "document": {
      "rank": 4
    }
  }
}
Response examples (200)
A successful response from `POST /_scripts/painless/_execute` with a `painless_test` context.
{
  "result": "0.1"
}
A successful response from `POST /_scripts/painless/_execute` with a `filter` context.
{
  "result": true
}
A successful response from `POST /_scripts/painless/_execute` with a `score` context.
{
  "result": 0.8
}









































































































































































Run a search

GET /{index}/_search

Get search hits that match the query defined in the request. You can provide search queries using the q query string parameter or the request body. If both are specified, only the query parameter is used.

If the Elasticsearch security features are enabled, you must have the read index privilege for the target data stream, index, or alias. For cross-cluster search, refer to the documentation about configuring CCS privileges. To search a point in time (PIT) for an alias, you must have the read index privilege for the alias's data streams or indices.

Search slicing

When paging through a large number of documents, it can be helpful to split the search into multiple slices to consume them independently with the slice and pit properties. By default the splitting is done first on the shards, then locally on each shard. The local splitting partitions the shard into contiguous ranges based on Lucene document IDs.

For instance if the number of shards is equal to 2 and you request 4 slices, the slices 0 and 2 are assigned to the first shard and the slices 1 and 3 are assigned to the second shard.

IMPORTANT: The same point-in-time ID should be used for all slices. If different PIT IDs are used, slices can overlap and miss documents. This situation can occur because the splitting criterion is based on Lucene document IDs, which are not stable across changes to the index.

External documentation

Path parameters

  • index string | array[string] Required

    A comma-separated list of data streams, indices, and aliases to search. It supports wildcards (*). To search all data streams and indices, omit this parameter or use * or _all.

Query parameters

  • If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • If true and there are shard request timeouts or shard failures, the request returns partial results. If false, it returns an error with no partial results.

    To override the default behavior, you can set the search.default_allow_partial_results cluster setting to false.

  • analyzer string

    The analyzer to use for the query string. This parameter can be used only when the q query string parameter is specified.

  • If true, wildcard and prefix queries are analyzed. This parameter can be used only when the q query string parameter is specified.

  • The number of shard results that should be reduced at once on the coordinating node. If the potential number of shards in the request can be large, this value should be used as a protection mechanism to reduce the memory overhead per search request.

  • If true, network round-trips between the coordinating node and the remote clusters are minimized when running cross-cluster search (CCS) requests.

  • The default operator for the query string query: AND or OR. This parameter can be used only when the q query string parameter is specified.

    Values are and, AND, or, or OR.

  • df string

    The field to use as a default when no field prefix is given in the query string. This parameter can be used only when the q query string parameter is specified.

  • docvalue_fields string | array[string]

    A comma-separated list of fields to return as the docvalue representation of a field for each hit.

  • expand_wildcards string | array[string]

    The type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. It supports comma-separated values such as open,hidden.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.
  • explain boolean

    If true, the request returns detailed information about score computation as part of a hit.

  • ignore_throttled boolean Deprecated

    If true, concrete, expanded or aliased indices will be ignored when frozen.

  • If false, the request returns an error if it targets a missing or closed index.

  • If true, the response includes the score contribution from any named queries.

    This functionality reruns each named query on every hit in a search response. Typically, this adds a small overhead to a request. However, using computationally expensive named queries on a large number of hits may add significant overhead.

  • lenient boolean

    If true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored. This parameter can be used only when the q query string parameter is specified.

  • The number of concurrent shard requests per node that the search runs concurrently. This value should be used to limit the impact of the search on the cluster in order to limit the number of concurrent shard requests.

  • The nodes and shards used for the search. By default, Elasticsearch selects from eligible nodes and shards using adaptive replica selection, accounting for allocation awareness. Valid values are:

    • _only_local to run the search only on shards on the local node.
    • _local to, if possible, run the search on shards on the local node, or if not, select shards using the default method.
    • _only_nodes:<node-id>,<node-id> to run the search on only the specified nodes IDs. If suitable shards exist on more than one selected node, use shards on those nodes using the default method. If none of the specified nodes are available, select shards from any available node using the default method.
    • _prefer_nodes:<node-id>,<node-id> to if possible, run the search on the specified nodes IDs. If not, select shards using the default method.
    • _shards:<shard>,<shard> to run the search only on the specified shards. You can combine this value with other preference values. However, the _shards value must come first. For example: _shards:2,3|_local.
    • <custom-string> (any string that does not start with _) to route searches with the same <custom-string> to the same shards in the same order.
  • A threshold that enforces a pre-filter roundtrip to prefilter search shards based on query rewriting if the number of shards the search request expands to exceeds the threshold. This filter roundtrip can limit the number of shards significantly if for instance a shard can not match any documents based on its rewrite method (if date filters are mandatory to match but the shard bounds and the query are disjoint). When unspecified, the pre-filter phase is executed if any of these conditions is met:

    • The request targets more than 128 shards.
    • The request targets one or more read-only index.
    • The primary sort of the query targets an indexed field.
  • If true, the caching of search results is enabled for requests where size is 0. It defaults to index level settings.

  • routing string

    A custom value that is used to route operations to a specific shard.

  • scroll string

    The period to retain the search context for scrolling. By default, this value cannot exceed 1d (24 hours). You can change this limit by using the search.max_keep_alive cluster-level setting.

  • Indicates how distributed term frequencies are calculated for relevance scoring.

    Supported values include:

    • query_then_fetch: Documents are scored using local term and document frequencies for the shard. This is usually faster but less accurate.
    • dfs_query_then_fetch: Documents are scored using global term and document frequencies across all shards. This is usually slower but more accurate.

    Values are query_then_fetch or dfs_query_then_fetch.

  • stats array[string]

    Specific tag of the request for logging and statistical purposes.

  • stored_fields string | array[string]

    A comma-separated list of stored fields to return as part of a hit. If no fields are specified, no stored fields are included in the response. If this field is specified, the _source parameter defaults to false. You can pass _source: true to return both source fields and stored fields in the search response.

  • The field to use for suggestions.

  • The suggest mode. This parameter can be used only when the suggest_field and suggest_text query string parameters are specified.

    Supported values include:

    • missing: Only generate suggestions for terms that are not in the shard.
    • popular: Only suggest terms that occur in more docs on the shard than the original term.
    • always: Suggest any matching suggestions based on terms in the suggest text.

    Values are missing, popular, or always.

  • The number of suggestions to return. This parameter can be used only when the suggest_field and suggest_text query string parameters are specified.

  • The source text for which the suggestions should be returned. This parameter can be used only when the suggest_field and suggest_text query string parameters are specified.

  • The maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting.

    IMPORTANT: Use with caution. Elasticsearch applies this parameter to each shard handling the request. When possible, let Elasticsearch perform early termination automatically. Avoid specifying this parameter for requests that target data streams with backing indices across multiple data tiers. If set to 0 (default), the query does not terminate early.

  • timeout string

    The period of time to wait for a response from each shard. If no response is received before the timeout expires, the request fails and returns an error. It defaults to no timeout.

  • track_total_hits boolean | number

    The number of hits matching the query to count accurately. If true, the exact number of hits is returned at the cost of some performance. If false, the response does not include the total number of hits matching the query.

  • If true, the request calculates and returns document scores, even if the scores are not used for sorting.

  • typed_keys boolean

    If true, aggregation and suggester names are be prefixed by their respective types in the response.

  • Indicates whether hits.total should be rendered as an integer or an object in the rest search response.

  • version boolean

    If true, the request returns the document version as part of a hit.

  • _source boolean | string | array[string]

    The source fields that are returned for matching documents. These fields are returned in the hits._source property of the search response. Valid values are:

    • true to return the entire document source.
    • false to not return the document source.
    • <string> to return the source fields that are specified as a comma-separated list that supports wildcard (*) patterns.
  • _source_excludes string | array[string]

    A comma-separated list of source fields to exclude from the response. You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter. If the _source parameter is false, this parameter is ignored.

  • _source_includes string | array[string]

    A comma-separated list of source fields to include in the response. If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the _source_excludes query parameter. If the _source parameter is false, this parameter is ignored.

  • If true, the request returns the sequence number and primary term of the last modification of each hit.

  • q string

    A query in the Lucene query string syntax. Query parameter searches do not support the full Elasticsearch Query DSL but are handy for testing.

    IMPORTANT: This parameter overrides the query parameter in the request body. If both parameters are specified, documents matching the query request body parameter are not returned.

  • size number

    The number of hits to return. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.

  • from number

    The starting document offset, which must be non-negative. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.

  • sort string | array[string]

    A comma-separated list of <field>:<direction> pairs.

application/json

Body

  • Defines the aggregations that are run as part of the search request.

    External documentation
  • collapse object
    External documentation
  • explain boolean

    If true, the request returns detailed information about score computation as part of a hit.

  • ext object

    Configuration of search extensions defined by Elasticsearch plugins.

    Hide ext attribute Show ext attribute object
    • * object Additional properties
  • from number

    The starting document offset, which must be non-negative. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.

  • Hide highlight attributes Show highlight attributes object
    • A string that contains each boundary character.

    • How far to scan for boundary characters.

    • Values are chars, sentence, or word.

    • Controls which locale is used to search for sentence and word boundaries. This parameter takes a form of a language tag, for example: "en-US", "fr-FR", "ja-JP".

    • force_source boolean Deprecated
    • Values are simple or span.

    • The size of the highlighted fragment in characters.

    • An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

      External documentation
    • If set to a non-negative value, highlighting stops at this defined maximum limit. The rest of the text is not processed, thus not highlighted and no error is returned The max_analyzed_offset query setting does not override the index.highlight.max_analyzed_offset setting, which prevails when it’s set to lower value than the query setting.

    • The amount of text you want to return from the beginning of the field if there are no matching fragments to highlight.

    • The maximum number of fragments to return. If the number of fragments is set to 0, no fragments are returned. Instead, the entire field contents are highlighted and returned. This can be handy when you need to highlight short texts such as a title or address, but fragmentation is not required. If number_of_fragments is 0, fragment_size is ignored.

    • options object
      Hide options attribute Show options attribute object
      • * object Additional properties
    • order string

      Value is score.

    • Controls the number of matching phrases in a document that are considered. Prevents the fvh highlighter from analyzing too many phrases and consuming too much memory. When using matched_fields, phrase_limit phrases per matched field are considered. Raising the limit increases query time and consumes more memory. Only supported by the fvh highlighter.

    • post_tags array[string]

      Use in conjunction with pre_tags to define the HTML tags to use for the highlighted text. By default, highlighted text is wrapped in <em> and </em> tags.

    • pre_tags array[string]

      Use in conjunction with post_tags to define the HTML tags to use for the highlighted text. By default, highlighted text is wrapped in <em> and </em> tags.

    • By default, only fields that contains a query match are highlighted. Set to false to highlight all fields.

    • Value is styled.

    • encoder string

      Values are default or html.

    • fields object Required
  • track_total_hits boolean | number

    Number of hits matching the query to count accurately. If true, the exact number of hits is returned at the cost of some performance. If false, the response does not include the total number of hits matching the query. Defaults to 10,000 hits.

  • indices_boost array[object]

    Boost the _score of documents from specified indices. The boost value is the factor by which scores are multiplied. A boost value greater than 1.0 increases the score. A boost value between 0 and 1.0 decreases the score.

    External documentation
    Hide indices_boost attribute Show indices_boost attribute object
    • * number Additional properties
  • docvalue_fields array[object]

    An array of wildcard (*) field patterns. The request returns doc values for field names matching these patterns in the hits.fields property of the response.

    External documentation
    Hide docvalue_fields attributes Show docvalue_fields attributes object
    • field string Required

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • format string

      The format in which the values are returned.

  • knn object | array[object]

    The approximate kNN search to run.

    One of:
    Hide attributes Show attributes
    • field string Required

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • query_vector array[number]
    • Hide query_vector_builder attribute Show query_vector_builder attribute object
    • k number

      The final number of nearest neighbors to return as top hits

    • The number of nearest neighbor candidates to consider per shard

    • boost number

      Boost value to apply to kNN scores

    • filter object | array[object]

      Filters for the kNN search query

      One of:

      An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

      External documentation
    • The minimum similarity for a vector to be considered a match

    • Hide inner_hits attributes Show inner_hits attributes object
      • name string
      • size number

        The maximum number of hits to return per inner_hits.

      • from number

        Inner hit starting document offset.

      • collapse object
        External documentation
      • docvalue_fields array[object]
        Hide docvalue_fields attributes Show docvalue_fields attributes object
        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • format string

          The format in which the values are returned.

      • explain boolean
      • Hide highlight attributes Show highlight attributes object
        • A string that contains each boundary character.

        • How far to scan for boundary characters.

        • Values are chars, sentence, or word.

        • Controls which locale is used to search for sentence and word boundaries. This parameter takes a form of a language tag, for example: "en-US", "fr-FR", "ja-JP".

        • force_source boolean Deprecated
        • Values are simple or span.

        • The size of the highlighted fragment in characters.

        • An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

          External documentation
        • If set to a non-negative value, highlighting stops at this defined maximum limit. The rest of the text is not processed, thus not highlighted and no error is returned The max_analyzed_offset query setting does not override the index.highlight.max_analyzed_offset setting, which prevails when it’s set to lower value than the query setting.

        • The amount of text you want to return from the beginning of the field if there are no matching fragments to highlight.

        • The maximum number of fragments to return. If the number of fragments is set to 0, no fragments are returned. Instead, the entire field contents are highlighted and returned. This can be handy when you need to highlight short texts such as a title or address, but fragmentation is not required. If number_of_fragments is 0, fragment_size is ignored.

        • options object
          Hide options attribute Show options attribute object
          • * object Additional properties
        • order string

          Value is score.

        • Controls the number of matching phrases in a document that are considered. Prevents the fvh highlighter from analyzing too many phrases and consuming too much memory. When using matched_fields, phrase_limit phrases per matched field are considered. Raising the limit increases query time and consumes more memory. Only supported by the fvh highlighter.

        • post_tags array[string]

          Use in conjunction with pre_tags to define the HTML tags to use for the highlighted text. By default, highlighted text is wrapped in <em> and </em> tags.

        • pre_tags array[string]

          Use in conjunction with post_tags to define the HTML tags to use for the highlighted text. By default, highlighted text is wrapped in <em> and </em> tags.

        • By default, only fields that contains a query match are highlighted. Set to false to highlight all fields.

        • Value is styled.

        • encoder string

          Values are default or html.

        • fields object Required
      • Hide script_fields attribute Show script_fields attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • script object Required
            Hide script attributes Show script attributes object
            • id string
            • params object

              Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            • options object
      • fields array[string]

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • sort string | object | array[string | object]

        One of:

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • _source boolean | object

        Defines how to fetch a source. Fetching can be disabled entirely, or the source can be filtered.

        One of:
      • stored_fields string | array[string]
      • version boolean
    • Hide rescore_vector attribute Show rescore_vector attribute object
      • oversample number Required

        Applies the specified oversample factor to k on the approximate kNN search

  • rank object
    Hide rank attribute Show rank attribute object
    • rrf object
      Hide rrf attributes Show rrf attributes object
      • How much influence documents in individual result sets per query have over the final ranked result set

      • Size of the individual result sets per query

  • The minimum _score for matching documents. Documents with a lower _score are not included in search results and results collected by aggregations.

  • An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

    External documentation
  • profile boolean

    Set to true to return detailed timing information about the execution of individual components in a search request. NOTE: This is a debugging tool and adds significant overhead to search execution.

  • query object

    An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

    External documentation
  • rescore object | array[object]

    Can be used to improve precision by reordering just the top (for example 100 - 500) documents returned by the query and post_filter phases.

    One of:
    Hide attributes Show attributes
    • query object
      Hide query attributes Show query attributes object
    • Hide learning_to_rank attributes Show learning_to_rank attributes object
      • model_id string Required

        The unique identifier of the trained model uploaded to Elasticsearch

      • params object

        Named parameters to be passed to the query templates used for feature

        Hide params attribute Show params attribute object
        • * object Additional properties
  • Hide retriever attributes Show retriever attributes object
  • Retrieve a script evaluation (based on different fields) for each hit.

    Hide script_fields attribute Show script_fields attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • script object Required
        Hide script attributes Show script attributes object
        • source string | object

          One of:
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          Hide params attribute Show params attribute object
          • * object Additional properties
        • lang string

          Any of:

          Values are painless, expression, mustache, or java.

        • options object
          Hide options attribute Show options attribute object
          • * string Additional properties
  • search_after array[number | string | boolean | null]

    A field value.

  • size number

    The number of hits to return, which must not be negative. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after property.

  • slice object
    Hide slice attributes Show slice attributes object
    • field string

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • id string Required
    • max number Required
  • sort string | object | array[string | object]

    One of:

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • _source boolean | object

    Defines how to fetch a source. Fetching can be disabled entirely, or the source can be filtered.

    One of:
  • fields array[object]

    An array of wildcard (*) field patterns. The request returns values for field names matching these patterns in the hits.fields property of the response.

    Hide fields attributes Show fields attributes object
    • field string Required

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • format string

      The format in which the values are returned.

  • suggest object
    Hide suggest attribute Show suggest attribute object
    • text string

      Global suggest text, to avoid repetition when the same text is used in several suggesters

  • The maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting.

    IMPORTANT: Use with caution. Elasticsearch applies this property to each shard handling the request. When possible, let Elasticsearch perform early termination automatically. Avoid specifying this property for requests that target data streams with backing indices across multiple data tiers.

    If set to 0 (default), the query does not terminate early.

  • timeout string

    The period of time to wait for a response from each shard. If no response is received before the timeout expires, the request fails and returns an error. Defaults to no timeout.

  • If true, calculate and return document scores, even if the scores are not used for sorting.

  • version boolean

    If true, the request returns the document version as part of a hit.

  • If true, the request returns sequence number and primary term of the last modification of each hit.

    External documentation
  • stored_fields string | array[string]
  • pit object
    Hide pit attributes Show pit attributes object
    • id string Required
    • A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • Hide runtime_mappings attribute Show runtime_mappings attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • fields object

        For type composite

        Hide fields attribute Show fields attribute object
        • * object Additional properties
          Hide * attribute Show * attribute object
          • type string Required

            Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

      • fetch_fields array[object]

        For type lookup

        Hide fetch_fields attributes Show fetch_fields attributes object
        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • format string
      • format string

        A custom format for date type runtime fields.

      • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • script object
        Hide script attributes Show script attributes object
        • source string | object

          One of:
        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          Hide params attribute Show params attribute object
          • * object Additional properties
        • lang string

          Any of:

          Values are painless, expression, mustache, or java.

        • options object
          Hide options attribute Show options attribute object
          • * string Additional properties
      • type string Required

        Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

  • stats array[string]

    The stats groups to associate with the search. Each group maintains a statistics aggregation for its associated searches. You can retrieve these stats using the indices stats API.

Responses

GET /{index}/_search
curl \
 --request GET 'http://api.example.com/{index}/_search' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"query\": {\n    \"term\": {\n      \"user.id\": \"kimchy\"\n    }\n  }\n}"'
Run `GET /my-index-000001/_search?from=40&size=20` to run a search.
{
  "query": {
    "term": {
      "user.id": "kimchy"
    }
  }
}
Run `POST /_search` to run a point in time search. The `id` parameter tells Elasticsearch to run the request using contexts from this open point in time. The `keep_alive` parameter tells Elasticsearch how long it should extend the time to live of the point in time.
{
    "size": 100,  
    "query": {
        "match" : {
            "title" : "elasticsearch"
        }
    },
    "pit": {
      "id":  "46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA==", 
      "keep_alive": "1m"  
    }
}
When paging through a large number of documents, it can be helpful to split the search into multiple slices to consume them independently. The result from running the first `GET /_search` request returns documents belonging to the first slice (`id: 0`). If you run a second request with `id` set to `1', it returns documents in the second slice. Since the maximum number of slices is set to `2`, the union of the results is equivalent to the results of a point-in-time search without slicing.
{
  "slice": {
    "id": 0,                      
    "max": 2                      
  },
  "query": {
    "match": {
      "message": "foo"
    }
  },
  "pit": {
    "id": "46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA=="
  }
}
Response examples (200)
An abbreviated response from `GET /my-index-000001/_search?from=40&size=20` with a simple term query.
{
  "took": 5,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 20,
      "relation": "eq"
    },
    "max_score": 1.3862942,
    "hits": [
      {
        "_index": "my-index-000001",
        "_id": "0",
        "_score": 1.3862942,
        "_source": {
          "@timestamp": "2099-11-15T14:12:12",
          "http": {
            "request": {
              "method": "get"
            },
            "response": {
              "status_code": 200,
              "bytes": 1070000
            },
            "version": "1.1"
          },
          "source": {
            "ip": "127.0.0.1"
          },
          "message": "GET /search HTTP/1.1 200 1070000",
          "user": {
            "id": "kimchy"
          }
        }
      }
    ]
  }
}






























































































Clear the cache Technical preview

POST /{index}/_searchable_snapshots/cache/clear

Clear indices and data streams from the shared cache for partially mounted indices.

External documentation

Path parameters

  • index string | array[string] Required

    A comma-separated list of data streams, indices, and aliases to clear from the cache. It supports wildcards (*).

Query parameters

  • expand_wildcards string | array[string]

    Whether to expand wildcard expression to concrete indices that are open, closed or both.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.
  • Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes _all string or when no indices have been specified)

  • Whether specified concrete indices should be ignored when unavailable (missing or closed)

Responses

POST /{index}/_searchable_snapshots/cache/clear
curl \
 --request POST 'http://api.example.com/{index}/_searchable_snapshots/cache/clear' \
 --header "Authorization: $API_KEY"

























































































Create a cross-cluster API key

POST /_security/cross_cluster/api_key

Create an API key of the cross_cluster type for the API key based remote cluster access. A cross_cluster API key cannot be used to authenticate through the REST interface.

IMPORTANT: To authenticate this request you must use a credential that is not an API key. Even if you use an API key that has the required privilege, the API returns an error.

Cross-cluster API keys are created by the Elasticsearch API key service, which is automatically enabled.

NOTE: Unlike REST API keys, a cross-cluster API key does not capture permissions of the authenticated user. The API key’s effective permission is exactly as specified with the access property.

A successful request returns a JSON structure that contains the API key, its unique ID, and its name. If applicable, it also returns expiration information for the API key in milliseconds.

By default, API keys never expire. You can specify expiration information when you create the API keys.

Cross-cluster API keys can only be updated with the update cross-cluster API key API. Attempting to update them with the update REST API key API or the bulk update REST API keys API will result in an error.

External documentation
application/json

Body Required

  • access object Required
    Hide access attributes Show access attributes object
    • replication array[object]

      A list of indices permission entries for cross-cluster replication.

      Hide replication attributes Show replication attributes object
  • A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • metadata object
    Hide metadata attribute Show metadata attribute object
    • * object Additional properties
  • name string Required

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • api_key string Required

      Generated API key.

    • Time unit for milliseconds

    • id string Required
    • name string Required
    • encoded string Required

      API key credentials which is the base64-encoding of the UTF-8 representation of id and api_key joined by a colon (:).

POST /_security/cross_cluster/api_key
curl \
 --request POST 'http://api.example.com/_security/cross_cluster/api_key' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"name\": \"my-cross-cluster-api-key\",\n  \"expiration\": \"1d\",   \n  \"access\": {\n    \"search\": [  \n      {\n        \"names\": [\"logs*\"]\n      }\n    ],\n    \"replication\": [  \n      {\n        \"names\": [\"archive*\"]\n      }\n    ]\n  },\n  \"metadata\": {\n    \"description\": \"phase one\",\n    \"environment\": {\n      \"level\": 1,\n      \"trusted\": true,\n      \"tags\": [\"dev\", \"staging\"]\n    }\n  }\n}"'
Request example
Run `POST /_security/cross_cluster/api_key` to create a cross-cluster API key.
{
  "name": "my-cross-cluster-api-key",
  "expiration": "1d",   
  "access": {
    "search": [  
      {
        "names": ["logs*"]
      }
    ],
    "replication": [  
      {
        "names": ["archive*"]
      }
    ]
  },
  "metadata": {
    "description": "phase one",
    "environment": {
      "level": 1,
      "trusted": true,
      "tags": ["dev", "staging"]
    }
  }
}
Response examples (200)
A successful response from `POST /_security/service/elastic/fleet-server/credential/token`.
{
  "created": true,
  "token": {
    "name": "Jk5J1HgBuyBK5TpDrdo4",
    "value": "AAEAAWVsYXN0aWM...vZmxlZXQtc2VydmVyL3Rva2VuMTo3TFdaSDZ"
  }
}




































Create or update roles

POST /_security/role/{name}

The role management APIs are generally the preferred way to manage roles in the native realm, rather than using file-based role management. The create or update roles API cannot update roles that are defined in roles files. File-based role management is not available in Elastic Serverless.

External documentation

Path parameters

  • name string Required

    The name of the role that is being created or updated. On Elasticsearch Serverless, the role name must begin with a letter or digit and can only contain letters, digits and the characters '_', '-', and '.'. Each role must have a unique name, as this will serve as the identifier for that role.

Query parameters

  • refresh string

    If true (the default) then refresh the affected shards to make this operation visible to search, if wait_for then wait for a refresh to make this operation visible to search, if false then do nothing with refreshes.

    Values are true, false, or wait_for.

application/json

Body Required

  • applications array[object]

    A list of application privilege entries.

    Hide applications attributes Show applications attributes object
    • application string Required

      The name of the application to which this entry applies.

    • privileges array[string] Required

      A list of strings, where each element is the name of an application privilege or action.

    • resources array[string] Required

      A list resources to which the privileges are applied.

  • cluster array[string]

    A list of cluster privileges. These privileges define the cluster-level actions for users with this role.

  • global object

    An object defining global privileges. A global privilege is a form of cluster privilege that is request-aware. Support for global privileges is currently limited to the management of application privileges.

    Hide global attribute Show global attribute object
    • * object Additional properties
  • indices array[object]

    A list of indices permissions entries.

    Hide indices attributes Show indices attributes object
    • Hide field_security attributes Show field_security attributes object
    • names string | array[string]

      A list of indices (or index name patterns) to which the permissions in this entry apply.

    • privileges array[string] Required

      The index level privileges that owners of the role have on the specified indices.

    • query string | object

      While creating or updating a role you can provide either a JSON structure or a string to the API. However, the response provided by Elasticsearch will only be string with a json-as-text content.

      Since this is embedded in IndicesPrivileges, the same structure is used for clarity in both contexts.

      One of:
    • Set to true if using wildcard or regular expressions for patterns that cover restricted indices. Implicitly, restricted indices have limited privileges that can cause pattern tests to fail. If restricted indices are explicitly included in the names list, Elasticsearch checks privileges against these indices regardless of the value set for allow_restricted_indices.

  • remote_indices array[object]

    A list of remote indices permissions entries.

    NOTE: Remote indices are effective for remote clusters configured with the API key based model. They have no effect for remote clusters configured with the certificate based model.

    Hide remote_indices attributes Show remote_indices attributes object
  • remote_cluster array[object]

    A list of remote cluster permissions entries.

    Hide remote_cluster attributes Show remote_cluster attributes object
    • clusters string | array[string] Required
    • privileges array[string] Required

      The cluster level privileges that owners of the role have on the remote cluster.

      Values are monitor_enrich or monitor_stats.

  • metadata object
    Hide metadata attribute Show metadata attribute object
    • * object Additional properties
  • run_as array[string]

    A list of users that the owners of this role can impersonate. Note: in Serverless, the run-as feature is disabled. For API compatibility, you can still specify an empty run_as field, but a non-empty list will be rejected.

  • Optional description of the role descriptor

  • Indicates roles that might be incompatible with the current cluster license, specifically roles with document and field level security. When the cluster license doesn’t allow certain features for a given role, this parameter is updated dynamically to list the incompatible features. If enabled is false, the role is ignored, but is still listed in the response from the authenticate API.

    Hide transient_metadata attribute Show transient_metadata attribute object
    • * object Additional properties

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • role object Required
      Hide role attribute Show role attribute object
POST /_security/role/{name}
curl \
 --request POST 'http://api.example.com/_security/role/{name}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"description\": \"Grants full access to all management features within the cluster.\",\n  \"cluster\": [\"all\"],\n  \"indices\": [\n    {\n      \"names\": [ \"index1\", \"index2\" ],\n      \"privileges\": [\"all\"],\n      \"field_security\" : { // optional\n        \"grant\" : [ \"title\", \"body\" ]\n      },\n      \"query\": \"{\\\"match\\\": {\\\"title\\\": \\\"foo\\\"}}\" // optional\n    }\n  ],\n  \"applications\": [\n    {\n      \"application\": \"myapp\",\n      \"privileges\": [ \"admin\", \"read\" ],\n      \"resources\": [ \"*\" ]\n    }\n  ],\n  \"run_as\": [ \"other_user\" ], // optional\n  \"metadata\" : { // optional\n    \"version\" : 1\n  }\n}"'
Run `POST /_security/role/my_admin_role` to create a role.
{
  "description": "Grants full access to all management features within the cluster.",
  "cluster": ["all"],
  "indices": [
    {
      "names": [ "index1", "index2" ],
      "privileges": ["all"],
      "field_security" : { // optional
        "grant" : [ "title", "body" ]
      },
      "query": "{\"match\": {\"title\": \"foo\"}}" // optional
    }
  ],
  "applications": [
    {
      "application": "myapp",
      "privileges": [ "admin", "read" ],
      "resources": [ "*" ]
    }
  ],
  "run_as": [ "other_user" ], // optional
  "metadata" : { // optional
    "version" : 1
  }
}
Run `POST /_security/role/cli_or_drivers_minimal` to configure a role that can run SQL in JDBC.
{
  "cluster": ["cluster:monitor/main"],
  "indices": [
    {
      "names": ["test"],
      "privileges": ["read", "indices:admin/get"]
    }
  ]
}
Run `POST /_security/role/only_remote_access_role` to configure a role with remote indices and remote cluster privileges for a remote cluster.
{
  "remote_indices": [
    {
      "clusters": ["my_remote"], 
      "names": ["logs*"], 
      "privileges": ["read", "read_cross_cluster", "view_index_metadata"] 
    }
  ],
  "remote_cluster": [
    {
      "clusters": ["my_remote"], 
      "privileges": ["monitor_stats"]  
    }
  ]
}
Response examples (200)
A successful response from `POST /_security/role/my_admin_role`.
{
  "role": {
    "created": true 
  }
}
















Delete role mappings Added in 5.5.0

DELETE /_security/role_mapping/{name}

Role mappings define which roles are assigned to each user. The role mapping APIs are generally the preferred way to manage role mappings rather than using role mapping files. The delete role mappings API cannot remove role mappings that are defined in role mapping files.

External documentation

Path parameters

  • name string Required

    The distinct name that identifies the role mapping. The name is used solely as an identifier to facilitate interaction via the API; it does not affect the behavior of the mapping in any way.

Query parameters

  • refresh string

    If true (the default) then refresh the affected shards to make this operation visible to search, if wait_for then wait for a refresh to make this operation visible to search, if false then do nothing with refreshes.

    Values are true, false, or wait_for.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • found boolean Required

      If the mapping is successfully deleted, found is true. Otherwise, found is false.

DELETE /_security/role_mapping/{name}
curl \
 --request DELETE 'http://api.example.com/_security/role_mapping/{name}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `DELETE /_security/role_mapping/mapping1`. If the mapping is successfully deleted, `found` is `true`.
{
  "found" : true
}




































































































































































Find API keys with a query Added in 7.15.0

GET /_security/_query/api_key

Get a paginated list of API keys and their information. You can optionally filter the results with a query.

To use this API, you must have at least the manage_own_api_key or the read_security cluster privileges. If you have only the manage_own_api_key privilege, this API returns only the API keys that you own. If you have the read_security, manage_api_key, or greater privileges (including manage_security), this API returns all API keys regardless of ownership.

Query parameters

  • Return the snapshot of the owner user's role descriptors associated with the API key. An API key's actual permission is the intersection of its assigned role descriptors and the owner user's role descriptors (effectively limited by it). An API key cannot retrieve any API key’s limited-by role descriptors (including itself) unless it has manage_api_key or higher privileges.

  • Determines whether to also retrieve the profile UID for the API key owner principal. If it exists, the profile UID is returned under the profile_uid response field for each API key.

  • typed_keys boolean

    Determines whether aggregation names are prefixed by their respective types in the response.

application/json

Body

  • Any aggregations to run over the corpus of returned API keys. Aggregations and queries work together. Aggregations are computed only on the API keys that match the query. This supports only a subset of aggregation types, namely: terms, range, date_range, missing, cardinality, value_count, composite, filter, and filters. Additionally, aggregations only run over the same subset of fields that query works with.

  • query object
    Hide query attributes Show query attributes object
    • match object

      Returns documents that match a provided text, number, date or boolean value. The provided text is analyzed before matching.

    • prefix object

      Returns documents that contain a specific prefix in a provided field.

    • range object

      Returns documents that contain terms within a provided range.

    • term object

      Returns documents that contain an exact term in a provided field. To return a document, the query term must exactly match the queried field's value, including whitespace and capitalization.

    • wildcard object

      Returns documents that contain terms matching a wildcard pattern.

  • from number

    The starting document offset. It must not be negative. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.

  • sort string | object | array[string | object]

    One of:

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • size number

    The number of hits to return. It must not be negative. The size parameter can be set to 0, in which case no API key matches are returned, only the aggregation results. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.

  • search_after array[number | string | boolean | null]

    A field value.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • total number Required

      The total number of API keys found.

    • count number Required

      The number of API keys returned in the response.

    • api_keys array[object] Required

      A list of API key information.

      Hide api_keys attributes Show api_keys attributes object
      • id string Required
      • name string Required
      • type string Required

        Values are rest or cross_cluster.

      • creation number

        Time unit for milliseconds

      • Time unit for milliseconds

      • invalidated boolean Required

        Invalidation status for the API key. If the key has been invalidated, it has a value of true. Otherwise, it is false.

      • Time unit for milliseconds

      • username string Required
      • realm string Required

        Realm name of the principal for which this API key was created.

      • Realm type of the principal for which this API key was created

      • metadata object Required
        Hide metadata attribute Show metadata attribute object
        • * object Additional properties
      • The role descriptors assigned to this API key when it was created or last updated. An empty role descriptor means the API key inherits the owner user’s permissions.

        Hide role_descriptors attribute Show role_descriptors attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • cluster array[string]

            A list of cluster privileges. These privileges define the cluster level actions that API keys are able to execute.

          • indices array[object]

            A list of indices permissions entries.

          • remote_indices array[object]

            A list of indices permissions for remote clusters.

          • remote_cluster array[object]

            A list of cluster permissions for remote clusters. NOTE: This is limited a subset of the cluster permissions.

          • global array[object] | object

            An object defining global privileges. A global privilege is a form of cluster privilege that is request-aware. Support for global privileges is currently limited to the management of application privileges.

          • applications array[object]

            A list of application privilege entries

          • metadata object
            Hide metadata attribute Show metadata attribute object
            • * object Additional properties
          • run_as array[string]

            A list of users that the API keys can impersonate. NOTE: In Elastic Cloud Serverless, the run-as feature is disabled. For API compatibility, you can still specify an empty run_as field, but a non-empty list will be rejected.

          • Optional description of the role descriptor

          • Hide restriction attribute Show restriction attribute object
            • workflows array[string] Required

              A list of workflows to which the API key is restricted. NOTE: In order to use a role restriction, an API key must be created with a single role descriptor.

          • Hide transient_metadata attribute Show transient_metadata attribute object
            • * object Additional properties
      • limited_by array[object]

        The owner user’s permissions associated with the API key. It is a point-in-time snapshot captured at creation and subsequent updates. An API key’s effective permissions are an intersection of its assigned privileges and the owner user’s permissions.

        Hide limited_by attribute Show limited_by attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • cluster array[string]

            A list of cluster privileges. These privileges define the cluster level actions that API keys are able to execute.

          • indices array[object]

            A list of indices permissions entries.

          • remote_indices array[object]

            A list of indices permissions for remote clusters.

          • remote_cluster array[object]

            A list of cluster permissions for remote clusters. NOTE: This is limited a subset of the cluster permissions.

          • applications array[object]

            A list of application privilege entries

          • metadata object
          • run_as array[string]

            A list of users that the API keys can impersonate. NOTE: In Elastic Cloud Serverless, the run-as feature is disabled. For API compatibility, you can still specify an empty run_as field, but a non-empty list will be rejected.

          • Optional description of the role descriptor

      • access object
        Hide access attributes Show access attributes object
        • replication array[object]

          A list of indices permission entries for cross-cluster replication.

          Hide replication attributes Show replication attributes object
      • The profile uid for the API key owner principal, if requested and if it exists

      • _sort array[number | string | boolean | null]

        A field value.

    • The aggregations result, if requested.

GET /_security/_query/api_key
curl \
 --request GET 'http://api.example.com/_security/_query/api_key' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"query\": {\n    \"ids\": {\n      \"values\": [\n        \"VuaCfGcBCdbkQm-e5aOx\"\n      ]\n    }\n  }\n}"'
Run `GET /_security/_query/api_key?with_limited_by=true` to retrieve an API key by ID.
{
  "query": {
    "ids": {
      "values": [
        "VuaCfGcBCdbkQm-e5aOx"
      ]
    }
  }
}
Run `GET /_security/_query/api_key`. Use a `bool` query to issue complex logical conditions and use `from`, `size`, and `sort` to help paginate the result. For example, the API key name must begin with `app1-key-` and must not be `app1-key-01`. It must be owned by a username with the wildcard pattern `org-*-user` and the `environment` metadata field must have a `production` value. The offset to begin the search result is the twentieth (zero-based index) API key. The page size of the response is 10 API keys. The result is first sorted by creation date in descending order, then by name in ascending order.
{
  "query": {
    "bool": {
      "must": [
        {
          "prefix": {
            "name": "app1-key-" 
          }
        },
        {
          "term": {
            "invalidated": "false" 
          }
        }
      ],
      "must_not": [
        {
          "term": {
            "name": "app1-key-01" 
          }
        }
      ],
      "filter": [
        {
          "wildcard": {
            "username": "org-*-user" 
          }
        },
        {
          "term": {
            "metadata.environment": "production" 
          }
        }
      ]
    }
  },
  "from": 20, 
  "size": 10, 
  "sort": [ 
    { "creation": { "order": "desc", "format": "date_time" } },
    "name"
  ]
}
Run `GET /_security/_query/api_key` to retrieve the API key by name.
{
  "query": {
    "term": {
      "name": {
        "value": "application-key-1"
      }
    }
  }
}
A successful response from `GET /_security/_query/api_key?with_limited_by=true`. The `limited_by` details are the owner user's permissions associated with the API key. It is a point-in-time snapshot captured at creation and subsequent updates. An API key's effective permissions are an intersection of its assigned privileges and the owner user's permissions.
{
  "api_keys": [
    {
      "id": "VuaCfGcBCdbkQm-e5aOx",
      "name": "application-key-1",
      "creation": 1548550550158,
      "expiration": 1548551550158,
      "invalidated": false,
      "username": "myuser",
      "realm": "native1",
      "realm_type": "native",
      "metadata": {
        "application": "my-application"
      },
      "role_descriptors": { },
      "limited_by": [ 
        {
          "role-power-user": {
            "cluster": [
              "monitor"
            ],
            "indices": [
              {
                "names": [
                  "*"
                ],
                "privileges": [
                  "read"
                ],
                "allow_restricted_indices": false
              }
            ],
            "applications": [ ],
            "run_as": [ ],
            "metadata": { },
            "transient_metadata": {
              "enabled": true
            }
          }
        }
      ]
    }
  ]
}
An abbreviated response from `GET /_security/_query/api_key` that contains a list of matched API keys along with their sort values. The first sort value is creation time, which is displayed in `date_time` format. The second sort value is the API key name.
{
  "total": 100,
  "count": 10,
  "api_keys": [
    {
      "id": "CLXgVnsBOGkf8IyjcXU7",
      "name": "app1-key-79",
      "creation": 1629250154811,
      "invalidated": false,
      "username": "org-admin-user",
      "realm": "native1",
      "metadata": {
        "environment": "production"
      },
      "role_descriptors": { },
      "_sort": [
        "2021-08-18T01:29:14.811Z",  
        "app1-key-79"  
      ]
    },
    {
      "id": "BrXgVnsBOGkf8IyjbXVB",
      "name": "app1-key-78",
      "creation": 1629250153794,
      "invalidated": false,
      "username": "org-admin-user",
      "realm": "native1",
      "metadata": {
        "environment": "production"
      },
      "role_descriptors": { },
      "_sort": [
        "2021-08-18T01:29:13.794Z",
        "app1-key-78"
      ]
    }
  ]
}
A successful response from `GET /_security/_query/api_key`. It includes the role descriptors that are assigned to each API key when it was created or last updated. Note that an API key's effective permissions are an intersection of its assigned privileges and the point-in-time snapshot of the owner user's permissions. An empty role descriptors object means the API key inherits the owner user's permissions.
{
  "total": 3,
  "count": 3,
  "api_keys": [ 
    {
      "id": "nkvrGXsB8w290t56q3Rg",
      "name": "my-api-key-1",
      "creation": 1628227480421,
      "expiration": 1629091480421,
      "invalidated": false,
      "username": "elastic",
      "realm": "reserved",
      "realm_type": "reserved",
      "metadata": {
        "letter": "a"
      },
      "role_descriptors": { 
        "role-a": {
          "cluster": [
            "monitor"
          ],
          "indices": [
            {
              "names": [
                "index-a"
              ],
              "privileges": [
                "read"
              ],
              "allow_restricted_indices": false
            }
          ],
          "applications": [ ],
          "run_as": [ ],
          "metadata": { },
          "transient_metadata": {
            "enabled": true
          }
        }
      }
    },
    {
      "id": "oEvrGXsB8w290t5683TI",
      "name": "my-api-key-2",
      "creation": 1628227498953,
      "expiration": 1628313898953,
      "invalidated": false,
      "username": "elastic",
      "realm": "reserved",
      "metadata": {
        "letter": "b"
      },
      "role_descriptors": { } 
    }
  ]
}

Find API keys with a query Added in 7.15.0

POST /_security/_query/api_key

Get a paginated list of API keys and their information. You can optionally filter the results with a query.

To use this API, you must have at least the manage_own_api_key or the read_security cluster privileges. If you have only the manage_own_api_key privilege, this API returns only the API keys that you own. If you have the read_security, manage_api_key, or greater privileges (including manage_security), this API returns all API keys regardless of ownership.

Query parameters

  • Return the snapshot of the owner user's role descriptors associated with the API key. An API key's actual permission is the intersection of its assigned role descriptors and the owner user's role descriptors (effectively limited by it). An API key cannot retrieve any API key’s limited-by role descriptors (including itself) unless it has manage_api_key or higher privileges.

  • Determines whether to also retrieve the profile UID for the API key owner principal. If it exists, the profile UID is returned under the profile_uid response field for each API key.

  • typed_keys boolean

    Determines whether aggregation names are prefixed by their respective types in the response.

application/json

Body

  • Any aggregations to run over the corpus of returned API keys. Aggregations and queries work together. Aggregations are computed only on the API keys that match the query. This supports only a subset of aggregation types, namely: terms, range, date_range, missing, cardinality, value_count, composite, filter, and filters. Additionally, aggregations only run over the same subset of fields that query works with.

  • query object
    Hide query attributes Show query attributes object
    • match object

      Returns documents that match a provided text, number, date or boolean value. The provided text is analyzed before matching.

    • prefix object

      Returns documents that contain a specific prefix in a provided field.

    • range object

      Returns documents that contain terms within a provided range.

    • term object

      Returns documents that contain an exact term in a provided field. To return a document, the query term must exactly match the queried field's value, including whitespace and capitalization.

    • wildcard object

      Returns documents that contain terms matching a wildcard pattern.

  • from number

    The starting document offset. It must not be negative. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.

  • sort string | object | array[string | object]

    One of:

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • size number

    The number of hits to return. It must not be negative. The size parameter can be set to 0, in which case no API key matches are returned, only the aggregation results. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.

  • search_after array[number | string | boolean | null]

    A field value.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • total number Required

      The total number of API keys found.

    • count number Required

      The number of API keys returned in the response.

    • api_keys array[object] Required

      A list of API key information.

      Hide api_keys attributes Show api_keys attributes object
      • id string Required
      • name string Required
      • type string Required

        Values are rest or cross_cluster.

      • creation number

        Time unit for milliseconds

      • Time unit for milliseconds

      • invalidated boolean Required

        Invalidation status for the API key. If the key has been invalidated, it has a value of true. Otherwise, it is false.

      • Time unit for milliseconds

      • username string Required
      • realm string Required

        Realm name of the principal for which this API key was created.

      • Realm type of the principal for which this API key was created

      • metadata object Required
        Hide metadata attribute Show metadata attribute object
        • * object Additional properties
      • The role descriptors assigned to this API key when it was created or last updated. An empty role descriptor means the API key inherits the owner user’s permissions.

        Hide role_descriptors attribute Show role_descriptors attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • cluster array[string]

            A list of cluster privileges. These privileges define the cluster level actions that API keys are able to execute.

          • indices array[object]

            A list of indices permissions entries.

          • remote_indices array[object]

            A list of indices permissions for remote clusters.

          • remote_cluster array[object]

            A list of cluster permissions for remote clusters. NOTE: This is limited a subset of the cluster permissions.

          • global array[object] | object

            An object defining global privileges. A global privilege is a form of cluster privilege that is request-aware. Support for global privileges is currently limited to the management of application privileges.

          • applications array[object]

            A list of application privilege entries

          • metadata object
            Hide metadata attribute Show metadata attribute object
            • * object Additional properties
          • run_as array[string]

            A list of users that the API keys can impersonate. NOTE: In Elastic Cloud Serverless, the run-as feature is disabled. For API compatibility, you can still specify an empty run_as field, but a non-empty list will be rejected.

          • Optional description of the role descriptor

          • Hide restriction attribute Show restriction attribute object
            • workflows array[string] Required

              A list of workflows to which the API key is restricted. NOTE: In order to use a role restriction, an API key must be created with a single role descriptor.

          • Hide transient_metadata attribute Show transient_metadata attribute object
            • * object Additional properties
      • limited_by array[object]

        The owner user’s permissions associated with the API key. It is a point-in-time snapshot captured at creation and subsequent updates. An API key’s effective permissions are an intersection of its assigned privileges and the owner user’s permissions.

        Hide limited_by attribute Show limited_by attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • cluster array[string]

            A list of cluster privileges. These privileges define the cluster level actions that API keys are able to execute.

          • indices array[object]

            A list of indices permissions entries.

          • remote_indices array[object]

            A list of indices permissions for remote clusters.

          • remote_cluster array[object]

            A list of cluster permissions for remote clusters. NOTE: This is limited a subset of the cluster permissions.

          • applications array[object]

            A list of application privilege entries

          • metadata object
          • run_as array[string]

            A list of users that the API keys can impersonate. NOTE: In Elastic Cloud Serverless, the run-as feature is disabled. For API compatibility, you can still specify an empty run_as field, but a non-empty list will be rejected.

          • Optional description of the role descriptor

      • access object
        Hide access attributes Show access attributes object
        • replication array[object]

          A list of indices permission entries for cross-cluster replication.

          Hide replication attributes Show replication attributes object
      • The profile uid for the API key owner principal, if requested and if it exists

      • _sort array[number | string | boolean | null]

        A field value.

    • The aggregations result, if requested.

POST /_security/_query/api_key
curl \
 --request POST 'http://api.example.com/_security/_query/api_key' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"query\": {\n    \"ids\": {\n      \"values\": [\n        \"VuaCfGcBCdbkQm-e5aOx\"\n      ]\n    }\n  }\n}"'
Run `GET /_security/_query/api_key?with_limited_by=true` to retrieve an API key by ID.
{
  "query": {
    "ids": {
      "values": [
        "VuaCfGcBCdbkQm-e5aOx"
      ]
    }
  }
}
Run `GET /_security/_query/api_key`. Use a `bool` query to issue complex logical conditions and use `from`, `size`, and `sort` to help paginate the result. For example, the API key name must begin with `app1-key-` and must not be `app1-key-01`. It must be owned by a username with the wildcard pattern `org-*-user` and the `environment` metadata field must have a `production` value. The offset to begin the search result is the twentieth (zero-based index) API key. The page size of the response is 10 API keys. The result is first sorted by creation date in descending order, then by name in ascending order.
{
  "query": {
    "bool": {
      "must": [
        {
          "prefix": {
            "name": "app1-key-" 
          }
        },
        {
          "term": {
            "invalidated": "false" 
          }
        }
      ],
      "must_not": [
        {
          "term": {
            "name": "app1-key-01" 
          }
        }
      ],
      "filter": [
        {
          "wildcard": {
            "username": "org-*-user" 
          }
        },
        {
          "term": {
            "metadata.environment": "production" 
          }
        }
      ]
    }
  },
  "from": 20, 
  "size": 10, 
  "sort": [ 
    { "creation": { "order": "desc", "format": "date_time" } },
    "name"
  ]
}
Run `GET /_security/_query/api_key` to retrieve the API key by name.
{
  "query": {
    "term": {
      "name": {
        "value": "application-key-1"
      }
    }
  }
}
A successful response from `GET /_security/_query/api_key?with_limited_by=true`. The `limited_by` details are the owner user's permissions associated with the API key. It is a point-in-time snapshot captured at creation and subsequent updates. An API key's effective permissions are an intersection of its assigned privileges and the owner user's permissions.
{
  "api_keys": [
    {
      "id": "VuaCfGcBCdbkQm-e5aOx",
      "name": "application-key-1",
      "creation": 1548550550158,
      "expiration": 1548551550158,
      "invalidated": false,
      "username": "myuser",
      "realm": "native1",
      "realm_type": "native",
      "metadata": {
        "application": "my-application"
      },
      "role_descriptors": { },
      "limited_by": [ 
        {
          "role-power-user": {
            "cluster": [
              "monitor"
            ],
            "indices": [
              {
                "names": [
                  "*"
                ],
                "privileges": [
                  "read"
                ],
                "allow_restricted_indices": false
              }
            ],
            "applications": [ ],
            "run_as": [ ],
            "metadata": { },
            "transient_metadata": {
              "enabled": true
            }
          }
        }
      ]
    }
  ]
}
An abbreviated response from `GET /_security/_query/api_key` that contains a list of matched API keys along with their sort values. The first sort value is creation time, which is displayed in `date_time` format. The second sort value is the API key name.
{
  "total": 100,
  "count": 10,
  "api_keys": [
    {
      "id": "CLXgVnsBOGkf8IyjcXU7",
      "name": "app1-key-79",
      "creation": 1629250154811,
      "invalidated": false,
      "username": "org-admin-user",
      "realm": "native1",
      "metadata": {
        "environment": "production"
      },
      "role_descriptors": { },
      "_sort": [
        "2021-08-18T01:29:14.811Z",  
        "app1-key-79"  
      ]
    },
    {
      "id": "BrXgVnsBOGkf8IyjbXVB",
      "name": "app1-key-78",
      "creation": 1629250153794,
      "invalidated": false,
      "username": "org-admin-user",
      "realm": "native1",
      "metadata": {
        "environment": "production"
      },
      "role_descriptors": { },
      "_sort": [
        "2021-08-18T01:29:13.794Z",
        "app1-key-78"
      ]
    }
  ]
}
A successful response from `GET /_security/_query/api_key`. It includes the role descriptors that are assigned to each API key when it was created or last updated. Note that an API key's effective permissions are an intersection of its assigned privileges and the point-in-time snapshot of the owner user's permissions. An empty role descriptors object means the API key inherits the owner user's permissions.
{
  "total": 3,
  "count": 3,
  "api_keys": [ 
    {
      "id": "nkvrGXsB8w290t56q3Rg",
      "name": "my-api-key-1",
      "creation": 1628227480421,
      "expiration": 1629091480421,
      "invalidated": false,
      "username": "elastic",
      "realm": "reserved",
      "realm_type": "reserved",
      "metadata": {
        "letter": "a"
      },
      "role_descriptors": { 
        "role-a": {
          "cluster": [
            "monitor"
          ],
          "indices": [
            {
              "names": [
                "index-a"
              ],
              "privileges": [
                "read"
              ],
              "allow_restricted_indices": false
            }
          ],
          "applications": [ ],
          "run_as": [ ],
          "metadata": { },
          "transient_metadata": {
            "enabled": true
          }
        }
      }
    },
    {
      "id": "oEvrGXsB8w290t5683TI",
      "name": "my-api-key-2",
      "creation": 1628227498953,
      "expiration": 1628313898953,
      "invalidated": false,
      "username": "elastic",
      "realm": "reserved",
      "metadata": {
        "letter": "b"
      },
      "role_descriptors": { } 
    }
  ]
}









































































Clone a snapshot Added in 7.10.0

PUT /_snapshot/{repository}/{snapshot}/_clone/{target_snapshot}

Clone part of all of a snapshot into another snapshot in the same repository.

Path parameters

  • repository string Required

    The name of the snapshot repository that both source and target snapshot belong to.

  • snapshot string Required

    The source snapshot name.

  • target_snapshot string Required

    The target snapshot name.

Query parameters

  • The period to wait for the master node. If the master node is not available before the timeout expires, the request fails and returns an error. To indicate that the request should never timeout, set it to -1.

application/json

Body Required

  • indices string Required

    A comma-separated list of indices to include in the snapshot. Multi-target syntax is supported.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /_snapshot/{repository}/{snapshot}/_clone/{target_snapshot}
curl \
 --request PUT 'http://api.example.com/_snapshot/{repository}/{snapshot}/_clone/{target_snapshot}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"indices\": \"index_a,index_b\"\n}"'
Request example
Run `PUT /_snapshot/my_repository/source_snapshot/_clone/target_snapshot` to clone the `source_snapshot` into a new `target_snapshot`.
{
  "indices": "index_a,index_b"
}








Create a snapshot Added in 0.0.0

POST /_snapshot/{repository}/{snapshot}

Take a snapshot of a cluster or of data streams and indices.

External documentation

Path parameters

  • repository string Required

    The name of the repository for the snapshot.

  • snapshot string Required

    The name of the snapshot. It supportes date math. It must be unique in the repository.

Query parameters

  • The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • If true, the request returns a response when the snapshot is complete. If false, the request returns a response when the snapshot initializes.

application/json

Body

  • expand_wildcards string | array[string]
  • feature_states array[string]

    The feature states to include in the snapshot. Each feature state includes one or more system indices containing related data. You can view a list of eligible features using the get features API.

    If include_global_state is true, all current feature states are included by default. If include_global_state is false, no feature states are included by default.

    Note that specifying an empty array will result in the default behavior. To exclude all feature states, regardless of the include_global_state value, specify an array with only the value none (["none"]).

  • If true, the request ignores data streams and indices in indices that are missing or closed. If false, the request returns an error for any data stream or index that is missing or closed.

  • If true, the current cluster state is included in the snapshot. The cluster state includes persistent cluster settings, composable index templates, legacy index templates, ingest pipelines, and ILM policies. It also includes data stored in system indices, such as Watches and task records (configurable via feature_states).

  • indices string | array[string]
  • metadata object
    Hide metadata attribute Show metadata attribute object
    • * object Additional properties
  • partial boolean

    If true, it enables you to restore a partial snapshot of indices with unavailable shards. Only shards that were successfully included in the snapshot will be restored. All missing shards will be recreated as empty.

    If false, the entire restore operation will fail if one or more indices included in the snapshot do not have all primary shards available.

Responses

POST /_snapshot/{repository}/{snapshot}
curl \
 --request POST 'http://api.example.com/_snapshot/{repository}/{snapshot}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"indices\": \"index_1,index_2\",\n  \"ignore_unavailable\": true,\n  \"include_global_state\": false,\n  \"metadata\": {\n    \"taken_by\": \"user123\",\n    \"taken_because\": \"backup before upgrading\"\n  }\n}"'
Request example
Run `PUT /_snapshot/my_repository/snapshot_2?wait_for_completion=true` to take a snapshot of `index_1` and `index_2`.
{
  "indices": "index_1,index_2",
  "ignore_unavailable": true,
  "include_global_state": false,
  "metadata": {
    "taken_by": "user123",
    "taken_because": "backup before upgrading"
  }
}
Response examples (200)
A successful response from `PUT /_snapshot/my_repository/snapshot_2?wait_for_completion=true`.
{
  "snapshot": {
    "snapshot": "snapshot_2",
    "uuid": "vdRctLCxSketdKb54xw67g",
    "repository": "my_repository",
    "version_id": <version_id>,
    "version": <version>,
    "indices": [],
    "data_streams": [],
    "feature_states": [],
    "include_global_state": false,
    "metadata": {
      "taken_by": "user123",
      "taken_because": "backup before upgrading"
    },
    "state": "SUCCESS",
    "start_time": "2020-06-25T14:00:28.850Z",
    "start_time_in_millis": 1593093628850,
    "end_time": "2020-06-25T14:00:28.850Z",
    "end_time_in_millis": 1593094752018,
    "duration_in_millis": 0,
    "failures": [],
    "shards": {
      "total": 0,
      "failed": 0,
      "successful": 0
    }
  }
}







































































































































Get a synonym rule Added in 8.10.0

GET /_synonyms/{set_id}/{rule_id}

Get a synonym rule from a synonym set.

Path parameters

  • set_id string Required

    The ID of the synonym set to retrieve the synonym rule from.

  • rule_id string Required

    The ID of the synonym rule to retrieve.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
GET /_synonyms/{set_id}/{rule_id}
curl \
 --request GET 'http://api.example.com/_synonyms/{set_id}/{rule_id}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _synonyms/my-synonyms-set/test-1`.
{
  "id": "test-1",
  "synonyms": "hello, hi"
}































































Delete a transform Added in 7.5.0

DELETE /_transform/{transform_id}

Path parameters

Query parameters

  • force boolean

    If this value is false, the transform must be stopped before it can be deleted. If true, the transform is deleted regardless of its current state.

  • If this value is true, the destination index is deleted together with the transform. If false, the destination index will not be deleted

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

DELETE /_transform/{transform_id}
curl \
 --request DELETE 'http://api.example.com/_transform/{transform_id}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response when the transform is deleted.
{
  "acknowledged": true
}








































Update a transform Added in 7.2.0

POST /_transform/{transform_id}/_update

Updates certain properties of a transform.

All updated properties except description do not take effect until after the transform starts the next checkpoint, thus there is data consistency in each checkpoint. To use this API, you must have read and view_index_metadata privileges for the source indices. You must also have index and read privileges for the destination index. When Elasticsearch security features are enabled, the transform remembers which roles the user who updated it had at the time of update and runs with those privileges.

Path parameters

Query parameters

  • When true, deferrable validations are not run. This behavior may be desired if the source index does not exist until after the transform is created.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

application/json

Body Required

  • dest object
    Hide dest attributes Show dest attributes object
  • Free text description of the transform.

  • A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • _meta object
    Hide _meta attribute Show _meta attribute object
    • * object Additional properties
  • source object
    Hide source attributes Show source attributes object
    • index string | array[string] Required
    • Hide runtime_mappings attribute Show runtime_mappings attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • fields object

          For type composite

          Hide fields attribute Show fields attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
            • type string Required

              Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

        • fetch_fields array[object]

          For type lookup

          Hide fetch_fields attributes Show fetch_fields attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • format string
        • format string

          A custom format for date type runtime fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • script object
          Hide script attributes Show script attributes object
          • source string | object

            One of:
          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            Hide params attribute Show params attribute object
            • * object Additional properties
          • lang string

            Any of:

            Values are painless, expression, mustache, or java.

          • options object
            Hide options attribute Show options attribute object
            • * string Additional properties
        • type string Required

          Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

    • query object

      A query clause that retrieves a subset of data from the source index.

      Query DSL
  • settings object
    Hide settings attributes Show settings attributes object
    • Specifies whether the transform checkpoint ranges should be optimized for performance. Such optimization can align checkpoint ranges with the date histogram interval when date histogram is specified as a group source in the transform config. As a result, less document updates in the destination index will be performed thus improving overall performance.

    • Defines if dates in the ouput should be written as ISO formatted string or as millis since epoch. epoch_millis was the default for transforms created before version 7.11. For compatible output set this value to true.

    • Specifies whether the transform should deduce the destination index mappings from the transform configuration.

    • Specifies a limit on the number of input documents per second. This setting throttles the transform by adding a wait time between search requests. The default value is null, which disables throttling.

    • Defines the initial page size to use for the composite aggregation for each checkpoint. If circuit breaker exceptions occur, the page size is dynamically adjusted to a lower value. The minimum value is 10 and the maximum is 65,536.

    • unattended boolean

      If true, the transform runs in unattended mode. In unattended mode, the transform retries indefinitely in case of an error which means the transform never fails. Setting the number of retries other than infinite fails in validation.

  • sync object
    Hide sync attribute Show sync attribute object
    • time object
      Hide time attributes Show time attributes object
      • delay string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • retention_policy object | string | null

    Defines a retention policy for the transform. Data that meets the defined criteria is deleted from the destination index.

    One of:
    Hide attribute Show attribute
    • time object
      Hide time attributes Show time attributes object
      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • max_age string Required

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • Hide authorization attributes Show authorization attributes object
      • api_key object
        Hide api_key attributes Show api_key attributes object
        • id string Required

          The identifier for the API key.

        • name string Required

          The name of the API key.

      • roles array[string]

        If a user ID was used for the most recent update to the transform, its roles at the time of the update are listed in the response.

      • If a service account was used for the most recent update to the transform, the account name is listed in the response.

    • create_time number Required
    • description string Required
    • dest object Required
      Hide dest attributes Show dest attributes object
    • A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • id string Required
    • latest object
      Hide latest attributes Show latest attributes object
      • sort string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • unique_key array[string] Required

        Specifies an array of one or more fields that are used to group the data.

    • pivot object
      Hide pivot attributes Show pivot attributes object
      • Defines how to aggregate the grouped data. The following aggregations are currently supported: average, bucket script, bucket selector, cardinality, filter, geo bounds, geo centroid, geo line, max, median absolute deviation, min, missing, percentiles, rare terms, scripted metric, stats, sum, terms, top metrics, value count, weighted average.

      • group_by object

        Defines how to group the data. More than one grouping can be defined per pivot. The following groupings are currently supported: date histogram, geotile grid, histogram, terms.

        Hide group_by attribute Show group_by attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • Hide date_histogram attributes Show date_histogram attributes object
            • Values are second, 1s, minute, 1m, hour, 1h, day, 1d, week, 1w, month, 1M, quarter, 1q, year, or 1y.

            • field string

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • format string

              The date format used to format key_as_string in the response. If no format is specified, the first date format specified in the field mapping is used.

            • interval string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • Only returns buckets that have min_doc_count number of documents. By default, all buckets between the first bucket that matches documents and the last one are returned.

            • offset string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • params object
            • script object
            • keyed boolean

              Set to true to associate a unique string key with each bucket and return the ranges as a hash rather than an array.

          • Hide geotile_grid attributes Show geotile_grid attributes object
            • field string

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • Allows for more accurate counting of the top cells returned in the final result the aggregation. Defaults to returning max(10,(size x number-of-shards)) buckets from each shard.

            • size number

              The maximum number of buckets to return.

          • Hide histogram attributes Show histogram attributes object
            • field string

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • interval number

              The interval for the buckets. Must be a positive decimal.

            • Only returns buckets that have min_doc_count number of documents. By default, the response will fill gaps in the histogram with empty buckets.

            • missing number

              The value to apply to documents that do not have a value. By default, documents without a value are ignored.

            • offset number

              By default, the bucket keys start with 0 and then continue in even spaced steps of interval. The bucket boundaries can be shifted by using the offset option.

            • script object
            • format string
            • keyed boolean

              If true, returns buckets as a hash instead of an array, keyed by the bucket keys.

    • Hide retention_policy attribute Show retention_policy attribute object
      • time object
        Hide time attributes Show time attributes object
        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • max_age string Required

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • settings object Required
      Hide settings attributes Show settings attributes object
      • Specifies whether the transform checkpoint ranges should be optimized for performance. Such optimization can align checkpoint ranges with the date histogram interval when date histogram is specified as a group source in the transform config. As a result, less document updates in the destination index will be performed thus improving overall performance.

      • Defines if dates in the ouput should be written as ISO formatted string or as millis since epoch. epoch_millis was the default for transforms created before version 7.11. For compatible output set this value to true.

      • Specifies whether the transform should deduce the destination index mappings from the transform configuration.

      • Specifies a limit on the number of input documents per second. This setting throttles the transform by adding a wait time between search requests. The default value is null, which disables throttling.

      • Defines the initial page size to use for the composite aggregation for each checkpoint. If circuit breaker exceptions occur, the page size is dynamically adjusted to a lower value. The minimum value is 10 and the maximum is 65,536.

      • unattended boolean

        If true, the transform runs in unattended mode. In unattended mode, the transform retries indefinitely in case of an error which means the transform never fails. Setting the number of retries other than infinite fails in validation.

    • source object Required
      Hide source attributes Show source attributes object
      • index string | array[string] Required
      • query object

        An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

        External documentation
      • remote object
        Hide remote attributes Show remote attributes object
        • A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • headers object

          An object containing the headers of the request.

          Hide headers attribute Show headers attribute object
          • * string Additional properties
        • host string Required
        • username string
        • password string
        • A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • size number

        The number of documents to index per batch. Use it when you are indexing from remote to ensure that the batches fit within the on-heap buffer, which defaults to a maximum size of 100 MB.

      • slice object
        Hide slice attributes Show slice attributes object
        • field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • id string Required
        • max number Required
      • sort string | object | array[string | object]

        One of:

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • _source string | array[string]
      • Hide runtime_mappings attribute Show runtime_mappings attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • fields object

            For type composite

            Hide fields attribute Show fields attribute object
            • * object Additional properties
              Hide * attribute Show * attribute object
              • type string Required

                Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

          • fetch_fields array[object]

            For type lookup

            Hide fetch_fields attributes Show fetch_fields attributes object
            • field string Required

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • format string
          • format string

            A custom format for date type runtime fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • script object
            Hide script attributes Show script attributes object
          • type string Required

            Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

    • sync object
      Hide sync attribute Show sync attribute object
      • time object
        Hide time attributes Show time attributes object
        • delay string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • version string Required
    • _meta object
      Hide _meta attribute Show _meta attribute object
      • * object Additional properties
POST /_transform/{transform_id}/_update
curl \
 --request POST 'http://api.example.com/_transform/{transform_id}/_update' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"source\": {\n    \"index\": \"kibana_sample_data_ecommerce\",\n    \"query\": {\n      \"term\": {\n        \"geoip.continent_name\": {\n          \"value\": \"Asia\"\n        }\n      }\n    }\n  },\n  \"pivot\": {\n    \"group_by\": {\n      \"customer_id\": {\n        \"terms\": {\n          \"field\": \"customer_id\",\n          \"missing_bucket\": true\n        }\n      }\n    },\n    \"aggregations\": {\n      \"max_price\": {\n        \"max\": {\n          \"field\": \"taxful_total_price\"\n        }\n      }\n    }\n  },\n  \"description\": \"Maximum priced ecommerce data by customer_id in Asia\",\n  \"dest\": {\n    \"index\": \"kibana_sample_data_ecommerce_transform1\",\n    \"pipeline\": \"add_timestamp_pipeline\"\n  },\n  \"frequency\": \"5m\",\n  \"sync\": {\n    \"time\": {\n      \"field\": \"order_date\",\n      \"delay\": \"60s\"\n    }\n  },\n  \"retention_policy\": {\n    \"time\": {\n      \"field\": \"order_date\",\n      \"max_age\": \"30d\"\n    }\n  }\n}"'
Request example
Run `POST _transform/simple-kibana-ecomm-pivot/_update` to update a transform that uses the pivot method.
{
  "source": {
    "index": "kibana_sample_data_ecommerce",
    "query": {
      "term": {
        "geoip.continent_name": {
          "value": "Asia"
        }
      }
    }
  },
  "pivot": {
    "group_by": {
      "customer_id": {
        "terms": {
          "field": "customer_id",
          "missing_bucket": true
        }
      }
    },
    "aggregations": {
      "max_price": {
        "max": {
          "field": "taxful_total_price"
        }
      }
    }
  },
  "description": "Maximum priced ecommerce data by customer_id in Asia",
  "dest": {
    "index": "kibana_sample_data_ecommerce_transform1",
    "pipeline": "add_timestamp_pipeline"
  },
  "frequency": "5m",
  "sync": {
    "time": {
      "field": "order_date",
      "delay": "60s"
    }
  },
  "retention_policy": {
    "time": {
      "field": "order_date",
      "max_age": "30d"
    }
  }
}
Response examples (200)
A successful response when creating a transform.
{
  "id": "simple-kibana-ecomm-pivot",
  "authorization": {
    "roles": [
      "superuser"
    ]
  },
  "version": "10.0.0",
  "create_time": 1712951576767,
  "source": {
    "index": [
      "kibana_sample_data_ecommerce"
    ],
    "query": {
      "term": {
        "geoip.continent_name": {
          "value": "Asia"
        }
      }
    }
  },
  "dest": {
    "index": "kibana_sample_data_ecommerce_transform_v2",
    "pipeline": "add_timestamp_pipeline"
  },
  "frequency": "15m",
  "sync": {
    "time": {
      "field": "order_date",
      "delay": "120s"
    }
  },
  "pivot": {
    "group_by": {
      "customer_id": {
        "terms": {
          "field": "customer_id",
          "missing_bucket": true
        }
      }
    },
    "aggregations": {
      "max_price": {
        "max": {
          "field": "taxful_total_price"
        }
      }
    }
  },
  "description": "Maximum priced ecommerce data by customer_id in Asia",
  "settings": {},
  "retention_policy": {
    "time": {
      "field": "order_date",
      "max_age": "30d"
    }
  }
}




Usage