Create or update a snapshot repository | Elasticsearch API documentation

Get index template information Added in 5.2.0

GET /_cat/templates

Get information about the index templates in a cluster. You can use index templates to apply index settings and field mappings to new indices at creation. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the get index template API.

Query parameters

h string | array[string]

List of columns to appear in the response. Supports simple wildcards.
s string | array[string]

List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.
local boolean

If true, the request computes the list of selected nodes from the local cluster state. If false the list of selected nodes are computed from the cluster state of the master node. In both cases the coordinating node will send requests for further information to each selected node.
master_timeout string

Period to wait for a connection to the master node.

Responses

200 application/json
Hide response attributes Show response attributes object
- name string
- index_patterns string
  
  The template index patterns.
- order string
  
  The template application order or priority number.
- version string | null
  
  The template version.
  
  One of:
  VersionString string string-2 string | null
- composed_of string
  
  The component templates that comprise the index template.

GET /_cat/templates

curl \
 --request GET 'http://api.example.com/_cat/templates' \
 --header "Authorization: $API_KEY"

Response examples (200)

A successful response from `GET _cat/templates/my-template-*?v=true&s=name&format=json`.

[
  {
    "name": "my-template-0",
    "index_patterns": "[te*]",
    "order": "500",
    "version": null,
    "composed_of": "[]"
  },
  {
    "name": "my-template-1",
    "index_patterns": "[tea*]",
    "order": "501",
    "version": null,
    "composed_of": "[]"
  },
  {
    "name": "my-template-2",
    "index_patterns": "[teak*]",
    "order": "502",
    "version": "7",
    "composed_of": "[]"
  }
]

Get the cluster health status Added in 1.3.0

GET /_cluster/health/{index}

Api key auth Basic auth Bearer auth

You can also use the API to get the health status of only specified data streams and indices. For data streams, the API retrieves the health status of the stream’s backing indices.

The cluster health status is: green, yellow or red. On the shard level, a red status indicates that the specific shard is not allocated in the cluster. Yellow means that the primary shard is allocated but replicas are not. Green means that all shards are allocated. The index level status is controlled by the worst shard status.

One of the main benefits of the API is the ability to wait until the cluster reaches a certain high watermark health level. The cluster status is controlled by the worst index status.

Path parameters

index string | array[string] Required

Comma-separated list of data streams, indices, and index aliases used to limit the request. Wildcard expressions (*) are supported. To target all data streams and indices in a cluster, omit this parameter or use _all or *.

Query parameters

expand_wildcards string | array[string]
Whether to expand wildcard expression to concrete indices that are open, closed or both.

Supported values include:
- all: Match any data stream or index, including hidden ones.
- open: Match open, non-hidden indices. Also matches any non-hidden data stream.
- closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
- hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
- none: Wildcard expressions are not accepted.
level string

Can be one of cluster, indices or shards. Controls the details level of the health information returned.

Values are cluster, indices, or shards.
local boolean

If true, the request retrieves information from the local node only. Defaults to false, which means information is retrieved from the master node.
master_timeout string

Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
timeout string

Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
wait_for_active_shards number | string

A number controlling to how many active shards to wait for, all to wait for all shards in the cluster to be active, or 0 to not wait.
wait_for_events string

Can be one of immediate, urgent, high, normal, low, languid. Wait until all currently queued events with the given priority are processed.

Values are immediate, urgent, high, normal, low, or languid.
wait_for_nodes string | number

The request waits until the specified number N of nodes is available. It also accepts >=N, <=N, >N and <N. Alternatively, it is possible to use ge(N), le(N), gt(N) and lt(N) notation.
wait_for_no_initializing_shards boolean

A boolean value which controls whether to wait (until the timeout provided) for the cluster to have no shard initializations. Defaults to false, which means it will not wait for initializing shards.
wait_for_no_relocating_shards boolean

A boolean value which controls whether to wait (until the timeout provided) for the cluster to have no shard relocations. Defaults to false, which means it will not wait for relocating shards.
wait_for_status string
One of green, yellow or red. Will wait (until the timeout provided) until the status of the cluster changes to the one provided or better, i.e. green > yellow > red. By default, will not wait for any status.

Supported values include:
- green (or GREEN): All shards are assigned.
- yellow (or YELLOW): All primary shards are assigned, but one or more replica shards are unassigned. If a node in the cluster fails, some data could be unavailable until that node is repaired.
- red (or RED): One or more primary shards are unassigned, so some data is unavailable. This can occur briefly during cluster startup as primary shards are assigned.
Values are green, GREEN, yellow, YELLOW, red, or RED.

Responses

200 application/json
Hide response attributes Show response attributes object
- active_primary_shards number Required
  
  The number of active primary shards.
- active_shards number Required
  
  The total number of active primary and replica shards.
- active_shards_percent string
  
  The ratio of active shards in the cluster expressed as a string formatted percentage.
- active_shards_percent_as_number number Required
  
  The ratio of active shards in the cluster expressed as a percentage.
- cluster_name string Required
- delayed_unassigned_shards number Required
  
  The number of shards whose allocation has been delayed by the timeout settings.
- indices object
  
  Hide indices attribute Show indices attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  active_primary_shards number Required
  
  active_shards number Required
  
  initializing_shards number Required
  
  number_of_replicas number Required
  
  number_of_shards number Required
  
  relocating_shards number Required
  
  shards object
  
  Hide shards attribute Show shards attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  active_shards number Required
  
  initializing_shards number Required
  
  primary_active boolean Required
  
  relocating_shards number Required
  
  status string Required
  
  Values are green, GREEN, yellow, YELLOW, red, or RED.
  
  unassigned_shards number Required
  
  unassigned_primary_shards number Required
  
  status string Required
  
  Values are green, GREEN, yellow, YELLOW, red, or RED.
  
  unassigned_shards number Required
  
  unassigned_primary_shards number Required
- initializing_shards number Required
  
  The number of shards that are under initialization.
- number_of_data_nodes number Required
  
  The number of nodes that are dedicated data nodes.
- number_of_in_flight_fetch number Required
  
  The number of unfinished fetches.
- number_of_nodes number Required
  
  The number of nodes within the cluster.
- number_of_pending_tasks number Required
  
  The number of cluster-level changes that have not yet been executed.
- relocating_shards number Required
  
  The number of shards that are under relocation.
- status string Required
  
  Values are green, GREEN, yellow, YELLOW, red, or RED.
- task_max_waiting_in_queue string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
- task_max_waiting_in_queue_millis number
  
  Time unit for milliseconds
- timed_out boolean Required
  
  If false the response returned within the period of time that is specified by the timeout parameter (30s by default)
- unassigned_primary_shards number Required
  
  The number of primary shards that are not allocated.
- unassigned_shards number Required
  
  The number of shards that are not allocated.

GET /_cluster/health/{index}

curl \
 --request GET 'http://api.example.com/_cluster/health/{index}' \
 --header "Authorization: $API_KEY"

Response examples (200)

A successful response from `GET _cluster/health`. It is the health status of a quiet single node cluster with a single index with one shard and one replica.

{
  "cluster_name" : "testcluster",
  "status" : "yellow",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 1,
  "active_shards" : 1,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 1,
  "delayed_unassigned_shards": 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch": 0,
  "task_max_waiting_in_queue_millis": 0,
  "active_shards_percent_as_number": 50.0
}

Get index recovery information

GET /{index}/_recovery

Api key auth Basic auth Bearer auth

Get information about ongoing and completed shard recoveries for one or more indices. For data streams, the API returns information for the stream's backing indices.

All recoveries, whether ongoing or complete, are kept in the cluster state and may be reported on at any time.

Shard recovery is the process of initializing a shard copy, such as restoring a primary shard from a snapshot or creating a replica shard from a primary shard. When a shard recovery completes, the recovered shard is available for search and indexing.

Recovery automatically occurs during the following processes:

When creating an index for the first time.
When a node rejoins the cluster and starts up any missing primary shard copies using the data that it holds in its data path.
Creation of new replica shard copies from the primary.
Relocation of a shard copy to a different node in the same cluster.
A snapshot restore operation.
A clone, shrink, or split operation.

You can determine the cause of a shard recovery using the recovery or cat recovery APIs.

The index recovery API reports information about completed recoveries only for shard copies that currently exist in the cluster. It only reports the last recovery for each shard copy and does not report historical information about earlier recoveries, nor does it report information about the recoveries of shard copies that no longer exist. This means that if a shard copy completes a recovery and then Elasticsearch relocates it onto a different node then the information about the original recovery will not be shown in the recovery API.

Path parameters

index string | array[string] Required

Comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.

Query parameters

active_only boolean

If true, the response only includes ongoing shard recoveries.
detailed boolean

If true, the response includes detailed information about shard recoveries.

Responses

200 application/json
Hide response attribute Show response attribute object
- * object Additional properties
  
  Hide * attribute Show * attribute object
  
  shards array[object] Required
  
  Hide shards attributes Show shards attributes object
  
  id number Required
  
  index object Required
  
  Hide index attributes Show index attributes object
  
  bytes object
  
  Hide bytes attributes Show bytes attributes object
  
  percent
  
  recovered
  
  recovered_in_bytes
  
  recovered_from_snapshot
  
  recovered_from_snapshot_in_bytes
  
  reused
  
  reused_in_bytes
  
  total
  
  total_in_bytes
  
  files object Required
  
  Hide files attributes Show files attributes object
  
  details array[object]
  
  percent
  
  recovered number Required
  
  reused number Required
  
  total number Required
  
  size object Required
  
  Hide size attributes Show size attributes object
  
  percent
  
  recovered
  
  recovered_in_bytes
  
  recovered_from_snapshot
  
  recovered_from_snapshot_in_bytes
  
  reused
  
  reused_in_bytes
  
  total
  
  total_in_bytes
  
  source_throttle_time string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  source_throttle_time_in_millis number
  
  Time unit for milliseconds
  
  target_throttle_time string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  target_throttle_time_in_millis number
  
  Time unit for milliseconds
  
  total_time string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  total_time_in_millis number
  
  Time unit for milliseconds
  
  primary boolean Required
  
  source object Required
  
  Hide source attributes Show source attributes object
  
  hostname string
  
  host string
  
  transport_address string
  
  id string
  
  ip string
  
  name string
  
  bootstrap_new_history_uuid boolean
  
  repository string
  
  snapshot string
  
  version string
  
  restoreUUID string
  
  index string
  
  stage string Required
  
  start object
  
  Hide start attributes Show start attributes object
  
  check_index_time string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  check_index_time_in_millis number
  
  Time unit for milliseconds
  
  total_time string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  total_time_in_millis number
  
  Time unit for milliseconds
  
  start_time string | number
  
  A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
  
  One of:
  DateTime string UnitMillis number
  
  start_time_in_millis number
  
  Time unit for milliseconds
  
  stop_time string | number
  
  A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
  
  One of:
  DateTime string UnitMillis number
  
  stop_time_in_millis number
  
  Time unit for milliseconds
  
  target object Required
  
  Hide target attributes Show target attributes object
  
  hostname string
  
  host string
  
  transport_address string
  
  id string
  
  ip string
  
  name string
  
  bootstrap_new_history_uuid boolean
  
  repository string
  
  snapshot string
  
  version string
  
  restoreUUID string
  
  index string
  
  total_time string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  total_time_in_millis number
  
  Time unit for milliseconds
  
  translog object Required
  
  Hide translog attributes Show translog attributes object
  
  percent string | number Required
  
  One of:
  Percentage string Percentage number
  
  recovered number Required
  
  total number Required
  
  total_on_start number Required
  
  total_time string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  total_time_in_millis number
  
  Time unit for milliseconds
  
  type string Required
  
  verify_index object Required
  
  Hide verify_index attributes Show verify_index attributes object
  
  check_index_time string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  check_index_time_in_millis number
  
  Time unit for milliseconds
  
  total_time string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  total_time_in_millis number
  
  Time unit for milliseconds

GET /{index}/_recovery

curl \
 --request GET 'http://api.example.com/{index}/_recovery' \
 --header "Authorization: $API_KEY"

Response examples (200)

A successful response from `GET /_recovery?human`, which gets information about ongoing and completed shard recoveries for all data streams and indices in a cluster. This example includes information about a single index recovering a single shard. The source of the recovery is a snapshot repository and the target of the recovery is the `my_es_node` node. The response also includes the number and percentage of files and bytes recovered.

{
  "index1" : {
    "shards" : [ {
      "id" : 0,
      "type" : "SNAPSHOT",
      "stage" : "INDEX",
      "primary" : true,
      "start_time" : "2014-02-24T12:15:59.716",
      "start_time_in_millis": 1393244159716,
      "stop_time" : "0s",
      "stop_time_in_millis" : 0,
      "total_time" : "2.9m",
      "total_time_in_millis" : 175576,
      "source" : {
        "repository" : "my_repository",
        "snapshot" : "my_snapshot",
        "index" : "index1",
        "version" : "{version}",
        "restoreUUID": "PDh1ZAOaRbiGIVtCvZOMww"
      },
      "target" : {
        "id" : "ryqJ5lO5S4-lSFbGntkEkg",
        "host" : "my.fqdn",
        "transport_address" : "my.fqdn",
        "ip" : "10.0.1.7",
        "name" : "my_es_node"
      },
      "index" : {
        "size" : {
          "total" : "75.4mb",
          "total_in_bytes" : 79063092,
          "reused" : "0b",
          "reused_in_bytes" : 0,
          "recovered" : "65.7mb",
          "recovered_in_bytes" : 68891939,
          "recovered_from_snapshot" : "0b",
          "recovered_from_snapshot_in_bytes" : 0,
          "percent" : "87.1%"
        },
        "files" : {
          "total" : 73,
          "reused" : 0,
          "recovered" : 69,
          "percent" : "94.5%"
        },
        "total_time" : "0s",
        "total_time_in_millis" : 0,
        "source_throttle_time" : "0s",
        "source_throttle_time_in_millis" : 0,
        "target_throttle_time" : "0s",
        "target_throttle_time_in_millis" : 0
      },
      "translog" : {
        "recovered" : 0,
        "total" : 0,
        "percent" : "100.0%",
        "total_on_start" : 0,
        "total_time" : "0s",
        "total_time_in_millis" : 0
      },
      "verify_index" : {
        "check_index_time" : "0s",
        "check_index_time_in_millis" : 0,
        "total_time" : "0s",
        "total_time_in_millis" : 0
      }
    } ]
  }
}

A successful response from `GET _recovery?human&detailed=true`. The response includes a listing of any physical files recovered and their sizes. The response also includes timings in milliseconds of the various stages of recovery: index retrieval, translog replay, and index start time. This response indicates the recovery is done.

{
  "index1" : {
    "shards" : [ {
      "id" : 0,
      "type" : "EXISTING_STORE",
      "stage" : "DONE",
      "primary" : true,
      "start_time" : "2014-02-24T12:38:06.349",
      "start_time_in_millis" : "1393245486349",
      "stop_time" : "2014-02-24T12:38:08.464",
      "stop_time_in_millis" : "1393245488464",
      "total_time" : "2.1s",
      "total_time_in_millis" : 2115,
      "source" : {
        "id" : "RGMdRc-yQWWKIBM4DGvwqQ",
        "host" : "my.fqdn",
        "transport_address" : "my.fqdn",
        "ip" : "10.0.1.7",
        "name" : "my_es_node"
      },
      "target" : {
        "id" : "RGMdRc-yQWWKIBM4DGvwqQ",
        "host" : "my.fqdn",
        "transport_address" : "my.fqdn",
        "ip" : "10.0.1.7",
        "name" : "my_es_node"
      },
      "index" : {
        "size" : {
          "total" : "24.7mb",
          "total_in_bytes" : 26001617,
          "reused" : "24.7mb",
          "reused_in_bytes" : 26001617,
          "recovered" : "0b",
          "recovered_in_bytes" : 0,
          "recovered_from_snapshot" : "0b",
          "recovered_from_snapshot_in_bytes" : 0,
          "percent" : "100.0%"
        },
        "files" : {
          "total" : 26,
          "reused" : 26,
          "recovered" : 0,
          "percent" : "100.0%",
          "details" : [ {
            "name" : "segments.gen",
            "length" : 20,
            "recovered" : 20
          }, {
            "name" : "_0.cfs",
            "length" : 135306,
            "recovered" : 135306,
            "recovered_from_snapshot": 0
          }, {
            "name" : "segments_2",
            "length" : 251,
            "recovered" : 251,
            "recovered_from_snapshot": 0
          }
          ]
        },
        "total_time" : "2ms",
        "total_time_in_millis" : 2,
        "source_throttle_time" : "0s",
        "source_throttle_time_in_millis" : 0,
        "target_throttle_time" : "0s",
        "target_throttle_time_in_millis" : 0
      },
      "translog" : {
        "recovered" : 71,
        "total" : 0,
        "percent" : "100.0%",
        "total_on_start" : 0,
        "total_time" : "2.0s",
        "total_time_in_millis" : 2025
      },
      "verify_index" : {
        "check_index_time" : 0,
        "check_index_time_in_millis" : 0,
        "total_time" : "88ms",
        "total_time_in_millis" : 88
      }
    } ]
  }
}

Simulate an index Added in 7.9.0

POST /_index_template/_simulate_index/{name}

Api key auth Basic auth Bearer auth

Get the index configuration that would be applied to the specified index from an existing index template.

Path parameters

name string Required

Name of the index to simulate

Query parameters

create boolean

Whether the index template we optionally defined in the body should only be dry-run added if new or can also replace an existing one
cause string

User defined reason for dry-run creating the new template for simulation purposes
master_timeout string

Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
include_defaults boolean

If true, returns all relevant default configurations for the index template.

Responses

200 application/json
Hide response attributes Show response attributes object
- overlapping array[object]
  
  Hide overlapping attributes Show overlapping attributes object
  
  name string Required
  
  index_patterns array[string] Required
- template object Required
  
  Hide template attributes Show template attributes object
  
  aliases object Required
  
  Hide aliases attribute Show aliases attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  filter object
  
  An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
  
  External documentation
  
  index_routing string
  
  is_hidden boolean
  
  If true, the alias is hidden. All indices for the alias must have the same is_hidden value.
  
  is_write_index boolean
  
  If true, the index is the write index for the alias.
  
  routing string
  
  search_routing string
  
  mappings object Required
  
  Hide mappings attributes Show mappings attributes object
  
  all_field object
  
  Hide all_field attributes Show all_field attributes object
  
  analyzer string Required
  
  enabled boolean Required
  
  omit_norms boolean Required
  
  search_analyzer string Required
  
  similarity string Required
  
  store boolean Required
  
  store_term_vector_offsets boolean Required
  
  store_term_vector_payloads boolean Required
  
  store_term_vector_positions boolean Required
  
  store_term_vectors boolean Required
  
  date_detection boolean
  
  dynamic string
  
  Values are strict, runtime, true, or false.
  
  dynamic_date_formats array[string]
  
  dynamic_templates array[object]
  
  _field_names object
  
  Hide _field_names attribute Show _field_names attribute object
  
  enabled boolean Required
  
  index_field object
  
  Hide index_field attribute Show index_field attribute object
  
  enabled boolean Required
  
  _meta object
  
  Hide _meta attribute Show _meta attribute object
  
  * object Additional properties
  
  numeric_detection boolean
  
  properties object
  
  _routing object
  
  Hide _routing attribute Show _routing attribute object
  
  required boolean Required
  
  _size object
  
  Hide _size attribute Show _size attribute object
  
  enabled boolean Required
  
  _source object
  
  Hide _source attributes Show _source attributes object
  
  compress boolean
  
  compress_threshold string
  
  enabled boolean
  
  excludes array[string]
  
  includes array[string]
  
  mode string
  
  Values are disabled, stored, or synthetic.
  
  runtime object
  
  Hide runtime attribute Show runtime attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  fields object
  
  For type composite
  
  Hide fields attribute Show fields attribute object
  
  * object Additional properties
  
  fetch_fields array[object]
  
  For type lookup
  
  format string
  
  A custom format for date type runtime fields.
  
  input_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_index string
  
  script object
  
  Hide script attributes Show script attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  type string Required
  
  Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.
  
  enabled boolean
  
  subobjects string
  
  Values are true or false.
  
  _data_stream_timestamp object
  
  Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
  
  enabled boolean Required
  
  settings object Required
  
  Hide settings attributes Show settings attributes object
  
  index object
  
  mode string
  
  routing_path string | array[string]
  
  One of:
  string-1 string array-2 array[string]
  
  soft_deletes object
  
  Hide soft_deletes attributes Show soft_deletes attributes object
  
  enabled boolean
  
  Indicates whether soft deletes are enabled on the index.
  
  retention_lease object
  
  Hide retention_lease attribute Show retention_lease attribute object
  
  period string Required
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  sort object
  
  Hide sort attributes Show sort attributes object
  
  field string | array[string]
  
  order string | array[string]
  
  Supported values include: asc (or ASC), desc (or DESC)
  
  One of:
  SegmentSortOrder string array-2 array[string]
  
  Values are asc, ASC, desc, or DESC.
  
  mode string | array[string]
  
  Supported values include: min (or MIN), max (or MAX)
  
  One of:
  SegmentSortMode string array-2 array[string]
  
  Values are min, MIN, max, or MAX.
  
  missing string | array[string]
  
  Supported values include: _last, _first
  
  One of:
  SegmentSortMissing string array-2 array[string]
  
  Values are _last or _first.
  
  number_of_shards number | string
  
  One of:
  number-1 number string-2 string
  
  number_of_replicas number | string
  
  One of:
  number-1 number string-2 string
  
  number_of_routing_shards number
  
  check_on_startup string
  
  Values are true, false, or checksum.
  
  codec string
  
  routing_partition_size number | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  Stringifiedinteger number Stringifiedinteger string
  
  load_fixed_bitset_filters_eagerly boolean
  
  hidden boolean | string
  
  One of:
  boolean-1 boolean string-2 string
  
  auto_expand_replicas string | null
  
  One of:
  string-1 string NullValue string | null
  
  merge object
  
  Hide merge attribute Show merge attribute object
  
  scheduler object
  
  Hide scheduler attributes Show scheduler attributes object
  
  max_thread_count number | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  Stringifiedinteger number Stringifiedinteger string
  
  max_merge_count number | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  Stringifiedinteger number Stringifiedinteger string
  
  search object
  
  Hide search attributes Show search attributes object
  
  idle object
  
  Hide idle attribute Show idle attribute object
  
  after string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  slowlog object
  
  Hide slowlog attributes Show slowlog attributes object
  
  level string
  
  source number
  
  reformat boolean
  
  threshold object
  
  Hide threshold attributes Show threshold attributes object
  
  query object
  
  fetch object
  
  refresh_interval string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  max_result_window number
  
  max_inner_result_window number
  
  max_rescore_window number
  
  max_docvalue_fields_search number
  
  max_script_fields number
  
  max_ngram_diff number
  
  max_shingle_diff number
  
  blocks object
  
  Hide blocks attributes Show blocks attributes object
  
  read_only boolean | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  Stringifiedboolean boolean Stringifiedboolean string
  
  read_only_allow_delete boolean | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  Stringifiedboolean boolean Stringifiedboolean string
  
  read boolean | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  Stringifiedboolean boolean Stringifiedboolean string
  
  write boolean | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  Stringifiedboolean boolean Stringifiedboolean string
  
  metadata boolean | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  Stringifiedboolean boolean Stringifiedboolean string
  
  max_refresh_listeners number
  
  analyze object
  
  Hide analyze attribute Show analyze attribute object
  
  max_token_count number | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  Stringifiedinteger number Stringifiedinteger string
  
  highlight object
  
  Hide highlight attribute Show highlight attribute object
  
  max_analyzed_offset number
  
  max_terms_count number
  
  max_regex_length number
  
  routing object
  
  Hide routing attributes Show routing attributes object
  
  allocation object
  
  Hide allocation attributes Show allocation attributes object
  
  enable string
  
  Values are all, primaries, new_primaries, or none.
  
  include object
  
  Hide include attributes Show include attributes object
  
  _tier_preference string
  
  _id string
  
  initial_recovery object
  
  Hide initial_recovery attribute Show initial_recovery attribute object
  
  _id string
  
  disk object
  
  Hide disk attribute Show disk attribute object
  
  threshold_enabled
  
  rebalance object
  
  Hide rebalance attribute Show rebalance attribute object
  
  enable string Required
  
  Values are all, primaries, replicas, or none.
  
  gc_deletes string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  default_pipeline string
  
  final_pipeline string
  
  lifecycle object
  
  Hide lifecycle attributes Show lifecycle attributes object
  
  name string
  
  indexing_complete boolean | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  Stringifiedboolean boolean Stringifiedboolean string
  
  origination_date number
  
  If specified, this is the timestamp used to calculate the index age for its phase transitions. Use this setting if you create a new index that contains old data and want to use the original creation date to calculate the index age. Specified as a Unix epoch value in milliseconds.
  
  parse_origination_date boolean
  
  Set to true to parse the origination date from the index name. This origination date is used to calculate the index age for its phase transitions. The index name must match the pattern ^{.*-{date_format}-\d+,} where the date_format is yyyy.MM.dd and the trailing digits are optional. An index that was rolled over would normally match the full format, for example logs-2016.10.31-000002). If the index name doesn’t match the pattern, index creation fails.
  
  step object
  
  Hide step attribute Show step attribute object
  
  wait_time_threshold string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  rollover_alias string
  
  The index alias to update when the index rolls over. Specify when using a policy that contains a rollover action. When the index rolls over, the alias is updated to reflect that the index is no longer the write index. For more information about rolling indices, see Rollover.
  
  prefer_ilm boolean | string
  
  Preference for the system that manages a data stream backing index (preferring ILM when both ILM and DLM are applicable for an index).
  
  One of:
  boolean-1 boolean string-2 string
  
  provided_name string
  
  creation_date number | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  UnitMillis number StringifiedEpochTimeUnitMillis string
  
  Time unit for milliseconds
  
  creation_date_string string | number
  
  A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
  
  One of:
  DateTime string UnitMillis number
  
  uuid string
  
  version object
  
  Hide version attributes Show version attributes object
  
  created string
  
  created_string string
  
  verified_before_close boolean | string
  
  One of:
  boolean-1 boolean string-2 string
  
  format string | number
  
  One of:
  string-1 string number-2 number
  
  max_slices_per_scroll number
  
  translog object
  
  Hide translog attributes Show translog attributes object
  
  sync_interval string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  durability string
  
  Values are request, REQUEST, async, or ASYNC.
  
  flush_threshold_size number | string
  
  One of:
  ByteSize number ByteSize string
  
  retention object
  
  Hide retention attributes Show retention attributes object
  
  size number | string
  
  One of:
  ByteSize number ByteSize string
  
  age string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  query_string object
  
  Hide query_string attribute Show query_string attribute object
  
  lenient boolean | string Required
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  Stringifiedboolean boolean Stringifiedboolean string
  
  priority number | string
  
  One of:
  number-1 number string-2 string
  
  top_metrics_max_size number
  
  analysis object
  
  Hide analysis attributes Show analysis attributes object
  
  analyzer object
  
  char_filter object
  
  filter object
  
  normalizer object
  
  tokenizer object
  
  settings object
  
  time_series object
  
  Hide time_series attributes Show time_series attributes object
  
  end_time string | number
  
  A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
  
  One of:
  DateTime string UnitMillis number
  
  start_time string | number
  
  A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
  
  One of:
  DateTime string UnitMillis number
  
  queries object
  
  Hide queries attribute Show queries attribute object
  
  cache object
  
  Hide cache attribute Show cache attribute object
  
  enabled boolean Required
  
  similarity object
  
  Configure custom similarity settings to customize how search results are scored.
  
  mapping object
  
  Hide mapping attributes Show mapping attributes object
  
  coerce boolean
  
  total_fields object
  
  Hide total_fields attributes Show total_fields attributes object
  
  limit number | string
  
  The maximum number of fields in an index. Field and object mappings, as well as field aliases count towards this limit. The limit is in place to prevent mappings and searches from becoming too large. Higher values can lead to performance degradations and memory issues, especially in clusters with a high load or few resources.
  
  One of:
  number-1 number string-2 string
  
  ignore_dynamic_beyond_limit boolean | string
  
  This setting determines what happens when a dynamically mapped field would exceed the total fields limit. When set to false (the default), the index request of the document that tries to add a dynamic field to the mapping will fail with the message Limit of total fields [X] has been exceeded. When set to true, the index request will not fail. Instead, fields that would exceed the limit are not added to the mapping, similar to dynamic: false. The fields that were not added to the mapping will be added to the _ignored field.
  
  One of:
  boolean-1 boolean string-2 string
  
  depth object
  
  Hide depth attribute Show depth attribute object
  
  limit number
  
  The maximum depth for a field, which is measured as the number of inner objects. For instance, if all fields are defined at the root object level, then the depth is 1. If there is one object mapping, then the depth is 2, etc.
  
  nested_fields object
  
  Hide nested_fields attribute Show nested_fields attribute object
  
  limit number
  
  The maximum number of distinct nested mappings in an index. The nested type should only be used in special cases, when arrays of objects need to be queried independently of each other. To safeguard against poorly designed mappings, this setting limits the number of unique nested types per index.
  
  nested_objects object
  
  Hide nested_objects attribute Show nested_objects attribute object
  
  limit number
  
  The maximum number of nested JSON objects that a single document can contain across all nested types. This limit helps to prevent out of memory errors when a document contains too many nested objects.
  
  field_name_length object
  
  Hide field_name_length attribute Show field_name_length attribute object
  
  limit number
  
  Setting for the maximum length of a field name. This setting isn’t really something that addresses mappings explosion but might still be useful if you want to limit the field length. It usually shouldn’t be necessary to set this setting. The default is okay unless a user starts to add a huge number of fields with really long names. Default is Long.MAX_VALUE (no limit).
  
  dimension_fields object
  
  Hide dimension_fields attribute Show dimension_fields attribute object
  
  limit number
  
  [preview] This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.
  
  source object
  
  Hide source attribute Show source attribute object
  
  mode string Required
  
  Values are disabled, stored, or synthetic.
  
  ignore_malformed boolean | string
  
  One of:
  boolean-1 boolean string-2 string
  
  indexing.slowlog object
  
  Hide indexing.slowlog attributes Show indexing.slowlog attributes object
  
  level string
  
  source number
  
  reformat boolean
  
  threshold object
  
  Hide threshold attribute Show threshold attribute object
  
  index object
  
  Hide index attributes Show index attributes object
  
  warn string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  info string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  debug string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  trace string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  indexing_pressure object
  
  Hide indexing_pressure attribute Show indexing_pressure attribute object
  
  memory object Required
  
  Hide memory attribute Show memory attribute object
  
  limit number
  
  Number of outstanding bytes that may be consumed by indexing requests. When this limit is reached or exceeded, the node will reject new coordinating and primary operations. When replica operations consume 1.5x this limit, the node will reject new replica operations. Defaults to 10% of the heap.
  
  store object
  
  Hide store attributes Show store attributes object
  
  type string Required
  
  Any of:
  StorageType string StorageType string
  
  Values are fs, niofs, mmapfs, or hybridfs.
  
  allow_mmap boolean
  
  You can restrict the use of the mmapfs and the related hybridfs store type via the setting node.store.allow_mmap. This is a boolean setting indicating whether or not memory-mapping is allowed. The default is to allow it. This setting is useful, for example, if you are in an environment where you can not control the ability to create a lot of memory maps so you need disable the ability to use memory-mapping.

POST /_index_template/_simulate_index/{name}

curl \
 --request POST 'http://api.example.com/_index_template/_simulate_index/{name}' \
 --header "Authorization: $API_KEY"

Response examples (200)

A successful response from `POST /_index_template/_simulate_index/my-index-000001`.

{
  "template" : {
    "settings" : {
      "index" : {
        "number_of_shards" : "2",
        "number_of_replicas" : "0",
        "routing" : {
          "allocation" : {
            "include" : {
              "_tier_preference" : "data_content"
            }
          }
        }
      }
    },
    "mappings" : {
      "properties" : {
        "@timestamp" : {
          "type" : "date"
        }
      }
    },
    "aliases" : { }
  },
  "overlapping" : [
    {
      "name" : "template_1",
      "index_patterns" : [
        "my-index-*"
      ]
    }
  ]
}

Get pipelines Added in 5.0.0

GET /_ingest/pipeline/{id}

Api key auth Basic auth Bearer auth

Get information about one or more ingest pipelines. This API returns a local reference of the pipeline.

External documentation

Path parameters

id string Required

Comma-separated list of pipeline IDs to retrieve. Wildcard (*) expressions are supported. To get all ingest pipelines, omit this parameter or use *.

Query parameters

master_timeout string

Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
summary boolean

Return pipelines without their definitions (default: false)

Responses

200 application/json
Hide response attribute Show response attribute object
- * object Additional properties
  
  Hide * attributes Show * attributes object
  
  description string
  
  Description of the ingest pipeline.
  
  on_failure array[object]
  
  Processors to run immediately after a processor failure.
  
  Hide on_failure attributes Show on_failure attributes object
  
  append object
  
  Hide append attributes Show append attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  value
  
  allow_duplicates boolean
  
  If false, the processor does not append values already present in the field.
  
  attachment object
  
  Hide attachment attributes Show attachment attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  indexed_chars number
  
  The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.
  
  indexed_chars_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  remove_binary boolean
  
  If true, the binary field will be removed from the document
  
  resource_name string
  
  Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.
  
  bytes object
  
  Hide bytes attributes Show bytes attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  circle object
  
  Hide circle attributes Show circle attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  error_distance number Required
  
  The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  shape_type string Required
  
  Values are geo_shape or shape.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  community_id object
  
  Hide community_id attributes Show community_id attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  source_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  source_port string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_port string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  iana_number string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  icmp_type string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  icmp_code string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  transport string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  seed number
  
  Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  convert object
  
  Hide convert attributes Show convert attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  type string Required
  
  Values are integer, long, double, float, boolean, ip, string, or auto.
  
  csv object
  
  Hide csv attributes Show csv attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  empty_value object
  
  Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  quote string
  
  Quote used in CSV, has to be single character string.
  
  separator string
  
  Separator used in CSV, has to be single character string.
  
  target_fields string | array[string] Required
  
  trim boolean
  
  Trim whitespaces in unquoted fields.
  
  date object
  
  Hide date attributes Show date attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  formats array[string] Required
  
  An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
  
  locale string
  
  The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  timezone string
  
  The timezone to use when parsing the date. Supports template snippets.
  
  output_format string
  
  The format to use when writing the date to target_field. Must be a valid java time pattern.
  
  date_index_name object
  
  Hide date_index_name attributes Show date_index_name attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  date_formats array[string]
  
  An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
  
  date_rounding string Required
  
  How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  index_name_format string
  
  The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.
  
  index_name_prefix string
  
  A prefix of the index name to be prepended before the printed date. Supports template snippets.
  
  locale string
  
  The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.
  
  timezone string
  
  The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.
  
  dissect object
  
  Hide dissect attributes Show dissect attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  append_separator string
  
  The character(s) that separate the appended fields.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern string Required
  
  The pattern to apply to the field.
  
  dot_expander object
  
  Hide dot_expander attributes Show dot_expander attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  override boolean
  
  Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.
  
  path string
  
  The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.
  
  drop object
  
  Hide drop attributes Show drop attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  enrich object
  
  Hide enrich attributes Show enrich attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  max_matches number
  
  The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.
  
  override boolean
  
  If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.
  
  policy_name string Required
  
  The name of the enrich policy to use.
  
  shape_relation string
  
  Values are intersects, disjoint, within, or contains.
  
  target_field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  fail object
  
  Hide fail attributes Show fail attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  message string Required
  
  The error message thrown by the processor. Supports template snippets.
  
  fingerprint object
  
  Hide fingerprint attributes Show fingerprint attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  fields string | array[string] Required
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  salt string
  
  Salt value for the hash function.
  
  method string
  
  Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.
  
  ignore_missing boolean
  
  If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.
  
  foreach object
  
  Hide foreach attributes Show foreach attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true, the processor silently exits without changing the document if the field is null or missing.
  
  processor object Required
  
  ip_location object
  
  Hide ip_location attributes Show ip_location attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  database_file string
  
  The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  first_only boolean
  
  If true, only the first found IP location data will be returned, even if the field contains an array.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  properties array[string]
  
  Controls what properties are added to the target_field based on the IP location lookup.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  download_database_on_pipeline_creation boolean
  
  If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.
  
  geo_grid object
  
  Hide geo_grid attributes Show geo_grid attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to interpret as a geo-tile.= The field format is determined by the tile_type.
  
  tile_type string Required
  
  Values are geotile, geohex, or geohash.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  parent_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  children_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  non_children_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  precision_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_format string
  
  Values are geojson or wkt.
  
  geoip object
  
  Hide geoip attributes Show geoip attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  database_file string
  
  The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  first_only boolean
  
  If true, only the first found geoip data will be returned, even if the field contains an array.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  properties array[string]
  
  Controls what properties are added to the target_field based on the geoip lookup.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  download_database_on_pipeline_creation boolean
  
  If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.
  
  grok object
  
  Hide grok attributes Show grok attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  ecs_compatibility string
  
  Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern_definitions object
  
  A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.
  
  patterns array[string] Required
  
  An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.
  
  trace_match boolean
  
  When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.
  
  gsub object
  
  Hide gsub attributes Show gsub attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern string Required
  
  The pattern to be replaced.
  
  replacement string Required
  
  The string to replace the matching patterns with.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  html_strip object
  
  Hide html_strip attributes Show html_strip attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document,
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  inference object
  
  Hide inference attributes Show inference attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  model_id string Required
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field_map object
  
  Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.
  
  inference_config object
  
  input_output
  
  ignore_missing boolean
  
  If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.
  
  join object
  
  Hide join attributes Show join attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  separator string Required
  
  The separator character.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  json object
  
  Hide json attributes Show json attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  add_to_root boolean
  
  Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.
  
  add_to_root_conflict_strategy string
  
  Values are replace or merge.
  
  allow_duplicate_keys boolean
  
  When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  kv object
  
  Hide kv attributes Show kv attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  exclude_keys array[string]
  
  List of keys to exclude from document.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field_split string Required
  
  Regex pattern to use for splitting key-value pairs.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  include_keys array[string]
  
  List of keys to filter and insert into document. Defaults to including all keys.
  
  prefix string
  
  Prefix to be added to extracted keys.
  
  strip_brackets boolean
  
  If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  trim_key string
  
  String of characters to trim from extracted keys.
  
  trim_value string
  
  String of characters to trim from extracted values.
  
  value_split string Required
  
  Regex pattern to use for splitting the key from the value within a key-value pair.
  
  lowercase object
  
  Hide lowercase attributes Show lowercase attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  network_direction object
  
  Hide network_direction attributes Show network_direction attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  source_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  internal_networks array[string]
  
  List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.
  
  internal_networks_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  pipeline object
  
  Hide pipeline attributes Show pipeline attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  name string Required
  
  ignore_missing_pipeline boolean
  
  Whether to ignore missing pipelines instead of failing.
  
  redact object
  
  Hide redact attributes Show redact attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  patterns array[string] Required
  
  A list of grok expressions to match and redact named captures with
  
  pattern_definitions object
  
  prefix string
  
  Start a redacted section with this token
  
  suffix string
  
  End a redacted section with this token
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  skip_if_unlicensed boolean
  
  If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document
  
  trace_redact boolean
  
  If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted
  
  registered_domain object
  
  Hide registered_domain attributes Show registered_domain attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  remove object
  
  Hide remove attributes Show remove attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string | array[string] Required
  
  keep string | array[string]
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  rename object
  
  Hide rename attributes Show rename attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  reroute object
  
  Hide reroute attributes Show reroute attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  destination string
  
  A static value for the target. Can’t be set when the dataset or namespace option is set.
  
  dataset
  
  namespace
  
  script object
  
  Hide script attributes Show script attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  id string
  
  lang
  
  params object
  
  Object containing parameters for the script.
  
  source
  
  set object
  
  Hide set attributes Show set attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  copy_from string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_empty_value boolean
  
  If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.
  
  media_type string
  
  The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.
  
  override boolean
  
  If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.
  
  value object
  
  The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.
  
  set_security_user object
  
  Hide set_security_user attributes Show set_security_user attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Controls what user related properties are added to the field.
  
  sort object
  
  Hide sort attributes Show sort attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  order string
  
  Values are asc or desc.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  split object
  
  Hide split attributes Show split attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  preserve_trailing boolean
  
  Preserves empty trailing fields, if any.
  
  separator string Required
  
  A regex which matches the separator, for example, , or \s+.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  terminate object
  
  Hide terminate attributes Show terminate attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  trim object
  
  Hide trim attributes Show trim attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  uppercase object
  
  Hide uppercase attributes Show uppercase attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  urldecode object
  
  Hide urldecode attributes Show urldecode attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  uri_parts object
  
  Hide uri_parts attributes Show uri_parts attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  keep_original boolean
  
  If true, the processor copies the unparsed URI to <target_field>.original.
  
  remove_if_successful boolean
  
  If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  user_agent object
  
  Hide user_agent attributes Show user_agent attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  regex_file string
  
  The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Controls what properties are added to target_field.
  
  Values are name, os, device, original, or version.
  
  extract_device_type boolean Beta
  
  Extracts device type from the user agent string on a best-effort basis.
  
  processors array[object]
  
  Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.
  
  Hide processors attributes Show processors attributes object
  
  append object
  
  Hide append attributes Show append attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  value
  
  allow_duplicates boolean
  
  If false, the processor does not append values already present in the field.
  
  attachment object
  
  Hide attachment attributes Show attachment attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  indexed_chars number
  
  The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.
  
  indexed_chars_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  remove_binary boolean
  
  If true, the binary field will be removed from the document
  
  resource_name string
  
  Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.
  
  bytes object
  
  Hide bytes attributes Show bytes attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  circle object
  
  Hide circle attributes Show circle attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  error_distance number Required
  
  The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  shape_type string Required
  
  Values are geo_shape or shape.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  community_id object
  
  Hide community_id attributes Show community_id attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  source_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  source_port string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_port string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  iana_number string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  icmp_type string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  icmp_code string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  transport string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  seed number
  
  Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  convert object
  
  Hide convert attributes Show convert attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  type string Required
  
  Values are integer, long, double, float, boolean, ip, string, or auto.
  
  csv object
  
  Hide csv attributes Show csv attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  empty_value object
  
  Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  quote string
  
  Quote used in CSV, has to be single character string.
  
  separator string
  
  Separator used in CSV, has to be single character string.
  
  target_fields string | array[string] Required
  
  trim boolean
  
  Trim whitespaces in unquoted fields.
  
  date object
  
  Hide date attributes Show date attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  formats array[string] Required
  
  An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
  
  locale string
  
  The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  timezone string
  
  The timezone to use when parsing the date. Supports template snippets.
  
  output_format string
  
  The format to use when writing the date to target_field. Must be a valid java time pattern.
  
  date_index_name object
  
  Hide date_index_name attributes Show date_index_name attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  date_formats array[string]
  
  An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
  
  date_rounding string Required
  
  How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  index_name_format string
  
  The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.
  
  index_name_prefix string
  
  A prefix of the index name to be prepended before the printed date. Supports template snippets.
  
  locale string
  
  The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.
  
  timezone string
  
  The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.
  
  dissect object
  
  Hide dissect attributes Show dissect attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  append_separator string
  
  The character(s) that separate the appended fields.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern string Required
  
  The pattern to apply to the field.
  
  dot_expander object
  
  Hide dot_expander attributes Show dot_expander attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  override boolean
  
  Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.
  
  path string
  
  The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.
  
  drop object
  
  Hide drop attributes Show drop attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  enrich object
  
  Hide enrich attributes Show enrich attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  max_matches number
  
  The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.
  
  override boolean
  
  If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.
  
  policy_name string Required
  
  The name of the enrich policy to use.
  
  shape_relation string
  
  Values are intersects, disjoint, within, or contains.
  
  target_field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  fail object
  
  Hide fail attributes Show fail attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  message string Required
  
  The error message thrown by the processor. Supports template snippets.
  
  fingerprint object
  
  Hide fingerprint attributes Show fingerprint attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  fields string | array[string] Required
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  salt string
  
  Salt value for the hash function.
  
  method string
  
  Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.
  
  ignore_missing boolean
  
  If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.
  
  foreach object
  
  Hide foreach attributes Show foreach attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true, the processor silently exits without changing the document if the field is null or missing.
  
  processor object Required
  
  ip_location object
  
  Hide ip_location attributes Show ip_location attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  database_file string
  
  The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  first_only boolean
  
  If true, only the first found IP location data will be returned, even if the field contains an array.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  properties array[string]
  
  Controls what properties are added to the target_field based on the IP location lookup.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  download_database_on_pipeline_creation boolean
  
  If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.
  
  geo_grid object
  
  Hide geo_grid attributes Show geo_grid attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to interpret as a geo-tile.= The field format is determined by the tile_type.
  
  tile_type string Required
  
  Values are geotile, geohex, or geohash.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  parent_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  children_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  non_children_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  precision_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_format string
  
  Values are geojson or wkt.
  
  geoip object
  
  Hide geoip attributes Show geoip attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  database_file string
  
  The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  first_only boolean
  
  If true, only the first found geoip data will be returned, even if the field contains an array.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  properties array[string]
  
  Controls what properties are added to the target_field based on the geoip lookup.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  download_database_on_pipeline_creation boolean
  
  If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.
  
  grok object
  
  Hide grok attributes Show grok attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  ecs_compatibility string
  
  Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern_definitions object
  
  A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.
  
  patterns array[string] Required
  
  An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.
  
  trace_match boolean
  
  When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.
  
  gsub object
  
  Hide gsub attributes Show gsub attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern string Required
  
  The pattern to be replaced.
  
  replacement string Required
  
  The string to replace the matching patterns with.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  html_strip object
  
  Hide html_strip attributes Show html_strip attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document,
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  inference object
  
  Hide inference attributes Show inference attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  model_id string Required
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field_map object
  
  Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.
  
  inference_config object
  
  input_output
  
  ignore_missing boolean
  
  If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.
  
  join object
  
  Hide join attributes Show join attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  separator string Required
  
  The separator character.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  json object
  
  Hide json attributes Show json attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  add_to_root boolean
  
  Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.
  
  add_to_root_conflict_strategy string
  
  Values are replace or merge.
  
  allow_duplicate_keys boolean
  
  When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  kv object
  
  Hide kv attributes Show kv attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  exclude_keys array[string]
  
  List of keys to exclude from document.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field_split string Required
  
  Regex pattern to use for splitting key-value pairs.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  include_keys array[string]
  
  List of keys to filter and insert into document. Defaults to including all keys.
  
  prefix string
  
  Prefix to be added to extracted keys.
  
  strip_brackets boolean
  
  If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  trim_key string
  
  String of characters to trim from extracted keys.
  
  trim_value string
  
  String of characters to trim from extracted values.
  
  value_split string Required
  
  Regex pattern to use for splitting the key from the value within a key-value pair.
  
  lowercase object
  
  Hide lowercase attributes Show lowercase attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  network_direction object
  
  Hide network_direction attributes Show network_direction attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  source_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  internal_networks array[string]
  
  List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.
  
  internal_networks_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  pipeline object
  
  Hide pipeline attributes Show pipeline attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  name string Required
  
  ignore_missing_pipeline boolean
  
  Whether to ignore missing pipelines instead of failing.
  
  redact object
  
  Hide redact attributes Show redact attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  patterns array[string] Required
  
  A list of grok expressions to match and redact named captures with
  
  pattern_definitions object
  
  prefix string
  
  Start a redacted section with this token
  
  suffix string
  
  End a redacted section with this token
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  skip_if_unlicensed boolean
  
  If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document
  
  trace_redact boolean
  
  If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted
  
  registered_domain object
  
  Hide registered_domain attributes Show registered_domain attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  remove object
  
  Hide remove attributes Show remove attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string | array[string] Required
  
  keep string | array[string]
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  rename object
  
  Hide rename attributes Show rename attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  reroute object
  
  Hide reroute attributes Show reroute attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  destination string
  
  A static value for the target. Can’t be set when the dataset or namespace option is set.
  
  dataset
  
  namespace
  
  script object
  
  Hide script attributes Show script attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  id string
  
  lang
  
  params object
  
  Object containing parameters for the script.
  
  source
  
  set object
  
  Hide set attributes Show set attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  copy_from string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_empty_value boolean
  
  If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.
  
  media_type string
  
  The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.
  
  override boolean
  
  If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.
  
  value object
  
  The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.
  
  set_security_user object
  
  Hide set_security_user attributes Show set_security_user attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Controls what user related properties are added to the field.
  
  sort object
  
  Hide sort attributes Show sort attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  order string
  
  Values are asc or desc.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  split object
  
  Hide split attributes Show split attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  preserve_trailing boolean
  
  Preserves empty trailing fields, if any.
  
  separator string Required
  
  A regex which matches the separator, for example, , or \s+.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  terminate object
  
  Hide terminate attributes Show terminate attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  trim object
  
  Hide trim attributes Show trim attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  uppercase object
  
  Hide uppercase attributes Show uppercase attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  urldecode object
  
  Hide urldecode attributes Show urldecode attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  uri_parts object
  
  Hide uri_parts attributes Show uri_parts attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  keep_original boolean
  
  If true, the processor copies the unparsed URI to <target_field>.original.
  
  remove_if_successful boolean
  
  If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  user_agent object
  
  Hide user_agent attributes Show user_agent attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  regex_file string
  
  The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Controls what properties are added to target_field.
  
  Values are name, os, device, original, or version.
  
  extract_device_type boolean Beta
  
  Extracts device type from the user agent string on a best-effort basis.
  
  version number
  
  deprecated boolean
  
  Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.
  
  _meta object
  
  Hide _meta attribute Show _meta attribute object
  
  * object Additional properties

GET /_ingest/pipeline/{id}

curl \
 --request GET 'http://api.example.com/_ingest/pipeline/{id}' \
 --header "Authorization: $API_KEY"

Response examples (200)

A successful response for retrieving information about an ingest pipeline.

{
  "my-pipeline-id" : {
    "description" : "describe pipeline",
    "version" : 123,
    "processors" : [
      {
        "set" : {
          "field" : "foo",
          "value" : "bar"
        }
      }
    ]
  }
}

Start datafeeds Added in 5.5.0

POST /_ml/datafeeds/{datafeed_id}/_start

Api key auth Basic auth Bearer auth

A datafeed must be started in order to retrieve data from Elasticsearch. A datafeed can be started and stopped multiple times throughout its lifecycle.

Before you can start a datafeed, the anomaly detection job must be open. Otherwise, an error occurs.

If you restart a stopped datafeed, it continues processing input data from the next millisecond after it was stopped. If new data was indexed for that exact millisecond between stopping and starting, it will be ignored.

When Elasticsearch security features are enabled, your datafeed remembers which roles the last user to create or update it had at the time of creation or update and runs the query using those same roles. If you provided secondary authorization headers when you created or updated the datafeed, those credentials are used instead.

Path parameters

datafeed_id string Required

A numerical character string that uniquely identifies the datafeed. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.

Query parameters

end string | number
The time that the datafeed should end, which can be specified by using one of the following formats:
- ISO 8601 format with milliseconds, for example 2017-01-22T06:00:00.000Z
- ISO 8601 format without milliseconds, for example 2017-01-22T06:00:00+00:00
- Milliseconds since the epoch, for example 1485061200000
Date-time arguments using either of the ISO 8601 formats must have a time zone designator, where Z is accepted as an abbreviation for UTC time. When a URL is expected (for example, in browsers), the + used in time zone designators must be encoded as %2B. The end time value is exclusive. If you do not specify an end time, the datafeed runs continuously.
start string | number

The time that the datafeed should begin, which can be specified by using the same formats as the end parameter. This value is inclusive. If you do not specify a start time and the datafeed is associated with a new anomaly detection job, the analysis starts from the earliest time for which data is available. If you restart a stopped datafeed and specify a start value that is earlier than the timestamp of the latest processed record, the datafeed continues from 1 millisecond after the timestamp of the latest processed record.
timeout string

Specifies the amount of time to wait until a datafeed starts.

application/json

Body

end string | number

A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

One of:
DateTime string UnitMillis number
start string | number

A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

One of:
DateTime string UnitMillis number
timeout string

A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

Responses

200 application/json
Hide response attributes Show response attributes object
- node string | array[string] Required
- started boolean Required
  
  For a successful response, this value is always true. On failure, an exception is returned instead.

POST /_ml/datafeeds/{datafeed_id}/_start

curl \
 --request POST 'http://api.example.com/_ml/datafeeds/{datafeed_id}/_start' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"":"string","timeout":"string"}'

Create or update a snapshot repository Added in 0.0.0

PUT /_snapshot/{repository}

Api key auth Basic auth Bearer auth

IMPORTANT: If you are migrating searchable snapshots, the repository name must be identical in the source and destination clusters. To register a snapshot repository, the cluster's global metadata must be writeable. Ensure there are no cluster blocks (for example, cluster.blocks.read_only and clsuter.blocks.read_only_allow_delete settings) that prevent write access.

Several options for this API can be specified using a query parameter or a request body parameter. If both parameters are specified, only the query parameter is used.

External documentation

Path parameters

repository string Required

The name of the snapshot repository to register or update.

Query parameters

master_timeout string

The period to wait for the master node. If the master node is not available before the timeout expires, the request fails and returns an error. To indicate that the request should never timeout, set it to -1.
timeout string

The period to wait for a response from all relevant nodes in the cluster after updating the cluster metadata. If no response is received before the timeout expires, the cluster metadata update still applies but the response will indicate that it was not completely acknowledged. To indicate that the request should never timeout, set it to -1.
verify boolean

If true, the request verifies the repository is functional on all master and data nodes in the cluster. If false, this verification is skipped. You can also perform this verification with the verify snapshot repository API.

application/json

Body object Required

uuid string
type string Required Discriminator

The Azure repository type.

Value is azure.

External documentation
settings object
Hide settings attributes Show settings attributes object
- chunk_size number | string
  
  One of:
  ByteSize number ByteSize string
- compress boolean
  
  When set to true, metadata files are stored in compressed format. This setting doesn't affect index files that are already compressed by default.
- max_restore_bytes_per_sec number | string
  
  One of:
  ByteSize number ByteSize string
- max_snapshot_bytes_per_sec number | string
  
  One of:
  ByteSize number ByteSize string
- base_path string
  
  The path to the repository data within the container. It defaults to the root directory.
  
  NOTE: Don't set base_path when configuring a snapshot repository for Elastic Cloud Enterprise. Elastic Cloud Enterprise automatically generates the base_path for each deployment so that multiple deployments can share the same bucket.
- client string
  
  The name of the Azure repository client to use.
- container string
  
  The Azure container.
- delete_objects_max_size number
  
  The maxmimum batch size, between 1 and 256, used for BlobBatch requests. Defaults to 256 which is the maximum number supported by the Azure blob batch API.
- location_mode string
  
  Either primary_only or secondary_only. Note that if you set it to secondary_only, it will force readonly to true.
- max_concurrent_batch_deletes number
  
  The maximum number of concurrent batch delete requests that will be submitted for any individual bulk delete with BlobBatch. Note that the effective number of concurrent deletes is further limited by the Azure client connection and event loop thread limits. Defaults to 10, minimum is 1, maximum is 100.
- readonly boolean
  
  If true, the repository is read-only. The cluster can retrieve and restore snapshots from the repository but not write to the repository or create snapshots in it.
  
  Only a cluster with write access can create snapshots in the repository. All other clusters connected to the repository should have the readonly parameter set to true. If false, the cluster can write to the repository and create snapshots in it.
  
  IMPORTANT: If you register the same snapshot repository with multiple clusters, only one cluster should have write access to the repository. Having multiple clusters write to the repository at the same time risks corrupting the contents of the repository.

uuid string
type string Required Discriminator

The Google Cloud Storage repository type.

Value is gcs.

External documentation
settings object
Hide settings attributes Show settings attributes object
- chunk_size number | string
  
  One of:
  ByteSize number ByteSize string
- compress boolean
  
  When set to true, metadata files are stored in compressed format. This setting doesn't affect index files that are already compressed by default.
- max_restore_bytes_per_sec number | string
  
  One of:
  ByteSize number ByteSize string
- max_snapshot_bytes_per_sec number | string
  
  One of:
  ByteSize number ByteSize string
- bucket string Required
  
  The name of the bucket to be used for snapshots.
- application_name string Deprecated
  
  The name used by the client when it uses the Google Cloud Storage service.
- base_path string
  
  The path to the repository data within the bucket. It defaults to the root of the bucket.
  
  NOTE: Don't set base_path when configuring a snapshot repository for Elastic Cloud Enterprise. Elastic Cloud Enterprise automatically generates the base_path for each deployment so that multiple deployments can share the same bucket.
- client string
  
  The name of the client to use to connect to Google Cloud Storage.
- readonly boolean
  
  If true, the repository is read-only. The cluster can retrieve and restore snapshots from the repository but not write to the repository or create snapshots in it.
  
  Only a cluster with write access can create snapshots in the repository. All other clusters connected to the repository should have the readonly parameter set to true.
  
  If false, the cluster can write to the repository and create snapshots in it.
  
  IMPORTANT: If you register the same snapshot repository with multiple clusters, only one cluster should have write access to the repository. Having multiple clusters write to the repository at the same time risks corrupting the contents of the repository.

uuid string
type string Required Discriminator

The S3 repository type.

Value is s3.

External documentation
settings object
Hide settings attributes Show settings attributes object
- chunk_size number | string
  
  One of:
  ByteSize number ByteSize string
- compress boolean
  
  When set to true, metadata files are stored in compressed format. This setting doesn't affect index files that are already compressed by default.
- max_restore_bytes_per_sec number | string
  
  One of:
  ByteSize number ByteSize string
- max_snapshot_bytes_per_sec number | string
  
  One of:
  ByteSize number ByteSize string
- bucket string Required
  
  The name of the S3 bucket to use for snapshots. The bucket name must adhere to Amazon's S3 bucket naming rules.
  
  External documentation
- base_path string
  
  The path to the repository data within its bucket. It defaults to an empty string, meaning that the repository is at the root of the bucket. The value of this setting should not start or end with a forward slash (/).
  
  NOTE: Don't set base_path when configuring a snapshot repository for Elastic Cloud Enterprise. Elastic Cloud Enterprise automatically generates the base_path for each deployment so that multiple deployments may share the same bucket.
- buffer_size number | string
  
  One of:
  ByteSize number ByteSize string
- canned_acl string
  
  The S3 repository supports all S3 canned ACLs: private, public-read, public-read-write, authenticated-read, log-delivery-write, bucket-owner-read, bucket-owner-full-control. You could specify a canned ACL using the canned_acl setting. When the S3 repository creates buckets and objects, it adds the canned ACL into the buckets and objects.
  
  External documentation
- client string
  
  The name of the S3 client to use to connect to S3.
- delete_objects_max_size number
  
  The maxmimum batch size, between 1 and 1000, used for DeleteObjects requests. Defaults to 1000 which is the maximum number supported by the AWS DeleteObjects API.
  
  External documentation
- get_register_retry_delay string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
- max_multipart_parts number
  
  The maximum number of parts that Elasticsearch will write during a multipart upload of a single object. Files which are larger than buffer_size × max_multipart_parts will be chunked into several smaller objects. Elasticsearch may also split a file across multiple objects to satisfy other constraints such as the chunk_size limit. Defaults to 10000 which is the maximum number of parts in a multipart upload in AWS S3.
- max_multipart_upload_cleanup_size number
  
  The maximum number of possibly-dangling multipart uploads to clean up in each batch of snapshot deletions. Defaults to 1000 which is the maximum number supported by the AWS ListMultipartUploads API. If set to 0, Elasticsearch will not attempt to clean up dangling multipart uploads.
  
  External documentation
- readonly boolean
  
  If true, the repository is read-only. The cluster can retrieve and restore snapshots from the repository but not write to the repository or create snapshots in it.
  
  Only a cluster with write access can create snapshots in the repository. All other clusters connected to the repository should have the readonly parameter set to true.
  
  If false, the cluster can write to the repository and create snapshots in it.
  
  IMPORTANT: If you register the same snapshot repository with multiple clusters, only one cluster should have write access to the repository. Having multiple clusters write to the repository at the same time risks corrupting the contents of the repository.
- server_side_encryption boolean
  
  When set to true, files are encrypted on server side using an AES256 algorithm.
- storage_class string
  
  The S3 storage class for objects written to the repository. Values may be standard, reduced_redundancy, standard_ia, onezone_ia, and intelligent_tiering.
  
  External documentation
- throttled_delete_retry.delay_increment string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
- throttled_delete_retry.maximum_delay string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
- throttled_delete_retry.maximum_number_of_retries number
  
  The number times to retry a throttled snapshot deletion. The default is 10 and the minimum value is 0 which will disable retries altogether. Note that if retries are enabled in the Azure client, each of these retries comprises that many client-level retries.

uuid string
type string Required Discriminator

The shared file system repository type.

Value is fs.

External documentation
settings object
Hide settings attributes Show settings attributes object
- chunk_size number | string
  
  One of:
  ByteSize number ByteSize string
- compress boolean
  
  When set to true, metadata files are stored in compressed format. This setting doesn't affect index files that are already compressed by default.
- max_restore_bytes_per_sec number | string
  
  One of:
  ByteSize number ByteSize string
- max_snapshot_bytes_per_sec number | string
  
  One of:
  ByteSize number ByteSize string
- location string Required
  
  The location of the shared filesystem used to store and retrieve snapshots. This location must be registered in the path.repo setting on all master and data nodes in the cluster. Unlike path.repo, this setting supports only a single file path.
- max_number_of_snapshots number
  
  The maximum number of snapshots the repository can contain. The default is Integer.MAX_VALUE, which is 2^31-1 or 2147483647.
- readonly boolean
  
  If true, the repository is read-only. The cluster can retrieve and restore snapshots from the repository but not write to the repository or create snapshots in it.
  
  Only a cluster with write access can create snapshots in the repository. All other clusters connected to the repository should have the readonly parameter set to true.
  
  If false, the cluster can write to the repository and create snapshots in it.
  
  IMPORTANT: If you register the same snapshot repository with multiple clusters, only one cluster should have write access to the repository. Having multiple clusters write to the repository at the same time risks corrupting the contents of the repository.

uuid string
type string Required Discriminator

The read-only URL repository type.

Value is url.

External documentation
settings object
Hide settings attributes Show settings attributes object
- chunk_size number | string
  
  One of:
  ByteSize number ByteSize string
- compress boolean
  
  When set to true, metadata files are stored in compressed format. This setting doesn't affect index files that are already compressed by default.
- max_restore_bytes_per_sec number | string
  
  One of:
  ByteSize number ByteSize string
- max_snapshot_bytes_per_sec number | string
  
  One of:
  ByteSize number ByteSize string
- http_max_retries number
  
  The maximum number of retries for HTTP and HTTPS URLs.
- http_socket_timeout string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
- max_number_of_snapshots number
  
  The maximum number of snapshots the repository can contain. The default is Integer.MAX_VALUE, which is 2^31-1 or 2147483647.
- url string Required
  The URL location of the root of the shared filesystem repository. The following protocols are supported:
  
  file
  
  ftp
  
  http
  
  https
  
  jar
  
  URLs using the HTTP, HTTPS, or FTP protocols must be explicitly allowed with the repositories.url.allowed_urls cluster setting. This setting supports wildcards in the place of a host, path, query, or fragment in the URL.
  
  URLs using the file protocol must point to the location of a shared filesystem accessible to all master and data nodes in the cluster. This location must be registered in the path.repo setting. You don't need to register URLs using the FTP, HTTP, HTTPS, or JAR protocols in the path.repo setting.

uuid string
type string Required Discriminator

The source-only repository type.

Value is source.

External documentation
settings object
Hide settings attributes Show settings attributes object
- chunk_size number | string
  
  One of:
  ByteSize number ByteSize string
- compress boolean
  
  When set to true, metadata files are stored in compressed format. This setting doesn't affect index files that are already compressed by default.
- max_restore_bytes_per_sec number | string
  
  One of:
  ByteSize number ByteSize string
- max_snapshot_bytes_per_sec number | string
  
  One of:
  ByteSize number ByteSize string
- delegate_type string
  
  The delegated repository type. For valid values, refer to the type parameter. Source repositories can use settings properties for its delegated repository type.
- max_number_of_snapshots number
  
  The maximum number of snapshots the repository can contain. The default is Integer.MAX_VALUE, which is 2^31-1 or 2147483647.
- read_only boolean
  
  If true, the repository is read-only. The cluster can retrieve and restore snapshots from the repository but not write to the repository or create snapshots in it.
  
  Only a cluster with write access can create snapshots in the repository. All other clusters connected to the repository should have the readonly parameter set to true.
  
  If false, the cluster can write to the repository and create snapshots in it.
  
  IMPORTANT: If you register the same snapshot repository with multiple clusters, only one cluster should have write access to the repository. Having multiple clusters write to the repository at the same time risks corrupting the contents of the repository.

Responses

200 application/json
Hide response attribute Show response attribute object
- acknowledged boolean Required
  
  For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /_snapshot/{repository}

curl \
 --request PUT 'http://api.example.com/_snapshot/{repository}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"type\": \"fs\",\n  \"settings\": {\n    \"location\": \"my_backup_location\"\n  }\n}"'

Request examples

Run `PUT /_snapshot/my_repository` to create or update a shared file system snapshot repository.

{
  "type": "fs",
  "settings": {
    "location": "my_backup_location"
  }
}

Run `PUT /_snapshot/my_repository` to create or update an Azure snapshot repository.

{
  "type": "azure",
  "settings": {
    "client": "secondary"
  }
}

Run `PUT /_snapshot/my_gcs_repository` to create or update a Google Cloud Storage snapshot repository.

{
  "type": "gcs",
  "settings": {
    "bucket": "my_other_bucket",
    "base_path": "dev"
  }
}

Run `PUT /_snapshot/my_s3_repository` to create or update an AWS S3 snapshot repository.

{
  "type": "s3",
  "settings": {
    "bucket": "my-bucket"
  }
}

Run `PUT _snapshot/my_src_only_repository` to create or update a source-only snapshot repository.

{
  "type": "source",
  "settings": {
    "delegate_type": "fs",
    "location": "my_backup_repository"
  }
}

Run `PUT _snapshot/my_read_only_url_repository` to create or update a read-only URL snapshot repository.

{
  "type": "url",
  "settings": {
    "url": "file:/mount/backups/my_fs_backup_location"
  }
}

Get index template information Added in 5.2.0

version string | null

Get the cluster health status Added in 1.3.0

start_time string | number

stop_time string | number

percent string | number Required

Simulate an index Added in 7.9.0

routing_path string | array[string]

order string | array[string]

mode string | array[string]

missing string | array[string]

number_of_shards number | string

number_of_replicas number | string

routing_partition_size number | string

hidden boolean | string

auto_expand_replicas string | null

max_thread_count number | string

max_merge_count number | string

read_only boolean | string

read_only_allow_delete boolean | string

read boolean | string

write boolean | string

metadata boolean | string

max_token_count number | string

indexing_complete boolean | string

prefer_ilm boolean | string

creation_date number | string

creation_date_string string | number

verified_before_close boolean | string

format string | number

flush_threshold_size number | string

size number | string

lenient boolean | string Required

priority number | string

end_time string | number

start_time string | number

limit number | string

ignore_dynamic_beyond_limit boolean | string

ignore_malformed boolean | string

type string Required

Get pipelines Added in 5.0.0

Start datafeeds Added in 5.5.0

end string | number

start string | number

Create or update a snapshot repository Added in 0.0.0

Body object Required

chunk_size number | string

max_restore_bytes_per_sec number | string

max_snapshot_bytes_per_sec number | string