Create or update an autoscaling policy Added in 7.11.0

PUT /_autoscaling/policy/{name}

NOTE: This feature is designed for indirect use by Elasticsearch Service, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.

External documentation

Path parameters

name string Required

the name of the autoscaling policy

Query parameters

master_timeout string

Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
timeout string

Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

application/json

Body Required

roles array[string] Required
deciders object Required

Decider settings.

External documentation
Hide deciders attribute Show deciders attribute object
- * object Additional properties

Responses

200 application/json
Hide response attribute Show response attribute object
- acknowledged boolean Required
  
  For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /_autoscaling/policy/{name}

curl \
 --request PUT 'http://api.example.com/_autoscaling/policy/{name}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"roles\": [],\n  \"deciders\": {\n    \"fixed\": {\n    }\n  }\n}"'

Request examples

{
  "roles": [],
  "deciders": {
    "fixed": {
    }
  }
}

The API method and path for this request: `PUT /_autoscaling/policy/my_autoscaling_policy`. It creates `my_autoscaling_policy` using the fixed autoscaling decider, applying to the set of nodes having (only) the `data_hot` role.

{
  "roles" : [ "data_hot" ],
  "deciders": {
    "fixed": {
    }
  }
}

Response examples (200)

{
  "acknowledged": true
}

Compact and aligned text (CAT)

The compact and aligned text (CAT) APIs aim are intended only for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, it's recommend to use a corresponding JSON API. All the cat commands accept a query string parameter help to see all the headers and info they provide, and the /_cat command alone lists all the available commands.

Get data frame analytics jobs Added in 7.7.0

GET /_cat/ml/data_frame/analytics/{id}

Api key auth Basic auth Bearer auth

Get configuration and usage information about data frame analytics jobs.

IMPORTANT: CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get data frame analytics jobs statistics API.

Path parameters

id string Required

The ID of the data frame analytics to fetch

Query parameters

allow_no_match boolean

Whether to ignore if a wildcard expression matches no configs. (This includes _all string or when no configs have been specified)
bytes string

The unit in which to display byte values

Values are b, kb, mb, gb, tb, or pb.
h string | array[string]
Comma-separated list of column names to display.

Supported values include:
- assignment_explanation (or ae): Contains messages relating to the selection of a node.
- create_time (or ct, createTime): The time when the data frame analytics job was created.
- description (or d): A description of a job.
- dest_index (or di, destIndex): Name of the destination index.
- failure_reason (or fr, failureReason): Contains messages about the reason why a data frame analytics job failed.
- id: Identifier for the data frame analytics job.
- model_memory_limit (or mml, modelMemoryLimit): The approximate maximum amount of memory resources that are permitted for the data frame analytics job.
- node.address (or na, nodeAddress): The network address of the node that the data frame analytics job is assigned to.
- node.ephemeral_id (or ne, nodeEphemeralId): The ephemeral ID of the node that the data frame analytics job is assigned to.
- node.id (or ni, nodeId): The unique identifier of the node that the data frame analytics job is assigned to.
- node.name (or nn, nodeName): The name of the node that the data frame analytics job is assigned to.
- progress (or p): The progress report of the data frame analytics job by phase.
- source_index (or si, sourceIndex): Name of the source index.
- state (or s): Current state of the data frame analytics job.
- type (or t): The type of analysis that the data frame analytics job performs.
- version (or v): The Elasticsearch version number in which the data frame analytics job was created.
s string | array[string]
Comma-separated list of column names or column aliases used to sort the response.

Supported values include:
- assignment_explanation (or ae): Contains messages relating to the selection of a node.
- create_time (or ct, createTime): The time when the data frame analytics job was created.
- description (or d): A description of a job.
- dest_index (or di, destIndex): Name of the destination index.
- failure_reason (or fr, failureReason): Contains messages about the reason why a data frame analytics job failed.
- id: Identifier for the data frame analytics job.
- model_memory_limit (or mml, modelMemoryLimit): The approximate maximum amount of memory resources that are permitted for the data frame analytics job.
- node.address (or na, nodeAddress): The network address of the node that the data frame analytics job is assigned to.
- node.ephemeral_id (or ne, nodeEphemeralId): The ephemeral ID of the node that the data frame analytics job is assigned to.
- node.id (or ni, nodeId): The unique identifier of the node that the data frame analytics job is assigned to.
- node.name (or nn, nodeName): The name of the node that the data frame analytics job is assigned to.
- progress (or p): The progress report of the data frame analytics job by phase.
- source_index (or si, sourceIndex): Name of the source index.
- state (or s): Current state of the data frame analytics job.
- type (or t): The type of analysis that the data frame analytics job performs.
- version (or v): The Elasticsearch version number in which the data frame analytics job was created.
time string

Unit used to display time values.

Values are nanos, micros, ms, s, m, h, or d.

Responses

200 application/json
Hide response attributes Show response attributes object
- id string
- type string
  
  The type of analysis that the job performs.
- create_time string
  
  The time when the job was created.
- version string
- source_index string
- dest_index string
- description string
  
  A description of the job.
- model_memory_limit string
  
  The approximate maximum amount of memory resources that are permitted for the job.
- state string
  
  The current status of the job.
- failure_reason string
  
  Messages about the reason why the job failed.
- progress string
  
  The progress report for the job by phase.
- assignment_explanation string
  
  Messages related to the selection of a node.
- node.id string
- node.name string
- node.ephemeral_id string
- node.address string
  
  The network address of the assigned node.

GET /_cat/ml/data_frame/analytics/{id}

curl \
 --request GET 'http://api.example.com/_cat/ml/data_frame/analytics/{id}' \
 --header "Authorization: $API_KEY"

Response examples (200)

A successful response from `GET _cat/ml/data_frame/analytics?v=true&format=json`.

[
  {
    "id": "classifier_job_1",
    "type": "classification",
    "create_time": "2020-02-12T11:49:09.594Z",
    "state": "stopped"
  },
    {
    "id": "classifier_job_2",
    "type": "classification",
    "create_time": "2020-02-12T11:49:14.479Z",
    "state": "stopped"
  },
  {
    "id": "classifier_job_3",
    "type": "classification",
    "create_time": "2020-02-12T11:49:16.928Z",
    "state": "stopped"
  },
  {
    "id": "classifier_job_4",
    "type": "classification",
    "create_time": "2020-02-12T11:49:19.127Z",
    "state": "stopped"
  },
  {
    "id": "classifier_job_5",
    "type": "classification",
    "create_time": "2020-02-12T11:49:21.349Z",
    "state": "stopped"
  }
]

Update the connector service type Beta

PUT /_connector/{connector_id}/_service_type

Api key auth Basic auth Bearer auth

Path parameters

connector_id string Required

The unique identifier of the connector to be updated

application/json

Body Required

service_type string Required

Responses

200 application/json
Hide response attribute Show response attribute object
- result string Required
  
  Values are created, updated, deleted, not_found, or noop.

PUT /_connector/{connector_id}/_service_type

curl \
 --request PUT 'http://api.example.com/_connector/{connector_id}/_service_type' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"service_type\": \"sharepoint_online\"\n}"'

Request example

{
    "service_type": "sharepoint_online"
}

Response examples (200)

{
  "result": "updated"
}

Bulk index or delete documents

POST /{index}/_bulk

Api key auth Basic auth Bearer auth

Perform multiple index, create, delete, and update actions in a single request. This reduces overhead and can greatly increase indexing speed.

If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:

To use the create action, you must have the create_doc, create, index, or write index privilege. Data streams support only the create action.
To use the index action, you must have the create, index, or write index privilege.
To use the delete action, you must have the delete or write index privilege.
To use the update action, you must have the index or write index privilege.
To automatically create a data stream or index with a bulk API request, you must have the auto_configure, create_index, or manage index privilege.
To make the result of a bulk operation visible to search using the refresh parameter, you must have the maintenance or manage index privilege.

Automatic data stream creation requires a matching index template with data stream enabled.

The actions are specified in the request body using a newline delimited JSON (NDJSON) structure:

action_and_meta_data\n
optional_source\n
action_and_meta_data\n
optional_source\n
....
action_and_meta_data\n
optional_source\n

The index and create actions expect a source on the next line and have the same semantics as the op_type parameter in the standard index API. A create action fails if a document with the same ID already exists in the target An index action adds or replaces a document as necessary.

NOTE: Data streams support only the create action. To update or delete a document in a data stream, you must target the backing index containing the document.

An update action expects that the partial doc, upsert, and script and its options are specified on the next line.

A delete action does not expect a source on the next line and has the same semantics as the standard delete API.

NOTE: The final line of data must end with a newline character (\n). Each newline character may be preceded by a carriage return (\r). When sending NDJSON data to the _bulk endpoint, use a Content-Type header of application/json or application/x-ndjson. Because this format uses literal newline characters (\n) as delimiters, make sure that the JSON actions and sources are not pretty printed.

If you provide a target in the request path, it is used for any actions that don't explicitly specify an _index argument.

A note on the format: the idea here is to make processing as fast as possible. As some of the actions are redirected to other shards on other nodes, only action_meta_data is parsed on the receiving node side.

Client libraries using this protocol should try and strive to do something similar on the client side, and reduce buffering as much as possible.

There is no "correct" number of actions to perform in a single bulk request. Experiment with different settings to find the optimal size for your particular workload. Note that Elasticsearch limits the maximum size of a HTTP request to 100mb by default so clients must ensure that no request exceeds this size. It is not possible to index a single document that exceeds the size limit, so you must pre-process any such documents into smaller pieces before sending them to Elasticsearch. For instance, split documents into pages or chapters before indexing them, or store raw binary data in a system outside Elasticsearch and replace the raw data with a link to the external system in the documents that you send to Elasticsearch.

Client suppport for bulk requests

Some of the officially supported clients provide helpers to assist with bulk requests and reindexing:

Go: Check out esutil.BulkIndexer
Perl: Check out Search::Elasticsearch::Client::5_0::Bulk and Search::Elasticsearch::Client::5_0::Scroll
Python: Check out elasticsearch.helpers.*
JavaScript: Check out client.helpers.*
.NET: Check out BulkAllObservable
PHP: Check out bulk indexing.

Submitting bulk requests with cURL

If you're providing text file input to curl, you must use the --data-binary flag instead of plain -d. The latter doesn't preserve newlines. For example:

$ cat requests
{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
$ curl -s -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/_bulk --data-binary "@requests"; echo
{"took":7, "errors": false, "items":[{"index":{"_index":"test","_id":"1","_version":1,"result":"created","forced_refresh":false}}]}

Optimistic concurrency control

Each index and delete action within a bulk API call may include the if_seq_no and if_primary_term parameters in their respective action and meta data lines. The if_seq_no and if_primary_term parameters control how operations are run, based on the last modification to existing documents. See Optimistic concurrency control for more details.

Versioning

Each bulk item can include the version value using the version field. It automatically follows the behavior of the index or delete operation based on the _version mapping. It also support the version_type.

Routing

Each bulk item can include the routing value using the routing field. It automatically follows the behavior of the index or delete operation based on the _routing mapping.

NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.

Wait for active shards

When making bulk calls, you can set the wait_for_active_shards parameter to require a minimum number of shard copies to be active before starting to process the bulk request.

Refresh

Control when the changes made by this request are visible to search.

NOTE: Only the shards that receive the bulk request will be affected by refresh. Imagine a _bulk?refresh=wait_for request with three documents in it that happen to be routed to different shards in an index with five shards. The request will only wait for those three shards to refresh. The other two shards that make up the index do not participate in the _bulk request at all.

Path parameters

index string Required

The name of the data stream, index, or index alias to perform bulk actions on.

Query parameters

include_source_on_error boolean

True or false if to include the document source in the error message in case of parsing errors.
list_executed_pipelines boolean

If true, the response will include the ingest pipelines that were run for each index or create.
pipeline string

The pipeline identifier to use to preprocess incoming documents. If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request. If a final pipeline is configured, it will always run regardless of the value of this parameter.
refresh string

If true, Elasticsearch refreshes the affected shards to make this operation visible to search. If wait_for, wait for a refresh to make this operation visible to search. If false, do nothing with refreshes. Valid values: true, false, wait_for.

Values are true, false, or wait_for.
routing string

A custom value that is used to route operations to a specific shard.
_source boolean | string | array[string]

Indicates whether to return the _source field (true or false) or contains a list of fields to return.
_source_excludes string | array[string]

A comma-separated list of source fields to exclude from the response. You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter. If the _source parameter is false, this parameter is ignored.
_source_includes string | array[string]

A comma-separated list of source fields to include in the response. If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the _source_excludes query parameter. If the _source parameter is false, this parameter is ignored.
timeout string

The period each action waits for the following operations: automatic index creation, dynamic mapping updates, and waiting for active shards. The default is 1m (one minute), which guarantees Elasticsearch waits for at least the timeout before failing. The actual wait time could be longer, particularly when multiple waits occur.
wait_for_active_shards number | string

The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). The default is 1, which waits for each primary shard to be active.
require_alias boolean

If true, the request's actions must target an index alias.
require_data_stream boolean

If true, the request's actions must target a data stream (existing or to be created).

application/json

Body object Required

index object
Hide index attributes Show index attributes object
- _id string
- _index string
- routing string
- if_primary_term number
- if_seq_no number
- version number
- version_type string
  
  Values are internal, external, external_gte, or force.
- dynamic_templates object
  
  A map from the full name of fields to the name of dynamic templates. It defaults to an empty map. If a name matches a dynamic template, that template will be applied regardless of other match predicates defined in the template. If a field is already defined in the mapping, then this parameter won't be used.
  Hide dynamic_templates attribute Show dynamic_templates attribute object
  
  * string Additional properties
- pipeline string
  
  The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request. If a final pipeline is configured, it will always run regardless of the value of this parameter.
- require_alias boolean
  
  If true, the request's actions must target an index alias.
create object
Hide create attributes Show create attributes object
- _id string
- _index string
- routing string
- if_primary_term number
- if_seq_no number
- version number
- version_type string
  
  Values are internal, external, external_gte, or force.
- dynamic_templates object
  
  A map from the full name of fields to the name of dynamic templates. It defaults to an empty map. If a name matches a dynamic template, that template will be applied regardless of other match predicates defined in the template. If a field is already defined in the mapping, then this parameter won't be used.
  Hide dynamic_templates attribute Show dynamic_templates attribute object
  
  * string Additional properties
- pipeline string
  
  The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request. If a final pipeline is configured, it will always run regardless of the value of this parameter.
- require_alias boolean
  
  If true, the request's actions must target an index alias.
update object
Hide update attributes Show update attributes object
- _id string
- _index string
- routing string
- if_primary_term number
- if_seq_no number
- version number
- version_type string
  
  Values are internal, external, external_gte, or force.
- require_alias boolean
  
  If true, the request's actions must target an index alias.
- retry_on_conflict number
  
  The number of times an update should be retried in the case of a version conflict.
delete object
Hide delete attributes Show delete attributes object
- _id string
- _index string
- routing string
- if_primary_term number
- if_seq_no number
- version number
- version_type string
  
  Values are internal, external, external_gte, or force.

detect_noop boolean

If true, the result in the response is set to 'noop' when no changes to the document occur.
doc object

A partial update to an existing document.
doc_as_upsert boolean

Set to true to use the contents of doc as the value of upsert.
script object
Hide script attributes Show script attributes object
- source string | object
  
  One of:
  ScriptSource string SearchRequestBody object
  
  Hide attributes Show attributes
  
  aggregations object
  
  Defines the aggregations that are run as part of the search request.
  
  External documentation
  
  collapse object
  External documentation
  
  explain boolean
  
  If true, the request returns detailed information about score computation as part of a hit.
  
  ext object
  
  Configuration of search extensions defined by Elasticsearch plugins.
  
  Hide ext attribute Show ext attribute object
  
  * object Additional properties
  
  from number
  
  The starting document offset, which must be non-negative. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.
  
  highlight object
  
  track_total_hits boolean | number
  
  Number of hits matching the query to count accurately. If true, the exact number of hits is returned at the cost of some performance. If false, the response does not include the total number of hits matching the query. Defaults to 10,000 hits.
  
  indices_boost array[object]
  
  Boost the _score of documents from specified indices. The boost value is the factor by which scores are multiplied. A boost value greater than 1.0 increases the score. A boost value between 0 and 1.0 decreases the score.
  
  External documentation
  
  docvalue_fields array[object]
  
  An array of wildcard (*) field patterns. The request returns doc values for field names matching these patterns in the hits.fields property of the response.
  
  External documentation
  
  knn object | array[object]
  
  The approximate kNN search to run.
  
  One of:
  KnnSearch object array-2 array[object]
  
  rank object
  
  Hide rank attribute Show rank attribute object
  
  rrf
  
  min_score number
  
  The minimum _score for matching documents. Documents with a lower _score are not included in search results or results collected by aggregations.
  
  post_filter object
  
  An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
  
  External documentation
  
  profile boolean
  
  Set to true to return detailed timing information about the execution of individual components in a search request. NOTE: This is a debugging tool and adds significant overhead to search execution.
  
  query object
  
  An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
  
  External documentation
  
  rescore array[object]
  
  retriever object
  
  Hide retriever attributes Show retriever attributes object
  
  standard
  
  knn
  
  rrf
  
  text_similarity_reranker
  
  rule
  
  script_fields object
  
  Retrieve a script evaluation (based on different fields) for each hit.
  
  Hide script_fields attribute Show script_fields attribute object
  
  * object Additional properties
  
  search_after array[number | string | boolean | null]
  
  A field value.
  
  size number
  
  The number of hits to return, which must not be negative. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after property.
  
  slice object
  
  Hide slice attributes Show slice attributes object
  
  field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  id string Required
  
  max number Required
  
  sort array[string | object]
  
  _source boolean | object
  
  Defines how to fetch a source. Fetching can be disabled entirely, or the source can be filtered.
  
  One of:
  SourceConfig boolean SourceFilter object
  
  fields array[object]
  
  An array of wildcard (*) field patterns. The request returns values for field names matching these patterns in the hits.fields property of the response.
  
  suggest object
  
  Hide suggest attribute Show suggest attribute object
  
  text string
  
  Global suggest text, to avoid repetition when the same text is used in several suggesters
  
  terminate_after number
  
  The maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting.
  
  IMPORTANT: Use with caution. Elasticsearch applies this property to each shard handling the request. When possible, let Elasticsearch perform early termination automatically. Avoid specifying this property for requests that target data streams with backing indices across multiple data tiers.
  
  If set to 0 (default), the query does not terminate early.
  
  timeout string
  
  The period of time to wait for a response from each shard. If no response is received before the timeout expires, the request fails and returns an error. Defaults to no timeout.
  
  track_scores boolean
  
  If true, calculate and return document scores, even if the scores are not used for sorting.
  
  version boolean
  
  If true, the request returns the document version as part of a hit.
  
  seq_no_primary_term boolean
  
  If true, the request returns sequence number and primary term of the last modification of each hit.
  
  External documentation
  
  stored_fields string | array[string]
  
  pit object
  
  Hide pit attributes Show pit attributes object
  
  id string Required
  
  keep_alive string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  runtime_mappings object
  
  Hide runtime_mappings attribute Show runtime_mappings attribute object
  
  * object Additional properties
  
  stats array[string]
  
  The stats groups to associate with the search. Each group maintains a statistics aggregation for its associated searches. You can retrieve these stats using the indices stats API.
- id string
- params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  Hide params attribute Show params attribute object
  
  * object Additional properties
- lang string
  
  Any of:
  ScriptLanguage string ScriptLanguage string
  
  Values are painless, expression, mustache, or java.
- options object
  Hide options attribute Show options attribute object
  
  * string Additional properties
scripted_upsert boolean

Set to true to run the script whether or not the document exists.
_source boolean | object

Defines how to fetch a source. Fetching can be disabled entirely, or the source can be filtered.
One of:
SourceConfig boolean SourceFilter object
Hide attributes Show attributes

excludes string | array[string]

includes string | array[string]
upsert object

If the document does not already exist, the contents of upsert are inserted as a new document. If the document exists, the script is run.

Responses

200 application/json
Hide response attributes Show response attributes object
- errors boolean Required
  
  If true, one or more of the operations in the bulk request did not complete successfully.
- items array[object] Required
  
  The result of each operation in the bulk request, in the order they were submitted.
  
  Hide items attribute Show items attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  _id string | null
  
  The document ID associated with the operation.
  
  One of:
  string-1 string string-2 string | null
  
  _index string Required
  
  The name of the index associated with the operation. If the operation targeted a data stream, this is the backing index into which the document was written.
  
  status number Required
  
  The HTTP status code returned for the operation.
  
  failure_store string
  
  Values are not_applicable_or_unknown, used, not_enabled, or failed.
  
  error object
  
  Hide error attributes Show error attributes object
  
  type string Required
  
  The type of error
  
  reason string
  
  A human-readable explanation of the error, in English.
  
  stack_trace string
  
  The server stack trace. Present only if the error_trace=true parameter was sent with the request.
  
  caused_by object
  
  root_cause array[object]
  
  suppressed array[object]
  
  _primary_term number
  
  The primary term assigned to the document for the operation. This property is returned only for successful operations.
  
  result string
  
  The result of the operation. Successful values are created, deleted, and updated.
  
  _seq_no number
  
  _shards object
  
  Hide _shards attributes Show _shards attributes object
  
  failed number Required
  
  successful number Required
  
  total number Required
  
  failures array[object]
  
  skipped number
  
  _version number
  
  forced_refresh boolean
  
  get object
  
  Hide get attributes Show get attributes object
  
  fields object
  
  Hide fields attribute Show fields attribute object
  
  * object Additional properties
  
  found boolean Required
  
  _seq_no number
  
  _primary_term number
  
  _routing string
  
  _source object
  
  Hide _source attribute Show _source attribute object
  
  * object Additional properties
- took number Required
  
  The length of time, in milliseconds, it took to process the bulk request.
- ingest_took number

POST /{index}/_bulk

curl \
 --request POST 'http://api.example.com/{index}/_bulk' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{ \"index\" : { \"_index\" : \"test\", \"_id\" : \"1\" } }\n{ \"field1\" : \"value1\" }\n{ \"delete\" : { \"_index\" : \"test\", \"_id\" : \"2\" } }\n{ \"create\" : { \"_index\" : \"test\", \"_id\" : \"3\" } }\n{ \"field1\" : \"value3\" }\n{ \"update\" : {\"_id\" : \"1\", \"_index\" : \"test\"} }\n{ \"doc\" : {\"field2\" : \"value2\"} }"'

Request examples

Run `POST _bulk` to perform multiple operations.

{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test", "_id" : "2" } }
{ "create" : { "_index" : "test", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_index" : "test"} }
{ "doc" : {"field2" : "value2"} }

When you run `POST _bulk` and use the `update` action, you can use `retry_on_conflict` as a field in the action itself (not in the extra payload line) to specify how many times an update should be retried in the case of a version conflict.

{ "update" : {"_id" : "1", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"} }
{ "update" : { "_id" : "0", "_index" : "index1", "retry_on_conflict" : 3} }
{ "script" : { "source": "ctx._source.counter += params.param1", "lang" : "painless", "params" : {"param1" : 1}}, "upsert" : {"counter" : 1}}
{ "update" : {"_id" : "2", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"}, "doc_as_upsert" : true }
{ "update" : {"_id" : "3", "_index" : "index1", "_source" : true} }
{ "doc" : {"field" : "value"} }
{ "update" : {"_id" : "4", "_index" : "index1"} }
{ "doc" : {"field" : "value"}, "_source": true}

To return only information about failed operations, run `POST /_bulk?filter_path=items.*.error`.

{ "update": {"_id": "5", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "update": {"_id": "6", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "create": {"_id": "7", "_index": "index1"} }
{ "my_field": "foo" }

Run `POST /_bulk` to perform a bulk request that consists of index and create actions with the `dynamic_templates` parameter. The bulk request creates two new fields `work_location` and `home_location` with type `geo_point` according to the `dynamic_templates` parameter. However, the `raw_location` field is created using default dynamic mapping rules, as a text field in that case since it is supplied as a string in the JSON document.

{ "index" : { "_index" : "my_index", "_id" : "1", "dynamic_templates": {"work_location": "geo_point"}} }
{ "field" : "value1", "work_location": "41.12,-71.34", "raw_location": "41.12,-71.34"}
{ "create" : { "_index" : "my_index", "_id" : "2", "dynamic_templates": {"home_location": "geo_point"}} }
{ "field" : "value2", "home_location": "41.12,-71.34"}

Response examples (200)

{
   "took": 30,
   "errors": false,
   "items": [
      {
         "index": {
            "_index": "test",
            "_id": "1",
            "_version": 1,
            "result": "created",
            "_shards": {
               "total": 2,
               "successful": 1,
               "failed": 0
            },
            "status": 201,
            "_seq_no" : 0,
            "_primary_term": 1
         }
      },
      {
         "delete": {
            "_index": "test",
            "_id": "2",
            "_version": 1,
            "result": "not_found",
            "_shards": {
               "total": 2,
               "successful": 1,
               "failed": 0
            },
            "status": 404,
            "_seq_no" : 1,
            "_primary_term" : 2
         }
      },
      {
         "create": {
            "_index": "test",
            "_id": "3",
            "_version": 1,
            "result": "created",
            "_shards": {
               "total": 2,
               "successful": 1,
               "failed": 0
            },
            "status": 201,
            "_seq_no" : 2,
            "_primary_term" : 3
         }
      },
      {
         "update": {
            "_index": "test",
            "_id": "1",
            "_version": 2,
            "result": "updated",
            "_shards": {
                "total": 2,
                "successful": 1,
                "failed": 0
            },
            "status": 200,
            "_seq_no" : 3,
            "_primary_term" : 4
         }
      }
   ]
}

If you run `POST /_bulk` with operations that update non-existent documents, the operations cannot complete successfully. The API returns a response with an `errors` property value `true`. The response also includes an error object for any failed operations. The error object contains additional information about the failure, such as the error type and reason.

{
  "took": 486,
  "errors": true,
  "items": [
    {
      "update": {
        "_index": "index1",
        "_id": "5",
        "status": 404,
        "error": {
          "type": "document_missing_exception",
          "reason": "[5]: document missing",
          "index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
          "shard": "0",
          "index": "index1"
        }
      }
    },
    {
      "update": {
        "_index": "index1",
        "_id": "6",
        "status": 404,
        "error": {
          "type": "document_missing_exception",
          "reason": "[6]: document missing",
          "index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
          "shard": "0",
          "index": "index1"
        }
      }
    },
    {
      "create": {
        "_index": "index1",
        "_id": "7",
        "_version": 1,
        "result": "created",
        "_shards": {
          "total": 2,
          "successful": 1,
          "failed": 0
        },
        "_seq_no": 0,
        "_primary_term": 1,
        "status": 201
      }
    }
  ]
}

An example response from `POST /_bulk?filter_path=items.*.error`, which returns only information about failed operations.

{
  "items": [
    {
      "update": {
        "error": {
          "type": "document_missing_exception",
          "reason": "[5]: document missing",
          "index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
          "shard": "0",
          "index": "index1"
        }
      }
    },
    {
      "update": {
        "error": {
          "type": "document_missing_exception",
          "reason": "[6]: document missing",
          "index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
          "shard": "0",
          "index": "index1"
        }
      }
    }
  ]
}

Create or update a document in an index

POST /{index}/_doc/{id}

Api key auth Basic auth Bearer auth

Add a JSON document to the specified data stream or index and make it searchable. If the target is an index and the document already exists, the request updates the document and increments its version.

NOTE: You cannot use this API to send update requests for existing documents in a data stream.

If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:

To add or overwrite a document using the PUT /<target>/_doc/<_id> request format, you must have the create, index, or write index privilege.
To add a document using the POST /<target>/_doc/ request format, you must have the create_doc, create, index, or write index privilege.
To automatically create a data stream or index with this API request, you must have the auto_configure, create_index, or manage index privilege.

Automatic data stream creation requires a matching index template with data stream enabled.

NOTE: Replica shards might not all be started when an indexing operation returns successfully. By default, only the primary is required. Set wait_for_active_shards to change this default behavior.

Automatically create data streams and indices

If the request's target doesn't exist and matches an index template with a data_stream definition, the index operation automatically creates the data stream.

If the target doesn't exist and doesn't match a data stream template, the operation automatically creates the index and applies any matching index templates.

NOTE: Elasticsearch includes several built-in index templates. To avoid naming collisions with these templates, refer to index pattern documentation.

If no mapping exists, the index operation creates a dynamic mapping. By default, new fields and objects are automatically added to the mapping if needed.

Automatic index creation is controlled by the action.auto_create_index setting. If it is true, any index can be created automatically. You can modify this setting to explicitly allow or block automatic creation of indices that match specified patterns or set it to false to turn off automatic index creation entirely. Specify a comma-separated list of patterns you want to allow or prefix each pattern with + or - to indicate whether it should be allowed or blocked. When a list is specified, the default behaviour is to disallow.

NOTE: The action.auto_create_index setting affects the automatic creation of indices only. It does not affect the creation of data streams.

Optimistic concurrency control

Index operations can be made conditional and only be performed if the last modification to the document was assigned the sequence number and primary term specified by the if_seq_no and if_primary_term parameters. If a mismatch is detected, the operation will result in a VersionConflictException and a status code of 409.

Routing

By default, shard placement — or routing — is controlled by using a hash of the document's ID value. For more explicit control, the value fed into the hash function used by the router can be directly specified on a per-operation basis using the routing parameter.

When setting up explicit mapping, you can also use the _routing field to direct the index operation to extract the routing value from the document itself. This does come at the (very minimal) cost of an additional document parsing pass. If the _routing mapping is defined and set to be required, the index operation will fail if no routing value is provided or extracted.

NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.

Distributed

The index operation is directed to the primary shard based on its route and performed on the actual node containing this shard. After the primary shard completes the operation, if needed, the update is distributed to applicable replicas.

Active shards

To improve the resiliency of writes to the system, indexing operations can be configured to wait for a certain number of active shard copies before proceeding with the operation. If the requisite number of active shard copies are not available, then the write operation must wait and retry, until either the requisite shard copies have started or a timeout occurs. By default, write operations only wait for the primary shards to be active before proceeding (that is to say wait_for_active_shards is 1). This default can be overridden in the index settings dynamically by setting index.write.wait_for_active_shards. To alter this behavior per operation, use the wait_for_active_shards request parameter.

Valid values are all or any positive integer up to the total number of configured copies per shard in the index (which is number_of_replicas+1). Specifying a negative value or a number greater than the number of shard copies will throw an error.

For example, suppose you have a cluster of three nodes, A, B, and C and you create an index index with the number of replicas set to 3 (resulting in 4 shard copies, one more copy than there are nodes). If you attempt an indexing operation, by default the operation will only ensure the primary copy of each shard is available before proceeding. This means that even if B and C went down and A hosted the primary shard copies, the indexing operation would still proceed with only one copy of the data. If wait_for_active_shards is set on the request to 3 (and all three nodes are up), the indexing operation will require 3 active shard copies before proceeding. This requirement should be met because there are 3 active nodes in the cluster, each one holding a copy of the shard. However, if you set wait_for_active_shards to all (or to 4, which is the same in this situation), the indexing operation will not proceed as you do not have all 4 copies of each shard active in the index. The operation will timeout unless a new node is brought up in the cluster to host the fourth copy of the shard.

It is important to note that this setting greatly reduces the chances of the write operation not writing to the requisite number of shard copies, but it does not completely eliminate the possibility, because this check occurs before the write operation starts. After the write operation is underway, it is still possible for replication to fail on any number of shard copies but still succeed on the primary. The _shards section of the API response reveals the number of shard copies on which replication succeeded and failed.

No operation (noop) updates

When updating a document by using this API, a new version of the document is always created even if the document hasn't changed. If this isn't acceptable use the _update API with detect_noop set to true. The detect_noop option isn't available on this API because it doesn’t fetch the old source and isn't able to compare it against the new source.

There isn't a definitive rule for when noop updates aren't acceptable. It's a combination of lots of factors like how frequently your data source sends updates that are actually noops and how many queries per second Elasticsearch runs on the shard receiving the updates.

Versioning

Each indexed document is given a version number. By default, internal versioning is used that starts at 1 and increments with each update, deletes included. Optionally, the version number can be set to an external value (for example, if maintained in a database). To enable this functionality, version_type should be set to external. The value provided must be a numeric, long value greater than or equal to 0, and less than around 9.2e+18.

NOTE: Versioning is completely real time, and is not affected by the near real time aspects of search operations. If no version is provided, the operation runs without any version checks.

When using the external version type, the system checks to see if the version number passed to the index request is greater than the version of the currently stored document. If true, the document will be indexed and the new version number used. If the value provided is less than or equal to the stored document's version number, a version conflict will occur and the index operation will fail. For example:

PUT my-index-000001/_doc/1?version=2&version_type=external
{
  "user": {
    "id": "elkbee"
  }
}

In this example, the operation will succeed since the supplied version of 2 is higher than the current document version of 1.
If the document was already updated and its version was set to 2 or higher, the indexing command will fail and result in a conflict (409 HTTP status code).

A nice side effect is that there is no need to maintain strict ordering of async indexing operations run as a result of changes to a source database, as long as version numbers from the source database are used.
Even the simple case of updating the Elasticsearch index using data from a database is simplified if external versioning is used, as only the latest version will be used if the index operations arrive out of order.

External documentation

Path parameters

index string Required

The name of the data stream or index to target. If the target doesn't exist and matches the name or wildcard (*) pattern of an index template with a data_stream definition, this request creates the data stream. If the target doesn't exist and doesn't match a data stream template, this request creates the index. You can check for existing targets with the resolve index API.
id string Required

A unique identifier for the document. To automatically generate a document ID, use the POST /<target>/_doc/ request format and omit this parameter.

Query parameters

if_primary_term number

Only perform the operation if the document has this primary term.
if_seq_no number

Only perform the operation if the document has this sequence number.
include_source_on_error boolean

True or false if to include the document source in the error message in case of parsing errors.
op_type string
Set to create to only index the document if it does not already exist (put if absent). If a document with the specified _id already exists, the indexing operation will fail. The behavior is the same as using the <index>/_create endpoint. If a document ID is specified, this paramater defaults to index. Otherwise, it defaults to create. If the request targets a data stream, an op_type of create is required.

Supported values include:
- index: Overwrite any documents that already exist.
- create: Only index documents that do not already exist.
Values are index or create.
pipeline string

The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, then setting the value to _none disables the default ingest pipeline for this request. If a final pipeline is configured it will always run, regardless of the value of this parameter.
refresh string

If true, Elasticsearch refreshes the affected shards to make this operation visible to search. If wait_for, it waits for a refresh to make this operation visible to search. If false, it does nothing with refreshes.

Values are true, false, or wait_for.
routing string

A custom value that is used to route operations to a specific shard.
timeout string

The period the request waits for the following operations: automatic index creation, dynamic mapping updates, waiting for active shards.

This parameter is useful for situations where the primary shard assigned to perform the operation might not be available when the operation runs. Some reasons for this might be that the primary shard is currently recovering from a gateway or undergoing relocation. By default, the operation will wait on the primary shard to become available for at least 1 minute before failing and responding with an error. The actual wait time could be longer, particularly when multiple waits occur.
version number

An explicit version number for concurrency control. It must be a non-negative long number.
version_type string
The version type.

Supported values include:
- internal: Use internal versioning that starts at 1 and increments with each update or delete.
- external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.
- external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document. NOTE: The external_gte version type is meant for special use cases and should be used with care. If used incorrectly, it can result in loss of data.
- force: This option is deprecated because it can cause primary and replica shards to diverge.
Values are internal, external, external_gte, or force.
wait_for_active_shards number | string

The number of shard copies that must be active before proceeding with the operation. You can set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). The default value of 1 means it waits for each primary shard to be active.
require_alias boolean

If true, the destination must be an index alias.

application/json

Body Required

object

Responses

200 application/json
Hide response attributes Show response attributes object
- _id string Required
- _index string Required
- _primary_term number
  
  The primary term assigned to the document for the indexing operation.
- result string Required
  
  Values are created, updated, deleted, not_found, or noop.
- _seq_no number
- _shards object Required
  
  Hide _shards attributes Show _shards attributes object
  
  failed number Required
  
  successful number Required
  
  total number Required
  
  failures array[object]
  
  Hide failures attributes Show failures attributes object
  
  index string
  
  node string
  
  reason object Required
  
  Hide reason attributes Show reason attributes object
  
  type string Required
  
  The type of error
  
  reason string
  
  A human-readable explanation of the error, in English.
  
  stack_trace string
  
  The server stack trace. Present only if the error_trace=true parameter was sent with the request.
  
  caused_by object
  
  root_cause array[object]
  
  suppressed array[object]
  
  shard number Required
  
  status string
  
  skipped number
- _version number Required
- forced_refresh boolean

POST /{index}/_doc/{id}

curl \
 --request POST 'http://api.example.com/{index}/_doc/{id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"@timestamp\": \"2099-11-15T13:12:00\",\n  \"message\": \"GET /search HTTP/1.1 200 1070000\",\n  \"user\": {\n    \"id\": \"kimchy\"\n  }\n}"'

Request examples

Run `POST my-index-000001/_doc/` to index a document. When you use the `POST /<target>/_doc/` request format, the `op_type` is automatically set to `create` and the index operation generates a unique ID for the document.

{
  "@timestamp": "2099-11-15T13:12:00",
  "message": "GET /search HTTP/1.1 200 1070000",
  "user": {
    "id": "kimchy"
  }
}

Run `PUT my-index-000001/_doc/1` to insert a JSON document into the `my-index-000001` index with an `_id` of 1.

{
  "@timestamp": "2099-11-15T13:12:00",
  "message": "GET /search HTTP/1.1 200 1070000",
  "user": {
    "id": "kimchy"
  }
}

Response examples (200)

A successful response from `POST my-index-000001/_doc/`, which contains an automated document ID.

{
  "_shards": {
    "total": 2,
    "failed": 0,
    "successful": 2
  },
  "_index": "my-index-000001",
  "_id": "W0tpsmIBdwcYyG50zbta",
  "_version": 1,
  "_seq_no": 0,
  "_primary_term": 1,
  "result": "created"
}

A successful response from `PUT my-index-000001/_doc/1`.

{
  "_shards": {
    "total": 2,
    "failed": 0,
    "successful": 2
  },
  "_index": "my-index-000001",
  "_id": "1",
  "_version": 1,
  "_seq_no": 0,
  "_primary_term": 1,
  "result": "created"
}

Run an async ES|QL query Added in 8.13.0

POST /_query/async

Api key auth Basic auth Bearer auth

Asynchronously run an ES|QL (Elasticsearch query language) query, monitor its progress, and retrieve results when they become available.

The API accepts the same parameters and request body as the synchronous query API, along with additional async related properties.

External documentation

Query parameters

delimiter string

The character to use between values within a CSV row. It is valid only for the CSV format.
drop_null_columns boolean

Indicates whether columns that are entirely null will be removed from the columns and values portion of the results. If true, the response will include an extra section under the name all_columns which has the name of all the columns.
format string

A short version of the Accept header, for example json or yaml.

Values are csv, json, tsv, txt, yaml, cbor, smile, or arrow.
keep_alive string

The period for which the query and its results are stored in the cluster. The default period is five days. When this period expires, the query and its results are deleted, even if the query is still ongoing. If the keep_on_completion parameter is false, Elasticsearch only stores async queries that do not complete within the period set by the wait_for_completion_timeout parameter, regardless of this value.
keep_on_completion boolean

Indicates whether the query and its results are stored in the cluster. If false, the query and its results are stored in the cluster only if the request does not complete during the period set by the wait_for_completion_timeout parameter.
wait_for_completion_timeout string

The period to wait for the request to finish. By default, the request waits for 1 second for the query results. If the query completes during this period, results are returned Otherwise, a query ID is returned that can later be used to retrieve the results.

application/json

Body Required

columnar boolean

By default, ES|QL returns results as rows. For example, FROM returns each individual document as one row. For the JSON, YAML, CBOR and smile formats, ES|QL can return the results in a columnar fashion where one row represents all the values of a certain column in the results.
filter object

An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

External documentation
locale string
params array[number | string | boolean | null]

To avoid any attempts of hacking or code injection, extract the values in a separate list of parameters. Use question mark placeholders (?) in the query string for each of the parameters.
profile boolean

If provided and true the response will include an extra profile object with information on how the query was executed. This information is for human debugging and its format can change at any time but it can give some insight into the performance of each part of the query.
query string Required

The ES|QL query API accepts an ES|QL query string in the query parameter, runs it, and returns the results.
tables object

Tables to use with the LOOKUP operation. The top level key is the table name and the next level key is the column name.
Hide tables attribute Show tables attribute object
- * object Additional properties
  Hide * attribute Show * attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  integer array[number | array]
  
  One of:
  TableValuesIntegerValue number TableValuesIntegerValue array[number]
  
  keyword array[string | array]
  
  One of:
  TableValuesKeywordValue string TableValuesKeywordValue array[string]
  
  long array[number | array]
  
  One of:
  TableValuesLongValue number TableValuesLongValue array[number]
  
  double array[number | array]
  
  One of:
  TableValuesLongDouble number TableValuesLongDouble array[number]
include_ccs_metadata boolean

When set to true and performing a cross-cluster query, the response will include an extra _clusters object with information about the clusters that participated in the search along with info such as shards count.
wait_for_completion_timeout string

A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

Responses

200 application/json
Hide response attributes Show response attributes object
- Time unit for milliseconds
- is_partial boolean
- all_columns array[object]
  
  Hide all_columns attributes Show all_columns attributes object
  
  name string Required
  
  type string Required
- columns array[object] Required
  
  Hide columns attributes Show columns attributes object
  
  name string Required
  
  type string Required
- values array[array] Required
  
  A field value.
  
  A field value.
- _clusters object
  
  Hide _clusters attributes Show _clusters attributes object
  
  total number Required
  
  successful number Required
  
  running number Required
  
  skipped number Required
  
  partial number Required
  
  failed number Required
  
  details object Required
  
  Hide details attribute Show details attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  status string Required
  
  Values are running, successful, partial, skipped, or failed.
  
  indices string Required
  
  Time unit for milliseconds
  
  _shards object
  
  Hide _shards attributes Show _shards attributes object
  
  total number Required
  
  successful number
  
  skipped number
  
  failed number
  
  failures array[object]
- profile object
  
  Profiling information. Present if profile was true in the request. The contents of this field are currently unstable.
- id string
- is_running boolean Required

POST /_query/async

curl \
 --request POST 'http://api.example.com/_query/async' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"query\": \"\"\"\n    FROM library,remote-*:library\n    | EVAL year = DATE_TRUNC(1 YEARS, release_date)\n    | STATS MAX(page_count) BY year\n    | SORT year\n    | LIMIT 5\n  \"\"\",\n  \"wait_for_completion_timeout\": \"2s\",\n  \"include_ccs_metadata\": true\n}"'

Request example

{
  "query": """
    FROM library,remote-*:library
    | EVAL year = DATE_TRUNC(1 YEARS, release_date)
    | STATS MAX(page_count) BY year
    | SORT year
    | LIMIT 5
  """,
  "wait_for_completion_timeout": "2s",
  "include_ccs_metadata": true
}

Run multiple Fleet searches Technical preview

GET /{index}/_fleet/_fleet_msearch

Api key auth Basic auth Bearer auth

Run several Fleet searches with a single API request. The API follows the same structure as the multi search API. However, similar to the Fleet search API, it supports the wait_for_checkpoints parameter.

Path parameters

index string Required

A single target to search. If the target is an index alias, it must resolve to a single index.

Query parameters

allow_no_indices boolean

If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.
ccs_minimize_roundtrips boolean

If true, network roundtrips between the coordinating node and remote clusters are minimized for cross-cluster search requests.
expand_wildcards string | array[string]
Type of index that wildcard expressions can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.

Supported values include:
- all: Match any data stream or index, including hidden ones.
- open: Match open, non-hidden indices. Also matches any non-hidden data stream.
- closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
- hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
- none: Wildcard expressions are not accepted.
ignore_throttled boolean

If true, concrete, expanded or aliased indices are ignored when frozen.
ignore_unavailable boolean

If true, missing or closed indices are not included in the response.
max_concurrent_searches number

Maximum number of concurrent searches the multi search API can execute.
max_concurrent_shard_requests number

Maximum number of concurrent shard requests that each sub-search request executes per node.
pre_filter_shard_size number

Defines a threshold that enforces a pre-filter roundtrip to prefilter search shards based on query rewriting if the number of shards the search request expands to exceeds the threshold. This filter roundtrip can limit the number of shards significantly if for instance a shard can not match any documents based on its rewrite method i.e., if date filters are mandatory to match but the shard bounds and the query are disjoint.
search_type string
Indicates whether global term and document frequencies should be used when scoring returned documents.

Supported values include:
- query_then_fetch: Documents are scored using local term and document frequencies for the shard. This is usually faster but less accurate.
- dfs_query_then_fetch: Documents are scored using global term and document frequencies across all shards. This is usually slower but more accurate.
Values are query_then_fetch or dfs_query_then_fetch.
rest_total_hits_as_int boolean

If true, hits.total are returned as an integer in the response. Defaults to false, which returns an object.
typed_keys boolean

Specifies whether aggregation and suggester names should be prefixed by their respective types in the response.
wait_for_checkpoints array[number]

A comma separated list of checkpoints. When configured, the search API will only be executed on a shard after the relevant checkpoint has become visible for search. Defaults to an empty list which will cause Elasticsearch to immediately execute the search.
allow_partial_search_results boolean

If true, returns partial results if there are shard request timeouts or shard failures. If false, returns an error with no partial results. Defaults to the configured cluster setting search.default_allow_partial_results, which is true by default.

application/json

Body object Required

allow_no_indices boolean
expand_wildcards string | array[string]
ignore_unavailable boolean
index string | array[string]
preference string
request_cache boolean
routing string
search_type string

Values are query_then_fetch or dfs_query_then_fetch.
ccs_minimize_roundtrips boolean
allow_partial_search_results boolean
ignore_throttled boolean

aggregations object

Defines the aggregations that are run as part of the search request.

External documentation
collapse object
External documentation
explain boolean

If true, the request returns detailed information about score computation as part of a hit.
ext object

Configuration of search extensions defined by Elasticsearch plugins.
Hide ext attribute Show ext attribute object
- * object Additional properties
from number

The starting document offset, which must be non-negative. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.
highlight object
Hide highlight attributes Show highlight attributes object
- type string
  
  Any of:
  HighlighterType string HighlighterType string
  
  Values are plain, fvh, or unified.
- boundary_chars string
  
  A string that contains each boundary character.
- boundary_max_scan number
  
  How far to scan for boundary characters.
- boundary_scanner string
  
  Values are chars, sentence, or word.
- boundary_scanner_locale string
  
  Controls which locale is used to search for sentence and word boundaries. This parameter takes a form of a language tag, for example: "en-US", "fr-FR", "ja-JP".
- force_source boolean Deprecated
- fragmenter string
  
  Values are simple or span.
- fragment_size number
  
  The size of the highlighted fragment in characters.
- highlight_filter boolean
- highlight_query object
  
  An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
  
  External documentation
- max_fragment_length number
- max_analyzed_offset number
  
  If set to a non-negative value, highlighting stops at this defined maximum limit. The rest of the text is not processed, thus not highlighted and no error is returned The max_analyzed_offset query setting does not override the index.highlight.max_analyzed_offset setting, which prevails when it’s set to lower value than the query setting.
- no_match_size number
  
  The amount of text you want to return from the beginning of the field if there are no matching fragments to highlight.
- number_of_fragments number
  
  The maximum number of fragments to return. If the number of fragments is set to 0, no fragments are returned. Instead, the entire field contents are highlighted and returned. This can be handy when you need to highlight short texts such as a title or address, but fragmentation is not required. If number_of_fragments is 0, fragment_size is ignored.
- options object
  Hide options attribute Show options attribute object
  
  * object Additional properties
- order string
  
  Value is score.
- phrase_limit number
  
  Controls the number of matching phrases in a document that are considered. Prevents the fvh highlighter from analyzing too many phrases and consuming too much memory. When using matched_fields, phrase_limit phrases per matched field are considered. Raising the limit increases query time and consumes more memory. Only supported by the fvh highlighter.
- post_tags array[string]
  
  Use in conjunction with pre_tags to define the HTML tags to use for the highlighted text. By default, highlighted text is wrapped in <em> and </em> tags.
- pre_tags array[string]
  
  Use in conjunction with post_tags to define the HTML tags to use for the highlighted text. By default, highlighted text is wrapped in <em> and </em> tags.
- require_field_match boolean
  
  By default, only fields that contains a query match are highlighted. Set to false to highlight all fields.
- tags_schema string
  
  Value is styled.
- encoder string
  
  Values are default or html.
- fields object Required
track_total_hits boolean | number

Number of hits matching the query to count accurately. If true, the exact number of hits is returned at the cost of some performance. If false, the response does not include the total number of hits matching the query. Defaults to 10,000 hits.
indices_boost array[object]

Boost the _score of documents from specified indices. The boost value is the factor by which scores are multiplied. A boost value greater than 1.0 increases the score. A boost value between 0 and 1.0 decreases the score.

External documentation
Hide indices_boost attribute Show indices_boost attribute object
- * number Additional properties
docvalue_fields array[object]

An array of wildcard (*) field patterns. The request returns doc values for field names matching these patterns in the hits.fields property of the response.

External documentation
Hide docvalue_fields attributes Show docvalue_fields attributes object
- field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
- format string
  
  The format in which the values are returned.
- include_unmapped boolean
knn object | array[object]

The approximate kNN search to run.
One of:
KnnSearch object array-2 array[object]
Hide attributes Show attributes

field string Required

Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

query_vector array[number]

query_vector_builder object

Hide query_vector_builder attribute Show query_vector_builder attribute object

text_embedding object

Hide text_embedding attributes Show text_embedding attributes object

model_id string Required

model_text string Required

k number

The final number of nearest neighbors to return as top hits

num_candidates number

The number of nearest neighbor candidates to consider per shard

boost number

Boost value to apply to kNN scores

filter object | array[object]

Filters for the kNN search query

One of:
QueryContainer object array-2 array[object]

An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

External documentation

similarity number

The minimum similarity for a vector to be considered a match

inner_hits object

Hide inner_hits attributes Show inner_hits attributes object

name string

size number

The maximum number of hits to return per inner_hits.

from number

Inner hit starting document offset.

collapse object
External documentation

docvalue_fields array[object]

explain boolean

highlight object

ignore_unmapped boolean

script_fields object

Hide script_fields attribute Show script_fields attribute object

* object Additional properties

seq_no_primary_term boolean

fields array[string]

Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

sort array[string | object]

_source boolean | object

Defines how to fetch a source. Fetching can be disabled entirely, or the source can be filtered.

One of:
SourceConfig boolean SourceFilter object

stored_fields string | array[string]

track_scores boolean

version boolean

rescore_vector object

Hide rescore_vector attribute Show rescore_vector attribute object

oversample number Required

Applies the specified oversample factor to k on the approximate kNN search
External documentation
Hide attributes Show attributes object

field string Required

Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

query_vector array[number]

query_vector_builder object

Hide query_vector_builder attribute Show query_vector_builder attribute object

text_embedding object

k number

The final number of nearest neighbors to return as top hits

num_candidates number

The number of nearest neighbor candidates to consider per shard

boost number

Boost value to apply to kNN scores

filter object | array[object]

Filters for the kNN search query

One of:
QueryContainer object array-2 array[object]

An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

similarity number

The minimum similarity for a vector to be considered a match

inner_hits object

Hide inner_hits attributes Show inner_hits attributes object

name string

size number

The maximum number of hits to return per inner_hits.

from number

Inner hit starting document offset.

collapse object

docvalue_fields array[object]

explain boolean

highlight

ignore_unmapped boolean

script_fields object

seq_no_primary_term boolean

fields array[string]

Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

sort

_source

stored_fields string | array[string]

track_scores boolean

version boolean

rescore_vector object

Hide rescore_vector attribute Show rescore_vector attribute object

oversample number Required

Applies the specified oversample factor to k on the approximate kNN search
rank object
Hide rank attribute Show rank attribute object
- rrf object
  Hide rrf attributes Show rrf attributes object
  
  rank_constant number
  
  How much influence documents in individual result sets per query have over the final ranked result set
  
  rank_window_size number
  
  Size of the individual result sets per query
min_score number

The minimum _score for matching documents. Documents with a lower _score are not included in search results or results collected by aggregations.
post_filter object

An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

External documentation
profile boolean

Set to true to return detailed timing information about the execution of individual components in a search request. NOTE: This is a debugging tool and adds significant overhead to search execution.
query object

An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

External documentation
rescore object | array[object]

Can be used to improve precision by reordering just the top (for example 100 - 500) documents returned by the query and post_filter phases.
One of:
Rescore object array-2 array[object]
Hide attributes Show attributes

window_size number

query object

Hide query attributes Show query attributes object

rescore_query object Required

An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

query_weight number

Relative importance of the original query versus the rescore query.

rescore_query_weight number

Relative importance of the rescore query versus the original query.

score_mode string

Values are avg, max, min, multiply, or total.

learning_to_rank object

Hide learning_to_rank attributes Show learning_to_rank attributes object

model_id string Required

The unique identifier of the trained model uploaded to Elasticsearch

params object

Named parameters to be passed to the query templates used for feature
Hide attributes Show attributes object

window_size number

query object

learning_to_rank object
retriever object
Hide retriever attributes Show retriever attributes object
- standard object
  Hide standard attributes Show standard attributes object
  
  filter object | array[object]
  
  Query to filter the documents that can match.
  
  One of:
  QueryContainer object array-2 array[object]
  
  An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
  
  min_score number
  
  Minimum _score for matching documents. Documents with a lower _score are not included in the top documents.
  
  query object
  
  An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
  
  External documentation
  
  search_after array[number | string | boolean | null]
  
  A field value.
  
  terminate_after number
  
  Maximum number of documents to collect for each shard.
  
  sort array[string | object]
  
  collapse object
  External documentation
- knn object
  Hide knn attributes Show knn attributes object
  
  filter object | array[object]
  
  Query to filter the documents that can match.
  
  One of:
  QueryContainer object array-2 array[object]
  
  An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
  
  min_score number
  
  Minimum _score for matching documents. Documents with a lower _score are not included in the top documents.
  
  field string Required
  
  The name of the vector field to search against.
  
  query_vector array[number]
  
  query_vector_builder object
  
  Hide query_vector_builder attribute Show query_vector_builder attribute object
  
  text_embedding object
  
  k number Required
  
  Number of nearest neighbors to return as top hits.
  
  num_candidates number Required
  
  Number of nearest neighbor candidates to consider per shard.
  
  similarity number
  
  The minimum similarity required for a document to be considered a match.
  
  rescore_vector object
  
  Hide rescore_vector attribute Show rescore_vector attribute object
  
  oversample number Required
  
  Applies the specified oversample factor to k on the approximate kNN search
- rrf object
  Hide rrf attributes Show rrf attributes object
  
  filter object | array[object]
  
  Query to filter the documents that can match.
  
  One of:
  QueryContainer object array-2 array[object]
  
  An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
  
  min_score number
  
  Minimum _score for matching documents. Documents with a lower _score are not included in the top documents.
  
  retrievers array[object] Required
  
  A list of child retrievers to specify which sets of returned top documents will have the RRF formula applied to them.
  
  rank_constant number
  
  This value determines how much influence documents in individual result sets per query have over the final ranked result set.
  
  rank_window_size number
  
  This value determines the size of the individual result sets per query.
- text_similarity_reranker object
  Hide text_similarity_reranker attributes Show text_similarity_reranker attributes object
  
  filter object | array[object]
  
  Query to filter the documents that can match.
  
  One of:
  QueryContainer object array-2 array[object]
  
  An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
  
  min_score number
  
  Minimum _score for matching documents. Documents with a lower _score are not included in the top documents.
  
  retriever object Required
  
  rank_window_size number
  
  This value determines how many documents we will consider from the nested retriever.
  
  inference_id string
  
  Unique identifier of the inference endpoint created using the inference API.
  
  inference_text string
  
  The text snippet used as the basis for similarity comparison
  
  field string
  
  The document field to be used for text similarity comparisons. This field should contain the text that will be evaluated against the inference_text
- rule object
  Hide rule attributes Show rule attributes object
  
  filter object | array[object]
  
  Query to filter the documents that can match.
  
  One of:
  QueryContainer object array-2 array[object]
  
  An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
  
  min_score number
  
  Minimum _score for matching documents. Documents with a lower _score are not included in the top documents.
  
  ruleset_ids array[string] Required
  
  The ruleset IDs containing the rules this retriever is evaluating against.
  
  match_criteria object Required
  
  The match criteria that will determine if a rule in the provided rulesets should be applied.
  
  retriever object Required
  
  rank_window_size number
  
  This value determines the size of the individual result set.
script_fields object

Retrieve a script evaluation (based on different fields) for each hit.
Hide script_fields attribute Show script_fields attribute object
- * object Additional properties
  Hide * attributes Show * attributes object
  
  script object Required
  
  Hide script attributes Show script attributes object
  
  source string | object
  
  One of:
  ScriptSource string SearchRequestBody object
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  Hide params attribute Show params attribute object
  
  * object Additional properties
  
  lang string
  
  Any of:
  ScriptLanguage string ScriptLanguage string
  
  Values are painless, expression, mustache, or java.
  
  options object
  
  Hide options attribute Show options attribute object
  
  * string Additional properties
  
  ignore_failure boolean
search_after array[number | string | boolean | null]

A field value.
size number

The number of hits to return, which must not be negative. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after property.
slice object
Hide slice attributes Show slice attributes object
- field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
- id string Required
- max number Required
sort string | object | array[string | object]
One of:
Field string SortOptions object Sort array[string | object]

Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
Hide attributes Show attributes

_score object

Hide _score attribute Show _score attribute object

order string

Values are asc or desc.

_doc object

Hide _doc attribute Show _doc attribute object

order string

Values are asc or desc.

_geo_distance object

Hide _geo_distance attributes Show _geo_distance attributes object

mode string

Values are min, max, sum, avg, or median.

distance_type string

Values are arc or plane.

ignore_unmapped boolean

order string

Values are asc or desc.

unit string

Values are in, ft, yd, mi, nmi, km, m, cm, or mm.

nested object

_script object

Hide _script attributes Show _script attributes object

order string

Values are asc or desc.

script object Required

type string

Values are string, number, or version.

mode string

Values are min, max, sum, avg, or median.

nested object
One of:
Field string SortOptions object

Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
_source boolean | object

Defines how to fetch a source. Fetching can be disabled entirely, or the source can be filtered.
One of:
SourceConfig boolean SourceFilter object
Hide attributes Show attributes

excludes string | array[string]

includes string | array[string]
fields array[object]

An array of wildcard (*) field patterns. The request returns values for field names matching these patterns in the hits.fields property of the response.
Hide fields attributes Show fields attributes object
- field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
- format string
  
  The format in which the values are returned.
- include_unmapped boolean
suggest object
Hide suggest attribute Show suggest attribute object
- text string
  
  Global suggest text, to avoid repetition when the same text is used in several suggesters
terminate_after number

The maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting.

IMPORTANT: Use with caution. Elasticsearch applies this property to each shard handling the request. When possible, let Elasticsearch perform early termination automatically. Avoid specifying this property for requests that target data streams with backing indices across multiple data tiers.

If set to 0 (default), the query does not terminate early.
timeout string

The period of time to wait for a response from each shard. If no response is received before the timeout expires, the request fails and returns an error. Defaults to no timeout.
track_scores boolean

If true, calculate and return document scores, even if the scores are not used for sorting.
version boolean

If true, the request returns the document version as part of a hit.
seq_no_primary_term boolean

If true, the request returns sequence number and primary term of the last modification of each hit.

External documentation
stored_fields string | array[string]
pit object
Hide pit attributes Show pit attributes object
- id string Required
- keep_alive string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
runtime_mappings object
Hide runtime_mappings attribute Show runtime_mappings attribute object
- * object Additional properties
  Hide * attributes Show * attributes object
  
  fields object
  
  For type composite
  
  Hide fields attribute Show fields attribute object
  
  * object Additional properties
  
  Hide * attribute Show * attribute object
  
  type string Required
  
  Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.
  
  fetch_fields array[object]
  
  For type lookup
  
  Hide fetch_fields attributes Show fetch_fields attributes object
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  format string
  
  format string
  
  A custom format for date type runtime fields.
  
  input_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_index string
  
  script object
  
  Hide script attributes Show script attributes object
  
  source string | object
  
  One of:
  ScriptSource string SearchRequestBody object
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  Hide params attribute Show params attribute object
  
  * object Additional properties
  
  lang string
  
  Any of:
  ScriptLanguage string ScriptLanguage string
  
  Values are painless, expression, mustache, or java.
  
  options object
  
  Hide options attribute Show options attribute object
  
  * string Additional properties
  
  type string Required
  
  Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.
stats array[string]

The stats groups to associate with the search. Each group maintains a statistics aggregation for its associated searches. You can retrieve these stats using the indices stats API.

Responses

200 application/json
Hide response attribute Show response attribute object
- docs array[object] Required
  
  One of:
  MultiSearchItem object ErrorResponseBase object
  
  Hide attributes Show attributes
  
  took number Required
  
  The number of milliseconds it took Elasticsearch to run the request. This value is calculated by measuring the time elapsed between receipt of a request on the coordinating node and the time at which the coordinating node is ready to send the response. It includes:
  
  Communication time between the coordinating node and data nodes
  
  Time the request spends in the search thread pool, queued for execution
  
  Actual run time
  
  It does not include:
  
  Time needed to send the request to Elasticsearch
  
  Time needed to serialize the JSON response
  
  Time needed to send the response to a client
  
  timed_out boolean Required
  
  If true, the request timed out before completion; returned results may be partial or empty.
  
  _shards object Required
  
  Hide _shards attributes Show _shards attributes object
  
  failed number Required
  
  successful number Required
  
  total number Required
  
  failures array[object]
  
  skipped number
  
  hits object Required
  
  Hide hits attributes Show hits attributes object
  
  total
  
  hits array[object] Required
  
  max_score
  
  aggregations object
  
  _clusters object
  
  Hide _clusters attributes Show _clusters attributes object
  
  skipped number Required
  
  successful number Required
  
  total number Required
  
  running number Required
  
  partial number Required
  
  failed number Required
  
  details object
  
  fields object
  
  Hide fields attribute Show fields attribute object
  
  * object Additional properties
  
  max_score number
  
  num_reduce_phases number
  
  profile object
  
  Hide profile attribute Show profile attribute object
  
  shards array[object] Required
  
  pit_id string
  
  _scroll_id string
  
  suggest object
  
  Hide suggest attribute Show suggest attribute object
  
  * array[object] Additional properties
  
  terminated_early boolean
  
  status number
  
  Hide attributes Show attributes
  
  error object Required
  
  Hide error attributes Show error attributes object
  
  type string Required
  
  The type of error
  
  reason string
  
  A human-readable explanation of the error, in English.
  
  stack_trace string
  
  The server stack trace. Present only if the error_trace=true parameter was sent with the request.
  
  caused_by object
  
  root_cause array[object]
  
  suppressed array[object]
  
  status number Required

GET /{index}/_fleet/_fleet_msearch

curl \
 --request GET 'http://api.example.com/{index}/_fleet/_fleet_msearch' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '[{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"index":"string","preference":"string","request_cache":true,"routing":"string","search_type":"query_then_fetch","ccs_minimize_roundtrips":true,"allow_partial_search_results":true,"ignore_throttled":true}]'

Explore graph analytics

POST /{index}/_graph/explore

Api key auth Basic auth Bearer auth

Extract and summarize information about the documents and terms in an Elasticsearch data stream or index. The easiest way to understand the behavior of this API is to use the Graph UI to explore connections. An initial request to the _explore API contains a seed query that identifies the documents of interest and specifies the fields that define the vertices and connections you want to include in the graph. Subsequent requests enable you to spider out from one more vertices of interest. You can exclude vertices that have already been returned.

External documentation

Path parameters

index string | array[string] Required

Name of the index.

Query parameters

routing string

Custom value used to route operations to a specific shard.
timeout string

Specifies the period of time to wait for a response from each shard. If no response is received before the timeout expires, the request fails and returns an error. Defaults to no timeout.

application/json

Body

connections object
Hide connections attributes Show connections attributes object
- connections object
- query object
  
  An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
  
  External documentation
- vertices array[object] Required
  
  Contains the fields you are interested in.
  Hide vertices attributes Show vertices attributes object
  
  exclude array[string]
  
  Prevents the specified terms from being included in the results.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  include array[object]
  
  Identifies the terms of interest that form the starting points from which you want to spider out.
  
  Hide include attributes Show include attributes object
  
  boost number
  
  term string Required
  
  min_doc_count number
  
  Specifies how many documents must contain a pair of terms before it is considered to be a useful connection. This setting acts as a certainty threshold.
  
  shard_min_doc_count number
  
  Controls how many documents on a particular shard have to contain a pair of terms before the connection is returned for global consideration.
  
  size number
  
  Specifies the maximum number of vertex terms returned for each field.
controls object
Hide controls attributes Show controls attributes object
- sample_diversity object
  Hide sample_diversity attributes Show sample_diversity attributes object
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  max_docs_per_value number Required
- sample_size number
  
  Each hop considers a sample of the best-matching documents on each shard. Using samples improves the speed of execution and keeps exploration focused on meaningfully-connected terms. Very small values (less than 50) might not provide sufficient weight-of-evidence to identify significant connections between terms. Very large sample sizes can dilute the quality of the results and increase execution times.
- timeout string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
- use_significance boolean Required
  
  Filters associated terms so only those that are significantly associated with your query are included.
query object

An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

External documentation
vertices array[object]

Specifies one or more fields that contain the terms you want to include in the graph as vertices.
Hide vertices attributes Show vertices attributes object
- exclude array[string]
  
  Prevents the specified terms from being included in the results.
- field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
- include array[object]
  
  Identifies the terms of interest that form the starting points from which you want to spider out.
  Hide include attributes Show include attributes object
  
  boost number
  
  term string Required
- min_doc_count number
  
  Specifies how many documents must contain a pair of terms before it is considered to be a useful connection. This setting acts as a certainty threshold.
- shard_min_doc_count number
  
  Controls how many documents on a particular shard have to contain a pair of terms before the connection is returned for global consideration.
- size number
  
  Specifies the maximum number of vertex terms returned for each field.

Responses

200 application/json
Hide response attributes Show response attributes object
- connections array[object] Required
  
  Hide connections attributes Show connections attributes object
  
  doc_count number Required
  
  source number Required
  
  target number Required
  
  weight number Required
- failures array[object] Required
  
  Hide failures attributes Show failures attributes object
  
  index string
  
  node string
  
  reason object Required
  
  Hide reason attributes Show reason attributes object
  
  type string Required
  
  The type of error
  
  reason string
  
  A human-readable explanation of the error, in English.
  
  stack_trace string
  
  The server stack trace. Present only if the error_trace=true parameter was sent with the request.
  
  caused_by object
  
  root_cause array[object]
  
  suppressed array[object]
  
  shard number Required
  
  status string
- timed_out boolean Required
- took number Required
- vertices array[object] Required
  
  Hide vertices attributes Show vertices attributes object
  
  depth number Required
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  term string Required
  
  weight number Required

POST /{index}/_graph/explore

curl \
 --request POST 'http://api.example.com/{index}/_graph/explore' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"query\": {\n    \"match\": {\n      \"query.raw\": \"midi\"\n    }\n  },\n  \"vertices\": [\n    {\n      \"field\": \"product\"\n    }\n  ],\n  \"connections\": {\n    \"vertices\": [\n      {\n        \"field\": \"query.raw\"\n      }\n    ]\n  }\n}"'

Request example

Run `POST clicklogs/_graph/explore` for a basic exploration An initial graph explore query typically begins with a query to identify strongly related terms. Seed the exploration with a query. This example is searching `clicklogs` for people who searched for the term `midi`.Identify the vertices to include in the graph. This example is looking for product codes that are significantly associated with searches for `midi`. Find the connections. This example is looking for other search terms that led people to click on the products that are associated with searches for `midi`.

{
  "query": {
    "match": {
      "query.raw": "midi"
    }
  },
  "vertices": [
    {
      "field": "product"
    }
  ],
  "connections": {
    "vertices": [
      {
        "field": "query.raw"
      }
    ]
  }
}

Get field usage stats Technical preview

GET /{index}/_field_usage_stats

Api key auth Basic auth Bearer auth

Get field usage information for each shard and field of an index. Field usage statistics are automatically captured when queries are running on a cluster. A shard-level search request that accesses a given field, even if multiple times during that request, is counted as a single use.

The response body reports the per-shard usage count of the data structures that back the fields in the index. A given request will increment each count by a maximum value of 1, even if the request accesses the same field multiple times.

Path parameters

index string | array[string] Required

Comma-separated list or wildcard expression of index names used to limit the request.

Query parameters

allow_no_indices boolean

If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.
expand_wildcards string | array[string]
Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

Supported values include:
- all: Match any data stream or index, including hidden ones.
- open: Match open, non-hidden indices. Also matches any non-hidden data stream.
- closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
- hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
- none: Wildcard expressions are not accepted.
ignore_unavailable boolean

If true, missing or closed indices are not included in the response.
fields string | array[string]

Comma-separated list or wildcard expressions of fields to include in the statistics.

Responses

200 application/json
Hide response attribute Show response attribute object
- _shards object Required
  
  Hide _shards attributes Show _shards attributes object
  
  failed number Required
  
  successful number Required
  
  total number Required
  
  failures array[object]
  
  Hide failures attributes Show failures attributes object
  
  index string
  
  node string
  
  reason object Required
  
  Hide reason attributes Show reason attributes object
  
  type string Required
  
  The type of error
  
  reason string
  
  A human-readable explanation of the error, in English.
  
  stack_trace string
  
  The server stack trace. Present only if the error_trace=true parameter was sent with the request.
  
  caused_by object
  
  root_cause array[object]
  
  suppressed array[object]
  
  shard number Required
  
  status string
  
  skipped number

GET /{index}/_field_usage_stats

curl \
 --request GET 'http://api.example.com/{index}/_field_usage_stats' \
 --header "Authorization: $API_KEY"

Response examples (200)

An abbreviated response from `GET /my-index-000001/_field_usage_stats`. The `all_fields` object reports the sums of the usage counts for all fields in the index (on the listed shard).

{
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "my-index-000001": {
    "shards": [
      {
        "tracking_id": "MpOl0QlTQ4SYYhEe6KgJoQ",
        "tracking_started_at_millis": 1625558985010,
        "routing": {
          "state": "STARTED",
          "primary": true,
          "node": "gA6KeeVzQkGURFCUyV-e8Q",
          "relocating_node": null
        },
        "stats": {
          "all_fields": {
            "any": "6",
            "inverted_index": {
              "terms": 1,
              "postings": 1,
              "proximity": 1,
              "positions": 0,
              "term_frequencies": 1,
              "offsets": 0,
              "payloads": 0
            },
            "stored_fields": 2,
            "doc_values": 1,
            "points": 0,
            "norms": 1,
            "term_vectors": 0,
            "knn_vectors": 0
          },
          "fields": {
            "_id": {
              "any": 1,
              "inverted_index": {
                "terms": 1,
                "postings": 1,
                "proximity": 1,
                "positions": 0,
                "term_frequencies": 1,
                "offsets": 0,
                "payloads": 0
              },
              "stored_fields": 1,
              "doc_values": 0,
              "points": 0,
              "norms": 0,
              "term_vectors": 0,
              "knn_vectors": 0
            },
            "_source": {},
            "context": {},
            "message.keyword": {}
          }
        }
      }
    ]
  }
}

Get index templates

GET /_template

Api key auth Basic auth Bearer auth

Get information about one or more index templates.

IMPORTANT: This documentation is about legacy index templates, which are deprecated and will be replaced by the composable templates introduced in Elasticsearch 7.8.

External documentation

Query parameters

flat_settings boolean

If true, returns settings in flat format.
local boolean

If true, the request retrieves information from the local node only.
master_timeout string

Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

Responses

200 application/json
Hide response attribute Show response attribute object
- * object Additional properties
  
  Hide * attributes Show * attributes object
  
  aliases object Required
  
  Hide aliases attribute Show aliases attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  filter object
  
  An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
  
  External documentation
  
  index_routing string
  
  is_hidden boolean
  
  If true, the alias is hidden. All indices for the alias must have the same is_hidden value.
  
  is_write_index boolean
  
  If true, the index is the write index for the alias.
  
  routing string
  
  search_routing string
  
  index_patterns array[string] Required
  
  mappings object Required
  
  Hide mappings attributes Show mappings attributes object
  
  all_field object
  
  Hide all_field attributes Show all_field attributes object
  
  analyzer string Required
  
  enabled boolean Required
  
  omit_norms boolean Required
  
  search_analyzer string Required
  
  similarity string Required
  
  store boolean Required
  
  store_term_vector_offsets boolean Required
  
  store_term_vector_payloads boolean Required
  
  store_term_vector_positions boolean Required
  
  store_term_vectors boolean Required
  
  date_detection boolean
  
  dynamic string
  
  Values are strict, runtime, true, or false.
  
  dynamic_date_formats array[string]
  
  dynamic_templates array[object]
  
  _field_names object
  
  Hide _field_names attribute Show _field_names attribute object
  
  enabled boolean Required
  
  index_field object
  
  Hide index_field attribute Show index_field attribute object
  
  enabled boolean Required
  
  _meta object
  
  Hide _meta attribute Show _meta attribute object
  
  * object Additional properties
  
  numeric_detection boolean
  
  properties object
  
  _routing object
  
  Hide _routing attribute Show _routing attribute object
  
  required boolean Required
  
  _size object
  
  Hide _size attribute Show _size attribute object
  
  enabled boolean Required
  
  _source object
  
  Hide _source attributes Show _source attributes object
  
  compress boolean
  
  compress_threshold string
  
  enabled boolean
  
  excludes array[string]
  
  includes array[string]
  
  mode string
  
  Values are disabled, stored, or synthetic.
  
  runtime object
  
  Hide runtime attribute Show runtime attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  fields object
  
  For type composite
  
  Hide fields attribute Show fields attribute object
  
  * object Additional properties
  
  fetch_fields array[object]
  
  For type lookup
  
  format string
  
  A custom format for date type runtime fields.
  
  input_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_index string
  
  script object
  
  Hide script attributes Show script attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  type string Required
  
  Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.
  
  enabled boolean
  
  subobjects string
  
  Values are true or false.
  
  _data_stream_timestamp object
  
  Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
  
  enabled boolean Required
  
  order number Required
  
  settings object Required
  
  Hide settings attribute Show settings attribute object
  
  * object Additional properties
  
  version number

GET /_template

curl \
 --request GET 'http://api.example.com/_template' \
 --header "Authorization: $API_KEY"

Get index shard stores

GET /{index}/_shard_stores

Api key auth Basic auth Bearer auth

Get store information about replica shards in one or more indices. For data streams, the API retrieves store information for the stream's backing indices.

The index shard stores API returns the following information:

The node on which each replica shard exists.
The allocation ID for each replica shard.
A unique ID for each replica shard.
Any errors encountered while opening the shard index or from an earlier failure.

By default, the API returns store information only for primary shards that are unassigned or have one or more unassigned replica shards.

Path parameters

index string | array[string] Required

List of data streams, indices, and aliases used to limit the request.

Query parameters

allow_no_indices boolean

If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.
expand_wildcards string | array[string]
Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.

Supported values include:
- all: Match any data stream or index, including hidden ones.
- open: Match open, non-hidden indices. Also matches any non-hidden data stream.
- closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
- hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
- none: Wildcard expressions are not accepted.
ignore_unavailable boolean

If true, missing or closed indices are not included in the response.
status string | array[string]
List of shard health statuses used to limit the request.

Supported values include:
- green: The primary shard and all replica shards are assigned.
- yellow: One or more replica shards are unassigned.
- red: The primary shard is unassigned.
- all: Return all shards, regardless of health status.

Responses

200 application/json
Hide response attribute Show response attribute object
- indices object Required
  
  Hide indices attribute Show indices attribute object
  
  * object Additional properties
  
  Hide * attribute Show * attribute object
  
  shards object Required
  
  Hide shards attribute Show shards attribute object
  
  * object Additional properties
  
  Hide * attribute Show * attribute object
  
  stores array[object] Required

GET /{index}/_shard_stores

curl \
 --request GET 'http://api.example.com/{index}/_shard_stores' \
 --header "Authorization: $API_KEY"

Response examples (200)

An abbreviated response from `GET /_shard_stores?status=green`.

{
  "indices": {
    "my-index-000001": {
      "shards": {
        "0": {
          "stores": [
            {
              "sPa3OgxLSYGvQ4oPs-Tajw": {
                "name": "node_t0",
                "ephemeral_id": "9NlXRFGCT1m8tkvYCMK-8A",
                "transport_address": "local[1]",
                "external_id": "node_t0",
                "attributes": {},
                "roles": [],
                "version": "8.10.0",
                "min_index_version": 7000099,
                "max_index_version": 8100099
              },
              "allocation_id": "2iNySv_OQVePRX-yaRH_lQ",
              "allocation": "primary",
              "store_exception": {}
            }
          ]
        }
      }
    }
  }
}

Create a Cohere inference endpoint Added in 8.13.0

PUT /_inference/{task_type}/{cohere_inference_id}

Api key auth Basic auth Bearer auth

Create an inference endpoint to perform an inference task with the cohere service.

When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running. After creating the endpoint, wait for the model deployment to complete before using it. To verify the deployment status, use the get trained model statistics API. Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count". Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.

Path parameters

task_type string Required

The type of the inference task that the model will perform.

Values are completion, rerank, or text_embedding.
cohere_inference_id string Required

The unique identifier of the inference endpoint.

application/json

Body

chunking_settings object
Hide chunking_settings attributes Show chunking_settings attributes object
- max_chunk_size number
  
  The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).
- overlap number
  
  The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.
- sentence_overlap number
  
  The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.
- strategy string
  
  The chunking strategy: sentence or word.
service string Required

Value is cohere.
service_settings object Required
Hide service_settings attributes Show service_settings attributes object
- api_key string Required
  
  A valid API key for your Cohere account. You can find or create your Cohere API keys on the Cohere API key settings page.
  
  IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key. After creating the inference model, you cannot change the associated API key. If you want to use a different API key, delete the inference model and recreate it with the same name and the updated API key.
  
  External documentation
- embedding_type string
  
  Values are byte, float, or int8.
- model_id string
  For a completion, rerank, or text_embedding task, the name of the model to use for the inference task.
  
  For the available completion models, refer to the Cohere command docs.
  
  For the available rerank models, refer to the Cohere rerank docs.
  
  For the available text_embedding models, refer to Cohere embed docs.
  
  The default value for a text embedding task is embed-english-v2.0.
- rate_limit object
  Hide rate_limit attribute Show rate_limit attribute object
  
  requests_per_minute number
  
  The number of requests allowed per minute.
- similarity string
  
  Values are cosine, dot_product, or l2_norm.
task_settings object
Hide task_settings attributes Show task_settings attributes object
- input_type string
  
  Values are classification, clustering, ingest, or search.
- return_documents boolean
  
  For a rerank task, return doc text within the results.
- top_n number
  
  For a rerank task, the number of most relevant documents to return. It defaults to the number of the documents. If this inference endpoint is used in a text_similarity_reranker retriever query and top_n is set, it must be greater than or equal to rank_window_size in the query.
- truncate string
  
  Values are END, NONE, or START.

Responses

200 application/json
Hide response attributes Show response attributes object
- chunking_settings object
  
  Hide chunking_settings attributes Show chunking_settings attributes object
  
  max_chunk_size number
  
  The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).
  
  overlap number
  
  The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.
  
  sentence_overlap number
  
  The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.
  
  strategy string
  
  The chunking strategy: sentence or word.
- service string Required
  
  The service type
- service_settings object Required
- task_settings object
- inference_id string Required
  
  The inference Id
- task_type string Required
  
  Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.

PUT /_inference/{task_type}/{cohere_inference_id}

curl \
 --request PUT 'http://api.example.com/_inference/{task_type}/{cohere_inference_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"service\": \"cohere\",\n    \"service_settings\": {\n        \"api_key\": \"Cohere-Api-key\",\n        \"model_id\": \"embed-english-light-v3.0\",\n        \"embedding_type\": \"byte\"\n    }\n}"'

Request examples

Run `PUT _inference/text_embedding/cohere-embeddings` to create an inference endpoint that performs a text embedding task.

{
    "service": "cohere",
    "service_settings": {
        "api_key": "Cohere-Api-key",
        "model_id": "embed-english-light-v3.0",
        "embedding_type": "byte"
    }
}

Run `PUT _inference/rerank/cohere-rerank` to create an inference endpoint that performs a rerank task.

{
    "service": "cohere",
    "service_settings": {
        "api_key": "Cohere-API-key",
        "model_id": "rerank-english-v3.0"
    },
    "task_settings": {
        "top_n": 10,
        "return_documents": true
    }
}

Create an Google AI Studio inference endpoint Added in 8.15.0

PUT /_inference/{task_type}/{googleaistudio_inference_id}

Api key auth Basic auth Bearer auth

Create an inference endpoint to perform an inference task with the googleaistudio service.

When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running. After creating the endpoint, wait for the model deployment to complete before using it. To verify the deployment status, use the get trained model statistics API. Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count". Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.

Path parameters

task_type string Required

The type of the inference task that the model will perform.

Values are completion or text_embedding.
googleaistudio_inference_id string Required

The unique identifier of the inference endpoint.

application/json

Body

chunking_settings object
Hide chunking_settings attributes Show chunking_settings attributes object
- max_chunk_size number
  
  The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).
- overlap number
  
  The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.
- sentence_overlap number
  
  The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.
- strategy string
  
  The chunking strategy: sentence or word.
service string Required

Value is googleaistudio.
service_settings object Required
Hide service_settings attributes Show service_settings attributes object
- api_key string Required
  
  A valid API key of your Google Gemini account.
- model_id string Required
  
  The name of the model to use for the inference task. Refer to the Google documentation for the list of supported models.
  
  External documentation
- rate_limit object
  Hide rate_limit attribute Show rate_limit attribute object
  
  requests_per_minute number
  
  The number of requests allowed per minute.

Responses

200 application/json
Hide response attributes Show response attributes object
- chunking_settings object
  
  Hide chunking_settings attributes Show chunking_settings attributes object
  
  max_chunk_size number
  
  The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).
  
  overlap number
  
  The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.
  
  sentence_overlap number
  
  The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.
  
  strategy string
  
  The chunking strategy: sentence or word.
- service string Required
  
  The service type
- service_settings object Required
- task_settings object
- inference_id string Required
  
  The inference Id
- task_type string Required
  
  Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.

PUT /_inference/{task_type}/{googleaistudio_inference_id}

curl \
 --request PUT 'http://api.example.com/_inference/{task_type}/{googleaistudio_inference_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"service\": \"googleaistudio\",\n    \"service_settings\": {\n        \"api_key\": \"api-key\",\n        \"model_id\": \"model-id\"\n    }\n}"'

Request example

Run `PUT _inference/completion/google_ai_studio_completion` to create an inference endpoint to perform a `completion` task type.

{
    "service": "googleaistudio",
    "service_settings": {
        "api_key": "api-key",
        "model_id": "model-id"
    }
}

Create an OpenAI inference endpoint Added in 8.12.0

PUT /_inference/{task_type}/{openai_inference_id}

Api key auth Basic auth Bearer auth

Create an inference endpoint to perform an inference task with the openai service.

When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running. After creating the endpoint, wait for the model deployment to complete before using it. To verify the deployment status, use the get trained model statistics API. Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count". Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.

Path parameters

task_type string Required

The type of the inference task that the model will perform. NOTE: The chat_completion task type only supports streaming and only through the _stream API.

Values are chat_completion, completion, or text_embedding.
openai_inference_id string Required

The unique identifier of the inference endpoint.

application/json

Body

chunking_settings object
Hide chunking_settings attributes Show chunking_settings attributes object
- max_chunk_size number
  
  The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).
- overlap number
  
  The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.
- sentence_overlap number
  
  The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.
- strategy string
  
  The chunking strategy: sentence or word.
service string Required

Value is openai.
service_settings object Required
Hide service_settings attributes Show service_settings attributes object
- api_key string Required
  
  A valid API key of your OpenAI account. You can find your OpenAI API keys in your OpenAI account under the API keys section.
  
  IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key. After creating the inference model, you cannot change the associated API key. If you want to use a different API key, delete the inference model and recreate it with the same name and the updated API key.
  
  External documentation
- dimensions number
  
  The number of dimensions the resulting output embeddings should have. It is supported only in text-embedding-3 and later models. If it is not set, the OpenAI defined default for the model is used.
- model_id string Required
  
  The name of the model to use for the inference task. Refer to the OpenAI documentation for the list of available text embedding models.
  
  External documentation
- organization_id string
  
  The unique identifier for your organization. You can find the Organization ID in your OpenAI account under Settings > Organizations.
- rate_limit object
  Hide rate_limit attribute Show rate_limit attribute object
  
  requests_per_minute number
  
  The number of requests allowed per minute.
- url string
  
  The URL endpoint to use for the requests. It can be changed for testing purposes.
task_settings object
Hide task_settings attribute Show task_settings attribute object
- user string
  
  For a completion or text_embedding task, specify the user issuing the request. This information can be used for abuse detection.

Responses

200 application/json
Hide response attributes Show response attributes object
- chunking_settings object
  
  Hide chunking_settings attributes Show chunking_settings attributes object
  
  max_chunk_size number
  
  The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).
  
  overlap number
  
  The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.
  
  sentence_overlap number
  
  The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.
  
  strategy string
  
  The chunking strategy: sentence or word.
- service string Required
  
  The service type
- service_settings object Required
- task_settings object
- inference_id string Required
  
  The inference Id
- task_type string Required
  
  Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.

PUT /_inference/{task_type}/{openai_inference_id}

curl \
 --request PUT 'http://api.example.com/_inference/{task_type}/{openai_inference_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"service\": \"openai\",\n    \"service_settings\": {\n        \"api_key\": \"OpenAI-API-Key\",\n        \"model_id\": \"text-embedding-3-small\",\n        \"dimensions\": 128\n    }\n}"'

Request examples

Run `PUT _inference/text_embedding/openai-embeddings` to create an inference endpoint that performs a `text_embedding` task. The embeddings created by requests to this endpoint will have 128 dimensions.

{
    "service": "openai",
    "service_settings": {
        "api_key": "OpenAI-API-Key",
        "model_id": "text-embedding-3-small",
        "dimensions": 128
    }
}

Run `PUT _inference/completion/amazon_bedrock_completion` to create an inference endpoint to perform a completion task.

{
    "service": "amazonbedrock",
    "service_settings": {
        "access_key": "AWS-access-key",
        "secret_key": "AWS-secret-key",
        "region": "us-east-1",
        "provider": "amazontitan",
        "model": "amazon.titan-text-premier-v1:0"
    }
}

Simulate a pipeline Added in 5.0.0

GET /_ingest/pipeline/{id}/_simulate

Api key auth Basic auth Bearer auth

Run an ingest pipeline against a set of provided documents. You can either specify an existing pipeline to use with the provided documents or supply a pipeline definition in the body of the request.

Path parameters

id string Required

The pipeline to test. If you don't specify a pipeline in the request body, this parameter is required.

Query parameters

verbose boolean

If true, the response includes output data for each processor in the executed pipeline.

application/json

Body Required

docs array[object] Required

Sample documents to test in the pipeline.
Hide docs attributes Show docs attributes object
- _id string
- _index string
- _source object Required
  
  JSON body for the document.
pipeline object Additional properties
Hide pipeline attributes Show pipeline attributes object
- description string
  
  Description of the ingest pipeline.
- on_failure array[object]
  
  Processors to run immediately after a processor failure.
  Hide on_failure attributes Show on_failure attributes object
  
  append object
  
  Hide append attributes Show append attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  value object | array[object] Required
  
  The value to be appended. Supports template snippets.
  
  One of:
  object-1 object array-2 array[object]
  
  allow_duplicates boolean
  
  If false, the processor does not append values already present in the field.
  
  attachment object
  
  Hide attachment attributes Show attachment attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  indexed_chars number
  
  The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.
  
  indexed_chars_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  remove_binary boolean
  
  If true, the binary field will be removed from the document
  
  resource_name string
  
  Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.
  
  bytes object
  
  Hide bytes attributes Show bytes attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  circle object
  
  Hide circle attributes Show circle attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  error_distance number Required
  
  The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  shape_type string Required
  
  Values are geo_shape or shape.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  community_id object
  
  Hide community_id attributes Show community_id attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  source_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  source_port string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_port string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  iana_number string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  icmp_type string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  icmp_code string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  transport string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  seed number
  
  Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  convert object
  
  Hide convert attributes Show convert attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  type string Required
  
  Values are integer, long, double, float, boolean, ip, string, or auto.
  
  csv object
  
  Hide csv attributes Show csv attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  empty_value object
  
  Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  quote string
  
  Quote used in CSV, has to be single character string.
  
  separator string
  
  Separator used in CSV, has to be single character string.
  
  target_fields string | array[string] Required
  
  trim boolean
  
  Trim whitespaces in unquoted fields.
  
  date object
  
  Hide date attributes Show date attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  formats array[string] Required
  
  An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
  
  locale string
  
  The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  timezone string
  
  The timezone to use when parsing the date. Supports template snippets.
  
  output_format string
  
  The format to use when writing the date to target_field. Must be a valid java time pattern.
  
  date_index_name object
  
  Hide date_index_name attributes Show date_index_name attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  date_formats array[string]
  
  An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
  
  date_rounding string Required
  
  How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  index_name_format string
  
  The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.
  
  index_name_prefix string
  
  A prefix of the index name to be prepended before the printed date. Supports template snippets.
  
  locale string
  
  The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.
  
  timezone string
  
  The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.
  
  dissect object
  
  Hide dissect attributes Show dissect attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  append_separator string
  
  The character(s) that separate the appended fields.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern string Required
  
  The pattern to apply to the field.
  
  dot_expander object
  
  Hide dot_expander attributes Show dot_expander attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  override boolean
  
  Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.
  
  path string
  
  The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.
  
  drop object
  
  Hide drop attributes Show drop attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  enrich object
  
  Hide enrich attributes Show enrich attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  max_matches number
  
  The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.
  
  override boolean
  
  If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.
  
  policy_name string Required
  
  The name of the enrich policy to use.
  
  shape_relation string
  
  Values are intersects, disjoint, within, or contains.
  
  target_field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  fail object
  
  Hide fail attributes Show fail attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  message string Required
  
  The error message thrown by the processor. Supports template snippets.
  
  fingerprint object
  
  Hide fingerprint attributes Show fingerprint attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  fields string | array[string] Required
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  salt string
  
  Salt value for the hash function.
  
  method string
  
  Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.
  
  ignore_missing boolean
  
  If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.
  
  foreach object
  
  Hide foreach attributes Show foreach attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true, the processor silently exits without changing the document if the field is null or missing.
  
  processor object Required
  
  ip_location object
  
  Hide ip_location attributes Show ip_location attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  database_file string
  
  The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  first_only boolean
  
  If true, only the first found IP location data will be returned, even if the field contains an array.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  properties array[string]
  
  Controls what properties are added to the target_field based on the IP location lookup.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  download_database_on_pipeline_creation boolean
  
  If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.
  
  geo_grid object
  
  Hide geo_grid attributes Show geo_grid attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to interpret as a geo-tile.= The field format is determined by the tile_type.
  
  tile_type string Required
  
  Values are geotile, geohex, or geohash.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  parent_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  children_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  non_children_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  precision_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_format string
  
  Values are geojson or wkt.
  
  geoip object
  
  Hide geoip attributes Show geoip attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  database_file string
  
  The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  first_only boolean
  
  If true, only the first found geoip data will be returned, even if the field contains an array.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  properties array[string]
  
  Controls what properties are added to the target_field based on the geoip lookup.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  download_database_on_pipeline_creation boolean
  
  If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.
  
  grok object
  
  Hide grok attributes Show grok attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  ecs_compatibility string
  
  Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern_definitions object
  
  A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.
  
  Hide pattern_definitions attribute Show pattern_definitions attribute object
  
  * string Additional properties
  
  patterns array[string] Required
  
  An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.
  
  trace_match boolean
  
  When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.
  
  gsub object
  
  Hide gsub attributes Show gsub attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern string Required
  
  The pattern to be replaced.
  
  replacement string Required
  
  The string to replace the matching patterns with.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  html_strip object
  
  Hide html_strip attributes Show html_strip attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document,
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  inference object
  
  Hide inference attributes Show inference attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  model_id string Required
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field_map object
  
  Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.
  
  Hide field_map attribute Show field_map attribute object
  
  * object Additional properties
  
  inference_config object
  
  Hide inference_config attributes Show inference_config attributes object
  
  regression object
  
  classification object
  
  input_output object | array[object]
  
  Input fields for inference and output (destination) fields for the inference results. This option is incompatible with the target_field and field_map options.
  
  One of:
  InputConfig object array-2 array[object]
  
  ignore_missing boolean
  
  If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.
  
  join object
  
  Hide join attributes Show join attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  separator string Required
  
  The separator character.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  json object
  
  Hide json attributes Show json attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  add_to_root boolean
  
  Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.
  
  add_to_root_conflict_strategy string
  
  Values are replace or merge.
  
  allow_duplicate_keys boolean
  
  When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  kv object
  
  Hide kv attributes Show kv attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  exclude_keys array[string]
  
  List of keys to exclude from document.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field_split string Required
  
  Regex pattern to use for splitting key-value pairs.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  include_keys array[string]
  
  List of keys to filter and insert into document. Defaults to including all keys.
  
  prefix string
  
  Prefix to be added to extracted keys.
  
  strip_brackets boolean
  
  If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  trim_key string
  
  String of characters to trim from extracted keys.
  
  trim_value string
  
  String of characters to trim from extracted values.
  
  value_split string Required
  
  Regex pattern to use for splitting the key from the value within a key-value pair.
  
  lowercase object
  
  Hide lowercase attributes Show lowercase attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  network_direction object
  
  Hide network_direction attributes Show network_direction attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  source_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  internal_networks array[string]
  
  List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.
  
  internal_networks_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  pipeline object
  
  Hide pipeline attributes Show pipeline attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  name string Required
  
  ignore_missing_pipeline boolean
  
  Whether to ignore missing pipelines instead of failing.
  
  redact object
  
  Hide redact attributes Show redact attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  patterns array[string] Required
  
  A list of grok expressions to match and redact named captures with
  
  pattern_definitions object
  
  Hide pattern_definitions attribute Show pattern_definitions attribute object
  
  * string Additional properties
  
  prefix string
  
  Start a redacted section with this token
  
  suffix string
  
  End a redacted section with this token
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  skip_if_unlicensed boolean
  
  If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document
  
  trace_redact boolean
  
  If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted
  
  registered_domain object
  
  Hide registered_domain attributes Show registered_domain attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  remove object
  
  Hide remove attributes Show remove attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string | array[string] Required
  
  keep string | array[string]
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  rename object
  
  Hide rename attributes Show rename attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  reroute object
  
  Hide reroute attributes Show reroute attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  destination string
  
  A static value for the target. Can’t be set when the dataset or namespace option is set.
  
  dataset string | array[string]
  
  Field references or a static value for the dataset part of the data stream name. In addition to the criteria for index names, cannot contain - and must be no longer than 100 characters. Example values are nginx.access and nginx.error.
  
  Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.
  
  default {{data_stream.dataset}}
  
  One of:
  string-1 string array-2 array[string]
  
  namespace string | array[string]
  
  Field references or a static value for the namespace part of the data stream name. See the criteria for index names for allowed characters. Must be no longer than 100 characters.
  
  Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.
  
  default {{data_stream.namespace}}
  
  One of:
  string-1 string array-2 array[string]
  
  script object
  
  Hide script attributes Show script attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  id string
  
  lang string
  
  Any of:
  ScriptLanguage string ScriptLanguage string
  
  Values are painless, expression, mustache, or java.
  
  params object
  
  Object containing parameters for the script.
  
  Hide params attribute Show params attribute object
  
  * object Additional properties
  
  source string | object
  
  One of:
  ScriptSource string SearchRequestBody object
  
  set object
  
  Hide set attributes Show set attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  copy_from string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_empty_value boolean
  
  If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.
  
  media_type string
  
  The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.
  
  override boolean
  
  If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.
  
  value object
  
  The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.
  
  set_security_user object
  
  Hide set_security_user attributes Show set_security_user attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Controls what user related properties are added to the field.
  
  sort object
  
  Hide sort attributes Show sort attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  order string
  
  Values are asc or desc.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  split object
  
  Hide split attributes Show split attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  preserve_trailing boolean
  
  Preserves empty trailing fields, if any.
  
  separator string Required
  
  A regex which matches the separator, for example, , or \s+.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  terminate object
  
  Hide terminate attributes Show terminate attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  trim object
  
  Hide trim attributes Show trim attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  uppercase object
  
  Hide uppercase attributes Show uppercase attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  urldecode object
  
  Hide urldecode attributes Show urldecode attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  uri_parts object
  
  Hide uri_parts attributes Show uri_parts attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  keep_original boolean
  
  If true, the processor copies the unparsed URI to <target_field>.original.
  
  remove_if_successful boolean
  
  If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  user_agent object
  
  Hide user_agent attributes Show user_agent attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  regex_file string
  
  The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Controls what properties are added to target_field.
  
  Values are name, os, device, original, or version.
  
  extract_device_type boolean Beta
  
  Extracts device type from the user agent string on a best-effort basis.
- processors array[object]
  
  Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.
  Hide processors attributes Show processors attributes object
  
  append object
  
  Hide append attributes Show append attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  value object | array[object] Required
  
  The value to be appended. Supports template snippets.
  
  One of:
  object-1 object array-2 array[object]
  
  allow_duplicates boolean
  
  If false, the processor does not append values already present in the field.
  
  attachment object
  
  Hide attachment attributes Show attachment attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  indexed_chars number
  
  The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.
  
  indexed_chars_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  remove_binary boolean
  
  If true, the binary field will be removed from the document
  
  resource_name string
  
  Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.
  
  bytes object
  
  Hide bytes attributes Show bytes attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  circle object
  
  Hide circle attributes Show circle attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  error_distance number Required
  
  The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  shape_type string Required
  
  Values are geo_shape or shape.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  community_id object
  
  Hide community_id attributes Show community_id attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  source_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  source_port string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_port string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  iana_number string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  icmp_type string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  icmp_code string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  transport string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  seed number
  
  Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  convert object
  
  Hide convert attributes Show convert attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  type string Required
  
  Values are integer, long, double, float, boolean, ip, string, or auto.
  
  csv object
  
  Hide csv attributes Show csv attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  empty_value object
  
  Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  quote string
  
  Quote used in CSV, has to be single character string.
  
  separator string
  
  Separator used in CSV, has to be single character string.
  
  target_fields string | array[string] Required
  
  trim boolean
  
  Trim whitespaces in unquoted fields.
  
  date object
  
  Hide date attributes Show date attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  formats array[string] Required
  
  An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
  
  locale string
  
  The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  timezone string
  
  The timezone to use when parsing the date. Supports template snippets.
  
  output_format string
  
  The format to use when writing the date to target_field. Must be a valid java time pattern.
  
  date_index_name object
  
  Hide date_index_name attributes Show date_index_name attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  date_formats array[string]
  
  An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
  
  date_rounding string Required
  
  How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  index_name_format string
  
  The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.
  
  index_name_prefix string
  
  A prefix of the index name to be prepended before the printed date. Supports template snippets.
  
  locale string
  
  The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.
  
  timezone string
  
  The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.
  
  dissect object
  
  Hide dissect attributes Show dissect attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  append_separator string
  
  The character(s) that separate the appended fields.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern string Required
  
  The pattern to apply to the field.
  
  dot_expander object
  
  Hide dot_expander attributes Show dot_expander attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  override boolean
  
  Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.
  
  path string
  
  The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.
  
  drop object
  
  Hide drop attributes Show drop attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  enrich object
  
  Hide enrich attributes Show enrich attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  max_matches number
  
  The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.
  
  override boolean
  
  If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.
  
  policy_name string Required
  
  The name of the enrich policy to use.
  
  shape_relation string
  
  Values are intersects, disjoint, within, or contains.
  
  target_field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  fail object
  
  Hide fail attributes Show fail attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  message string Required
  
  The error message thrown by the processor. Supports template snippets.
  
  fingerprint object
  
  Hide fingerprint attributes Show fingerprint attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  fields string | array[string] Required
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  salt string
  
  Salt value for the hash function.
  
  method string
  
  Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.
  
  ignore_missing boolean
  
  If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.
  
  foreach object
  
  Hide foreach attributes Show foreach attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true, the processor silently exits without changing the document if the field is null or missing.
  
  processor object Required
  
  ip_location object
  
  Hide ip_location attributes Show ip_location attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  database_file string
  
  The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  first_only boolean
  
  If true, only the first found IP location data will be returned, even if the field contains an array.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  properties array[string]
  
  Controls what properties are added to the target_field based on the IP location lookup.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  download_database_on_pipeline_creation boolean
  
  If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.
  
  geo_grid object
  
  Hide geo_grid attributes Show geo_grid attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to interpret as a geo-tile.= The field format is determined by the tile_type.
  
  tile_type string Required
  
  Values are geotile, geohex, or geohash.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  parent_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  children_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  non_children_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  precision_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_format string
  
  Values are geojson or wkt.
  
  geoip object
  
  Hide geoip attributes Show geoip attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  database_file string
  
  The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  first_only boolean
  
  If true, only the first found geoip data will be returned, even if the field contains an array.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  properties array[string]
  
  Controls what properties are added to the target_field based on the geoip lookup.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  download_database_on_pipeline_creation boolean
  
  If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.
  
  grok object
  
  Hide grok attributes Show grok attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  ecs_compatibility string
  
  Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern_definitions object
  
  A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.
  
  Hide pattern_definitions attribute Show pattern_definitions attribute object
  
  * string Additional properties
  
  patterns array[string] Required
  
  An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.
  
  trace_match boolean
  
  When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.
  
  gsub object
  
  Hide gsub attributes Show gsub attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern string Required
  
  The pattern to be replaced.
  
  replacement string Required
  
  The string to replace the matching patterns with.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  html_strip object
  
  Hide html_strip attributes Show html_strip attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document,
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  inference object
  
  Hide inference attributes Show inference attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  model_id string Required
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field_map object
  
  Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.
  
  Hide field_map attribute Show field_map attribute object
  
  * object Additional properties
  
  inference_config object
  
  Hide inference_config attributes Show inference_config attributes object
  
  regression object
  
  classification object
  
  input_output object | array[object]
  
  Input fields for inference and output (destination) fields for the inference results. This option is incompatible with the target_field and field_map options.
  
  One of:
  InputConfig object array-2 array[object]
  
  ignore_missing boolean
  
  If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.
  
  join object
  
  Hide join attributes Show join attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  separator string Required
  
  The separator character.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  json object
  
  Hide json attributes Show json attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  add_to_root boolean
  
  Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.
  
  add_to_root_conflict_strategy string
  
  Values are replace or merge.
  
  allow_duplicate_keys boolean
  
  When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  kv object
  
  Hide kv attributes Show kv attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  exclude_keys array[string]
  
  List of keys to exclude from document.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field_split string Required
  
  Regex pattern to use for splitting key-value pairs.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  include_keys array[string]
  
  List of keys to filter and insert into document. Defaults to including all keys.
  
  prefix string
  
  Prefix to be added to extracted keys.
  
  strip_brackets boolean
  
  If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  trim_key string
  
  String of characters to trim from extracted keys.
  
  trim_value string
  
  String of characters to trim from extracted values.
  
  value_split string Required
  
  Regex pattern to use for splitting the key from the value within a key-value pair.
  
  lowercase object
  
  Hide lowercase attributes Show lowercase attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  network_direction object
  
  Hide network_direction attributes Show network_direction attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  source_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  internal_networks array[string]
  
  List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.
  
  internal_networks_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  pipeline object
  
  Hide pipeline attributes Show pipeline attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  name string Required
  
  ignore_missing_pipeline boolean
  
  Whether to ignore missing pipelines instead of failing.
  
  redact object
  
  Hide redact attributes Show redact attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  patterns array[string] Required
  
  A list of grok expressions to match and redact named captures with
  
  pattern_definitions object
  
  Hide pattern_definitions attribute Show pattern_definitions attribute object
  
  * string Additional properties
  
  prefix string
  
  Start a redacted section with this token
  
  suffix string
  
  End a redacted section with this token
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  skip_if_unlicensed boolean
  
  If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document
  
  trace_redact boolean
  
  If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted
  
  registered_domain object
  
  Hide registered_domain attributes Show registered_domain attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  remove object
  
  Hide remove attributes Show remove attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string | array[string] Required
  
  keep string | array[string]
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  rename object
  
  Hide rename attributes Show rename attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  reroute object
  
  Hide reroute attributes Show reroute attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  destination string
  
  A static value for the target. Can’t be set when the dataset or namespace option is set.
  
  dataset string | array[string]
  
  Field references or a static value for the dataset part of the data stream name. In addition to the criteria for index names, cannot contain - and must be no longer than 100 characters. Example values are nginx.access and nginx.error.
  
  Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.
  
  default {{data_stream.dataset}}
  
  One of:
  string-1 string array-2 array[string]
  
  namespace string | array[string]
  
  Field references or a static value for the namespace part of the data stream name. See the criteria for index names for allowed characters. Must be no longer than 100 characters.
  
  Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.
  
  default {{data_stream.namespace}}
  
  One of:
  string-1 string array-2 array[string]
  
  script object
  
  Hide script attributes Show script attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  id string
  
  lang string
  
  Any of:
  ScriptLanguage string ScriptLanguage string
  
  Values are painless, expression, mustache, or java.
  
  params object
  
  Object containing parameters for the script.
  
  Hide params attribute Show params attribute object
  
  * object Additional properties
  
  source string | object
  
  One of:
  ScriptSource string SearchRequestBody object
  
  set object
  
  Hide set attributes Show set attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  copy_from string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_empty_value boolean
  
  If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.
  
  media_type string
  
  The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.
  
  override boolean
  
  If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.
  
  value object
  
  The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.
  
  set_security_user object
  
  Hide set_security_user attributes Show set_security_user attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Controls what user related properties are added to the field.
  
  sort object
  
  Hide sort attributes Show sort attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  order string
  
  Values are asc or desc.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  split object
  
  Hide split attributes Show split attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  preserve_trailing boolean
  
  Preserves empty trailing fields, if any.
  
  separator string Required
  
  A regex which matches the separator, for example, , or \s+.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  terminate object
  
  Hide terminate attributes Show terminate attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  trim object
  
  Hide trim attributes Show trim attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  uppercase object
  
  Hide uppercase attributes Show uppercase attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  urldecode object
  
  Hide urldecode attributes Show urldecode attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  uri_parts object
  
  Hide uri_parts attributes Show uri_parts attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  keep_original boolean
  
  If true, the processor copies the unparsed URI to <target_field>.original.
  
  remove_if_successful boolean
  
  If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  user_agent object
  
  Hide user_agent attributes Show user_agent attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  Hide if attributes Show if attributes object
  
  source
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  lang
  
  options object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  regex_file string
  
  The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Controls what properties are added to target_field.
  
  Values are name, os, device, original, or version.
  
  extract_device_type boolean Beta
  
  Extracts device type from the user agent string on a best-effort basis.
- version number
- deprecated boolean
  
  Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.
- _meta object
  Hide _meta attribute Show _meta attribute object
  
  * object Additional properties

Responses

200 application/json
Hide response attribute Show response attribute object
- docs array[object] Required
  
  Hide docs attributes Show docs attributes object
  
  doc object
  
  Hide doc attributes Show doc attributes object
  
  _id string Required
  
  _index string Required
  
  _ingest object Required
  
  Hide _ingest attributes Show _ingest attributes object
  
  _redact object
  
  Hide _redact attribute Show _redact attribute object
  
  _is_redacted boolean Required
  
  indicates if document has been redacted
  
  timestamp string
  
  pipeline string
  
  _routing string
  
  Value used to send the document to a specific primary shard.
  
  _source object Required
  
  JSON body for the document.
  
  Hide _source attribute Show _source attribute object
  
  * object Additional properties
  
  _version number | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  VersionNumber number StringifiedVersionNumber string
  
  _version_type string
  
  Values are internal, external, external_gte, or force.
  
  error object
  
  Hide error attributes Show error attributes object
  
  type string Required
  
  The type of error
  
  reason string
  
  A human-readable explanation of the error, in English.
  
  stack_trace string
  
  The server stack trace. Present only if the error_trace=true parameter was sent with the request.
  
  caused_by object
  
  root_cause array[object]
  
  suppressed array[object]
  
  processor_results array[object]
  
  Hide processor_results attributes Show processor_results attributes object
  
  doc object
  
  Hide doc attributes Show doc attributes object
  
  _id string Required
  
  _index string Required
  
  _ingest object Required
  
  _routing string
  
  Value used to send the document to a specific primary shard.
  
  _source object Required
  
  JSON body for the document.
  
  _version
  
  _version_type string
  
  Values are internal, external, external_gte, or force.
  
  tag string
  
  processor_type string
  
  status string
  
  Values are success, error, error_ignored, skipped, or dropped.
  
  description string
  
  ignored_error object
  
  Hide ignored_error attributes Show ignored_error attributes object
  
  type string Required
  
  The type of error
  
  reason string
  
  A human-readable explanation of the error, in English.
  
  stack_trace string
  
  The server stack trace. Present only if the error_trace=true parameter was sent with the request.
  
  caused_by object
  
  root_cause array[object]
  
  suppressed array[object]
  
  error object
  
  Hide error attributes Show error attributes object
  
  type string Required
  
  The type of error
  
  reason string
  
  A human-readable explanation of the error, in English.
  
  stack_trace string
  
  The server stack trace. Present only if the error_trace=true parameter was sent with the request.
  
  caused_by object
  
  root_cause array[object]
  
  suppressed array[object]

GET /_ingest/pipeline/{id}/_simulate

curl \
 --request GET 'http://api.example.com/_ingest/pipeline/{id}/_simulate' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"pipeline\" :\n  {\n    \"description\": \"_description\",\n    \"processors\": [\n      {\n        \"set\" : {\n          \"field\" : \"field2\",\n          \"value\" : \"_value\"\n        }\n      }\n    ]\n  },\n  \"docs\": [\n    {\n      \"_index\": \"index\",\n      \"_id\": \"id\",\n      \"_source\": {\n        \"foo\": \"bar\"\n      }\n    },\n    {\n      \"_index\": \"index\",\n      \"_id\": \"id\",\n      \"_source\": {\n        \"foo\": \"rab\"\n      }\n    }\n  ]\n}"'

Request example

You can specify the used pipeline either in the request body or as a path parameter.

{
  "pipeline" :
  {
    "description": "_description",
    "processors": [
      {
        "set" : {
          "field" : "field2",
          "value" : "_value"
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "rab"
      }
    }
  ]
}

Response examples (200)

A successful response for running an ingest pipeline against a set of provided documents.

{
   "docs": [
      {
         "doc": {
            "_id": "id",
            "_index": "index",
            "_version": "-3",
            "_source": {
               "field2": "_value",
               "foo": "bar"
            },
            "_ingest": {
               "timestamp": "2017-05-04T22:30:03.187Z"
            }
         }
      },
      {
         "doc": {
            "_id": "id",
            "_index": "index",
            "_version": "-3",
            "_source": {
               "field2": "_value",
               "foo": "rab"
            },
            "_ingest": {
               "timestamp": "2017-05-04T22:30:03.188Z"
            }
         }
      }
   ]
}

Create or update an autoscaling policy Added in 7.11.0

Body Required

Get data frame analytics jobs Added in 7.7.0

Update the connector service type Beta

Body Required

Body object Required

_id string | null

Body Required

Run an async ES|QL query Added in 8.13.0

Body Required

Run multiple Fleet searches Technical preview

Body object Required

_source boolean | object

filter object | array[object]

filter object | array[object]

filter object | array[object]

filter object | array[object]

filter object | array[object]

source string | object

lang string

source string | object

lang string

Get field usage stats Technical preview

Create a Cohere inference endpoint Added in 8.13.0

Create an Google AI Studio inference endpoint Added in 8.15.0

Create an OpenAI inference endpoint Added in 8.12.0

Simulate a pipeline Added in 5.0.0

Body Required

value object | array[object] Required

input_output object | array[object]

dataset string | array[string]

namespace string | array[string]

lang string

source string | object

value object | array[object] Required

input_output object | array[object]

dataset string | array[string]

namespace string | array[string]

lang string

source string | object

_version number | string