Run a watch | Elasticsearch API documentation (v8)

Get component templates Generally available; Added in 5.1.0

GET /_cat/component_templates/{name}

All methods and paths for this operation:

GET /_cat/component_templates

GET /_cat/component_templates/{name}

Get information about component templates in a cluster. Component templates are building blocks for constructing index templates that specify index mappings, settings, and aliases.

IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the get component template API.

Required authorization

Cluster privileges: monitor

Path parameters

name string Required

The name of the component template. It accepts wildcard expressions. If it is omitted, all component templates are returned.

Query parameters

h string | array[string]
A comma-separated list of columns names to display. It supports simple wildcards.

Supported values include:
- name (or n): The name of the component template.
- version (or v): The version number of the component template.
- alias_count (or a): The number of aliases in the component template.
- mapping_count (or m): The number of mappings in the component template.
- settings_count (or s): The number of settings in the component template.
- metadata_count (or me): The number of metadata entries in the component template.
- included_in (or i): The index templates that include this component template.
Values are name, n, version, v, alias_count, a, mapping_count, m, settings_count, s, metadata_count, me, included_in, or i.
s string | array[string]

List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.
local boolean

If true, the request computes the list of selected nodes from the local cluster state. If false the list of selected nodes are computed from the cluster state of the master node. In both cases the coordinating node will send requests for further information to each selected node.
master_timeout string

The period to wait for a connection to the master node.

Values are -1 or 0.

Responses

200 application/json
Hide response attributes Show response attributes object
- name string Required
- version string | null Required
  
  One of:
  string-1 string string-2 string | null
- alias_count string Required
- mapping_count string Required
- settings_count string Required
- metadata_count string Required
- included_in string Required

GET /_cat/component_templates/{name}

GET _cat/component_templates/my-template-*?v=true&s=name&format=json

resp = client.cat.component_templates(
    name="my-template-*",
    v=True,
    s="name",
    format="json",
)

const response = await client.cat.componentTemplates({
  name: "my-template-*",
  v: "true",
  s: "name",
  format: "json",
});

response = client.cat.component_templates(
  name: "my-template-*",
  v: "true",
  s: "name",
  format: "json"
)

$resp = $client->cat()->componentTemplates([
    "name" => "my-template-*",
    "v" => "true",
    "s" => "name",
    "format" => "json",
]);

curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_cat/component_templates/my-template-*?v=true&s=name&format=json"

client.cat().componentTemplates();

Response examples (200)

A successful response from `GET _cat/component_templates/my-template-*?v=true&s=name&format=json`.

[
  {
    "name": "my-template-1",
    "version": "null",
    "alias_count": "0",
    "mapping_count": "0",
    "settings_count": "1",
    "metadata_count": "0",
    "included_in": "[my-index-template]"
  },
    {
    "name": "my-template-2",
    "version": null,
    "alias_count": "0",
    "mapping_count": "3",
    "settings_count": "0",
    "metadata_count": "0",
    "included_in": "[my-index-template]"
  }
]

Get field data cache information Generally available

GET /_cat/fielddata/{fields}

Api key auth Basic auth Bearer auth

All methods and paths for this operation:

GET /_cat/fielddata

GET /_cat/fielddata/{fields}

Get the amount of heap memory currently used by the field data cache on every data node in the cluster.

IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the nodes stats API.

Required authorization

Cluster privileges: monitor

Path parameters

fields string | array[string] Required

Comma-separated list of fields used to limit returned information. To retrieve all fields, omit this parameter.

Query parameters

bytes string

The unit used to display byte values.

Values are b, kb, mb, gb, tb, or pb.
fields string | array[string]

Comma-separated list of fields used to limit returned information.
h string | array[string]
A comma-separated list of columns names to display. It supports simple wildcards.

Supported values include:
- id: The node ID.
- host (or h): The host name of the node.
- ip: The IP address of the node.
- node (or n): The node name.
- field (or f): The field name.
- size (or s): The field data usage.
Values are id, host, h, ip, node, n, field, f, size, or s.
s string | array[string]

List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

Responses

200 application/json
Hide response attributes Show response attributes object
- id string
  
  node id
- host string
  
  host name
- ip string
  
  ip address
- node string
  
  node name
- field string
  
  field name
- size string
  
  field data usage

GET /_cat/fielddata/{fields}

GET /_cat/fielddata?v=true&fields=body&format=json

resp = client.cat.fielddata(
    v=True,
    fields="body",
    format="json",
)

const response = await client.cat.fielddata({
  v: "true",
  fields: "body",
  format: "json",
});

response = client.cat.fielddata(
  v: "true",
  fields: "body",
  format: "json"
)

$resp = $client->cat()->fielddata([
    "v" => "true",
    "fields" => "body",
    "format" => "json",
]);

curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_cat/fielddata?v=true&fields=body&format=json"

client.cat().fielddata();

Response examples (200)

A successful response from `GET /_cat/fielddata?v=true&fields=body&format=json`. You can specify an individual field in the request body or URL path. This example retrieves heap memory size information for the `body` field.

[
  {
    "id": "Nqk-6inXQq-OxUfOUI8jNQ",
    "host": "127.0.0.1",
    "ip": "127.0.0.1",
    "node": "Nqk-6in",
    "field": "body",
    "size": "544b"
  }
]

A successful response from `GET /_cat/fielddata/body,soul?v=true&format=json`. You can specify a comma-separated list of fields in the request body or URL path. This example retrieves heap memory size information for the `body` and `soul` fields. To get information for all fields, run `GET /_cat/fielddata?v=true`.

[
  {
    "id": "Nqk-6inXQq-OxUfOUI8jNQ",
    "host": "1127.0.0.1",
    "ip": "127.0.0.1",
    "node": "Nqk-6in",
    "field": "body",
    "size": "544b"
  },
  {
    "id": "Nqk-6inXQq-OxUfOUI8jNQ",
    "host": "127.0.0.1",
    "ip": "127.0.0.1",
    "node": "Nqk-6in",
    "field": "soul",
    "size": "480b"
  }
]

Get shard information Generally available

GET /_cat/shards/{index}

Api key auth Basic auth Bearer auth

All methods and paths for this operation:

GET /_cat/shards

GET /_cat/shards/{index}

Get information about the shards in a cluster. For data streams, the API returns information about the backing indices. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications.

Required authorization

Index privileges: monitor
Cluster privileges: monitor

Path parameters

index string | array[string] Required

A comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.

Query parameters

bytes string

The unit used to display byte values.

Values are b, kb, mb, gb, tb, or pb.
h string | array[string]
List of columns to appear in the response. Supports simple wildcards.

Supported values include:
- completion.size (or cs, completionSize): Size of completion. For example: 0b.
- dataset.size: Disk space used by the shard’s dataset, which may or may not be the size on disk, but includes space used by the shard on object storage. Reported as a size value for example: 5kb.
- dense_vector.value_count (or dvc, denseVectorCount): Number of indexed dense vectors.
- docs (or d, dc): Number of documents in shard, for example: 25.
- fielddata.evictions (or fe, fielddataEvictions): Fielddata cache evictions, for example: 0.
- fielddata.memory_size (or fm, fielddataMemory): Used fielddata cache memory, for example: 0b.
- flush.total (or ft, flushTotal): Number of flushes, for example: 1.
- flush.total_time (or ftt, flushTotalTime): Time spent in flush, for example: 1.
- get.current (or gc, getCurrent): Number of current get operations, for example: 0.
- get.exists_time (or geti, getExistsTime): Time spent in successful gets, for example: 14ms.
- get.exists_total (or geto, getExistsTotal): Number of successful get operations, for example: 2.
- get.missing_time (or gmti, getMissingTime): Time spent in failed gets, for example: 0s.
- get.missing_total (or gmto, getMissingTotal): Number of failed get operations, for example: 1.
- get.time (or gti, getTime): Time spent in get, for example: 14ms.
- get.total (or gto, getTotal): Number of get operations, for example: 2.
- id: ID of the node, for example: k0zy.
- index (or i, idx): Name of the index.
- indexing.delete_current (or idc, indexingDeleteCurrent): Number of current deletion operations, for example: 0.
- indexing.delete_time (or idti, indexingDeleteTime): Time spent in deletions, for example: 2ms.
- indexing.delete_total (or idto, indexingDeleteTotal): Number of deletion operations, for example: 2.
- indexing.index_current (or iic, indexingIndexCurrent): Number of current indexing operations, for example: 0.
- indexing.index_failed_due_to_version_conflict (or iifvc, indexingIndexFailedDueToVersionConflict): Number of failed indexing operations due to version conflict, for example: 0.
- indexing.index_failed (or iif, indexingIndexFailed): Number of failed indexing operations, for example: 0.
- indexing.index_time (or iiti, indexingIndexTime): Time spent in indexing, such as for example: 134ms.
- indexing.index_total (or iito, indexingIndexTotal): Number of indexing operations, for example: 1.
- ip: IP address of the node, for example: 127.0.1.1.
- merges.current (or mc, mergesCurrent): Number of current merge operations, for example: 0.
- merges.current_docs (or mcd, mergesCurrentDocs): Number of current merging documents, for example: 0.
- merges.current_size (or mcs, mergesCurrentSize): Size of current merges, for example: 0b.
- merges.total (or mt, mergesTotal): Number of completed merge operations, for example: 0.
- merges.total_docs (or mtd, mergesTotalDocs): Number of merged documents, for example: 0.
- merges.total_size (or mts, mergesTotalSize): Size of current merges, for example: 0b.
- merges.total_time (or mtt, mergesTotalTime): Time spent merging documents, for example: 0s.
- node (or n): Node name, for example: I8hydUG.
- prirep (or p, pr, primaryOrReplica): Shard type. Returned values are primary or replica.
- query_cache.evictions (or qce, queryCacheEvictions): Query cache evictions, for example: 0.
- query_cache.memory_size (or qcm, queryCacheMemory): Used query cache memory, for example: 0b.
- recoverysource.type (or rs): Type of recovery source.
- refresh.time (or rti, refreshTime): Time spent in refreshes, for example: 91ms.
- refresh.total (or rto, refreshTotal): Number of refreshes, for example: 16.
- search.fetch_current (or sfc, searchFetchCurrent): Current fetch phase operations, for example: 0.
- search.fetch_time (or sfti, searchFetchTime): Time spent in fetch phase, for example: 37ms.
- search.fetch_total (or sfto, searchFetchTotal): Number of fetch operations, for example: 7.
- search.open_contexts (or so, searchOpenContexts): Open search contexts, for example: 0.
- search.query_current (or sqc, searchQueryCurrent): Current query phase operations, for example: 0.
- search.query_time (or sqti, searchQueryTime): Time spent in query phase, for example: 43ms.
- search.query_total (or sqto, searchQueryTotal): Number of query operations, for example: 9.
- search.scroll_current (or scc, searchScrollCurrent): Open scroll contexts, for example: 2.
- search.scroll_time (or scti, searchScrollTime): Time scroll contexts held open, for example: 2m.
- search.scroll_total (or scto, searchScrollTotal): Completed scroll contexts, for example: 1.
- segments.count (or sc, segmentsCount): Number of segments, for example: 4.
- segments.fixed_bitset_memory (or sfbm, fixedBitsetMemory): Memory used by fixed bit sets for nested object field types and type filters for types referred in join fields, for example: 1.0kb.
- segments.index_writer_memory (or siwm, segmentsIndexWriterMemory): Memory used by index writer, for example: 18mb.
- segments.memory (or sm, segmentsMemory): Memory used by segments, for example: 1.4kb.
- segments.version_map_memory (or svmm, segmentsVersionMapMemory): Memory used by version map, for example: 1.0kb.
- seq_no.global_checkpoint (or sqg, globalCheckpoint): Global checkpoint.
- seq_no.local_checkpoint (or sql, localCheckpoint): Local checkpoint.
- seq_no.max (or sqm, maxSeqNo): Maximum sequence number.
- shard (or s, sh): Name of the shard.
- dsparse_vector.value_count (or svc, sparseVectorCount): Number of indexed sparse vectors.
- state (or st): State of the shard. Returned values are:
  - INITIALIZING: The shard is recovering from a peer shard or gateway.
  - RELOCATING: The shard is relocating.
  - STARTED: The shard has started.
  - UNASSIGNED: The shard is not assigned to any node.
- store (or sto): Disk space used by the shard, for example: 5kb.
- suggest.current (or suc, suggestCurrent): Number of current suggest operations, for example: 0.
- suggest.time (or suti, suggestTime): Time spent in suggest, for example: 0.
- suggest.total (or suto, suggestTotal): Number of suggest operations, for example: 0.
- sync_id: Sync ID of the shard.
- unassigned.at (or ua): Time at which the shard became unassigned in Coordinated Universal Time (UTC).
- unassigned.details (or ud): Details about why the shard became unassigned. This does not explain why the shard is currently unassigned. To understand why a shard is not assigned, use the Cluster allocation explain API.
- unassigned.for (or uf): Time at which the shard was requested to be unassigned in Coordinated Universal Time (UTC).
- unassigned.reason (or ur): Indicates the reason for the last change to the state of this unassigned shard. This does not explain why the shard is currently unassigned. To understand why a shard is not assigned, use the Cluster allocation explain API. Returned values include:
  - ALLOCATION_FAILED: Unassigned as a result of a failed allocation of the shard.
  - CLUSTER_RECOVERED: Unassigned as a result of a full cluster recovery.
  - DANGLING_INDEX_IMPORTED: Unassigned as a result of importing a dangling index.
  - EXISTING_INDEX_RESTORED: Unassigned as a result of restoring into a closed index.
  - FORCED_EMPTY_PRIMARY: The shard’s allocation was last modified by forcing an empty primary using the Cluster reroute API.
  - INDEX_CLOSED: Unassigned because the index was closed.
  - INDEX_CREATED: Unassigned as a result of an API creation of an index.
  - INDEX_REOPENED: Unassigned as a result of opening a closed index.
  - MANUAL_ALLOCATION: The shard’s allocation was last modified by the Cluster reroute API.
  - NEW_INDEX_RESTORED: Unassigned as a result of restoring into a new index.
  - NODE_LEFT: Unassigned as a result of the node hosting it leaving the cluster.
  - NODE_RESTARTING: Similar to NODE_LEFT, except that the node was registered as restarting using the Node shutdown API.
  - PRIMARY_FAILED: The shard was initializing as a replica, but the primary shard failed before the initialization completed.
  - REALLOCATED_REPLICA: A better replica location is identified and causes the existing replica allocation to be cancelled.
  - REINITIALIZED: When a shard moves from started back to initializing.
  - REPLICA_ADDED: Unassigned as a result of explicit addition of a replica.
  - REROUTE_CANCELLED: Unassigned as a result of explicit cancel reroute command.
Values are completion.size, cs, completionSize, dataset.size, dense_vector.value_count, dvc, denseVectorCount, docs, d, dc, fielddata.evictions, fe, fielddataEvictions, fielddata.memory_size, fm, fielddataMemory, flush.total, ft, flushTotal, flush.total_time, ftt, flushTotalTime, get.current, gc, getCurrent, get.exists_time, geti, getExistsTime, get.exists_total, geto, getExistsTotal, get.missing_time, gmti, getMissingTime, get.missing_total, gmto, getMissingTotal, get.time, gti, getTime, get.total, gto, getTotal, id, index, i, idx, indexing.delete_current, idc, indexingDeleteCurrent, indexing.delete_time, idti, indexingDeleteTime, indexing.delete_total, idto, indexingDeleteTotal, indexing.index_current, iic, indexingIndexCurrent, indexing.index_failed_due_to_version_conflict, iifvc, indexingIndexFailedDueToVersionConflict, indexing.index_failed, iif, indexingIndexFailed, indexing.index_time, iiti, indexingIndexTime, indexing.index_total, iito, indexingIndexTotal, ip, merges.current, mc, mergesCurrent, merges.current_docs, mcd, mergesCurrentDocs, merges.current_size, mcs, mergesCurrentSize, merges.total, mt, mergesTotal, merges.total_docs, mtd, mergesTotalDocs, merges.total_size, mts, mergesTotalSize, merges.total_time, mtt, mergesTotalTime, node, n, prirep, p, pr, primaryOrReplica, query_cache.evictions, qce, queryCacheEvictions, query_cache.memory_size, qcm, queryCacheMemory, recoverysource.type, rs, refresh.time, rti, refreshTime, refresh.total, rto, refreshTotal, search.fetch_current, sfc, searchFetchCurrent, search.fetch_time, sfti, searchFetchTime, search.fetch_total, sfto, searchFetchTotal, search.open_contexts, so, searchOpenContexts, search.query_current, sqc, searchQueryCurrent, search.query_time, sqti, searchQueryTime, search.query_total, sqto, searchQueryTotal, search.scroll_current, scc, searchScrollCurrent, search.scroll_time, scti, searchScrollTime, search.scroll_total, scto, searchScrollTotal, segments.count, sc, segmentsCount, segments.fixed_bitset_memory, sfbm, fixedBitsetMemory, segments.index_writer_memory, siwm, segmentsIndexWriterMemory, segments.memory, sm, segmentsMemory, segments.version_map_memory, svmm, segmentsVersionMapMemory, seq_no.global_checkpoint, sqg, globalCheckpoint, seq_no.local_checkpoint, sql, localCheckpoint, seq_no.max, sqm, maxSeqNo, shard, s, sh, dsparse_vector.value_count, svc, sparseVectorCount, state, st, store, sto, suggest.current, suc, suggestCurrent, suggest.time, suti, suggestTime, suggest.total, suto, suggestTotal, sync_id, unassigned.at, ua, unassigned.details, ud, unassigned.for, uf, unassigned.reason, or ur.
s string | array[string]

A comma-separated list of column names or aliases that determines the sort order. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.
master_timeout string

The period to wait for a connection to the master node.

Values are -1 or 0.
time string

The unit used to display time values.

Values are nanos, micros, ms, s, m, h, or d.

Responses

200 application/json
Hide response attributes Show response attributes object
- index string
  
  The index name.
- shard string
  
  The shard name.
- prirep string
  
  The shard type: primary or replica.
- state string
  
  The shard state. Returned values include: INITIALIZING: The shard is recovering from a peer shard or gateway. RELOCATING: The shard is relocating. STARTED: The shard has started. UNASSIGNED: The shard is not assigned to any node.
- docs string | null
  
  The number of documents in the shard.
  
  One of:
  string-1 string string-2 string | null
- store string | null
  
  The disk space used by the shard.
  
  One of:
  string-1 string string-2 string | null
- dataset string | null
  
  total size of dataset (including the cache for partially mounted indices)
  
  One of:
  string-1 string string-2 string | null
- ip string | null
  
  The IP address of the node.
  
  One of:
  string-1 string string-2 string | null
- id string
  
  The unique identifier for the node.
- node string | null
  
  The name of node.
  
  One of:
  string-1 string string-2 string | null
- sync_id string
  
  The sync identifier.
- unassigned.reason string
  
  The reason for the last change to the state of an unassigned shard. It does not explain why the shard is currently unassigned; use the cluster allocation explain API for that information. Returned values include: ALLOCATION_FAILED: Unassigned as a result of a failed allocation of the shard. CLUSTER_RECOVERED: Unassigned as a result of a full cluster recovery. DANGLING_INDEX_IMPORTED: Unassigned as a result of importing a dangling index. EXISTING_INDEX_RESTORED: Unassigned as a result of restoring into a closed index. FORCED_EMPTY_PRIMARY: The shard’s allocation was last modified by forcing an empty primary using the cluster reroute API. INDEX_CLOSED: Unassigned because the index was closed. INDEX_CREATED: Unassigned as a result of an API creation of an index. INDEX_REOPENED: Unassigned as a result of opening a closed index. MANUAL_ALLOCATION: The shard’s allocation was last modified by the cluster reroute API. NEW_INDEX_RESTORED: Unassigned as a result of restoring into a new index. NODE_LEFT: Unassigned as a result of the node hosting it leaving the cluster. NODE_RESTARTING: Similar to NODE_LEFT, except that the node was registered as restarting using the node shutdown API. PRIMARY_FAILED: The shard was initializing as a replica, but the primary shard failed before the initialization completed. REALLOCATED_REPLICA: A better replica location is identified and causes the existing replica allocation to be cancelled. REINITIALIZED: When a shard moves from started back to initializing. REPLICA_ADDED: Unassigned as a result of explicit addition of a replica. REROUTE_CANCELLED: Unassigned as a result of explicit cancel reroute command.
- unassigned.at string
  
  The time at which the shard became unassigned in Coordinated Universal Time (UTC).
- unassigned.for string
  
  The time at which the shard was requested to be unassigned in Coordinated Universal Time (UTC).
- unassigned.details string
  
  Additional details as to why the shard became unassigned. It does not explain why the shard is not assigned; use the cluster allocation explain API for that information.
- recoverysource.type string
  
  The type of recovery source.
- completion.size string
  
  The size of completion.
- fielddata.memory_size string
  
  The used fielddata cache memory.
- fielddata.evictions string
  
  The fielddata cache evictions.
- query_cache.memory_size string
  
  The used query cache memory.
- query_cache.evictions string
  
  The query cache evictions.
- flush.total string
  
  The number of flushes.
- flush.total_time string
  
  The time spent in flush.
- get.current string
  
  The number of current get operations.
- get.time string
  
  The time spent in get operations.
- get.total string
  
  The number of get operations.
- get.exists_time string
  
  The time spent in successful get operations.
- get.exists_total string
  
  The number of successful get operations.
- get.missing_time string
  
  The time spent in failed get operations.
- get.missing_total string
  
  The number of failed get operations.
- indexing.delete_current string
  
  The number of current deletion operations.
- indexing.delete_time string
  
  The time spent in deletion operations.
- indexing.delete_total string
  
  The number of delete operations.
- indexing.index_current string
  
  The number of current indexing operations.
- indexing.index_time string
  
  The time spent in indexing operations.
- indexing.index_total string
  
  The number of indexing operations.
- indexing.index_failed string
  
  The number of failed indexing operations.
- merges.current string
  
  The number of current merge operations.
- merges.current_docs string
  
  The number of current merging documents.
- merges.current_size string
  
  The size of current merge operations.
- merges.total string
  
  The number of completed merge operations.
- merges.total_docs string
  
  The nuber of merged documents.
- merges.total_size string
  
  The size of current merges.
- merges.total_time string
  
  The time spent merging documents.
- refresh.total string
  
  The total number of refreshes.
- refresh.time string
  
  The time spent in refreshes.
- refresh.external_total string
  
  The total nunber of external refreshes.
- refresh.external_time string
  
  The time spent in external refreshes.
- refresh.listeners string
  
  The number of pending refresh listeners.
- search.fetch_current string
  
  The current fetch phase operations.
- search.fetch_time string
  
  The time spent in fetch phase.
- search.fetch_total string
  
  The total number of fetch operations.
- search.open_contexts string
  
  The number of open search contexts.
- search.query_current string
  
  The current query phase operations.
- search.query_time string
  
  The time spent in query phase.
- search.query_total string
  
  The total number of query phase operations.
- search.scroll_current string
  
  The open scroll contexts.
- search.scroll_time string
  
  The time scroll contexts were held open.
- search.scroll_total string
  
  The number of completed scroll contexts.
- segments.count string
  
  The number of segments.
- segments.memory string
  
  The memory used by segments.
- segments.index_writer_memory string
  
  The memory used by the index writer.
- segments.version_map_memory string
  
  The memory used by the version map.
- segments.fixed_bitset_memory string
  
  The memory used by fixed bit sets for nested object field types and export type filters for types referred in _parent fields.
- seq_no.max string
  
  The maximum sequence number.
- seq_no.local_checkpoint string
  
  The local checkpoint.
- seq_no.global_checkpoint string
  
  The global checkpoint.
- warmer.current string
  
  The number of current warmer operations.
- warmer.total string
  
  The total number of warmer operations.
- warmer.total_time string
  
  The time spent in warmer operations.
- path.data string
  
  The shard data path.
- path.state string
  
  The shard state path.
- bulk.total_operations string
  
  The number of bulk shard operations.
- bulk.total_time string
  
  The time spent in shard bulk operations.
- bulk.total_size_in_bytes string
  
  The total size in bytes of shard bulk operations.
- bulk.avg_time string
  
  The average time spent in shard bulk operations.
- bulk.avg_size_in_bytes string
  
  The average size in bytes of shard bulk operations.

GET /_cat/shards/{index}

GET _cat/shards?format=json

resp = client.cat.shards(
    format="json",
)

const response = await client.cat.shards({
  format: "json",
});

response = client.cat.shards(
  format: "json"
)

$resp = $client->cat()->shards([
    "format" => "json",
]);

curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_cat/shards?format=json"

client.cat().shards();

Response examples (200)

A successful response from `GET _cat/shards?format=json`.

[
  {
    "index": "my-index-000001",
    "shard": "0",
    "prirep": "p",
    "state": "STARTED",
    "docs": "3014",
    "store": "31.1mb",
    "dataset": "249b",
    "ip": "192.168.56.10",
    "node": "H5dfFeA"
  }
]

A successful response from `GET _cat/shards/my-index-*?format=json`. It returns information for any data streams or indices beginning with `my-index-`.

[
  {
    "index": "my-index-000001",
    "shard": "0",
    "prirep": "p",
    "state": "STARTED",
    "docs": "3014",
    "store": "31.1mb",
    "dataset": "249b",
    "ip": "192.168.56.10",
    "node": "H5dfFeA"
  }
]

A successful response from `GET _cat/shards?format=json`. The `RELOCATING` value in the `state` column indicates the index shard is relocating.

[
  {
    "index": "my-index-000001",
    "shard": "0",
    "prirep": "p",
    "state": "RELOCATING",
    "docs": "3014",
    "store": "31.1mb",
    "dataset": "249b",
    "ip": "192.168.56.10",
    "node": "H5dfFeA -> -> 192.168.56.30 bGG90GE"
  }
]

A successful response from `GET _cat/shards?format=json`. Before a shard is available for use, it goes through an `INITIALIZING` state. You can use the cat shards API to see which shards are initializing.

[
  {
    "index": "my-index-000001",
    "shard": "0",
    "prirep": "p",
    "state": "STARTED",
    "docs": "3014",
    "store": "31.1mb",
    "dataset": "249b",
    "ip": "192.168.56.10",
    "node": "H5dfFeA"
  },
  {
    "index": "my-index-000001",
    "shard": "0",
    "prirep": "r",
    "state": "INITIALIZING",
    "docs": "0",
    "store": "14.3mb",
    "dataset": "249b",
    "ip": "192.168.56.30",
    "node": "bGG90GE"
  }
]

A successful response from `GET _cat/shards?h=index,shard,prirep,state,unassigned.reason&format=json`. It includes the `unassigned.reason` column, which indicates why a shard is unassigned.

[
  {
    "index": "my-index-000001",
    "shard": "0",
    "prirep": "p",
    "state": "STARTED",
    "unassigned.reason": "3014 31.1mb 192.168.56.10 H5dfFeA"
  },
  {
    "index": "my-index-000001",
    "shard": "0",
    "prirep": "r",
    "state": "STARTED",
    "unassigned.reason": "3014 31.1mb 192.168.56.30 bGG90GE"
  },
  {
    "index": "my-index-000001",
    "shard": "0",
    "prirep": "r",
    "state": "STARTED",
    "unassigned.reason": "3014 31.1mb 192.168.56.20 I8hydUG"
  },
  {
    "index": "my-index-000001",
    "shard": "0",
    "prirep": "r",
    "state": "UNASSIGNED",
    "unassigned.reason": "ALLOCATION_FAILED"
  }
]

Get the cluster health Generally available; Added in 8.7.0

GET /_health_report/{feature}

Api key auth Basic auth Bearer auth

All methods and paths for this operation:

GET /_health_report

GET /_health_report/{feature}

Get a report with the health status of an Elasticsearch cluster. The report contains a list of indicators that compose Elasticsearch functionality.

Each indicator has a health status of: green, unknown, yellow or red. The indicator will provide an explanation and metadata describing the reason for its current health status.

The cluster’s status is controlled by the worst indicator status.

In the event that an indicator’s status is non-green, a list of impacts may be present in the indicator result which detail the functionalities that are negatively affected by the health issue. Each impact carries with it a severity level, an area of the system that is affected, and a simple description of the impact on the system.

Some health indicators can determine the root cause of a health problem and prescribe a set of steps that can be performed in order to improve the health of the system. The root cause and remediation steps are encapsulated in a diagnosis. A diagnosis contains a cause detailing a root cause analysis, an action containing a brief description of the steps to take to fix the problem, the list of affected resources (if applicable), and a detailed step-by-step troubleshooting guide to fix the diagnosed problem.

NOTE: The health indicators perform root cause analysis of non-green health statuses. This can be computationally expensive when called frequently. When setting up automated polling of the API for health status, set verbose to false to disable the more expensive analysis logic.

Path parameters

feature string | array[string] Required

A feature of the cluster, as returned by the top-level health report API.

Query parameters

timeout string

Explicit operation timeout.

Values are -1 or 0.
verbose boolean

Opt-in for more information about the health of the system.
size number

Limit the number of affected resources the health report API returns.

Responses

200 application/json
Hide response attributes Show response attributes object
- cluster_name string Required
- indicators object Required
  
  Hide indicators attributes Show indicators attributes object
  
  master_is_stable object
  
  MASTER_IS_STABLE
  
  Hide master_is_stable attributes Show master_is_stable attributes object
  
  symptom string Required
  
  impacts array[object]
  
  diagnosis array[object]
  
  shards_availability object
  
  SHARDS_AVAILABILITY
  
  Hide shards_availability attributes Show shards_availability attributes object
  
  symptom string Required
  
  impacts array[object]
  
  diagnosis array[object]
  
  disk object
  
  DISK
  
  Hide disk attributes Show disk attributes object
  
  symptom string Required
  
  impacts array[object]
  
  diagnosis array[object]
  
  repository_integrity object
  
  REPOSITORY_INTEGRITY
  
  Hide repository_integrity attributes Show repository_integrity attributes object
  
  symptom string Required
  
  impacts array[object]
  
  diagnosis array[object]
  
  data_stream_lifecycle object
  
  DATA_STREAM_LIFECYCLE
  
  Hide data_stream_lifecycle attributes Show data_stream_lifecycle attributes object
  
  symptom string Required
  
  impacts array[object]
  
  diagnosis array[object]
  
  ilm object
  
  ILM
  
  Hide ilm attributes Show ilm attributes object
  
  symptom string Required
  
  impacts array[object]
  
  diagnosis array[object]
  
  slm object
  
  SLM
  
  Hide slm attributes Show slm attributes object
  
  symptom string Required
  
  impacts array[object]
  
  diagnosis array[object]
  
  shards_capacity object
  
  SHARDS_CAPACITY
  
  Hide shards_capacity attributes Show shards_capacity attributes object
  
  symptom string Required
  
  impacts array[object]
  
  diagnosis array[object]
- status string
  
  Values are green, yellow, red, unknown, or unavailable.

GET /_health_report/{feature}

GET _health_report

resp = client.health_report()

const response = await client.healthReport();

response = client.health_report

$resp = $client->healthReport();

curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_health_report"

client.healthReport(h -> h);

Get all connector sync jobs Beta; Added in 8.12.0

GET /_connector/_sync_job

Api key auth Basic auth Bearer auth

Get information about all stored connector sync jobs listed by their creation date in ascending order.

Query parameters

from number

Starting offset (default: 0)
size number

Specifies a max number of results to get
status string

A sync job status to fetch connector sync jobs for

Values are canceling, canceled, completed, error, in_progress, pending, or suspended.
connector_id string

A connector id to fetch connector sync jobs for
job_type string | array[string]

A comma-separated list of job types to fetch the sync jobs for

Supported values include: full, incremental, access_control

Values are full, incremental, or access_control.

Responses

200 application/json
Hide response attributes Show response attributes object
- count number Required
- results array[object] Required
  
  Hide results attributes Show results attributes object
  
  cancelation_requested_at string | number
  
  One of:
  string-1 string UnitMillis number
  
  canceled_at string | number
  
  One of:
  string-1 string UnitMillis number
  
  completed_at string | number
  
  One of:
  string-1 string UnitMillis number
  
  connector object Required
  
  Hide connector attributes Show connector attributes object
  
  configuration object Required
  
  filtering object Required
  
  id string Required
  
  index_name string Required
  
  language string
  
  pipeline object
  
  service_type string Required
  
  sync_cursor object
  
  created_at string | number
  
  One of:
  string-1 string UnitMillis number
  
  deleted_document_count number Required
  
  error string
  
  id string Required
  
  indexed_document_count number Required
  
  indexed_document_volume number Required
  
  job_type string Required
  
  Values are full, incremental, or access_control.
  
  last_seen string | number
  
  One of:
  string-1 string UnitMillis number
  
  metadata object Required
  
  Hide metadata attribute Show metadata attribute object
  
  * object Additional properties
  
  started_at string | number
  
  One of:
  string-1 string UnitMillis number
  
  status string Required
  
  Values are canceling, canceled, completed, error, in_progress, pending, or suspended.
  
  total_document_count number Required
  
  trigger_method string Required
  
  Values are on_demand or scheduled.
  
  worker_hostname string

GET /_connector/_sync_job

GET _connector/_sync_job?connector_id=my-connector-id&size=1

resp = client.connector.sync_job_list(
    connector_id="my-connector-id",
    size="1",
)

const response = await client.connector.syncJobList({
  connector_id: "my-connector-id",
  size: 1,
});

response = client.connector.sync_job_list(
  connector_id: "my-connector-id",
  size: "1"
)

$resp = $client->connector()->syncJobList([
    "connector_id" => "my-connector-id",
    "size" => "1",
]);

curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_connector/_sync_job?connector_id=my-connector-id&size=1"

client.connector().syncJobList(s -> s
    .connectorId("my-connector-id")
    .size(1)
);

Executes several fleet searches with a single API request Technical preview; Added in 7.16.0

POST /{index}/_fleet/_fleet_msearch

Api key auth Basic auth Bearer auth

All methods and paths for this operation:

GET /_fleet/_fleet_msearch

POST /_fleet/_fleet_msearch

GET /{index}/_fleet/_fleet_msearch

POST /{index}/_fleet/_fleet_msearch

The API follows the same structure as the multi search (_msearch) API. However, similar to the fleet search API, it supports the wait_for_checkpoints parameter.

Required authorization

Index privileges: read

External documentation

Path parameters

index string Required

A single target to search. If the target is an index alias, it must resolve to a single index.

Query parameters

allow_no_indices boolean

If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.
ccs_minimize_roundtrips boolean

If true, network roundtrips between the coordinating node and remote clusters are minimized for cross-cluster search requests.
expand_wildcards string | array[string]
Type of index that wildcard expressions can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.

Supported values include:
- all: Match any data stream or index, including hidden ones.
- open: Match open, non-hidden indices. Also matches any non-hidden data stream.
- closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
- hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
- none: Wildcard expressions are not accepted.
Values are all, open, closed, hidden, or none.
ignore_throttled boolean

If true, concrete, expanded or aliased indices are ignored when frozen.
ignore_unavailable boolean

If true, missing or closed indices are not included in the response.
max_concurrent_searches number

Maximum number of concurrent searches the multi search API can execute.
max_concurrent_shard_requests number

Maximum number of concurrent shard requests that each sub-search request executes per node.
pre_filter_shard_size number

Defines a threshold that enforces a pre-filter roundtrip to prefilter search shards based on query rewriting if the number of shards the search request expands to exceeds the threshold. This filter roundtrip can limit the number of shards significantly if for instance a shard can not match any documents based on its rewrite method i.e., if date filters are mandatory to match but the shard bounds and the query are disjoint.
search_type string
Indicates whether global term and document frequencies should be used when scoring returned documents.

Supported values include:
- query_then_fetch: Documents are scored using local term and document frequencies for the shard. This is usually faster but less accurate.
- dfs_query_then_fetch: Documents are scored using global term and document frequencies across all shards. This is usually slower but more accurate.
Values are query_then_fetch or dfs_query_then_fetch.
rest_total_hits_as_int boolean

If true, hits.total are returned as an integer in the response. Defaults to false, which returns an object.
typed_keys boolean

Specifies whether aggregation and suggester names should be prefixed by their respective types in the response.
wait_for_checkpoints array[number]

A comma separated list of checkpoints. When configured, the search API will only be executed on a shard after the relevant checkpoint has become visible for search. Defaults to an empty list which will cause Elasticsearch to immediately execute the search.
allow_partial_search_results boolean

If true, returns partial results if there are shard request timeouts or shard failures. If false, returns an error with no partial results. Defaults to the configured cluster setting search.default_allow_partial_results which is true by default.

application/json

Body object Required

Contains parameters used to limit or change the subsequent search body request.

allow_no_indices boolean
expand_wildcards string | array[string]
Supported values include:
- all: Match any data stream or index, including hidden ones.
- open: Match open, non-hidden indices. Also matches any non-hidden data stream.
- closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
- hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
- none: Wildcard expressions are not accepted.
ignore_unavailable boolean
index string | array[string]
preference string
request_cache boolean
routing string
search_type string
Supported values include:
- query_then_fetch: Documents are scored using local term and document frequencies for the shard. This is usually faster but less accurate.
- dfs_query_then_fetch: Documents are scored using global term and document frequencies across all shards. This is usually slower but more accurate.
Values are query_then_fetch or dfs_query_then_fetch.
ccs_minimize_roundtrips boolean
allow_partial_search_results boolean
ignore_throttled boolean

aggregations object
External documentation
collapse object
External documentation
query object

Defines the search definition using the Query DSL.

External documentation
explain boolean

If true, returns detailed information about score computation as part of a hit.

Default value is false.
ext object

Configuration of search extensions defined by Elasticsearch plugins.
Hide ext attribute Show ext attribute object
- * object Additional properties
stored_fields string | array[string]

List of stored fields to return as part of a hit. If no fields are specified, no stored fields are included in the response. If this field is specified, the _source parameter defaults to false. You can pass _source: true to return both source fields and stored fields in the search response.
docvalue_fields array[object]

Array of wildcard (*) patterns. The request returns doc values for field names matching these patterns in the hits.fields property of the response.

A reference to a field with formatting instructions on how to return the value
Hide docvalue_fields attributes Show docvalue_fields attributes object
- field string Required
  
  A wildcard pattern. The request returns values for field names matching this pattern.
- format string
  
  The format in which the values are returned.
- include_unmapped boolean
knn object | array[object]

Defines the approximate kNN search to run.
One of:
KnnSearch object array-2 array[object]
Hide attributes Show attributes

field string Required

The name of the vector field to search against

query_vector array[number]

The query vector

query_vector_builder object

The query vector builder. You must provide a query_vector_builder or query_vector, but not both.

k number

The final number of nearest neighbors to return as top hits

num_candidates number

The number of nearest neighbor candidates to consider per shard

boost number

Boost value to apply to kNN scores

filter object | array[object]

Filters for the kNN search query

One of:
QueryContainer object array-2 array[object]

An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

External documentation

similarity number

The minimum similarity for a vector to be considered a match

inner_hits object

If defined, each search hit will contain inner hits.

Hide inner_hits attributes Show inner_hits attributes object

size number

The maximum number of hits to return per inner_hits.

Default value is 3.

from number

Inner hit starting document offset.

Default value is 0.

docvalue_fields array[object]

explain boolean

ignore_unmapped boolean

script_fields object

seq_no_primary_term boolean

fields array[string]

Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

track_scores boolean

Default value is false.

version boolean

rescore_vector object

Apply oversampling and rescoring to quantized vectors *

Hide rescore_vector attribute Show rescore_vector attribute object

oversample number Required

Applies the specified oversample factor to k on the approximate kNN search
Hide attributes Show attributes object

field string Required

The name of the vector field to search against

query_vector array[number]

The query vector

query_vector_builder object

The query vector builder. You must provide a query_vector_builder or query_vector, but not both.

k number

The final number of nearest neighbors to return as top hits

num_candidates number

The number of nearest neighbor candidates to consider per shard

boost number

Boost value to apply to kNN scores

filter object | array[object]

Filters for the kNN search query

One of:
QueryContainer object array-2 array[object]

An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

similarity number

The minimum similarity for a vector to be considered a match

inner_hits object

If defined, each search hit will contain inner hits.

rescore_vector object

Apply oversampling and rescoring to quantized vectors *
from number

Starting document offset. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.

Default value is 0.
highlight object
Hide highlight attributes Show highlight attributes object
- type
- boundary_chars string
  
  A string that contains each boundary character.
  
  Default value is .,!? \t\n.
- boundary_max_scan number
  
  How far to scan for boundary characters.
  
  Default value is 20.
- boundary_scanner string
  Specifies how to break the highlighted fragments: chars, sentence, or word. Only valid for the unified and fvh highlighters. Defaults to sentence for the unified highlighter. Defaults to chars for the fvh highlighter.
  
  Supported values include:
  
  chars: Use the characters specified by boundary_chars as highlighting boundaries. The boundary_max_scan setting controls how far to scan for boundary characters. Only valid for the fvh highlighter.
  
  sentence: Break highlighted fragments at the next sentence boundary, as determined by Java’s BreakIterator. You can specify the locale to use with boundary_scanner_locale. When used with the unified highlighter, the sentence scanner splits sentences bigger than fragment_size at the first word boundary next to fragment_size. You can set fragment_size to 0 to never split any sentence.
  
  word: Break highlighted fragments at the next word boundary, as determined by Java’s BreakIterator. You can specify the locale to use with boundary_scanner_locale.
  Values are chars, sentence, or word.
- boundary_scanner_locale string
  
  Controls which locale is used to search for sentence and word boundaries. This parameter takes a form of a language tag, for example: "en-US", "fr-FR", "ja-JP".
  
  Default value is Locale.ROOT.
- force_source boolean Deprecated
- fragmenter string
  
  Specifies how text should be broken up in highlight snippets: simple or span. Only valid for the plain highlighter.
  
  Values are simple or span.
- fragment_size number
  
  The size of the highlighted fragment in characters.
  
  Default value is 100.
- highlight_filter boolean
- highlight_query object
  
  Highlight matches for a query other than the search query. This is especially useful if you use a rescore query because those are not taken into account by highlighting by default.
- max_fragment_length number
- max_analyzed_offset number
  
  If set to a non-negative value, highlighting stops at this defined maximum limit. The rest of the text is not processed, thus not highlighted and no error is returned The max_analyzed_offset query setting does not override the index.highlight.max_analyzed_offset setting, which prevails when it’s set to lower value than the query setting.
- no_match_size number
  
  The amount of text you want to return from the beginning of the field if there are no matching fragments to highlight.
  
  Default value is 0.
- number_of_fragments number
  
  The maximum number of fragments to return. If the number of fragments is set to 0, no fragments are returned. Instead, the entire field contents are highlighted and returned. This can be handy when you need to highlight short texts such as a title or address, but fragmentation is not required. If number_of_fragments is 0, fragment_size is ignored.
  
  Default value is 5.
- options object
  Hide options attribute Show options attribute object
  
  * object Additional properties
- order string
  
  Sorts highlighted fragments by score when set to score. By default, fragments will be output in the order they appear in the field (order: none). Setting this option to score will output the most relevant fragments first. Each highlighter applies its own logic to compute relevancy scores.
  
  Value is score.
- phrase_limit number
  
  Controls the number of matching phrases in a document that are considered. Prevents the fvh highlighter from analyzing too many phrases and consuming too much memory. When using matched_fields, phrase_limit phrases per matched field are considered. Raising the limit increases query time and consumes more memory. Only supported by the fvh highlighter.
  
  Default value is 256.
- post_tags array[string]
  
  Use in conjunction with pre_tags to define the HTML tags to use for the highlighted text. By default, highlighted text is wrapped in <em> and </em> tags.
- pre_tags array[string]
  
  Use in conjunction with post_tags to define the HTML tags to use for the highlighted text. By default, highlighted text is wrapped in <em> and </em> tags.
- require_field_match boolean
  
  By default, only fields that contains a query match are highlighted. Set to false to highlight all fields.
  
  Default value is true.
- tags_schema string
  
  Set to styled to use the built-in tag schema.
  
  Value is styled.
- encoder string
  
  Values are default or html.
- fields object Required
indices_boost array[object]

Boosts the _score of documents from specified indices.
Hide indices_boost attribute Show indices_boost attribute object
- * number Additional properties
min_score number

The minimum _score for matching documents. Documents with a lower _score are not included in search results and results collected by aggregations.
post_filter object

An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

External documentation
profile boolean
rescore object | array[object]
One of:
object-2 object array-2 array[object]
Hide attributes Show attributes

window_size number

query object

learning_to_rank object
Hide attribute Show attribute object

window_size number
script_fields object

Retrieve a script evaluation (based on different fields) for each hit.
Hide script_fields attribute Show script_fields attribute object
- * object Additional properties
  Hide * attributes Show * attributes object
  
  script object Required
  
  Hide script attributes Show script attributes object
  
  source string
  
  The script source.
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  options object
  
  ignore_failure boolean
search_after array[number | string | boolean | null | object]

A field value.

A field value.

One of:
number-1 number number-2 number string-3 string boolean-4 boolean string-5 string | null object-6 object
size number

The number of hits to return. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.

Default value is 10.
sort string | object | array[string | object]

One of:
Field string SortOptions object array-2 array[string | object]

Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
_source boolean | object

Indicates which source fields are returned for matching documents. These fields are returned in the hits._source property of the search response.
One of:
boolean-1 boolean SourceFilter object

Indicates which source fields are returned for matching documents. These fields are returned in the hits._source property of the search response.
Indicates which source fields are returned for matching documents. These fields are returned in the hits._source property of the search response.
Hide attributes Show attributes

exclude_vectors boolean

If true, vector fields are excluded from the returned source.

This option takes precedence over includes: any vector field will remain excluded even if it matches an includes rule.

excludes string | array[string]

A list of fields to exclude from the returned source.

includes string | array[string]

A list of fields to include in the returned source.
fields array[object]

Array of wildcard (*) patterns. The request returns values for field names matching these patterns in the hits.fields property of the response.

A reference to a field with formatting instructions on how to return the value
Hide fields attributes Show fields attributes object
- field string Required
  
  A wildcard pattern. The request returns values for field names matching this pattern.
- format string
  
  The format in which the values are returned.
- include_unmapped boolean
terminate_after number

Maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting. Defaults to 0, which does not terminate query execution early.

Default value is 0.
stats array[string]

Stats groups to associate with the search. Each group maintains a statistics aggregation for its associated searches. You can retrieve these stats using the indices stats API.
timeout string

Specifies the period of time to wait for a response from each shard. If no response is received before the timeout expires, the request fails and returns an error. Defaults to no timeout.
track_scores boolean

If true, calculate and return document scores, even if the scores are not used for sorting.

Default value is false.
track_total_hits boolean | number

Number of hits matching the query to count accurately. If true, the exact number of hits is returned at the cost of some performance. If false, the response does not include the total number of hits matching the query. Defaults to 10,000 hits.
version boolean

If true, returns document version as part of a hit.

Default value is false.
runtime_mappings object

Defines one or more runtime fields in the search request. These fields take precedence over mapped fields with the same name.
Hide runtime_mappings attribute Show runtime_mappings attribute object
- * object Additional properties
  Hide * attributes Show * attributes object
  
  fields object
  
  For type composite
  
  Hide fields attribute Show fields attribute object
  
  * object Additional properties
  
  fetch_fields array[object]
  
  For type lookup
  
  format string
  
  A custom format for date type runtime fields.
  
  input_field string
  
  For type lookup
  
  target_field string
  
  For type lookup
  
  target_index string
  
  For type lookup
  
  script object
  
  Painless script executed at query time.
  
  type string Required
  
  Field type, which can be: boolean, composite, date, double, geo_point, ip,keyword, long, or lookup.
  
  Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.
seq_no_primary_term boolean

If true, returns sequence number and primary term of the last modification of each hit. See Optimistic concurrency control.
pit object

Limits the search to a point in time (PIT). If you provide a PIT, you cannot specify an in the request path.
Hide pit attributes Show pit attributes object
- id string Required
- keep_alive string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
suggest object
Hide suggest attribute Show suggest attribute object
- text string
  
  Global suggest text, to avoid repetition when the same text is used in several suggesters

Responses

200 application/json
Hide response attribute Show response attribute object
- docs array[object] Required
  
  One of:
  object-2 object ErrorResponseBase object
  
  Hide attributes Show attributes
  
  took number Required
  
  The number of milliseconds it took Elasticsearch to run the request. This value is calculated by measuring the time elapsed between receipt of a request on the coordinating node and the time at which the coordinating node is ready to send the response. It includes:
  
  Communication time between the coordinating node and data nodes
  
  Time the request spends in the search thread pool, queued for execution
  
  Actual run time
  
  It does not include:
  
  Time needed to send the request to Elasticsearch
  
  Time needed to serialize the JSON response
  
  Time needed to send the response to a client
  
  timed_out boolean Required
  
  If true, the request timed out before completion; returned results may be partial or empty.
  
  _shards object Required
  
  A count of shards used for the request.
  
  hits object Required
  
  The returned documents and metadata.
  
  aggregations object
  
  _clusters object
  
  fields object
  
  Hide fields attribute Show fields attribute object
  
  * object Additional properties
  
  max_score number
  
  num_reduce_phases number
  
  profile object
  
  pit_id string
  
  _scroll_id string
  
  The identifier for the search and its search context. You can use this scroll ID with the scroll API to retrieve the next batch of search results for the request. This property is returned only if the scroll query parameter is specified in the request.
  
  suggest object
  
  Hide suggest attribute Show suggest attribute object
  
  * array[object] Additional properties
  
  terminated_early boolean
  
  status number
  
  The response returned by Elasticsearch when request execution did not succeed.
  
  Hide attributes Show attributes
  
  error object Required
  
  Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.
  
  Hide error attributes Show error attributes object
  
  type string Required
  
  The type of error
  
  reason
  
  stack_trace string
  
  The server stack trace. Present only if the error_trace=true parameter was sent with the request.
  
  root_cause array[object]
  
  suppressed array[object]
  
  status number Required

POST /{index}/_fleet/_fleet_msearch

curl \
 --request POST 'http://api.example.com/{index}/_fleet/_fleet_msearch' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '[{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"index":"string","preference":"string","request_cache":true,"routing":"string","search_type":"query_then_fetch","ccs_minimize_roundtrips":true,"allow_partial_search_results":true,"ignore_throttled":true}]'

Simulate an index template Generally available

POST /_index_template/_simulate/{name}

Api key auth Basic auth Bearer auth

All methods and paths for this operation:

POST /_index_template/_simulate

POST /_index_template/_simulate/{name}

Get the index configuration that would be applied by a particular index template.

Required authorization

Cluster privileges: manage_index_templates

Path parameters

name string Required

Name of the index template to simulate. To test a template configuration before you add it to the cluster, omit this parameter and specify the template configuration in the request body.

Query parameters

create boolean

If true, the template passed in the body is only used if no existing templates match the same index patterns. If false, the simulation uses the template with the highest priority. Note that the template is not permanently added or updated in either case; it is only used for the simulation.
cause string

User defined reason for dry-run creating the new template for simulation purposes
master_timeout string

Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

Values are -1 or 0.
include_defaults boolean Generally available; Added in 8.11.0

If true, returns all relevant default configurations for the index template.

application/json

Body

allow_auto_create boolean

This setting overrides the value of the action.auto_create_index cluster setting. If set to true in a template, then indices can be automatically created using that template even if auto-creation of indices is disabled via actions.auto_create_index. If set to false, then indices or data streams matching the template must always be explicitly created, and may never be automatically created.
index_patterns string | array[string]

Array of wildcard (*) expressions used to match the names of data streams and indices during creation.
composed_of array[string]

An ordered list of component template names. Component templates are merged in the order specified, meaning that the last component template specified has the highest precedence.
template object

Template to be applied. It may optionally include an aliases, mappings, or settings configuration.
Hide template attributes Show template attributes object
- aliases object
  
  Aliases to add. If the index template includes a data_stream object, these are data stream aliases. Otherwise, these are index aliases. Data stream aliases ignore the index_routing, routing, and search_routing options.
  Hide aliases attribute Show aliases attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  filter object
  
  Query used to limit documents the alias can access.
  
  External documentation
  
  index_routing string
  
  Value used to route indexing operations to a specific shard. If specified, this overwrites the routing value for indexing operations.
  
  is_hidden boolean
  
  If true, the alias is hidden. All indices for the alias must have the same is_hidden value.
  
  Default value is false.
  
  is_write_index boolean
  
  If true, the index is the write index for the alias.
  
  Default value is false.
  
  routing string
  
  Value used to route indexing and search operations to a specific shard.
  
  search_routing string
  
  Value used to route search operations to a specific shard. If specified, this overwrites the routing value for search operations.
- mappings object
  
  Mapping for fields in the index. If specified, this mapping can include field names, field data types, and mapping parameters.
  Hide mappings attributes Show mappings attributes object
  
  all_field object
  
  Hide all_field attributes Show all_field attributes object
  
  analyzer string Required
  
  enabled boolean Required
  
  omit_norms boolean Required
  
  search_analyzer string Required
  
  similarity string Required
  
  store boolean Required
  
  store_term_vector_offsets boolean Required
  
  store_term_vector_payloads boolean Required
  
  store_term_vector_positions boolean Required
  
  store_term_vectors boolean Required
  
  date_detection boolean
  
  dynamic string
  
  Values are strict, runtime, true, or false.
  
  dynamic_date_formats array[string]
  
  dynamic_templates array[object]
  
  _field_names object
  
  Hide _field_names attribute Show _field_names attribute object
  
  enabled boolean Required
  
  index_field object
  
  Hide index_field attribute Show index_field attribute object
  
  enabled boolean Required
  
  _meta object
  
  Hide _meta attribute Show _meta attribute object
  
  * object Additional properties
  
  numeric_detection boolean
  
  properties object
  
  _routing object
  
  Hide _routing attribute Show _routing attribute object
  
  required boolean Required
  
  _size object
  
  Hide _size attribute Show _size attribute object
  
  enabled boolean Required
  
  _source object
  
  Hide _source attributes Show _source attributes object
  
  compress boolean
  
  compress_threshold string
  
  enabled boolean
  
  excludes array[string]
  
  includes array[string]
  
  runtime object
  
  Hide runtime attribute Show runtime attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  fields object
  
  For type composite
  
  fetch_fields array[object]
  
  For type lookup
  
  format string
  
  A custom format for date type runtime fields.
  
  enabled boolean
  
  subobjects string
  
  Values are true or false.
  
  _data_stream_timestamp object
  
  Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
  
  enabled boolean Required
- settings object
  
  Configuration options for the index.
  
  Index settings
- lifecycle object
  
  Data stream lifecycle denotes that a data stream is managed by the data stream lifecycle and contains the configuration.
  Hide lifecycle attributes Show lifecycle attributes object
  
  data_retention string
  
  If defined, every document added to this data stream will be stored at least for this time frame. Any time after this duration the document could be deleted. When empty, every document in this data stream will be stored indefinitely.
  
  downsampling object
  
  The downsampling configuration to execute for the managed backing index after rollover.
  
  Hide downsampling attribute Show downsampling attribute object
  
  rounds array[object] Required
  
  The list of downsampling rounds to execute as part of this downsampling configuration
  
  enabled boolean
  
  If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle that's disabled (enabled: false) will have no effect on the data stream.
  
  Default value is true.
data_stream object

If this object is included, the template is used to create data streams and their backing indices. Supports an empty object. Data streams require a matching index template with a data_stream object.
Hide data_stream attributes Show data_stream attributes object
- hidden boolean
- allow_custom_routing boolean
priority number

Priority to determine index template precedence when a new data stream or index is created. The index template with the highest priority is chosen. If no priority is specified the template is treated as though it is of priority 0 (lowest priority). This number is not automatically generated by Elasticsearch.
version number

Version number used to manage index templates externally. This number is not automatically generated by Elasticsearch.
_meta object

Optional user metadata about the index template. May have any contents. This map is not automatically generated by Elasticsearch.
Hide _meta attribute Show _meta attribute object
- * object Additional properties
ignore_missing_component_templates array[string]

The configuration option ignore_missing_component_templates can be used when an index template references a component template that might not exist
deprecated boolean

Marks this index template as deprecated. When creating or updating a non-deprecated index template that uses deprecated components, Elasticsearch will emit a deprecation warning.

Responses

200 application/json
Hide response attributes Show response attributes object
- overlapping array[object]
  
  Hide overlapping attributes Show overlapping attributes object
  
  name string Required
  
  index_patterns array[string] Required
- template object Required
  
  Hide template attributes Show template attributes object
  
  aliases object Required
  
  Hide aliases attribute Show aliases attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  filter object
  
  Query used to limit documents the alias can access.
  
  index_routing string
  
  Value used to route indexing operations to a specific shard. If specified, this overwrites the routing value for indexing operations.
  
  is_hidden boolean
  
  If true, the alias is hidden. All indices for the alias must have the same is_hidden value.
  
  Default value is false.
  
  is_write_index boolean
  
  If true, the index is the write index for the alias.
  
  Default value is false.
  
  routing string
  
  Value used to route indexing and search operations to a specific shard.
  
  search_routing string
  
  Value used to route search operations to a specific shard. If specified, this overwrites the routing value for search operations.
  
  mappings object Required
  
  Hide mappings attributes Show mappings attributes object
  
  all_field object
  
  date_detection boolean
  
  dynamic string
  
  Values are strict, runtime, true, or false.
  
  dynamic_date_formats array[string]
  
  dynamic_templates array[object]
  
  _field_names object
  
  index_field object
  
  _meta object
  
  numeric_detection boolean
  
  properties object
  
  _routing object
  
  _size object
  
  _source object
  
  runtime object
  
  Hide runtime attribute Show runtime attribute object
  
  * object Additional properties
  
  enabled boolean
  
  subobjects string
  
  Values are true or false.
  
  _data_stream_timestamp object
  
  settings object Required
  Index settings

POST /_index_template/_simulate/{name}

POST /_index_template/_simulate
{
  "index_patterns": ["my-index-*"],
  "composed_of": ["ct2"],
  "priority": 10,
  "template": {
    "settings": {
      "index.number_of_replicas": 1
    }
  }
}

resp = client.indices.simulate_template(
    index_patterns=[
        "my-index-*"
    ],
    composed_of=[
        "ct2"
    ],
    priority=10,
    template={
        "settings": {
            "index.number_of_replicas": 1
        }
    },
)

const response = await client.indices.simulateTemplate({
  index_patterns: ["my-index-*"],
  composed_of: ["ct2"],
  priority: 10,
  template: {
    settings: {
      "index.number_of_replicas": 1,
    },
  },
});

response = client.indices.simulate_template(
  body: {
    "index_patterns": [
      "my-index-*"
    ],
    "composed_of": [
      "ct2"
    ],
    "priority": 10,
    "template": {
      "settings": {
        "index.number_of_replicas": 1
      }
    }
  }
)

$resp = $client->indices()->simulateTemplate([
    "body" => [
        "index_patterns" => array(
            "my-index-*",
        ),
        "composed_of" => array(
            "ct2",
        ),
        "priority" => 10,
        "template" => [
            "settings" => [
                "index.number_of_replicas" => 1,
            ],
        ],
    ],
]);

curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"index_patterns":["my-index-*"],"composed_of":["ct2"],"priority":10,"template":{"settings":{"index.number_of_replicas":1}}}' "$ELASTICSEARCH_URL/_index_template/_simulate"

client.indices().simulateTemplate(s -> s
    .composedOf("ct2")
    .indexPatterns("my-index-*")
    .priority(10L)
    .template(t -> t
        .settings(se -> se
            .otherSettings("index.number_of_replicas", JsonData.fromJson("1"))
        )
    )
);

Request example

To see what settings will be applied by a template before you add it to the cluster, you can pass a template configuration in the request body. The specified template is used for the simulation if it has a higher priority than existing templates.

{
  "index_patterns": ["my-index-*"],
  "composed_of": ["ct2"],
  "priority": 10,
  "template": {
    "settings": {
      "index.number_of_replicas": 1
    }
  }
}

Response examples (200)

A successful response from `POST /_index_template/_simulate` with a template configuration in the request body. The response shows any overlapping templates with a lower priority.

{
  "template" : {
    "settings" : {
      "index" : {
        "number_of_replicas" : "1",
        "routing" : {
          "allocation" : {
            "include" : {
              "_tier_preference" : "data_content"
            }
          }
        }
      }
    },
    "mappings" : {
      "properties" : {
        "@timestamp" : {
          "type" : "date"
        }
      }
    },
    "aliases" : { }
  },
  "overlapping" : [
    {
      "name" : "final-template",
      "index_patterns" : [
        "my-index-*"
      ]
    }
  ]
}

Start the ILM plugin Generally available; Added in 6.6.0

POST /_ilm/start

Api key auth Basic auth Bearer auth

Start the index lifecycle management plugin if it is currently stopped. ILM is started automatically when the cluster is formed. Restarting ILM is necessary only when it has been stopped using the stop ILM API.

Required authorization

Cluster privileges: manage_ilm

Query parameters

master_timeout string

Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

Values are -1 or 0.
timeout string

Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

Values are -1 or 0.

Responses

200 application/json
Hide response attribute Show response attribute object
- acknowledged boolean Required
  
  For a successful response, this value is always true. On failure, an exception is returned instead.

POST /_ilm/start

POST _ilm/start

resp = client.ilm.start()

const response = await client.ilm.start();

response = client.ilm.start

$resp = $client->ilm()->start();

curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_ilm/start"

client.ilm().start(s -> s);

Response examples (200)

A successful response when stating the ILM plugin.

{
  "acknowledged": true
}

Create or update an IP geolocation database configuration Generally available; Added in 8.15.0

PUT /_ingest/ip_location/database/{id}

Api key auth Basic auth Bearer auth

Required authorization

Cluster privileges: manage

Path parameters

id string Required

The database configuration identifier.

Query parameters

master_timeout string

The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error. A value of -1 indicates that the request should never time out.

Values are -1 or 0.
timeout string

The period to wait for a response from all relevant nodes in the cluster after updating the cluster metadata. If no response is received before the timeout expires, the cluster metadata update still applies but the response indicates that it was not completely acknowledged. A value of -1 indicates that the request should never time out.

Values are -1 or 0.

application/json

Body Required

The configuration necessary to identify which IP geolocation provider to use to download a database, as well as any provider-specific configuration necessary for such downloading. At present, the only supported providers are maxmind and ipinfo, and the maxmind provider requires that an account_id (string) is configured. A provider (either maxmind or ipinfo) must be specified. The web and local providers can be returned as read only configurations.

name string Required

The provider-assigned name of the IP geolocation database to download.
maxmind object
Hide maxmind attribute Show maxmind attribute object
- account_id string Required
ipinfo object

Responses

200 application/json
Hide response attribute Show response attribute object
- acknowledged boolean Required
  
  For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /_ingest/ip_location/database/{id}

PUT _ingest/ip_location/database/my-database-1
{
  "name": "GeoIP2-Domain",
  "maxmind": {
    "account_id": "1234567"
  }
}

resp = client.ingest.put_ip_location_database(
    id="my-database-1",
    configuration={
        "name": "GeoIP2-Domain",
        "maxmind": {
            "account_id": "1234567"
        }
    },
)

const response = await client.ingest.putIpLocationDatabase({
  id: "my-database-1",
  configuration: {
    name: "GeoIP2-Domain",
    maxmind: {
      account_id: "1234567",
    },
  },
});

response = client.ingest.put_ip_location_database(
  id: "my-database-1",
  body: {
    "name": "GeoIP2-Domain",
    "maxmind": {
      "account_id": "1234567"
    }
  }
)

$resp = $client->ingest()->putIpLocationDatabase([
    "id" => "my-database-1",
    "body" => [
        "name" => "GeoIP2-Domain",
        "maxmind" => [
            "account_id" => "1234567",
        ],
    ],
]);

curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"name":"GeoIP2-Domain","maxmind":{"account_id":"1234567"}}' "$ELASTICSEARCH_URL/_ingest/ip_location/database/my-database-1"

client.ingest().putIpLocationDatabase(p -> p
    .id("my-database-1")
    .configuration(c -> c
        .maxmind(m -> m
            .accountId("1234567")
        )
        .name("GeoIP2-Domain")
    )
);

Request example

An example body for a `PUT _ingest/ip_location/database/my-database-1` request.

{
  "name": "GeoIP2-Domain",
  "maxmind": {
    "account_id": "1234567"
  }
}

Create or update a pipeline Generally available; Added in 5.0.0

PUT /_ingest/pipeline/{id}

Api key auth Basic auth Bearer auth

Changes made using this API take effect immediately.

External documentation

Path parameters

id string Required

ID of the ingest pipeline to create or update.

Query parameters

master_timeout string

Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

Values are -1 or 0.
timeout string

Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

Values are -1 or 0.
if_version number

Required version for optimistic concurrency control for pipeline updates

application/json

Body Required

_meta object

Optional metadata about the ingest pipeline. May have any contents. This map is not automatically generated by Elasticsearch.
Hide _meta attribute Show _meta attribute object
- * object Additional properties
description string

Description of the ingest pipeline.
on_failure array[object]

Processors to run immediately after a processor failure. Each processor supports a processor-level on_failure value. If a processor without an on_failure value fails, Elasticsearch uses this pipeline-level parameter as a fallback. The processors in this parameter run sequentially in the order specified. Elasticsearch will not attempt to run the pipeline's remaining processors.
Hide on_failure attributes Show on_failure attributes object
- append object
  
  Appends one or more values to an existing array if the field already exists and it is an array. Converts a scalar to an array and appends one or more values to it if the field exists and it is a scalar. Creates an array containing the provided values if the field doesn’t exist. Accepts a single value or an array of values.
  Hide append attributes Show append attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to be appended to. Supports template snippets.
  
  value object | array[object] Required
  
  The value to be appended. Supports template snippets.
  
  One of:
  object-1 object array-2 array[object]
  
  allow_duplicates boolean
  
  If false, the processor does not append values already present in the field.
  
  Default value is true.
- attachment object
  
  The attachment processor lets Elasticsearch extract file attachments in common formats (such as PPT, XLS, and PDF) by using the Apache text extraction library Tika.
  Hide attachment attributes Show attachment attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to get the base64 encoded field from.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  indexed_chars number
  
  The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.
  
  Default value is 100000.
  
  indexed_chars_field string
  
  Field name from which you can overwrite the number of chars being used for extraction.
  
  properties array[string]
  
  Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.
  
  target_field string
  
  The field that will hold the attachment information.
  
  remove_binary boolean
  
  If true, the binary field will be removed from the document
  
  Default value is false.
  
  resource_name string
  
  Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.
- bytes object
  
  Converts a human readable byte value (for example 1kb) to its value in bytes (for example 1024). If the field is an array of strings, all members of the array will be converted. Supported human readable units are "b", "kb", "mb", "gb", "tb", "pb" case insensitive. An error will occur if the field is not a supported format or resultant value exceeds 2^63.
  Hide bytes attributes Show bytes attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to convert.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  target_field string
  
  The field to assign the converted value to. By default, the field is updated in-place.
- circle object
  
  Converts circle definitions of shapes to regular polygons which approximate them.
  Hide circle attributes Show circle attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  error_distance number Required
  
  The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).
  
  field string Required
  
  The field to interpret as a circle. Either a string in WKT format or a map for GeoJSON.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  shape_type string Required
  
  Which field mapping type is to be used when processing the circle: geo_shape or shape.
  
  Values are geo_shape or shape.
  
  target_field string
  
  The field to assign the polygon shape to By default, the field is updated in-place.
- community_id object
  
  Computes the Community ID for network flow data as defined in the Community ID Specification. You can use a community ID to correlate network events related to a single flow.
  Hide community_id attributes Show community_id attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  source_ip string
  
  Field containing the source IP address.
  
  source_port string
  
  Field containing the source port.
  
  destination_ip string
  
  Field containing the destination IP address.
  
  destination_port string
  
  Field containing the destination port.
  
  iana_number string
  
  Field containing the IANA number.
  
  icmp_type string
  
  Field containing the ICMP type.
  
  icmp_code string
  
  Field containing the ICMP code.
  
  transport string
  
  Field containing the transport protocol name or number. Used only when the iana_number field is not present. The following protocol names are currently supported: eigrp, gre, icmp, icmpv6, igmp, ipv6-icmp, ospf, pim, sctp, tcp, udp
  
  target_field string
  
  Output field for the community ID.
  
  seed number
  
  Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.
  
  Default value is 0.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  Default value is true.
- convert object
  
  Converts a field in the currently ingested document to a different type, such as converting a string to an integer. If the field value is an array, all members will be converted.
  Hide convert attributes Show convert attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field whose value is to be converted.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  target_field string
  
  The field to assign the converted value to. By default, the field is updated in-place.
  
  type string Required
  
  The type to convert the existing value to.
  
  Values are integer, long, double, float, boolean, ip, string, or auto.
- csv object
  
  Extracts fields from CSV line out of a single text field within a document. Any empty field in CSV will be skipped.
  Hide csv attributes Show csv attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  empty_value object
  
  Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").
  
  field string Required
  
  The field to extract data from.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  quote string
  
  Quote used in CSV, has to be single character string.
  
  Default value is ".
  
  separator string
  
  Separator used in CSV, has to be single character string.
  
  Default value is ,.
  
  target_fields string | array[string] Required
  
  The array of fields to assign extracted values to.
  
  trim boolean
  
  Trim whitespaces in unquoted fields.
- date object
  
  Parses dates from fields, and then uses the date or timestamp as the timestamp for the document.
  Hide date attributes Show date attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to get the date from.
  
  formats array[string] Required
  
  An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
  
  locale string
  
  The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.
  
  Default value is ENGLISH.
  
  target_field string
  
  The field that will hold the parsed date.
  
  timezone string
  
  The timezone to use when parsing the date. Supports template snippets.
  
  Default value is UTC.
  
  output_format string
  
  The format to use when writing the date to target_field. Must be a valid java time pattern.
  
  Default value is yyyy-MM-dd'T'HH:mm:ss.SSSXXX.
- date_index_name object
  
  The purpose of this processor is to point documents to the right time based index based on a date or timestamp field in a document by using the date math index name support.
  Hide date_index_name attributes Show date_index_name attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  date_formats array[string] Required
  
  An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
  
  date_rounding string Required
  
  How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.
  
  field string Required
  
  The field to get the date or timestamp from.
  
  index_name_format string
  
  The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.
  
  Default value is yyyy-MM-dd.
  
  index_name_prefix string
  
  A prefix of the index name to be prepended before the printed date. Supports template snippets.
  
  locale string
  
  The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.
  
  Default value is ENGLISH.
  
  timezone string
  
  The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.
  
  Default value is UTC.
- dissect object
  
  Extracts structured fields out of a single text field by matching the text field against a delimiter-based pattern.
  Hide dissect attributes Show dissect attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  append_separator string
  
  The character(s) that separate the appended fields.
  
  Default value is "".
  
  field string Required
  
  The field to dissect.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  pattern string Required
  
  The pattern to apply to the field.
- dot_expander object
  
  Expands a field with dots into an object field. This processor allows fields with dots in the name to be accessible by other processors in the pipeline. Otherwise these fields can’t be accessed by any processor.
  Hide dot_expander attributes Show dot_expander attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to expand into an object field. If set to *, all top-level fields will be expanded.
  
  override boolean
  
  Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.
  
  Default value is false.
  
  path string
  
  The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.
- drop object
  
  Drops the document without raising any errors. This is useful to prevent the document from getting indexed based on some condition.
  Hide drop attributes Show drop attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
- enrich object
  
  The enrich processor can enrich documents with data from another index.
  Hide enrich attributes Show enrich attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field in the input document that matches the policies match_field used to retrieve the enrichment data. Supports template snippets.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  max_matches number
  
  The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.
  
  Default value is 1.
  
  override boolean
  
  If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.
  
  Default value is true.
  
  policy_name string Required
  
  The name of the enrich policy to use.
  
  shape_relation string
  
  A spatial relation operator used to match the geoshape of incoming documents to documents in the enrich index. This option is only used for geo_match enrich policy types.
  
  Supported values include:
  
  intersects: Return all documents whose geo_shape or geo_point field intersects the query geometry.
  
  disjoint: Return all documents whose geo_shape or geo_point field has nothing in common with the query geometry.
  
  within: Return all documents whose geo_shape or geo_point field is within the query geometry. Line geometries are not supported.
  
  contains: Return all documents whose geo_shape or geo_point field contains the query geometry.
  
  Values are intersects, disjoint, within, or contains.
  
  target_field string Required
  
  Field added to incoming documents to contain enrich data. This field contains both the match_field and enrich_fields specified in the enrich policy. Supports template snippets.
- fail object
  
  Raises an exception. This is useful for when you expect a pipeline to fail and want to relay a specific message to the requester.
  Hide fail attributes Show fail attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  message string Required
  
  The error message thrown by the processor. Supports template snippets.
- fingerprint object
  
  Computes a hash of the document’s content. You can use this hash for content fingerprinting.
  Hide fingerprint attributes Show fingerprint attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  fields string | array[string] Required
  
  Array of fields to include in the fingerprint. For objects, the processor hashes both the field key and value. For other fields, the processor hashes only the field value.
  
  target_field string
  
  Output field for the fingerprint.
  
  salt string
  
  Salt value for the hash function.
  
  method string
  
  The hash method used to compute the fingerprint. Must be one of MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.
  
  Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.
  
  ignore_missing boolean
  
  If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.
  
  Default value is false.
- foreach object
  
  Runs an ingest processor on each element of an array or object.
  Hide foreach attributes Show foreach attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Field containing array or object values.
  
  ignore_missing boolean
  
  If true, the processor silently exits without changing the document if the field is null or missing.
  
  Default value is false.
  
  processor object Required
  
  Ingest processor to run on each element.
- ip_location object
  
  Currently an undocumented alias for GeoIP Processor.
  Hide ip_location attributes Show ip_location attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  database_file string
  
  The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.
  
  Default value is GeoLite2-City.mmdb.
  
  field string Required
  
  The field to get the ip address from for the geographical lookup.
  
  first_only boolean
  
  If true, only the first found IP location data will be returned, even if the field contains an array.
  
  Default value is true.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  properties array[string]
  
  Controls what properties are added to the target_field based on the IP location lookup.
  
  target_field string
  
  The field that will hold the geographical information looked up from the MaxMind database.
  
  download_database_on_pipeline_creation boolean
  
  If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.
- geo_grid object
  
  Converts geo-grid definitions of grid tiles or cells to regular bounding boxes or polygons which describe their shape. This is useful if there is a need to interact with the tile shapes as spatially indexable fields.
  Hide geo_grid attributes Show geo_grid attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to interpret as a geo-tile.= The field format is determined by the tile_type.
  
  tile_type string Required
  
  Three tile formats are understood: geohash, geotile and geohex.
  
  Values are geotile, geohex, or geohash.
  
  target_field string
  
  The field to assign the polygon shape to, by default, the field is updated in-place.
  
  parent_field string
  
  If specified and a parent tile exists, save that tile address to this field.
  
  children_field string
  
  If specified and children tiles exist, save those tile addresses to this field as an array of strings.
  
  non_children_field string
  
  If specified and intersecting non-child tiles exist, save their addresses to this field as an array of strings.
  
  precision_field string
  
  If specified, save the tile precision (zoom) as an integer to this field.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  target_format string
  
  Which format to save the generated polygon in.
  
  Values are geojson or wkt.
- geoip object
  
  The geoip processor adds information about the geographical location of an IPv4 or IPv6 address.
  Hide geoip attributes Show geoip attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  database_file string
  
  The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.
  
  Default value is GeoLite2-City.mmdb.
  
  field string Required
  
  The field to get the ip address from for the geographical lookup.
  
  first_only boolean
  
  If true, only the first found geoip data will be returned, even if the field contains an array.
  
  Default value is true.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  properties array[string]
  
  Controls what properties are added to the target_field based on the geoip lookup.
  
  target_field string
  
  The field that will hold the geographical information looked up from the MaxMind database.
  
  download_database_on_pipeline_creation boolean
  
  If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.
- grok object
  
  Extracts structured fields out of a single text field within a document. You choose which field to extract matched fields from, as well as the grok pattern you expect will match. A grok pattern is like a regular expression that supports aliased expressions that can be reused.
  Hide grok attributes Show grok attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  ecs_compatibility string
  
  Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.
  
  Default value is disabled.
  
  field string Required
  
  The field to use for grok expression parsing.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  pattern_definitions object
  
  A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.
  
  Hide pattern_definitions attribute Show pattern_definitions attribute object
  
  * string Additional properties
  
  patterns array[string] Required
  
  An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.
  
  trace_match boolean
  
  When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.
  
  Default value is false.
- gsub object
  
  Converts a string field by applying a regular expression and a replacement. If the field is an array of string, all members of the array will be converted. If any non-string values are encountered, the processor will throw an exception.
  Hide gsub attributes Show gsub attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to apply the replacement to.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  pattern string Required
  
  The pattern to be replaced.
  
  replacement string Required
  
  The string to replace the matching patterns with.
  
  target_field string
  
  The field to assign the converted value to By default, the field is updated in-place.
- html_strip object
  
  Removes HTML tags from the field. If the field is an array of strings, HTML tags will be removed from all members of the array.
  Hide html_strip attributes Show html_strip attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The string-valued field to remove HTML tags from.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document,
  
  Default value is false.
  
  target_field string
  
  The field to assign the converted value to By default, the field is updated in-place.
- inference object
  
  Uses a pre-trained data frame analytics model or a model deployed for natural language processing tasks to infer against the data that is being ingested in the pipeline.
  Hide inference attributes Show inference attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  model_id string Required
  
  The ID or alias for the trained model, or the ID of the deployment.
  
  target_field string
  
  Field added to incoming documents to contain results objects.
  
  field_map object
  
  Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.
  
  Hide field_map attribute Show field_map attribute object
  
  * object Additional properties
  
  inference_config object
  
  Contains the inference type and its options.
  
  input_output object | array[object]
  
  Input fields for inference and output (destination) fields for the inference results. This option is incompatible with the target_field and field_map options.
  
  One of:
  InputConfig object array-2 array[object]
  
  ignore_missing boolean
  
  If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.
- join object
  
  Joins each element of an array into a single string using a separator character between each element. Throws an error when the field is not an array.
  Hide join attributes Show join attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Field containing array values to join.
  
  separator string Required
  
  The separator character.
  
  target_field string
  
  The field to assign the joined value to. By default, the field is updated in-place.
- json object
  
  Converts a JSON string into a structured JSON object.
  Hide json attributes Show json attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  add_to_root boolean
  
  Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.
  
  Default value is false.
  
  add_to_root_conflict_strategy string
  
  When set to replace, root fields that conflict with fields from the parsed JSON will be overridden. When set to merge, conflicting fields will be merged. Only applicable if add_to_root is set to true.
  
  Supported values include:
  
  replace: Root fields that conflict with fields from the parsed JSON will be overridden.
  
  merge: Conflicting fields will be merged.
  
  Values are replace or merge.
  
  allow_duplicate_keys boolean
  
  When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.
  
  Default value is false.
  
  field string Required
  
  The field to be parsed.
  
  target_field string
  
  The field that the converted structured object will be written into. Any existing content in this field will be overwritten.
- kv object
  
  This processor helps automatically parse messages (or specific event fields) which are of the foo=bar variety.
  Hide kv attributes Show kv attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  exclude_keys array[string]
  
  List of keys to exclude from document.
  
  field string Required
  
  The field to be parsed. Supports template snippets.
  
  field_split string Required
  
  Regex pattern to use for splitting key-value pairs.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  include_keys array[string]
  
  List of keys to filter and insert into document. Defaults to including all keys.
  
  prefix string
  
  Prefix to be added to extracted keys.
  
  Default value is null.
  
  strip_brackets boolean
  
  If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.
  
  Default value is false.
  
  target_field string
  
  The field to insert the extracted keys into. Defaults to the root of the document. Supports template snippets.
  
  trim_key string
  
  String of characters to trim from extracted keys.
  
  trim_value string
  
  String of characters to trim from extracted values.
  
  value_split string Required
  
  Regex pattern to use for splitting the key from the value within a key-value pair.
- lowercase object
  
  Converts a string to its lowercase equivalent. If the field is an array of strings, all members of the array will be converted.
  Hide lowercase attributes Show lowercase attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to make lowercase.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  target_field string
  
  The field to assign the converted value to. By default, the field is updated in-place.
- network_direction object
  
  Calculates the network direction given a source IP address, destination IP address, and a list of internal networks.
  Hide network_direction attributes Show network_direction attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  source_ip string
  
  Field containing the source IP address.
  
  destination_ip string
  
  Field containing the destination IP address.
  
  target_field string
  
  Output field for the network direction.
  
  internal_networks array[string]
  
  List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.
  
  internal_networks_field string
  
  A field on the given document to read the internal_networks configuration from.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  Default value is true.
- pipeline object
  
  Executes another pipeline.
  Hide pipeline attributes Show pipeline attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  name string Required
  
  The name of the pipeline to execute. Supports template snippets.
  
  ignore_missing_pipeline boolean
  
  Whether to ignore missing pipelines instead of failing.
  
  Default value is false.
- redact object
  
  The Redact processor uses the Grok rules engine to obscure text in the input document matching the given Grok patterns. The processor can be used to obscure Personal Identifying Information (PII) by configuring it to detect known patterns such as email or IP addresses. Text that matches a Grok pattern is replaced with a configurable string such as <EMAIL> where an email address is matched or simply replace all matches with the text <REDACTED> if preferred.
  Hide redact attributes Show redact attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to be redacted
  
  patterns array[string] Required
  
  A list of grok expressions to match and redact named captures with
  
  pattern_definitions object
  
  Hide pattern_definitions attribute Show pattern_definitions attribute object
  
  * string Additional properties
  
  prefix string
  
  Start a redacted section with this token
  
  Default value is <.
  
  suffix string
  
  End a redacted section with this token
  
  Default value is >.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  skip_if_unlicensed boolean
  
  If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document
  
  Default value is false.
  
  trace_redact boolean Generally available; Added in 8.16.0
  
  If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted
  
  Default value is false.
- registered_domain object
  
  Extracts the registered domain (also known as the effective top-level domain or eTLD), sub-domain, and top-level domain from a fully qualified domain name (FQDN). Uses the registered domains defined in the Mozilla Public Suffix List.
  Hide registered_domain attributes Show registered_domain attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Field containing the source FQDN.
  
  target_field string
  
  Object field containing extracted domain components. If an empty string, the processor adds components to the document’s root.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  Default value is true.
- remove object
  
  Removes existing fields. If one field doesn’t exist, an exception will be thrown.
  Hide remove attributes Show remove attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string | array[string] Required
  
  Fields to be removed. Supports template snippets.
  
  keep string | array[string]
  
  Fields to be kept. When set, all fields other than those specified are removed.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
- rename object
  
  Renames an existing field. If the field doesn’t exist or the new name is already used, an exception will be thrown.
  Hide rename attributes Show rename attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to be renamed. Supports template snippets.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  target_field string Required
  
  The new name of the field. Supports template snippets.
- reroute object
  
  Routes a document to another target index or data stream. When setting the destination option, the target is explicitly specified and the dataset and namespace options can’t be set. When the destination option is not set, this processor is in a data stream mode. Note that in this mode, the reroute processor can only be used on data streams that follow the data stream naming scheme.
  Hide reroute attributes Show reroute attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  destination string
  
  A static value for the target. Can’t be set when the dataset or namespace option is set.
  
  dataset string | array[string]
  
  Field references or a static value for the dataset part of the data stream name. In addition to the criteria for index names, cannot contain - and must be no longer than 100 characters. Example values are nginx.access and nginx.error.
  
  Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.
  
  default {{data_stream.dataset}}
  
  One of:
  string-1 string array-2 array[string]
  
  namespace string | array[string]
  
  Field references or a static value for the namespace part of the data stream name. See the criteria for index names for allowed characters. Must be no longer than 100 characters.
  
  Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.
  
  default {{data_stream.namespace}}
  
  One of:
  string-1 string array-2 array[string]
- script object
  
  Runs an inline or stored script on incoming documents. The script runs in the ingest context.
  Hide script attributes Show script attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  id string
  
  ID of a stored script. If no source is specified, this parameter is required.
  
  lang string
  
  Script language.
  
  Default value is painless.
  
  params object
  
  Object containing parameters for the script.
  
  Hide params attribute Show params attribute object
  
  * object Additional properties
  
  source string
  
  Inline script. If no id is specified, this parameter is required.
- set object
  
  Adds a field with the specified value. If the field already exists, its value will be replaced with the provided one.
  Hide set attributes Show set attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  copy_from string
  
  The origin field which will be copied to field, cannot set value simultaneously. Supported data types are boolean, number, array, object, string, date, etc.
  
  field string Required
  
  The field to insert, upsert, or update. Supports template snippets.
  
  ignore_empty_value boolean
  
  If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  media_type string
  
  The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.
  
  override boolean
  
  If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.
  
  Default value is true.
  
  value object
  
  The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.
- set_security_user object
  
  Sets user-related details (such as username, roles, email, full_name, metadata, api_key, realm and authentication_type) from the current authenticated user to the current document by pre-processing the ingest.
  Hide set_security_user attributes Show set_security_user attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to store the user information into.
  
  properties array[string]
  
  Controls what user related properties are added to the field.
- sort object
  
  Sorts the elements of an array ascending or descending. Homogeneous arrays of numbers will be sorted numerically, while arrays of strings or heterogeneous arrays of strings + numbers will be sorted lexicographically. Throws an error when the field is not an array.
  Hide sort attributes Show sort attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to be sorted.
  
  order string
  
  The sort order to use. Accepts "asc" or "desc".
  
  Supported values include:
  
  asc: Ascending (smallest to largest)
  
  desc: Descending (largest to smallest)
  
  Values are asc or desc.
  
  target_field string
  
  The field to assign the sorted value to. By default, the field is updated in-place.
- split object
  
  Splits a field into an array using a separator character. Only works on string fields.
  Hide split attributes Show split attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to split.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  preserve_trailing boolean
  
  Preserves empty trailing fields, if any.
  
  Default value is false.
  
  separator string Required
  
  A regex which matches the separator, for example, , or \s+.
  
  target_field string
  
  The field to assign the split value to. By default, the field is updated in-place.
- terminate object
  
  Terminates the current ingest pipeline, causing no further processors to be run. This will normally be executed conditionally, using the if option.
  Hide terminate attributes Show terminate attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
- trim object
  
  Trims whitespace from a field. If the field is an array of strings, all members of the array will be trimmed. This only works on leading and trailing whitespace.
  Hide trim attributes Show trim attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The string-valued field to trim whitespace from.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  target_field string
  
  The field to assign the trimmed value to. By default, the field is updated in-place.
- uppercase object
  
  Converts a string to its uppercase equivalent. If the field is an array of strings, all members of the array will be converted.
  Hide uppercase attributes Show uppercase attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to make uppercase.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  target_field string
  
  The field to assign the converted value to. By default, the field is updated in-place.
- urldecode object
  
  URL-decodes a string. If the field is an array of strings, all members of the array will be decoded.
  Hide urldecode attributes Show urldecode attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to decode.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  target_field string
  
  The field to assign the converted value to. By default, the field is updated in-place.
- uri_parts object
  
  Parses a Uniform Resource Identifier (URI) string and extracts its components as an object. This URI object includes properties for the URI’s domain, path, fragment, port, query, scheme, user info, username, and password.
  Hide uri_parts attributes Show uri_parts attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Field containing the URI string.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  keep_original boolean
  
  If true, the processor copies the unparsed URI to <target_field>.original.
  
  Default value is true.
  
  remove_if_successful boolean
  
  If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.
  
  Default value is false.
  
  target_field string
  
  Output field for the URI object.
- user_agent object
  
  The user_agent processor extracts details from the user agent string a browser sends with its web requests. This processor adds this information by default under the user_agent field.
  Hide user_agent attributes Show user_agent attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field containing the user agent string.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  regex_file string
  
  The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.
  
  target_field string
  
  The field that will be filled with the user agent details.
  
  properties array[string]
  
  Controls what properties are added to target_field.
  
  Values are name, os, device, original, or version. Default value is ["name", "major", "minor", "patch", "build", "os", "os_name", "os_major", "os_minor", "device"].
  
  extract_device_type boolean Beta; Added in 8.9.0
  
  Extracts device type from the user agent string on a best-effort basis.
  
  Default value is false.
processors array[object]

Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.
Hide processors attributes Show processors attributes object
- append object
  
  Appends one or more values to an existing array if the field already exists and it is an array. Converts a scalar to an array and appends one or more values to it if the field exists and it is a scalar. Creates an array containing the provided values if the field doesn’t exist. Accepts a single value or an array of values.
  Hide append attributes Show append attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to be appended to. Supports template snippets.
  
  value object | array[object] Required
  
  The value to be appended. Supports template snippets.
  
  One of:
  object-1 object array-2 array[object]
  
  allow_duplicates boolean
  
  If false, the processor does not append values already present in the field.
  
  Default value is true.
- attachment object
  
  The attachment processor lets Elasticsearch extract file attachments in common formats (such as PPT, XLS, and PDF) by using the Apache text extraction library Tika.
  Hide attachment attributes Show attachment attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to get the base64 encoded field from.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  indexed_chars number
  
  The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.
  
  Default value is 100000.
  
  indexed_chars_field string
  
  Field name from which you can overwrite the number of chars being used for extraction.
  
  properties array[string]
  
  Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.
  
  target_field string
  
  The field that will hold the attachment information.
  
  remove_binary boolean
  
  If true, the binary field will be removed from the document
  
  Default value is false.
  
  resource_name string
  
  Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.
- bytes object
  
  Converts a human readable byte value (for example 1kb) to its value in bytes (for example 1024). If the field is an array of strings, all members of the array will be converted. Supported human readable units are "b", "kb", "mb", "gb", "tb", "pb" case insensitive. An error will occur if the field is not a supported format or resultant value exceeds 2^63.
  Hide bytes attributes Show bytes attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to convert.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  target_field string
  
  The field to assign the converted value to. By default, the field is updated in-place.
- circle object
  
  Converts circle definitions of shapes to regular polygons which approximate them.
  Hide circle attributes Show circle attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  error_distance number Required
  
  The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).
  
  field string Required
  
  The field to interpret as a circle. Either a string in WKT format or a map for GeoJSON.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  shape_type string Required
  
  Which field mapping type is to be used when processing the circle: geo_shape or shape.
  
  Values are geo_shape or shape.
  
  target_field string
  
  The field to assign the polygon shape to By default, the field is updated in-place.
- community_id object
  
  Computes the Community ID for network flow data as defined in the Community ID Specification. You can use a community ID to correlate network events related to a single flow.
  Hide community_id attributes Show community_id attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  source_ip string
  
  Field containing the source IP address.
  
  source_port string
  
  Field containing the source port.
  
  destination_ip string
  
  Field containing the destination IP address.
  
  destination_port string
  
  Field containing the destination port.
  
  iana_number string
  
  Field containing the IANA number.
  
  icmp_type string
  
  Field containing the ICMP type.
  
  icmp_code string
  
  Field containing the ICMP code.
  
  transport string
  
  Field containing the transport protocol name or number. Used only when the iana_number field is not present. The following protocol names are currently supported: eigrp, gre, icmp, icmpv6, igmp, ipv6-icmp, ospf, pim, sctp, tcp, udp
  
  target_field string
  
  Output field for the community ID.
  
  seed number
  
  Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.
  
  Default value is 0.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  Default value is true.
- convert object
  
  Converts a field in the currently ingested document to a different type, such as converting a string to an integer. If the field value is an array, all members will be converted.
  Hide convert attributes Show convert attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field whose value is to be converted.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  target_field string
  
  The field to assign the converted value to. By default, the field is updated in-place.
  
  type string Required
  
  The type to convert the existing value to.
  
  Values are integer, long, double, float, boolean, ip, string, or auto.
- csv object
  
  Extracts fields from CSV line out of a single text field within a document. Any empty field in CSV will be skipped.
  Hide csv attributes Show csv attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  empty_value object
  
  Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").
  
  field string Required
  
  The field to extract data from.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  quote string
  
  Quote used in CSV, has to be single character string.
  
  Default value is ".
  
  separator string
  
  Separator used in CSV, has to be single character string.
  
  Default value is ,.
  
  target_fields string | array[string] Required
  
  The array of fields to assign extracted values to.
  
  trim boolean
  
  Trim whitespaces in unquoted fields.
- date object
  
  Parses dates from fields, and then uses the date or timestamp as the timestamp for the document.
  Hide date attributes Show date attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to get the date from.
  
  formats array[string] Required
  
  An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
  
  locale string
  
  The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.
  
  Default value is ENGLISH.
  
  target_field string
  
  The field that will hold the parsed date.
  
  timezone string
  
  The timezone to use when parsing the date. Supports template snippets.
  
  Default value is UTC.
  
  output_format string
  
  The format to use when writing the date to target_field. Must be a valid java time pattern.
  
  Default value is yyyy-MM-dd'T'HH:mm:ss.SSSXXX.
- date_index_name object
  
  The purpose of this processor is to point documents to the right time based index based on a date or timestamp field in a document by using the date math index name support.
  Hide date_index_name attributes Show date_index_name attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  date_formats array[string] Required
  
  An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
  
  date_rounding string Required
  
  How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.
  
  field string Required
  
  The field to get the date or timestamp from.
  
  index_name_format string
  
  The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.
  
  Default value is yyyy-MM-dd.
  
  index_name_prefix string
  
  A prefix of the index name to be prepended before the printed date. Supports template snippets.
  
  locale string
  
  The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.
  
  Default value is ENGLISH.
  
  timezone string
  
  The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.
  
  Default value is UTC.
- dissect object
  
  Extracts structured fields out of a single text field by matching the text field against a delimiter-based pattern.
  Hide dissect attributes Show dissect attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  append_separator string
  
  The character(s) that separate the appended fields.
  
  Default value is "".
  
  field string Required
  
  The field to dissect.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  pattern string Required
  
  The pattern to apply to the field.
- dot_expander object
  
  Expands a field with dots into an object field. This processor allows fields with dots in the name to be accessible by other processors in the pipeline. Otherwise these fields can’t be accessed by any processor.
  Hide dot_expander attributes Show dot_expander attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to expand into an object field. If set to *, all top-level fields will be expanded.
  
  override boolean
  
  Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.
  
  Default value is false.
  
  path string
  
  The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.
- drop object
  
  Drops the document without raising any errors. This is useful to prevent the document from getting indexed based on some condition.
  Hide drop attributes Show drop attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
- enrich object
  
  The enrich processor can enrich documents with data from another index.
  Hide enrich attributes Show enrich attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field in the input document that matches the policies match_field used to retrieve the enrichment data. Supports template snippets.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  max_matches number
  
  The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.
  
  Default value is 1.
  
  override boolean
  
  If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.
  
  Default value is true.
  
  policy_name string Required
  
  The name of the enrich policy to use.
  
  shape_relation string
  
  A spatial relation operator used to match the geoshape of incoming documents to documents in the enrich index. This option is only used for geo_match enrich policy types.
  
  Supported values include:
  
  intersects: Return all documents whose geo_shape or geo_point field intersects the query geometry.
  
  disjoint: Return all documents whose geo_shape or geo_point field has nothing in common with the query geometry.
  
  within: Return all documents whose geo_shape or geo_point field is within the query geometry. Line geometries are not supported.
  
  contains: Return all documents whose geo_shape or geo_point field contains the query geometry.
  
  Values are intersects, disjoint, within, or contains.
  
  target_field string Required
  
  Field added to incoming documents to contain enrich data. This field contains both the match_field and enrich_fields specified in the enrich policy. Supports template snippets.
- fail object
  
  Raises an exception. This is useful for when you expect a pipeline to fail and want to relay a specific message to the requester.
  Hide fail attributes Show fail attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  message string Required
  
  The error message thrown by the processor. Supports template snippets.
- fingerprint object
  
  Computes a hash of the document’s content. You can use this hash for content fingerprinting.
  Hide fingerprint attributes Show fingerprint attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  fields string | array[string] Required
  
  Array of fields to include in the fingerprint. For objects, the processor hashes both the field key and value. For other fields, the processor hashes only the field value.
  
  target_field string
  
  Output field for the fingerprint.
  
  salt string
  
  Salt value for the hash function.
  
  method string
  
  The hash method used to compute the fingerprint. Must be one of MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.
  
  Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.
  
  ignore_missing boolean
  
  If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.
  
  Default value is false.
- foreach object
  
  Runs an ingest processor on each element of an array or object.
  Hide foreach attributes Show foreach attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Field containing array or object values.
  
  ignore_missing boolean
  
  If true, the processor silently exits without changing the document if the field is null or missing.
  
  Default value is false.
  
  processor object Required
  
  Ingest processor to run on each element.
- ip_location object
  
  Currently an undocumented alias for GeoIP Processor.
  Hide ip_location attributes Show ip_location attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  database_file string
  
  The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.
  
  Default value is GeoLite2-City.mmdb.
  
  field string Required
  
  The field to get the ip address from for the geographical lookup.
  
  first_only boolean
  
  If true, only the first found IP location data will be returned, even if the field contains an array.
  
  Default value is true.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  properties array[string]
  
  Controls what properties are added to the target_field based on the IP location lookup.
  
  target_field string
  
  The field that will hold the geographical information looked up from the MaxMind database.
  
  download_database_on_pipeline_creation boolean
  
  If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.
- geo_grid object
  
  Converts geo-grid definitions of grid tiles or cells to regular bounding boxes or polygons which describe their shape. This is useful if there is a need to interact with the tile shapes as spatially indexable fields.
  Hide geo_grid attributes Show geo_grid attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to interpret as a geo-tile.= The field format is determined by the tile_type.
  
  tile_type string Required
  
  Three tile formats are understood: geohash, geotile and geohex.
  
  Values are geotile, geohex, or geohash.
  
  target_field string
  
  The field to assign the polygon shape to, by default, the field is updated in-place.
  
  parent_field string
  
  If specified and a parent tile exists, save that tile address to this field.
  
  children_field string
  
  If specified and children tiles exist, save those tile addresses to this field as an array of strings.
  
  non_children_field string
  
  If specified and intersecting non-child tiles exist, save their addresses to this field as an array of strings.
  
  precision_field string
  
  If specified, save the tile precision (zoom) as an integer to this field.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  target_format string
  
  Which format to save the generated polygon in.
  
  Values are geojson or wkt.
- geoip object
  
  The geoip processor adds information about the geographical location of an IPv4 or IPv6 address.
  Hide geoip attributes Show geoip attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  database_file string
  
  The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.
  
  Default value is GeoLite2-City.mmdb.
  
  field string Required
  
  The field to get the ip address from for the geographical lookup.
  
  first_only boolean
  
  If true, only the first found geoip data will be returned, even if the field contains an array.
  
  Default value is true.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  properties array[string]
  
  Controls what properties are added to the target_field based on the geoip lookup.
  
  target_field string
  
  The field that will hold the geographical information looked up from the MaxMind database.
  
  download_database_on_pipeline_creation boolean
  
  If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.
- grok object
  
  Extracts structured fields out of a single text field within a document. You choose which field to extract matched fields from, as well as the grok pattern you expect will match. A grok pattern is like a regular expression that supports aliased expressions that can be reused.
  Hide grok attributes Show grok attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  ecs_compatibility string
  
  Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.
  
  Default value is disabled.
  
  field string Required
  
  The field to use for grok expression parsing.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  pattern_definitions object
  
  A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.
  
  Hide pattern_definitions attribute Show pattern_definitions attribute object
  
  * string Additional properties
  
  patterns array[string] Required
  
  An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.
  
  trace_match boolean
  
  When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.
  
  Default value is false.
- gsub object
  
  Converts a string field by applying a regular expression and a replacement. If the field is an array of string, all members of the array will be converted. If any non-string values are encountered, the processor will throw an exception.
  Hide gsub attributes Show gsub attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to apply the replacement to.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  pattern string Required
  
  The pattern to be replaced.
  
  replacement string Required
  
  The string to replace the matching patterns with.
  
  target_field string
  
  The field to assign the converted value to By default, the field is updated in-place.
- html_strip object
  
  Removes HTML tags from the field. If the field is an array of strings, HTML tags will be removed from all members of the array.
  Hide html_strip attributes Show html_strip attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The string-valued field to remove HTML tags from.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document,
  
  Default value is false.
  
  target_field string
  
  The field to assign the converted value to By default, the field is updated in-place.
- inference object
  
  Uses a pre-trained data frame analytics model or a model deployed for natural language processing tasks to infer against the data that is being ingested in the pipeline.
  Hide inference attributes Show inference attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  model_id string Required
  
  The ID or alias for the trained model, or the ID of the deployment.
  
  target_field string
  
  Field added to incoming documents to contain results objects.
  
  field_map object
  
  Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.
  
  Hide field_map attribute Show field_map attribute object
  
  * object Additional properties
  
  inference_config object
  
  Contains the inference type and its options.
  
  input_output object | array[object]
  
  Input fields for inference and output (destination) fields for the inference results. This option is incompatible with the target_field and field_map options.
  
  One of:
  InputConfig object array-2 array[object]
  
  ignore_missing boolean
  
  If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.
- join object
  
  Joins each element of an array into a single string using a separator character between each element. Throws an error when the field is not an array.
  Hide join attributes Show join attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Field containing array values to join.
  
  separator string Required
  
  The separator character.
  
  target_field string
  
  The field to assign the joined value to. By default, the field is updated in-place.
- json object
  
  Converts a JSON string into a structured JSON object.
  Hide json attributes Show json attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  add_to_root boolean
  
  Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.
  
  Default value is false.
  
  add_to_root_conflict_strategy string
  
  When set to replace, root fields that conflict with fields from the parsed JSON will be overridden. When set to merge, conflicting fields will be merged. Only applicable if add_to_root is set to true.
  
  Supported values include:
  
  replace: Root fields that conflict with fields from the parsed JSON will be overridden.
  
  merge: Conflicting fields will be merged.
  
  Values are replace or merge.
  
  allow_duplicate_keys boolean
  
  When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.
  
  Default value is false.
  
  field string Required
  
  The field to be parsed.
  
  target_field string
  
  The field that the converted structured object will be written into. Any existing content in this field will be overwritten.
- kv object
  
  This processor helps automatically parse messages (or specific event fields) which are of the foo=bar variety.
  Hide kv attributes Show kv attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  exclude_keys array[string]
  
  List of keys to exclude from document.
  
  field string Required
  
  The field to be parsed. Supports template snippets.
  
  field_split string Required
  
  Regex pattern to use for splitting key-value pairs.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  include_keys array[string]
  
  List of keys to filter and insert into document. Defaults to including all keys.
  
  prefix string
  
  Prefix to be added to extracted keys.
  
  Default value is null.
  
  strip_brackets boolean
  
  If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.
  
  Default value is false.
  
  target_field string
  
  The field to insert the extracted keys into. Defaults to the root of the document. Supports template snippets.
  
  trim_key string
  
  String of characters to trim from extracted keys.
  
  trim_value string
  
  String of characters to trim from extracted values.
  
  value_split string Required
  
  Regex pattern to use for splitting the key from the value within a key-value pair.
- lowercase object
  
  Converts a string to its lowercase equivalent. If the field is an array of strings, all members of the array will be converted.
  Hide lowercase attributes Show lowercase attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to make lowercase.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  target_field string
  
  The field to assign the converted value to. By default, the field is updated in-place.
- network_direction object
  
  Calculates the network direction given a source IP address, destination IP address, and a list of internal networks.
  Hide network_direction attributes Show network_direction attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  source_ip string
  
  Field containing the source IP address.
  
  destination_ip string
  
  Field containing the destination IP address.
  
  target_field string
  
  Output field for the network direction.
  
  internal_networks array[string]
  
  List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.
  
  internal_networks_field string
  
  A field on the given document to read the internal_networks configuration from.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  Default value is true.
- pipeline object
  
  Executes another pipeline.
  Hide pipeline attributes Show pipeline attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  name string Required
  
  The name of the pipeline to execute. Supports template snippets.
  
  ignore_missing_pipeline boolean
  
  Whether to ignore missing pipelines instead of failing.
  
  Default value is false.
- redact object
  
  The Redact processor uses the Grok rules engine to obscure text in the input document matching the given Grok patterns. The processor can be used to obscure Personal Identifying Information (PII) by configuring it to detect known patterns such as email or IP addresses. Text that matches a Grok pattern is replaced with a configurable string such as <EMAIL> where an email address is matched or simply replace all matches with the text <REDACTED> if preferred.
  Hide redact attributes Show redact attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to be redacted
  
  patterns array[string] Required
  
  A list of grok expressions to match and redact named captures with
  
  pattern_definitions object
  
  Hide pattern_definitions attribute Show pattern_definitions attribute object
  
  * string Additional properties
  
  prefix string
  
  Start a redacted section with this token
  
  Default value is <.
  
  suffix string
  
  End a redacted section with this token
  
  Default value is >.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  skip_if_unlicensed boolean
  
  If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document
  
  Default value is false.
  
  trace_redact boolean Generally available; Added in 8.16.0
  
  If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted
  
  Default value is false.
- registered_domain object
  
  Extracts the registered domain (also known as the effective top-level domain or eTLD), sub-domain, and top-level domain from a fully qualified domain name (FQDN). Uses the registered domains defined in the Mozilla Public Suffix List.
  Hide registered_domain attributes Show registered_domain attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Field containing the source FQDN.
  
  target_field string
  
  Object field containing extracted domain components. If an empty string, the processor adds components to the document’s root.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  Default value is true.
- remove object
  
  Removes existing fields. If one field doesn’t exist, an exception will be thrown.
  Hide remove attributes Show remove attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string | array[string] Required
  
  Fields to be removed. Supports template snippets.
  
  keep string | array[string]
  
  Fields to be kept. When set, all fields other than those specified are removed.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
- rename object
  
  Renames an existing field. If the field doesn’t exist or the new name is already used, an exception will be thrown.
  Hide rename attributes Show rename attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to be renamed. Supports template snippets.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  target_field string Required
  
  The new name of the field. Supports template snippets.
- reroute object
  
  Routes a document to another target index or data stream. When setting the destination option, the target is explicitly specified and the dataset and namespace options can’t be set. When the destination option is not set, this processor is in a data stream mode. Note that in this mode, the reroute processor can only be used on data streams that follow the data stream naming scheme.
  Hide reroute attributes Show reroute attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  destination string
  
  A static value for the target. Can’t be set when the dataset or namespace option is set.
  
  dataset string | array[string]
  
  Field references or a static value for the dataset part of the data stream name. In addition to the criteria for index names, cannot contain - and must be no longer than 100 characters. Example values are nginx.access and nginx.error.
  
  Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.
  
  default {{data_stream.dataset}}
  
  One of:
  string-1 string array-2 array[string]
  
  namespace string | array[string]
  
  Field references or a static value for the namespace part of the data stream name. See the criteria for index names for allowed characters. Must be no longer than 100 characters.
  
  Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.
  
  default {{data_stream.namespace}}
  
  One of:
  string-1 string array-2 array[string]
- script object
  
  Runs an inline or stored script on incoming documents. The script runs in the ingest context.
  Hide script attributes Show script attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  id string
  
  ID of a stored script. If no source is specified, this parameter is required.
  
  lang string
  
  Script language.
  
  Default value is painless.
  
  params object
  
  Object containing parameters for the script.
  
  Hide params attribute Show params attribute object
  
  * object Additional properties
  
  source string
  
  Inline script. If no id is specified, this parameter is required.
- set object
  
  Adds a field with the specified value. If the field already exists, its value will be replaced with the provided one.
  Hide set attributes Show set attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  copy_from string
  
  The origin field which will be copied to field, cannot set value simultaneously. Supported data types are boolean, number, array, object, string, date, etc.
  
  field string Required
  
  The field to insert, upsert, or update. Supports template snippets.
  
  ignore_empty_value boolean
  
  If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  media_type string
  
  The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.
  
  override boolean
  
  If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.
  
  Default value is true.
  
  value object
  
  The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.
- set_security_user object
  
  Sets user-related details (such as username, roles, email, full_name, metadata, api_key, realm and authentication_type) from the current authenticated user to the current document by pre-processing the ingest.
  Hide set_security_user attributes Show set_security_user attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to store the user information into.
  
  properties array[string]
  
  Controls what user related properties are added to the field.
- sort object
  
  Sorts the elements of an array ascending or descending. Homogeneous arrays of numbers will be sorted numerically, while arrays of strings or heterogeneous arrays of strings + numbers will be sorted lexicographically. Throws an error when the field is not an array.
  Hide sort attributes Show sort attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to be sorted.
  
  order string
  
  The sort order to use. Accepts "asc" or "desc".
  
  Supported values include:
  
  asc: Ascending (smallest to largest)
  
  desc: Descending (largest to smallest)
  
  Values are asc or desc.
  
  target_field string
  
  The field to assign the sorted value to. By default, the field is updated in-place.
- split object
  
  Splits a field into an array using a separator character. Only works on string fields.
  Hide split attributes Show split attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to split.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  preserve_trailing boolean
  
  Preserves empty trailing fields, if any.
  
  Default value is false.
  
  separator string Required
  
  A regex which matches the separator, for example, , or \s+.
  
  target_field string
  
  The field to assign the split value to. By default, the field is updated in-place.
- terminate object
  
  Terminates the current ingest pipeline, causing no further processors to be run. This will normally be executed conditionally, using the if option.
  Hide terminate attributes Show terminate attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
- trim object
  
  Trims whitespace from a field. If the field is an array of strings, all members of the array will be trimmed. This only works on leading and trailing whitespace.
  Hide trim attributes Show trim attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The string-valued field to trim whitespace from.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  target_field string
  
  The field to assign the trimmed value to. By default, the field is updated in-place.
- uppercase object
  
  Converts a string to its uppercase equivalent. If the field is an array of strings, all members of the array will be converted.
  Hide uppercase attributes Show uppercase attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to make uppercase.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  target_field string
  
  The field to assign the converted value to. By default, the field is updated in-place.
- urldecode object
  
  URL-decodes a string. If the field is an array of strings, all members of the array will be decoded.
  Hide urldecode attributes Show urldecode attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to decode.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  target_field string
  
  The field to assign the converted value to. By default, the field is updated in-place.
- uri_parts object
  
  Parses a Uniform Resource Identifier (URI) string and extracts its components as an object. This URI object includes properties for the URI’s domain, path, fragment, port, query, scheme, user info, username, and password.
  Hide uri_parts attributes Show uri_parts attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Field containing the URI string.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  keep_original boolean
  
  If true, the processor copies the unparsed URI to <target_field>.original.
  
  Default value is true.
  
  remove_if_successful boolean
  
  If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.
  
  Default value is false.
  
  target_field string
  
  Output field for the URI object.
- user_agent object
  
  The user_agent processor extracts details from the user agent string a browser sends with its web requests. This processor adds this information by default under the user_agent field.
  Hide user_agent attributes Show user_agent attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field containing the user agent string.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  Default value is false.
  
  regex_file string
  
  The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.
  
  target_field string
  
  The field that will be filled with the user agent details.
  
  properties array[string]
  
  Controls what properties are added to target_field.
  
  Values are name, os, device, original, or version. Default value is ["name", "major", "minor", "patch", "build", "os", "os_name", "os_major", "os_minor", "device"].
  
  extract_device_type boolean Beta; Added in 8.9.0
  
  Extracts device type from the user agent string on a best-effort basis.
  
  Default value is false.
version number

Version number used by external systems to track ingest pipelines. This parameter is intended for external systems only. Elasticsearch does not use or validate pipeline version numbers.
deprecated boolean

Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.

Default value is false.

Responses

200 application/json
Hide response attribute Show response attribute object
- acknowledged boolean Required
  
  For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /_ingest/pipeline/{id}

PUT _ingest/pipeline/my-pipeline-id
{
  "description" : "My optional pipeline description",
  "processors" : [
    {
      "set" : {
        "description" : "My optional processor description",
        "field": "my-keyword-field",
        "value": "foo"
      }
    }
  ]
}

resp = client.ingest.put_pipeline(
    id="my-pipeline-id",
    description="My optional pipeline description",
    processors=[
        {
            "set": {
                "description": "My optional processor description",
                "field": "my-keyword-field",
                "value": "foo"
            }
        }
    ],
)

const response = await client.ingest.putPipeline({
  id: "my-pipeline-id",
  description: "My optional pipeline description",
  processors: [
    {
      set: {
        description: "My optional processor description",
        field: "my-keyword-field",
        value: "foo",
      },
    },
  ],
});

response = client.ingest.put_pipeline(
  id: "my-pipeline-id",
  body: {
    "description": "My optional pipeline description",
    "processors": [
      {
        "set": {
          "description": "My optional processor description",
          "field": "my-keyword-field",
          "value": "foo"
        }
      }
    ]
  }
)

$resp = $client->ingest()->putPipeline([
    "id" => "my-pipeline-id",
    "body" => [
        "description" => "My optional pipeline description",
        "processors" => array(
            [
                "set" => [
                    "description" => "My optional processor description",
                    "field" => "my-keyword-field",
                    "value" => "foo",
                ],
            ],
        ),
    ],
]);

curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"description":"My optional pipeline description","processors":[{"set":{"description":"My optional processor description","field":"my-keyword-field","value":"foo"}}]}' "$ELASTICSEARCH_URL/_ingest/pipeline/my-pipeline-id"

client.ingest().putPipeline(p -> p
    .description("My optional pipeline description")
    .id("my-pipeline-id")
    .processors(pr -> pr
        .set(s -> s
            .field("my-keyword-field")
            .value(JsonData.fromJson("\"foo\""))
            .description("My optional processor description")
        )
    )
);

Request examples

{
  "description" : "My optional pipeline description",
  "processors" : [
    {
      "set" : {
        "description" : "My optional processor description",
        "field": "my-keyword-field",
        "value": "foo"
      }
    }
  ]
}

You can use the `_meta` parameter to add arbitrary metadata to a pipeline.

{
  "description" : "My optional pipeline description",
  "processors" : [
    {
      "set" : {
        "description" : "My optional processor description",
        "field": "my-keyword-field",
        "value": "foo"
      }
    }
  ],
  "_meta": {
    "reason": "set my-keyword-field to foo",
    "serialization": {
      "class": "MyPipeline",
      "id": 10
    }
  }
}

Revert to a snapshot Generally available; Added in 5.4.0

POST /_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}/_revert

Api key auth Basic auth Bearer auth

The machine learning features react quickly to anomalous input, learning new behaviors in data. Highly anomalous input increases the variance in the models whilst the system learns whether this is a new step-change in behavior or a one-off event. In the case where this anomalous input is known to be a one-off, then it might be appropriate to reset the model state to a time before this event. For example, you might consider reverting to a saved snapshot after Black Friday or a critical system failure.

Required authorization

Cluster privileges: manage_ml

Path parameters

job_id string Required

Identifier for the anomaly detection job.
snapshot_id string Required

You can specify empty as the . Reverting to the empty snapshot means the anomaly detection job starts learning a new model from scratch when it is started.

Query parameters

delete_intervening_results boolean

If true, deletes the results in the time period between the latest results and the time of the reverted snapshot. It also resets the model to accept records for this time period. If you choose not to delete intervening results when reverting a snapshot, the job will not accept input data that is older than the current time. If you want to resend data, then delete the intervening results.

application/json

Body

delete_intervening_results boolean

Refer to the description for the delete_intervening_results query parameter.

Default value is false.

Responses

200 application/json
Hide response attribute Show response attribute object
- model object Required
  
  Hide model attributes Show model attributes object
  
  description string
  
  An optional description of the job.
  
  job_id string Required
  
  A numerical character string that uniquely identifies the job that the snapshot was created for.
  
  latest_record_time_stamp number
  
  The timestamp of the latest processed record.
  
  latest_result_time_stamp number
  
  The timestamp of the latest bucket result.
  
  min_version string Required
  
  The minimum version required to be able to restore the model snapshot.
  
  model_size_stats object
  
  Summary information describing the model.
  
  Hide model_size_stats attributes Show model_size_stats attributes object
  
  bucket_allocation_failures_count number Required
  
  job_id string Required
  
  log_time
  
  memory_status string Required
  
  Values are ok, soft_limit, or hard_limit.
  
  model_bytes
  
  model_bytes_exceeded
  
  model_bytes_memory_limit
  
  output_memory_allocator_bytes
  
  peak_model_bytes
  
  assignment_memory_basis string
  
  result_type string Required
  
  total_by_field_count number Required
  
  total_over_field_count number Required
  
  total_partition_field_count number Required
  
  categorization_status string Required
  
  Values are ok or warn.
  
  categorized_doc_count number Required
  
  dead_category_count number Required
  
  failed_category_count number Required
  
  frequent_category_count number Required
  
  rare_category_count number Required
  
  total_category_count number Required
  
  timestamp number
  
  retain boolean Required
  
  If true, this snapshot will not be deleted during automatic cleanup of snapshots older than model_snapshot_retention_days. However, this snapshot will be deleted when the job is deleted. The default value is false.
  
  snapshot_doc_count number Required
  
  For internal use only.
  
  snapshot_id string Required
  
  A numerical character string that uniquely identifies the model snapshot.
  
  timestamp number Required
  
  The creation timestamp for the snapshot.

POST /_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}/_revert

POST _ml/anomaly_detectors/low_request_rate/model_snapshots/1637092688/_revert
{
  "delete_intervening_results": true
}

resp = client.ml.revert_model_snapshot(
    job_id="low_request_rate",
    snapshot_id="1637092688",
    delete_intervening_results=True,
)

const response = await client.ml.revertModelSnapshot({
  job_id: "low_request_rate",
  snapshot_id: 1637092688,
  delete_intervening_results: true,
});

response = client.ml.revert_model_snapshot(
  job_id: "low_request_rate",
  snapshot_id: "1637092688",
  body: {
    "delete_intervening_results": true
  }
)

$resp = $client->ml()->revertModelSnapshot([
    "job_id" => "low_request_rate",
    "snapshot_id" => "1637092688",
    "body" => [
        "delete_intervening_results" => true,
    ],
]);

curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"delete_intervening_results":true}' "$ELASTICSEARCH_URL/_ml/anomaly_detectors/low_request_rate/model_snapshots/1637092688/_revert"

client.ml().revertModelSnapshot(r -> r
    .deleteInterveningResults(true)
    .jobId("low_request_rate")
    .snapshotId("1637092688")
);

Request example

An example body for a `POST _ml/anomaly_detectors/low_request_rate/model_snapshots/1637092688/_revert` request.

{
  "delete_intervening_results": true
}

Get rollup job information Deprecated Technical preview; Added in 6.3.0

GET /_rollup/job/{id}

Api key auth Basic auth Bearer auth

All methods and paths for this operation:

GET /_rollup/job

GET /_rollup/job/{id}

Get the configuration, stats, and status of rollup jobs.

NOTE: This API returns only active (both STARTED and STOPPED) jobs. If a job was created, ran for a while, then was deleted, the API does not return any details about it. For details about a historical rollup job, the rollup capabilities API may be more useful.

Required authorization

Cluster privileges: monitor_rollup

Path parameters

id string Required

Identifier for the rollup job. If it is _all or omitted, the API returns all rollup jobs.

Responses

200 application/json
Hide response attribute Show response attribute object
- jobs array[object] Required
  
  Hide jobs attributes Show jobs attributes object
  
  config object Required
  
  The rollup job configuration.
  
  Hide config attributes Show config attributes object
  
  cron string Required
  
  groups object Required
  
  id string Required
  
  index_pattern string Required
  
  metrics array[object] Required
  
  page_size number Required
  
  rollup_index string Required
  
  timeout string Required
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  stats object Required
  
  Transient statistics about the rollup job, such as how many documents have been processed and how many rollup summary docs have been indexed. These stats are not persisted. If a node is restarted, these stats are reset.
  
  Hide stats attributes Show stats attributes object
  
  documents_processed number Required
  
  index_failures number Required
  
  index_total number Required
  
  pages_processed number Required
  
  rollups_indexed number Required
  
  search_failures number Required
  
  search_total number Required
  
  trigger_count number Required
  
  processing_total number Required
  
  status object Required
  
  The current status of the indexer for the rollup job.
  
  Hide status attributes Show status attributes object
  
  current_position object
  
  Hide current_position attribute Show current_position attribute object
  
  * object Additional properties
  
  job_state string Required
  
  Values are started, indexing, stopping, stopped, or aborting.
  
  upgraded_doc_id boolean

GET /_rollup/job/{id}

GET _rollup/job/sensor

resp = client.rollup.get_jobs(
    id="sensor",
)

const response = await client.rollup.getJobs({
  id: "sensor",
});

response = client.rollup.get_jobs(
  id: "sensor"
)

$resp = $client->rollup()->getJobs([
    "id" => "sensor",
]);

curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_rollup/job/sensor"

client.rollup().getJobs(g -> g
    .id("sensor")
);

Response examples (200)

A successful response from `GET _rollup/job/sensor`.

{
  "jobs": [
    {
      "config": {
        "id": "sensor",
        "index_pattern": "sensor-*",
        "rollup_index": "sensor_rollup",
        "cron": "*/30 * * * * ?",
        "groups": {
          "date_histogram": {
            "fixed_interval": "1h",
            "delay": "7d",
            "field": "timestamp",
            "time_zone": "UTC"
          },
          "terms": {
            "fields": [
              "node"
            ]
          }
        },
        "metrics": [
          {
            "field": "temperature",
            "metrics": [
              "min",
              "max",
              "sum"
            ]
          },
          {
            "field": "voltage",
            "metrics": [
              "avg"
            ]
          }
        ],
        "timeout": "20s",
        "page_size": 1000
      },
      "status": {
        "job_state": "stopped"
      },
      "stats": {
        "pages_processed": 0,
        "documents_processed": 0,
        "rollups_indexed": 0,
        "trigger_count": 0,
        "index_failures": 0,
        "index_time_in_ms": 0,
        "index_total": 0,
        "search_failures": 0,
        "search_time_in_ms": 0,
        "search_total": 0,
        "processing_time_in_ms": 0,
        "processing_total": 0
      }
    }
  ]
}

Get async search results Generally available; Added in 7.7.0

GET /_async_search/{id}

Api key auth Basic auth Bearer auth

Retrieve the results of a previously submitted asynchronous search request. If the Elasticsearch security features are enabled, access to the results of a specific async search is restricted to the user or API key that submitted it.

Path parameters

id string

A unique identifier for the async search.

Query parameters

keep_alive string

The length of time that the async search should be available in the cluster. When not specified, the keep_alive set with the corresponding submit async request will be used. Otherwise, it is possible to override the value and extend the validity of the request. When this period expires, the search, if still running, is cancelled. If the search is completed, its saved results are deleted.

Values are -1 or 0.
typed_keys boolean

Specify whether aggregation and suggester names should be prefixed by their respective types in the response
wait_for_completion_timeout string

Specifies to wait for the search to be completed up until the provided timeout. Final results will be returned if available before the timeout expires, otherwise the currently available results will be returned once the timeout expires. By default no timeout is set meaning that the currently available results will be returned without any additional wait.

Values are -1 or 0.

Responses

200 application/json
Hide response attributes Show response attributes object
- id string
- is_partial boolean Required
  
  When the query is no longer running, this property indicates whether the search failed or was successfully completed on all shards. While the query is running, is_partial is always set to true.
- is_running boolean Required
  
  Indicates whether the search is still running or has completed.
  
  If the search failed after some shards returned their results or the node that is coordinating the async search dies, results may be partial even though is_running is false.
- expiration_time string | number
  
  Indicates when the async search will expire.
  
  One of:
  string-1 string UnitMillis number
  
  Indicates when the async search will expire.
- Time unit for milliseconds
- start_time string | number
  
  One of:
  string-1 string UnitMillis number
- Time unit for milliseconds
- completion_time string | number
  
  Indicates when the async search completed. It is present only when the search has completed.
  
  One of:
  string-1 string UnitMillis number
  
  Indicates when the async search completed. It is present only when the search has completed.
- Time unit for milliseconds
- response object Required
  
  Hide response attributes Show response attributes object
  
  aggregations object
  
  Partial aggregations results, coming from the shards that have already completed running the query.
  
  _clusters object
  
  Hide _clusters attributes Show _clusters attributes object
  
  skipped number Required
  
  successful number Required
  
  total number Required
  
  running number Required
  
  partial number Required
  
  failed number Required
  
  details object
  
  fields object
  
  Hide fields attribute Show fields attribute object
  
  * object Additional properties
  
  hits object Required
  
  Hide hits attributes Show hits attributes object
  
  total
  
  hits array[object] Required
  
  max_score
  
  max_score number
  
  num_reduce_phases number
  
  Indicates how many reductions of the results have been performed. If this number increases compared to the last retrieved results for a get asynch search request, you can expect additional results included in the search response.
  
  profile object
  
  Hide profile attribute Show profile attribute object
  
  shards array[object] Required
  
  pit_id string
  
  _scroll_id string
  
  _shards object Required
  
  Indicates how many shards have run the query. Note that in order for shard results to be included in the search response, they need to be reduced first.
  
  Hide _shards attribute Show _shards attribute object
  
  failures array[object]
  
  suggest object
  
  Hide suggest attribute Show suggest attribute object
  
  * array[object] Additional properties
  
  terminated_early boolean
  
  timed_out boolean Required
  
  took number Required

GET /_async_search/{id}

GET /_async_search/FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=

resp = client.async_search.get(
    id="FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=",
)

const response = await client.asyncSearch.get({
  id: "FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=",
});

response = client.async_search.get(
  id: "FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc="
)

$resp = $client->asyncSearch()->get([
    "id" => "FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=",
]);

curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_async_search/FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc="

client.asyncSearch().get(g -> g
    .id("FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=")
);

Response examples (200)

A succesful response from `GET /_async_search/FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=`.

{
  "id" : "FmRldE8zREVEUzA2ZVpUeGs2ejJFUFEaMkZ5QTVrSTZSaVN3WlNFVmtlWHJsdzoxMDc=",
  "is_partial" : false, 
  "is_running" : false, 
  "start_time_in_millis" : 1583945890986,
  "expiration_time_in_millis" : 1584377890986, 
  "completion_time_in_millis" : 1583945903130, 
  "response" : {
    "took" : 12144,
    "timed_out" : false,
    "num_reduce_phases" : 46, 
    "_shards" : {
      "total" : 562,
      "successful" : 188, 
      "skipped" : 0,
      "failed" : 0
    },
    "hits" : {
      "total" : {
        "value" : 456433,
        "relation" : "eq"
      },
      "max_score" : null,
      "hits" : [ ]
    },
    "aggregations" : { 
      "sale_date" :  {
        "buckets" : []
      }
    }
  }
}

Close a point in time Generally available; Added in 7.10.0

DELETE /_pit

Api key auth Basic auth Bearer auth

A point in time must be opened explicitly before being used in search requests. The keep_alive parameter tells Elasticsearch how long it should persist. A point in time is automatically closed when the keep_alive period has elapsed. However, keeping points in time has a cost; close them as soon as they are no longer required for search requests.

application/json

Body

id string Required

The ID of the point-in-time.

Responses

200 application/json
Hide response attributes Show response attributes object
- succeeded boolean Required
  
  If true, all search contexts associated with the point-in-time ID were successfully closed.
- num_freed number Required
  
  The number of search contexts that were successfully closed.

DELETE /_pit

DELETE /_pit
{
  "id": "46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA=="
}

resp = client.close_point_in_time(
    id="46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA==",
)

const response = await client.closePointInTime({
  id: "46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA==",
});

response = client.close_point_in_time(
  body: {
    "id": "46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA=="
  }
)

$resp = $client->closePointInTime([
    "body" => [
        "id" => "46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA==",
    ],
]);

curl -X DELETE -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"id":"46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA=="}' "$ELASTICSEARCH_URL/_pit"

client.closePointInTime(c -> c
    .id("46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA==")
);

Request example

Run `DELETE /_pit` to close a point-in-time.

{
  "id": "46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA=="
}

Response examples (200)

A successful response from `DELETE /_pit`.

{
  "succeeded": true, 
  "num_freed": 3     
}

Create or update users Generally available

POST /_security/user/{username}

Api key auth Basic auth Bearer auth

All methods and paths for this operation:

PUT /_security/user/{username}

POST /_security/user/{username}

Add and update users in the native realm. A password is required for adding a new user but is optional when updating an existing user. To change a user's password without updating any other fields, use the change password API.

Required authorization

Cluster privileges: manage_security

Path parameters

username string Required

An identifier for the user.

NOTE: Usernames must be at least 1 and no more than 507 characters. They can contain alphanumeric characters (a-z, A-Z, 0-9), spaces, punctuation, and printable symbols in the Basic Latin (ASCII) block. Leading or trailing whitespace is not allowed.

Query parameters

refresh string

Valid values are true, false, and wait_for. These values have the same meaning as in the index API, but the default value for this API is true.

Values are true, false, or wait_for.

application/json

Body Required

username string
email string | null

The email of the user.

One of:
string-1 string string-2 string | null
full_name string | null

The full name of the user.

One of:
string-1 string string-2 string | null
metadata object

Arbitrary metadata that you want to associate with the user.
Hide metadata attribute Show metadata attribute object
- * object Additional properties
password string

The user's password. Passwords must be at least 6 characters long. When adding a user, one of password or password_hash is required. When updating an existing user, the password is optional, so that other fields on the user (such as their roles) may be updated without modifying the user's password
password_hash string

A hash of the user's password. This must be produced using the same hashing algorithm as has been configured for password storage. For more details, see the explanation of the xpack.security.authc.password_hashing.algorithm setting in the user cache and password hash algorithm documentation. Using this parameter allows the client to pre-hash the password for performance and/or confidentiality reasons. The password parameter and the password_hash parameter cannot be used in the same request.

External documentation
roles array[string]

A set of roles the user has. The roles determine the user's access permissions. To create a user without any roles, specify an empty list ([]).
enabled boolean

Specifies whether the user is enabled.

Default value is true.

Responses

200 application/json
Hide response attribute Show response attribute object
- created boolean Required
  
  A successful call returns a JSON structure that shows whether the user has been created or updated. When an existing user is updated, created is set to false.

POST /_security/user/{username}

POST /_security/user/jacknich
{
  "password" : "l0ng-r4nd0m-p@ssw0rd",
  "roles" : [ "admin", "other_role1" ],
  "full_name" : "Jack Nicholson",
  "email" : "jacknich@example.com",
  "metadata" : {
    "intelligence" : 7
  }
}

resp = client.security.put_user(
    username="jacknich",
    password="l0ng-r4nd0m-p@ssw0rd",
    roles=[
        "admin",
        "other_role1"
    ],
    full_name="Jack Nicholson",
    email="jacknich@example.com",
    metadata={
        "intelligence": 7
    },
)

const response = await client.security.putUser({
  username: "jacknich",
  password: "l0ng-r4nd0m-p@ssw0rd",
  roles: ["admin", "other_role1"],
  full_name: "Jack Nicholson",
  email: "jacknich@example.com",
  metadata: {
    intelligence: 7,
  },
});

response = client.security.put_user(
  username: "jacknich",
  body: {
    "password": "l0ng-r4nd0m-p@ssw0rd",
    "roles": [
      "admin",
      "other_role1"
    ],
    "full_name": "Jack Nicholson",
    "email": "jacknich@example.com",
    "metadata": {
      "intelligence": 7
    }
  }
)

$resp = $client->security()->putUser([
    "username" => "jacknich",
    "body" => [
        "password" => "l0ng-r4nd0m-p@ssw0rd",
        "roles" => array(
            "admin",
            "other_role1",
        ),
        "full_name" => "Jack Nicholson",
        "email" => "jacknich@example.com",
        "metadata" => [
            "intelligence" => 7,
        ],
    ],
]);

curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"password":"l0ng-r4nd0m-p@ssw0rd","roles":["admin","other_role1"],"full_name":"Jack Nicholson","email":"jacknich@example.com","metadata":{"intelligence":7}}' "$ELASTICSEARCH_URL/_security/user/jacknich"

client.security().putUser(p -> p
    .email("jacknich@example.com")
    .fullName("Jack Nicholson")
    .metadata("intelligence", JsonData.fromJson("7"))
    .password("l0ng-r4nd0m-p@ssw0rd")
    .roles(List.of("admin","other_role1"))
    .username("jacknich")
);

Request example

Run `POST /_security/user/jacknich` to activate a user profile.

{
  "password" : "l0ng-r4nd0m-p@ssw0rd",
  "roles" : [ "admin", "other_role1" ],
  "full_name" : "Jack Nicholson",
  "email" : "jacknich@example.com",
  "metadata" : {
    "intelligence" : 7
  }
}

Response examples (200)

A successful response from `POST /_security/user/jacknich`. When an existing user is updated, `created` is set to `false`.

{
  "created": true 
}

Get service account credentials Generally available; Added in 7.13.0

GET /_security/service/{namespace}/{service}/credential

Api key auth Basic auth Bearer auth

To use this API, you must have at least the read_security cluster privilege (or a greater privilege such as manage_service_account or manage_security).

The response includes service account tokens that were created with the create service account tokens API as well as file-backed tokens from all nodes of the cluster.

NOTE: For tokens backed by the service_tokens file, the API collects them from all nodes of the cluster. Tokens with the same name from different nodes are assumed to be the same token and are only counted once towards the total number of service tokens.

Required authorization

Cluster privileges: read_security

External documentation

Path parameters

namespace string Required

The name of the namespace.
service string Required

The service name.

Responses

200 application/json
Hide response attributes Show response attributes object
- service_account string Required
- count number Required
- tokens object Required
  
  Hide tokens attribute Show tokens attribute object
  
  * object
  
  Hide * attribute Show * attribute object
  
  * object Additional properties
- nodes_credentials object Required
  
  Service account credentials collected from all nodes of the cluster.
  
  Hide nodes_credentials attributes Show nodes_credentials attributes object
  
  _nodes object Required
  
  General status showing how nodes respond to the above collection request
  
  Hide _nodes attributes Show _nodes attributes object
  
  failures array[object]
  
  Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.
  
  Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.
  
  total number Required
  
  Total number of nodes selected by the request.
  
  successful number Required
  
  Number of nodes that responded successfully to the request.
  
  failed number Required
  
  Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.
  
  file_tokens object Required
  
  File-backed tokens collected from all nodes
  
  Hide file_tokens attribute Show file_tokens attribute object
  
  * object Additional properties
  
  Hide * attribute Show * attribute object
  
  nodes array[string] Required

GET /_security/service/{namespace}/{service}/credential

GET /_security/service/elastic/fleet-server/credential

resp = client.security.get_service_credentials(
    namespace="elastic",
    service="fleet-server",
)

const response = await client.security.getServiceCredentials({
  namespace: "elastic",
  service: "fleet-server",
});

response = client.security.get_service_credentials(
  namespace: "elastic",
  service: "fleet-server"
)

$resp = $client->security()->getServiceCredentials([
    "namespace" => "elastic",
    "service" => "fleet-server",
]);

curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_security/service/elastic/fleet-server/credential"

client.security().getServiceCredentials(g -> g
    .namespace("elastic")
    .service("fleet-server")
);

Response examples (200)

A successful response from `GET /_security/service/elastic/fleet-server/credential`. The response contains all credentials for the `elastic/fleet-server` service account.

{
  "service_account": "elastic/fleet-server",
  "count": 3,
  "tokens": {
    "token1": {},        
    "token42": {}        
  },
  "nodes_credentials": { 
    "_nodes": {          
      "total": 3,
      "successful": 3,
      "failed": 0
    },
    "file_tokens": {      
      "my-token": {
        "nodes": [ "node0", "node1" ] 
      }
    }
  }
}

Suggest a user profile Generally available; Added in 8.2.0

POST /_security/profile/_suggest

Api key auth Basic auth Bearer auth

All methods and paths for this operation:

GET /_security/profile/_suggest

POST /_security/profile/_suggest

Get suggestions for user profiles that match specified search criteria.

NOTE: The user profile feature is designed only for use by Kibana and Elastic's Observability, Enterprise Search, and Elastic Security solutions. Individual users and external applications should not call this API directly. Elastic reserves the right to change or remove this feature in future releases without prior notice.

Required authorization

Cluster privileges: read_security

Query parameters

data string | array[string]

A comma-separated list of filters for the data field of the profile document. To return all content use data=*. To return a subset of content, use data=<key> to retrieve content nested under the specified <key>. By default, the API returns no data content. It is an error to specify data as both the query parameter and the request body field.

application/json

Body

name string

A query string used to match name-related fields in user profile documents. Name-related fields are the user's username, full_name, and email.
size number

The number of profiles to return.

Default value is 10.
data string | array[string]

A comma-separated list of filters for the data field of the profile document. To return all content use data=*. To return a subset of content, use data=<key> to retrieve content nested under the specified <key>. By default, the API returns no data content. It is an error to specify data as both the query parameter and the request body field.

One of:
string-1 string array-2 array[string]
hint object

Extra search criteria to improve relevance of the suggestion result. Profiles matching the spcified hint are ranked higher in the response. Profiles not matching the hint aren't excluded from the response as long as the profile matches the name field query.
Hide hint attributes Show hint attributes object
- uids array[string]
  
  A list of profile UIDs to match against.
- labels object
  
  A single key-value pair to match against the labels section of a profile. A profile is considered matching if it matches at least one of the strings.

Responses

200 application/json
Hide response attributes Show response attributes object
- total object Required
  
  Metadata about the number of matching profiles.
  
  Hide total attributes Show total attributes object
  
  value number Required
  
  relation string Required
- took number Required
  
  The number of milliseconds it took Elasticsearch to run the request.
- profiles array[object] Required
  
  A list of profile documents, ordered by relevance, that match the search criteria.
  
  Hide profiles attributes Show profiles attributes object
  
  uid string Required
  
  user object Required
  
  Hide user attributes Show user attributes object
  
  email string | null
  
  One of:
  string-1 string string-2 string | null
  
  full_name string | null
  
  One of:
  Name string string-2 string | null
  
  realm_name string Required
  
  realm_domain string
  
  roles array[string] Required
  
  username string Required
  
  data object Required
  
  Hide data attribute Show data attribute object
  
  * object Additional properties
  
  labels object Required
  
  Hide labels attribute Show labels attribute object
  
  * object Additional properties
  
  enabled boolean

POST /_security/profile/_suggest

POST /_security/profile/_suggest
{
  "name": "jack",  
  "hint": {
    "uids": [  
      "u_8RKO7AKfEbSiIHZkZZ2LJy2MUSDPWDr3tMI_CkIGApU_0",
      "u_79HkWkwmnBH5gqFKwoxggWPjEBOur1zLPXQPEl1VBW0_0"
    ],
    "labels": {
      "direction": ["north", "east"]  
    }
  }
}

resp = client.security.suggest_user_profiles(
    name="jack",
    hint={
        "uids": [
            "u_8RKO7AKfEbSiIHZkZZ2LJy2MUSDPWDr3tMI_CkIGApU_0",
            "u_79HkWkwmnBH5gqFKwoxggWPjEBOur1zLPXQPEl1VBW0_0"
        ],
        "labels": {
            "direction": [
                "north",
                "east"
            ]
        }
    },
)

const response = await client.security.suggestUserProfiles({
  name: "jack",
  hint: {
    uids: [
      "u_8RKO7AKfEbSiIHZkZZ2LJy2MUSDPWDr3tMI_CkIGApU_0",
      "u_79HkWkwmnBH5gqFKwoxggWPjEBOur1zLPXQPEl1VBW0_0",
    ],
    labels: {
      direction: ["north", "east"],
    },
  },
});

response = client.security.suggest_user_profiles(
  body: {
    "name": "jack",
    "hint": {
      "uids": [
        "u_8RKO7AKfEbSiIHZkZZ2LJy2MUSDPWDr3tMI_CkIGApU_0",
        "u_79HkWkwmnBH5gqFKwoxggWPjEBOur1zLPXQPEl1VBW0_0"
      ],
      "labels": {
        "direction": [
          "north",
          "east"
        ]
      }
    }
  }
)

$resp = $client->security()->suggestUserProfiles([
    "body" => [
        "name" => "jack",
        "hint" => [
            "uids" => array(
                "u_8RKO7AKfEbSiIHZkZZ2LJy2MUSDPWDr3tMI_CkIGApU_0",
                "u_79HkWkwmnBH5gqFKwoxggWPjEBOur1zLPXQPEl1VBW0_0",
            ),
            "labels" => [
                "direction" => array(
                    "north",
                    "east",
                ),
            ],
        ],
    ],
]);

curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"name":"jack","hint":{"uids":["u_8RKO7AKfEbSiIHZkZZ2LJy2MUSDPWDr3tMI_CkIGApU_0","u_79HkWkwmnBH5gqFKwoxggWPjEBOur1zLPXQPEl1VBW0_0"],"labels":{"direction":["north","east"]}}}' "$ELASTICSEARCH_URL/_security/profile/_suggest"

client.security().suggestUserProfiles(s -> s
    .hint(h -> h
        .uids(List.of("u_8RKO7AKfEbSiIHZkZZ2LJy2MUSDPWDr3tMI_CkIGApU_0","u_79HkWkwmnBH5gqFKwoxggWPjEBOur1zLPXQPEl1VBW0_0"))
        .labels("direction", List.of("north","east"))
    )
    .name("jack")
);

Request example

Run `POST /_security/profile/_suggest` to get suggestions for profile documents with name-related fields matching `jack`. It specifies both `uids` and `labels` hints for better relevance. The `labels` hint ranks profiles higher if their `direction` label matches either `north` or `east`.

{
  "name": "jack",  
  "hint": {
    "uids": [  
      "u_8RKO7AKfEbSiIHZkZZ2LJy2MUSDPWDr3tMI_CkIGApU_0",
      "u_79HkWkwmnBH5gqFKwoxggWPjEBOur1zLPXQPEl1VBW0_0"
    ],
    "labels": {
      "direction": ["north", "east"]  
    }
  }
}

Response examples (200)

A successful response from `GET /_security/saml/metadata/saml1`. It contains the SAML metadata that was generated for the SAML realm as an XML string.

{
    "metadata" : "<?xml version=\"1.0\" encoding=\"UTF-8\"?><md:EntityDescriptor xmlns:md=\"urn:oasis:names:tc:SAML:2.0:metadata\" entityID=\"https://kibana.org\"><md:SPSSODescriptor AuthnRequestsSigned=\"false\" WantAssertionsSigned=\"true\" protocolSupportEnumeration=\"urn:oasis:names:tc:SAML:2.0:protocol\"><md:SingleLogoutService Binding=\"urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect\" Location=\"https://kibana.org/logout\"/><md:AssertionConsumerService Binding=\"urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST\" Location=\"https://kibana.org/api/security/saml/callback\" index=\"1\" isDefault=\"true\"/></md:SPSSODescriptor></md:EntityDescriptor>"
}

Analyze a snapshot repository Generally available; Added in 7.12.0

POST /_snapshot/{repository}/_analyze

Api key auth Basic auth Bearer auth

Analyze the performance characteristics and any incorrect behaviour found in a repository.

The response exposes implementation details of the analysis which may change from version to version. The response body format is therefore not considered stable and may be different in newer versions.

There are a large number of third-party storage systems available, not all of which are suitable for use as a snapshot repository by Elasticsearch. Some storage systems behave incorrectly, or perform poorly, especially when accessed concurrently by multiple clients as the nodes of an Elasticsearch cluster do. This API performs a collection of read and write operations on your repository which are designed to detect incorrect behaviour and to measure the performance characteristics of your storage system.

The default values for the parameters are deliberately low to reduce the impact of running an analysis inadvertently and to provide a sensible starting point for your investigations. Run your first analysis with the default parameter values to check for simple problems. If successful, run a sequence of increasingly large analyses until you encounter a failure or you reach a blob_count of at least 2000, a max_blob_size of at least 2gb, a max_total_data_size of at least 1tb, and a register_operation_count of at least 100. Always specify a generous timeout, possibly 1h or longer, to allow time for each analysis to run to completion. Perform the analyses using a multi-node cluster of a similar size to your production cluster so that it can detect any problems that only arise when the repository is accessed by many nodes at once.

If the analysis fails, Elasticsearch detected that your repository behaved unexpectedly. This usually means you are using a third-party storage system with an incorrect or incompatible implementation of the API it claims to support. If so, this storage system is not suitable for use as a snapshot repository. You will need to work with the supplier of your storage system to address the incompatibilities that Elasticsearch detects.

If the analysis is successful, the API returns details of the testing process, optionally including how long each operation took. You can use this information to determine the performance of your storage system. If any operation fails or returns an incorrect result, the API returns an error. If the API returns an error, it may not have removed all the data it wrote to the repository. The error will indicate the location of any leftover data and this path is also recorded in the Elasticsearch logs. You should verify that this location has been cleaned up correctly. If there is still leftover data at the specified location, you should manually remove it.

If the connection from your client to Elasticsearch is closed while the client is waiting for the result of the analysis, the test is cancelled. Some clients are configured to close their connection if no response is received within a certain timeout. An analysis takes a long time to complete so you might need to relax any such client-side timeouts. On cancellation the analysis attempts to clean up the data it was writing, but it may not be able to remove it all. The path to the leftover data is recorded in the Elasticsearch logs. You should verify that this location has been cleaned up correctly. If there is still leftover data at the specified location, you should manually remove it.

If the analysis is successful then it detected no incorrect behaviour, but this does not mean that correct behaviour is guaranteed. The analysis attempts to detect common bugs but it does not offer 100% coverage. Additionally, it does not test the following:

Your repository must perform durable writes. Once a blob has been written it must remain in place until it is deleted, even after a power loss or similar disaster.
Your repository must not suffer from silent data corruption. Once a blob has been written, its contents must remain unchanged until it is deliberately modified or deleted.
Your repository must behave correctly even if connectivity from the cluster is disrupted. Reads and writes may fail in this case, but they must not return incorrect results.

IMPORTANT: An analysis writes a substantial amount of data to your repository and then reads it back again. This consumes bandwidth on the network between the cluster and the repository, and storage space and I/O bandwidth on the repository itself. You must ensure this load does not affect other users of these systems. Analyses respect the repository settings max_snapshot_bytes_per_sec and max_restore_bytes_per_sec if available and the cluster setting indices.recovery.max_bytes_per_sec which you can use to limit the bandwidth they consume.

NOTE: This API is intended for exploratory use by humans. You should expect the request parameters and the response format to vary in future versions.

NOTE: Different versions of Elasticsearch may perform different checks for repository compatibility, with newer versions typically being stricter than older ones. A storage system that passes repository analysis with one version of Elasticsearch may fail with a different version. This indicates it behaves incorrectly in ways that the former version did not detect. You must work with the supplier of your storage system to address the incompatibilities detected by the repository analysis API in any version of Elasticsearch.

NOTE: This API may not work correctly in a mixed-version cluster.

Implementation details

NOTE: This section of documentation describes how the repository analysis API works in this version of Elasticsearch, but you should expect the implementation to vary between versions. The request parameters and response format depend on details of the implementation so may also be different in newer versions.

The analysis comprises a number of blob-level tasks, as set by the blob_count parameter and a number of compare-and-exchange operations on linearizable registers, as set by the register_operation_count parameter. These tasks are distributed over the data and master-eligible nodes in the cluster for execution.

For most blob-level tasks, the executing node first writes a blob to the repository and then instructs some of the other nodes in the cluster to attempt to read the data it just wrote. The size of the blob is chosen randomly, according to the max_blob_size and max_total_data_size parameters. If any of these reads fails then the repository does not implement the necessary read-after-write semantics that Elasticsearch requires.

For some blob-level tasks, the executing node will instruct some of its peers to attempt to read the data before the writing process completes. These reads are permitted to fail, but must not return partial data. If any read returns partial data then the repository does not implement the necessary atomicity semantics that Elasticsearch requires.

For some blob-level tasks, the executing node will overwrite the blob while its peers are reading it. In this case the data read may come from either the original or the overwritten blob, but the read operation must not return partial data or a mix of data from the two blobs. If any of these reads returns partial data or a mix of the two blobs then the repository does not implement the necessary atomicity semantics that Elasticsearch requires for overwrites.

The executing node will use a variety of different methods to write the blob. For instance, where applicable, it will use both single-part and multi-part uploads. Similarly, the reading nodes will use a variety of different methods to read the data back again. For instance they may read the entire blob from start to end or may read only a subset of the data.

For some blob-level tasks, the executing node will cancel the write before it is complete. In this case, it still instructs some of the other nodes in the cluster to attempt to read the blob but all of these reads must fail to find the blob.

Linearizable registers are special blobs that Elasticsearch manipulates using an atomic compare-and-exchange operation. This operation ensures correct and strongly-consistent behavior even when the blob is accessed by multiple nodes at the same time. The detailed implementation of the compare-and-exchange operation on linearizable registers varies by repository type. Repository analysis verifies that that uncontended compare-and-exchange operations on a linearizable register blob always succeed. Repository analysis also verifies that contended operations either succeed or report the contention but do not return incorrect results. If an operation fails due to contention, Elasticsearch retries the operation until it succeeds. Most of the compare-and-exchange operations performed by repository analysis atomically increment a counter which is represented as an 8-byte blob. Some operations also verify the behavior on small blobs with sizes other than 8 bytes.

Required authorization

Cluster privileges: manage

Path parameters

repository string Required

The name of the repository.

Query parameters

blob_count number

The total number of blobs to write to the repository during the test. For realistic experiments, you should set it to at least 2000.
concurrency number

The number of operations to run concurrently during the test.
detailed boolean

Indicates whether to return detailed results, including timing information for every operation performed during the analysis. If false, it returns only a summary of the analysis.
early_read_node_count number

The number of nodes on which to perform an early read operation while writing each blob. Early read operations are only rarely performed.
max_blob_size number | string

The maximum size of a blob to be written during the test. For realistic experiments, you should set it to at least 2gb.
max_total_data_size number | string

An upper limit on the total size of all the blobs written during the test. For realistic experiments, you should set it to at least 1tb.
rare_action_probability number

The probability of performing a rare action such as an early read, an overwrite, or an aborted write on each blob.
rarely_abort_writes boolean

Indicates whether to rarely cancel writes before they complete.
read_node_count number

The number of nodes on which to read a blob after writing.
register_operation_count number

The minimum number of linearizable register operations to perform in total. For realistic experiments, you should set it to at least 100.
seed number

The seed for the pseudo-random number generator used to generate the list of operations performed during the test. To repeat the same set of operations in multiple experiments, use the same seed in each experiment. Note that the operations are performed concurrently so might not always happen in the same order on each run.
timeout string

The period of time to wait for the test to complete. If no response is received before the timeout expires, the test is cancelled and returns an error.

Values are -1 or 0.

Responses

200 application/json
Hide response attributes Show response attributes object
- blob_count number Required
  
  The number of blobs written to the repository during the test.
- blob_path string Required
  
  The path in the repository under which all the blobs were written during the test.
- concurrency number Required
  
  The number of write operations performed concurrently during the test.
- coordinating_node object Required
  
  The node that coordinated the analysis and performed the final cleanup.
  
  Hide coordinating_node attributes Show coordinating_node attributes object
  
  id string Required
  
  name string Required
- delete_elapsed string Required
  
  The time it took to delete all the blobs in the container.
- delete_elapsed_nanos number
  
  Time unit for nanoseconds
- details object Required
  
  A description of every read and write operation performed during the test.
  
  Hide details attributes Show details attributes object
  
  blob object Required
  
  A description of the blob that was written and read.
  
  Hide blob attributes Show blob attributes object
  
  name string Required
  
  The name of the blob.
  
  overwritten boolean Required
  
  Indicates whether the blob was overwritten while the read operations were ongoing. /**
  
  read_early boolean Required
  
  read_end number Required
  
  The position, in bytes, at which read operations completed.
  
  read_start number Required
  
  The position, in bytes, at which read operations started.
  
  reads object Required
  
  A description of every read operation performed on the blob.
  
  size
  
  size_bytes number Required
  
  The size of the blob in bytes.
  
  overwrite_elapsed string
  
  The elapsed time spent overwriting the blob. If the blob was not overwritten, this information is omitted.
  
  Time unit for nanoseconds
  
  write_elapsed string Required
  
  The elapsed time spent writing the blob.
  
  Time unit for nanoseconds
  
  write_throttled string Required
  
  The length of time spent waiting for the max_snapshot_bytes_per_sec (or indices.recovery.max_bytes_per_sec if the recovery settings for managed services are set) throttle while writing the blob.
  
  Time unit for nanoseconds
  
  writer_node object Required
  
  The node which wrote the blob and coordinated the read operations.
  
  Hide writer_node attributes Show writer_node attributes object
  
  id string Required
  
  name string Required
- early_read_node_count number Required
  
  The limit on the number of nodes on which early read operations were performed after writing each blob.
- issues_detected array[string] Required
  
  A list of correctness issues detected, which is empty if the API succeeded. It is included to emphasize that a successful response does not guarantee correct behaviour in future.
- listing_elapsed string Required
  
  The time it took to retrieve a list of all the blobs in the container.
- listing_elapsed_nanos number
  
  Time unit for nanoseconds
- max_blob_size number | string
  
  The limit on the size of a blob written during the test.
  
  One of:
  number-1 number string-2 string
  
  The limit on the size of a blob written during the test.
- max_blob_size_bytes number Required
  
  The limit, in bytes, on the size of a blob written during the test.
- max_total_data_size number | string
  
  The limit on the total size of all blob written during the test.
  
  One of:
  number-1 number string-2 string
  
  The limit on the total size of all blob written during the test.
- max_total_data_size_bytes number Required
  
  The limit, in bytes, on the total size of all blob written during the test.
- rare_action_probability number Required
  
  The probability of performing rare actions during the test.
- read_node_count number Required
  
  The limit on the number of nodes on which read operations were performed after writing each blob.
- repository string Required
  
  The name of the repository that was the subject of the analysis.
- seed number Required
  
  The seed for the pseudo-random number generator used to generate the operations used during the test.
- summary object Required
  
  A collection of statistics that summarize the results of the test.
  
  Hide summary attributes Show summary attributes object
  
  read object Required
  
  A collection of statistics that summarise the results of the read operations in the test.
  
  Hide read attributes Show read attributes object
  
  count number Required
  
  The number of read operations performed in the test.
  
  max_wait string Required
  
  The maximum time spent waiting for the first byte of any read request to be received.
  
  total_elapsed string Required
  
  The total elapsed time spent on reading blobs in the test.
  
  total_size
  
  total_size_bytes number Required
  
  The total size of all the blobs or partial blobs read in the test, in bytes.
  
  total_throttled string Required
  
  The total time spent waiting due to the max_restore_bytes_per_sec or indices.recovery.max_bytes_per_sec throttles.
  
  total_wait string Required
  
  The total time spent waiting for the first byte of each read request to be received.
  
  write object Required
  
  A collection of statistics that summarise the results of the write operations in the test.
  
  Hide write attributes Show write attributes object
  
  count number Required
  
  The number of write operations performed in the test.
  
  total_elapsed string Required
  
  The total elapsed time spent on writing blobs in the test.
  
  total_size
  
  total_size_bytes number Required
  
  The total size of all the blobs written in the test, in bytes.
  
  total_throttled string Required
  
  The total time spent waiting due to the max_snapshot_bytes_per_sec throttle.
  
  total_throttled_nanos number Required
  
  The total time spent waiting due to the max_snapshot_bytes_per_sec throttle, in nanoseconds.

POST /_snapshot/{repository}/_analyze

POST /_snapshot/my_repository/_analyze?blob_count=10&max_blob_size=1mb&timeout=120s

resp = client.snapshot.repository_analyze(
    name="my_repository",
    blob_count="10",
    max_blob_size="1mb",
    timeout="120s",
)

const response = await client.snapshot.repositoryAnalyze({
  name: "my_repository",
  blob_count: 10,
  max_blob_size: "1mb",
  timeout: "120s",
});

response = client.snapshot.repository_analyze(
  repository: "my_repository",
  blob_count: "10",
  max_blob_size: "1mb",
  timeout: "120s"
)

$resp = $client->snapshot()->repositoryAnalyze([
    "repository" => "my_repository",
    "blob_count" => "10",
    "max_blob_size" => "1mb",
    "timeout" => "120s",
]);

curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_snapshot/my_repository/_analyze?blob_count=10&max_blob_size=1mb&timeout=120s"

client.snapshot().repositoryAnalyze(r -> r
    .blobCount(10)
    .maxBlobSize("1mb")
    .name("my_repository")
    .timeout(t -> t
        .offset(120)
    )
);

Get snapshot lifecycle management statistics Generally available; Added in 7.5.0

GET /_slm/stats

Api key auth Basic auth Bearer auth

Get global and policy-level statistics about actions taken by snapshot lifecycle management.

Required authorization

Cluster privileges: manage_slm

Query parameters

master_timeout string

Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

Values are -1 or 0.
timeout string

Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

Values are -1 or 0.

Responses

200 application/json
Hide response attributes Show response attributes object
- retention_deletion_time string Required
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
- retention_deletion_time_millis number
  
  Time unit for milliseconds
- retention_failed number Required
- retention_runs number Required
- retention_timed_out number Required
- total_snapshots_deleted number Required
- total_snapshot_deletion_failures number Required
- total_snapshots_failed number Required
- total_snapshots_taken number Required
- policy_stats array[object] Required
  
  Hide policy_stats attributes Show policy_stats attributes object
  
  policy string Required
  
  snapshots_taken number Required
  
  snapshots_failed number Required
  
  snapshots_deleted number Required
  
  snapshot_deletion_failures number Required

GET /_slm/stats

GET /_slm/stats

resp = client.slm.get_stats()

const response = await client.slm.getStats();

response = client.slm.get_stats

$resp = $client->slm()->getStats();

curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_slm/stats"

client.slm().getStats(g -> g);

Response examples (200)

A successful response from `GET /_slm/stats`.

{
  "retention_runs": 13,
  "retention_failed": 0,
  "retention_timed_out": 0,
  "retention_deletion_time": "1.4s",
  "retention_deletion_time_millis": 1404,
  "policy_stats": [ ],
  "total_snapshots_taken": 1,
  "total_snapshots_failed": 1,
  "total_snapshots_deleted": 0,
  "total_snapshot_deletion_failures": 0
}

Get transform stats Generally available; Added in 7.5.0

GET /_transform/{transform_id}/_stats

Api key auth Basic auth Bearer auth

Get usage information for transforms.

Required authorization

Index privileges: read,view_index_metadata
Cluster privileges: monitor_transform

Path parameters

transform_id string | array[string] Required

Identifier for the transform. It can be a transform identifier or a wildcard expression. You can get information for all transforms by using _all, by specifying * as the <transform_id>, or by omitting the <transform_id>.

Query parameters

allow_no_match boolean
Specifies what to do when the request:
1. Contains wildcard expressions and there are no transforms that match.
2. Contains the _all string or no identifiers and there are no matches.
3. Contains wildcard expressions and there are only partial matches.
If this parameter is false, the request returns a 404 status code when there are no matches or only partial matches.
from number

Skips the specified number of transforms.
size number

Specifies the maximum number of transforms to obtain.
timeout string

Controls the time to wait for the stats

Values are -1 or 0.

Responses

200 application/json
Hide response attributes Show response attributes object
- count number Required
- transforms array[object] Required
  
  Hide transforms attributes Show transforms attributes object
  
  checkpointing object Required
  
  Hide checkpointing attributes Show checkpointing attributes object
  
  changes_last_detected_at number
  
  changes_last_detected_at_string
  
  last object Required
  
  next object
  
  operations_behind number
  
  last_search_time number
  
  last_search_time_string
  
  health object
  
  Hide health attributes Show health attributes object
  
  status string Required
  
  Supported values include:
  
  green (or GREEN): All shards are assigned.
  
  yellow (or YELLOW): All primary shards are assigned, but one or more replica shards are unassigned. If a node in the cluster fails, some data could be unavailable until that node is repaired.
  
  red (or RED): One or more primary shards are unassigned, so some data is unavailable. This can occur briefly during cluster startup as primary shards are assigned.
  
  unknown
  
  unavailable
  
  Values are green, GREEN, yellow, YELLOW, red, RED, unknown, or unavailable.
  
  issues array[object]
  
  If a non-healthy status is returned, contains a list of issues of the transform.
  
  id string Required
  
  node object
  
  Hide node attributes Show node attributes object
  
  attributes object Required
  
  Lists node attributes.
  
  Hide attributes attribute Show attributes attribute object
  
  * string Additional properties
  
  ephemeral_id string Required
  
  The ephemeral ID of the node.
  
  id string
  
  The unique identifier of the node.
  
  name string Required
  
  The unique identifier of the node.
  
  transport_address string Required
  
  The host and port where transport HTTP connections are accepted.
  
  reason string
  
  state string Required
  
  stats object Required
  
  Hide stats attributes Show stats attributes object
  
  documents_indexed number Required
  
  documents_deleted number
  
  documents_processed number Required
  
  exponential_avg_documents_indexed number Required
  
  exponential_avg_documents_processed number Required
  
  index_failures number Required
  
  index_total number Required
  
  pages_processed number Required
  
  processing_total number Required
  
  search_failures number Required
  
  search_total number Required
  
  trigger_count number Required

GET /_transform/{transform_id}/_stats

GET _transform/ecommerce-customer-transform/_stats

resp = client.transform.get_transform_stats(
    transform_id="ecommerce-customer-transform",
)

const response = await client.transform.getTransformStats({
  transform_id: "ecommerce-customer-transform",
});

response = client.transform.get_transform_stats(
  transform_id: "ecommerce-customer-transform"
)

$resp = $client->transform()->getTransformStats([
    "transform_id" => "ecommerce-customer-transform",
]);

curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_transform/ecommerce-customer-transform/_stats"

client.transform().getTransformStats(g -> g
    .transformId("ecommerce-customer-transform")
);

Response examples (200)

A successful response that contains usage information for a transform.

{
  "count": 1,
  "transforms": [
    {
      "id": "ecommerce-customer-transform",
      "state": "started",
      "node": {
        "id": "cpTIGMsVQ8Gqwqlxxxxxxx",
        "name": "my.home",
        "ephemeral_id": "5-L21nFsQxxxxxxxxxx-xx",
        "transport_address": "127.0.0.1:9300",
        "attributes": {}
      },
      "stats": {
        "pages_processed": 78,
        "documents_processed": 6027,
        "documents_indexed": 68,
        "documents_deleted": 22,
        "delete_time_in_ms": 214,
        "trigger_count": 168,
        "index_time_in_ms": 412,
        "index_total": 20,
        "index_failures": 0,
        "search_time_in_ms": 353,
        "search_total": 78,
        "search_failures": 0,
        "processing_time_in_ms": 8,
        "processing_total": 78,
        "exponential_avg_checkpoint_duration_ms": 97.30637923893185,
        "exponential_avg_documents_indexed": 2.2064915040974062,
        "exponential_avg_documents_processed": 179.89419945785045
      },
      "checkpointing": {
        "last": {
          "checkpoint": 20,
          "timestamp_millis": 1585344558220,
          "time_upper_bound_millis": 1585344498220
        },
        "changes_last_detected_at": 1585344558219
      },
      "health": {
        "status": "green"
      }
    }
  ]
}

Run a watch Generally available

POST /_watcher/watch/{id}/_execute

Api key auth Basic auth Bearer auth

All methods and paths for this operation:

PUT /_watcher/watch/_execute

POST /_watcher/watch/_execute

PUT /_watcher/watch/{id}/_execute

POST /_watcher/watch/{id}/_execute

This API can be used to force execution of the watch outside of its triggering logic or to simulate the watch execution for debugging purposes.

For testing and debugging purposes, you also have fine-grained control on how the watch runs. You can run the watch without running all of its actions or alternatively by simulating them. You can also force execution by ignoring the watch condition and control whether a watch record would be written to the watch history after it runs.

You can use the run watch API to run watches that are not yet registered by specifying the watch definition inline. This serves as great tool for testing and debugging your watches prior to adding them to Watcher.

When Elasticsearch security features are enabled on your cluster, watches are run with the privileges of the user that stored the watches. If your user is allowed to read index a, but not index b, then the exact same set of rules will apply during execution of a watch.

When using the run watch API, the authorization data of the user that called the API will be used as a base, instead of the information who stored the watch.

Required authorization

Cluster privileges: manage_watcher

Path parameters

id string Required

The watch identifier.

Query parameters

debug boolean

Defines whether the watch runs in debug mode.

application/json

Body

action_modes object

Determines how to handle the watch actions as part of the watch execution.
Hide action_modes attribute Show action_modes attribute object
- * string Additional properties
  
  Values are simulate, force_simulate, execute, force_execute, or skip.
alternative_input object

When present, the watch uses this object as a payload instead of executing its own input.
Hide alternative_input attribute Show alternative_input attribute object
- * object Additional properties
ignore_condition boolean

When set to true, the watch execution uses the always condition. This can also be specified as an HTTP parameter.

Default value is false.
record_execution boolean

When set to true, the watch record representing the watch execution result is persisted to the .watcher-history index for the current time. In addition, the status of the watch is updated, possibly throttling subsequent runs. This can also be specified as an HTTP parameter.

Default value is false.
simulated_actions object
Hide simulated_actions attributes Show simulated_actions attributes object
- actions array[string] Required
- all object Required
- use_all boolean Required
trigger_data object

This structure is parsed as the data of the trigger event that will be used during the watch execution.
Hide trigger_data attributes Show trigger_data attributes object
- scheduled_time string | number
  
  One of:
  string-1 string UnitMillis number
- triggered_time string | number
  
  One of:
  string-1 string UnitMillis number
watch object

When present, this watch is used instead of the one specified in the request. This watch is not persisted to the index and record_execution cannot be set.
Hide watch attributes Show watch attributes object
- actions object Required
  Hide actions attribute Show actions attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  action_type string
  
  Values are email, webhook, index, logging, slack, or pagerduty.
  
  condition object
  
  Hide condition attributes Show condition attributes object
  
  array_compare object
  
  compare object
  
  foreach string
  
  max_iterations number
  
  name string
  
  throttle_period string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  Time unit for milliseconds
  
  transform object
  
  Hide transform attribute Show transform attribute object
  
  chain array[object]
  
  index object
  
  logging object
  
  Hide logging attributes Show logging attributes object
  
  level string
  
  text string Required
  
  category string
  
  slack object
  
  Hide slack attribute Show slack attribute object
  
  account string
- condition object Required
  Hide condition attributes Show condition attributes object
  
  always object
  
  array_compare object
  
  Hide array_compare attribute Show array_compare attribute object
  
  * object Additional properties
  
  Hide * attribute Show * attribute object
  
  path string Required
  
  compare object
  
  Hide compare attribute Show compare attribute object
  
  * object Additional properties
  
  never object
  
  script object
  
  Hide script attributes Show script attributes object
  
  lang string
  
  Default value is painless.
  
  params object
  
  source string
  
  id string
- input object Required
  Hide input attributes Show input attributes object
  
  chain object
  
  Hide chain attribute Show chain attribute object
  
  inputs array[object] Required
  
  http object
  
  Hide http attribute Show http attribute object
  
  extract array[string]
  
  search object
  
  Hide search attribute Show search attribute object
  
  extract array[string]
  
  simple object
  
  Hide simple attribute Show simple attribute object
  
  * object Additional properties
- metadata object
  Hide metadata attribute Show metadata attribute object
  
  * object Additional properties
- status object
  Hide status attributes Show status attributes object
  
  actions object Required
  
  Hide actions attribute Show actions attribute object
  
  * object Additional properties
  
  last_checked string
  
  last_met_condition string
  
  state object Required
  
  Hide state attribute Show state attribute object
  
  active boolean Required
  
  version number Required
  
  execution_state string
- throttle_period string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
- Time unit for milliseconds
- transform object
  Hide transform attributes Show transform attributes object
  
  chain array[object]
  
  script object
  
  Hide script attributes Show script attributes object
  
  lang string
  
  Default value is painless.
  
  params object
  
  source string
  
  id string
  
  search object
- trigger object Required
  Hide trigger attribute Show trigger attribute object
  
  schedule object
  
  Hide schedule attributes Show schedule attributes object
  
  timezone string
  
  monthly
  
  weekly
  
  yearly

Responses

200 application/json
Hide response attributes Show response attributes object
- _id string Required
  
  The watch record identifier as it would be stored in the .watcher-history index.
- watch_record object Required
  
  The watch record document as it would be stored in the .watcher-history index.
  
  Hide watch_record attributes Show watch_record attributes object
  
  condition object Required
  
  Hide condition attributes Show condition attributes object
  
  always object
  
  array_compare object
  
  Hide array_compare attribute Show array_compare attribute object
  
  * object Additional properties
  
  compare object
  
  Hide compare attribute Show compare attribute object
  
  * object Additional properties
  
  never object
  
  script object
  
  input object Required
  
  Hide input attributes Show input attributes object
  
  chain object
  
  http object
  
  search object
  
  simple object
  
  Hide simple attribute Show simple attribute object
  
  * object Additional properties
  
  messages array[string] Required
  
  metadata object
  
  Hide metadata attribute Show metadata attribute object
  
  * object Additional properties
  
  node string Required
  
  result object Required
  
  Hide result attributes Show result attributes object
  
  actions array[object] Required
  
  condition object Required
  
  execution_time
  
  input object Required
  
  state string Required
  
  Values are awaits_execution, checking, execution_not_needed, throttled, executed, failed, deleted_while_queued, or not_executed_already_queued.
  
  trigger_event object Required
  
  Hide trigger_event attributes Show trigger_event attributes object
  
  manual object Required
  
  triggered_time
  
  type string Required
  
  user string Required
  
  watch_id string Required
  
  status object
  
  Hide status attributes Show status attributes object
  
  actions object Required
  
  last_checked
  
  last_met_condition
  
  state object Required
  
  version number Required
  
  execution_state string

POST /_watcher/watch/{id}/_execute

POST _watcher/watch/my_watch/_execute
{
  "trigger_data" : { 
    "triggered_time" : "now",
    "scheduled_time" : "now"
  },
  "alternative_input" : { 
    "foo" : "bar"
  },
  "ignore_condition" : true, 
  "action_modes" : {
    "my-action" : "force_simulate" 
  },
  "record_execution" : true 
}

resp = client.watcher.execute_watch(
    id="my_watch",
    trigger_data={
        "triggered_time": "now",
        "scheduled_time": "now"
    },
    alternative_input={
        "foo": "bar"
    },
    ignore_condition=True,
    action_modes={
        "my-action": "force_simulate"
    },
    record_execution=True,
)

const response = await client.watcher.executeWatch({
  id: "my_watch",
  trigger_data: {
    triggered_time: "now",
    scheduled_time: "now",
  },
  alternative_input: {
    foo: "bar",
  },
  ignore_condition: true,
  action_modes: {
    "my-action": "force_simulate",
  },
  record_execution: true,
});

response = client.watcher.execute_watch(
  id: "my_watch",
  body: {
    "trigger_data": {
      "triggered_time": "now",
      "scheduled_time": "now"
    },
    "alternative_input": {
      "foo": "bar"
    },
    "ignore_condition": true,
    "action_modes": {
      "my-action": "force_simulate"
    },
    "record_execution": true
  }
)

$resp = $client->watcher()->executeWatch([
    "id" => "my_watch",
    "body" => [
        "trigger_data" => [
            "triggered_time" => "now",
            "scheduled_time" => "now",
        ],
        "alternative_input" => [
            "foo" => "bar",
        ],
        "ignore_condition" => true,
        "action_modes" => [
            "my-action" => "force_simulate",
        ],
        "record_execution" => true,
    ],
]);

curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"trigger_data":{"triggered_time":"now","scheduled_time":"now"},"alternative_input":{"foo":"bar"},"ignore_condition":true,"action_modes":{"my-action":"force_simulate"},"record_execution":true}' "$ELASTICSEARCH_URL/_watcher/watch/my_watch/_execute"

client.watcher().executeWatch(e -> e
    .actionModes("my-action", ActionExecutionMode.ForceSimulate)
    .alternativeInput("foo", JsonData.fromJson("\"bar\""))
    .id("my_watch")
    .ignoreCondition(true)
    .recordExecution(true)
    .triggerData(t -> t
        .scheduledTime(DateTime.of("now"))
        .triggeredTime(DateTime.of("now"))
    )
);

Request examples

Run `POST _watcher/watch/my_watch/_execute` to run a watch. The input defined in the watch is ignored and the `alternative_input` is used as the payload. The condition as defined by the watch is ignored and is assumed to evaluate to true. The `force_simulate` action forces the simulation of `my-action`. Forcing the simulation means that throttling is ignored and the watch is simulated by Watcher instead of being run normally.

{
  "trigger_data" : { 
    "triggered_time" : "now",
    "scheduled_time" : "now"
  },
  "alternative_input" : { 
    "foo" : "bar"
  },
  "ignore_condition" : true, 
  "action_modes" : {
    "my-action" : "force_simulate" 
  },
  "record_execution" : true 
}

Run `POST _watcher/watch/my_watch/_execute` and set a different mode for each action.

{
  "action_modes" : {
    "action1" : "force_simulate",
    "action2" : "skip"
  }
}

Run `POST _watcher/watch/_execute` to run a watch inline. All other settings for this API still apply when inlining a watch. In this example, while the inline watch defines a compare condition, during the execution this condition will be ignored.

{
  "watch" : {
    "trigger" : { "schedule" : { "interval" : "10s" } },
    "input" : {
      "search" : {
        "request" : {
          "indices" : [ "logs" ],
          "body" : {
            "query" : {
              "match" : { "message": "error" }
            }
          }
        }
      }
    },
    "condition" : {
      "compare" : { "ctx.payload.hits.total" : { "gt" : 0 }}
    },
    "actions" : {
      "log_error" : {
        "logging" : {
          "text" : "Found {{ctx.payload.hits.total}} errors in the logs"
        }
      }
    }
  }
}

Response examples (200)

A successful response from `POST _watcher/watch/my_watch/_execute`.

{
  "_id": "my_watch_0-2015-06-02T23:17:55.124Z", 
  "watch_record": { 
    "@timestamp": "2015-06-02T23:17:55.124Z",
    "watch_id": "my_watch",
    "node": "my_node",
    "messages": [],
    "trigger_event": {
      "type": "manual",
      "triggered_time": "2015-06-02T23:17:55.124Z",
      "manual": {
        "schedule": {
          "scheduled_time": "2015-06-02T23:17:55.124Z"
        }
      }
    },
    "state": "executed",
    "status": {
      "version": 1,
      "execution_state": "executed",
      "state": {
        "active": true,
        "timestamp": "2015-06-02T23:17:55.111Z"
      },
      "last_checked": "2015-06-02T23:17:55.124Z",
      "last_met_condition": "2015-06-02T23:17:55.124Z",
      "actions": {
        "test_index": {
          "ack": {
            "timestamp": "2015-06-02T23:17:55.124Z",
            "state": "ackable"
          },
          "last_execution": {
            "timestamp": "2015-06-02T23:17:55.124Z",
            "successful": true
          },
          "last_successful_execution": {
            "timestamp": "2015-06-02T23:17:55.124Z",
            "successful": true
          }
        }
      }
    },
    "input": {
      "simple": {
        "payload": {
          "send": "yes"
        }
      }
    },
    "condition": {
      "always": {}
    },
    "result": { 
      "execution_time": "2015-06-02T23:17:55.124Z",
      "execution_duration": 12608,
      "input": {
        "type": "simple",
        "payload": {
          "foo": "bar"
        },
        "status": "success"
      },
      "condition": {
        "type": "always",
        "met": true,
        "status": "success"
      },
      "actions": [
        {
          "id": "test_index",
          "index": {
            "response": {
              "index": "test",
              "version": 1,
              "created": true,
              "result": "created",
              "id": "AVSHKzPa9zx62AzUzFXY"
            }
          },
          "status": "success",
          "type": "index"
        }
      ]
    },
    "user": "test_admin" 
  }
}

Get component templates Generally available; Added in 5.1.0

Required authorization

Path parameters

Query parameters

Responses

version string | null Required

Get field data cache information Generally available

Required authorization

Path parameters

Query parameters

Responses

Get shard information Generally available

Required authorization

Path parameters

Query parameters

Responses

docs string | null

store string | null

dataset string | null

ip string | null

node string | null

Get the cluster health Generally available; Added in 8.7.0

Path parameters

Query parameters

Responses

Get all connector sync jobs Beta; Added in 8.12.0

Query parameters

Responses

cancelation_requested_at string | number

canceled_at string | number

completed_at string | number

created_at string | number

last_seen string | number

started_at string | number

Executes several fleet searches with a single API request Technical preview; Added in 7.16.0

Required authorization

Path parameters

Query parameters

Body object Required

Responses

Simulate an index template Generally available

Required authorization

Path parameters

Query parameters

Body

Responses

Start the ILM plugin Generally available; Added in 6.6.0

Required authorization

Query parameters

Responses

Create or update an IP geolocation database configuration Generally available; Added in 8.15.0

Required authorization

Path parameters

Query parameters

Body Required

Responses

Create or update a pipeline Generally available; Added in 5.0.0

Path parameters

Query parameters

Body Required

value object | array[object] Required

input_output object | array[object]

dataset string | array[string]

namespace string | array[string]

value object | array[object] Required

input_output object | array[object]

dataset string | array[string]

namespace string | array[string]

Responses

Revert to a snapshot Generally available; Added in 5.4.0

Required authorization

Path parameters

Query parameters

Body

Responses

Get rollup job information Deprecated Technical preview; Added in 6.3.0

Required authorization

Path parameters

Responses

Get async search results Generally available; Added in 7.7.0