Create or update an autoscaling policy
Added in 7.11.0
NOTE: This feature is designed for indirect use by Elasticsearch Service, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.
Path parameters
-
name
string Required the name of the autoscaling policy
Query parameters
-
master_timeout
string Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1
or0
. -
timeout
string Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1
or0
.
Body
Required
-
roles
array[string] Required -
deciders
object Required Decider settings.
External documentation
PUT /_autoscaling/policy/<name>
{
"roles": [],
"deciders": {
"fixed": {
}
}
}
curl \
--request PUT 'http://api.example.com/_autoscaling/policy/{name}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"roles\": [],\n \"deciders\": {\n \"fixed\": {\n }\n }\n}"'
{
"roles": [],
"deciders": {
"fixed": {
}
}
}
{
"roles" : [ "data_hot" ],
"deciders": {
"fixed": {
}
}
}
{
"acknowledged": true
}
Get the autoscaling capacity
Added in 7.11.0
NOTE: This feature is designed for indirect use by Elasticsearch Service, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.
This API gets the current autoscaling capacity based on the configured autoscaling policy. It will return information to size the cluster appropriately to the current workload.
The required_capacity
is calculated as the maximum of the required_capacity
result of all individual deciders that are enabled for the policy.
The operator should verify that the current_nodes
match the operator’s knowledge of the cluster to avoid making autoscaling decisions based on stale or incomplete information.
The response contains decider-specific information you can use to diagnose how and why autoscaling determined a certain capacity was required. This information is provided for diagnosis only. Do not use this information to make autoscaling decisions.
Query parameters
-
master_timeout
string Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1
or0
.
GET /_autoscaling/capacity
curl \
--request GET 'http://api.example.com/_autoscaling/capacity' \
--header "Authorization: $API_KEY"
{
policies: {}
}
Compact and aligned text (CAT)
The compact and aligned text (CAT) APIs aim are intended only for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, it's recommend to use a corresponding JSON API.
All the cat commands accept a query string parameter help
to see all the headers and info they provide, and the /_cat
command alone lists all the available commands.
Get component templates
Added in 5.1.0
Get information about component templates in a cluster. Component templates are building blocks for constructing index templates that specify index mappings, settings, and aliases.
IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the get component template API.
Query parameters
-
h
string | array[string] List of columns to appear in the response. Supports simple wildcards.
-
s
string | array[string] List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting
:asc
or:desc
as a suffix to the column name. -
local
boolean If
true
, the request computes the list of selected nodes from the local cluster state. Iffalse
the list of selected nodes are computed from the cluster state of the master node. In both cases the coordinating node will send requests for further information to each selected node. -
master_timeout
string The period to wait for a connection to the master node.
Values are
-1
or0
.
GET _cat/component_templates/my-template-*?v=true&s=name&format=json
curl \
--request GET 'http://api.example.com/_cat/component_templates' \
--header "Authorization: $API_KEY"
[
{
"name": "my-template-1",
"version": "null",
"alias_count": "0",
"mapping_count": "0",
"settings_count": "1",
"metadata_count": "0",
"included_in": "[my-index-template]"
},
{
"name": "my-template-2",
"version": null,
"alias_count": "0",
"mapping_count": "3",
"settings_count": "0",
"metadata_count": "0",
"included_in": "[my-index-template]"
}
]
Get a document count
Get quick access to a document count for a data stream, an index, or an entire cluster. The document count only includes live documents, not deleted documents which have not yet been removed by the merge process.
IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the count API.
Path parameters
-
index
string | array[string] Required A comma-separated list of data streams, indices, and aliases used to limit the request. It supports wildcards (
*
). To target all data streams and indices, omit this parameter or use*
or_all
.
GET /_cat/count/my-index-000001?v=true&format=json
curl \
--request GET 'http://api.example.com/_cat/count/{index}' \
--header "Authorization: $API_KEY"
[
{
"epoch": "1475868259",
"timestamp": "15:24:20",
"count": "120"
}
]
[
{
"epoch": "1475868259",
"timestamp": "15:24:20",
"count": "121"
}
]
Get node information
Get information about the nodes in a cluster. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the nodes info API.
Query parameters
-
bytes
string The unit used to display byte values.
Values are
b
,kb
,mb
,gb
,tb
, orpb
. -
full_id
boolean | string If
true
, return the full node ID. Iffalse
, return the shortened node ID. -
include_unloaded_segments
boolean If true, the response includes information from segments that are not loaded into memory.
-
h
string | array[string] A comma-separated list of columns names to display. It supports simple wildcards.
Supported values include:
build
(orb
): The Elasticsearch build hash. For example:5c03844
.completion.size
(orcs
,completionSize
): The size of completion. For example:0b
.cpu
: The percentage of recent system CPU used.disk.avail
(ord
,disk
,diskAvail
): The available disk space. For example:198.4gb
.disk.total
(ordt
,diskTotal
): The total disk space. For example:458.3gb
.disk.used
(ordu
,diskUsed
): The used disk space. For example:259.8gb
.disk.used_percent
(ordup
,diskUsedPercent
): The percentage of disk space used.fielddata.evictions
(orfe
,fielddataEvictions
): The number of fielddata cache evictions.fielddata.memory_size
(orfm
,fielddataMemory
): The fielddata cache memory used. For example:0b
.file_desc.current
(orfdc
,fileDescriptorCurrent
): The number of file descriptors used.file_desc.max
(orfdm
,fileDescriptorMax
): The maximum number of file descriptors.file_desc.percent
(orfdp
,fileDescriptorPercent
): The percentage of file descriptors used.flush.total
(orft
,flushTotal
): The number of flushes.flush.total_time
(orftt
,flushTotalTime
): The amount of time spent in flush.get.current
(orgc
,getCurrent
): The number of current get operations.get.exists_time
(orgeti
,getExistsTime
): The time spent in successful get operations. For example:14ms
.get.exists_total
(orgeto
,getExistsTotal
): The number of successful get operations.get.missing_time
(orgmti
,getMissingTime
): The time spent in failed get operations. For example:0s
.get.missing_total
(orgmto
,getMissingTotal
): The number of failed get operations.get.time
(orgti
,getTime
): The amount of time spent in get operations. For example:14ms
.get.total
(orgto
,getTotal
): The number of get operations.heap.current
(orhc
,heapCurrent
): The used heap size. For example:311.2mb
.heap.max
(orhm
,heapMax
): The total heap size. For example:4gb
.heap.percent
(orhp
,heapPercent
): The used percentage of total allocated Elasticsearch JVM heap. This value reflects only the Elasticsearch process running within the operating system and is the most direct indicator of its JVM, heap, or memory resource performance.http_address
(orhttp
): The bound HTTP address.id
(ornodeId
): The identifier for the node.indexing.delete_current
(oridc
,indexingDeleteCurrent
): The number of current deletion operations.indexing.delete_time
(oridti
,indexingDeleteTime
): The time spent in deletion operations. For example:2ms
.indexing.delete_total
(oridto
,indexingDeleteTotal
): The number of deletion operations.indexing.index_current
(oriic
,indexingIndexCurrent
): The number of current indexing operations.indexing.index_failed
(oriif
,indexingIndexFailed
): The number of failed indexing operations.indexing.index_failed_due_to_version_conflict
(oriifvc
,indexingIndexFailedDueToVersionConflict
): The number of indexing operations that failed due to version conflict.indexing.index_time
(oriiti
,indexingIndexTime
): The time spent in indexing operations. For example:134ms
.indexing.index_total
(oriito
,indexingIndexTotal
): The number of indexing operations.ip
(ori
): The IP address.jdk
(orj
): The Java version. For example:1.8.0
.load_1m
(orl
): The most recent load average. For example:0.22
.load_5m
(orl
): The load average for the last five minutes. For example:0.78
.load_15m
(orl
): The load average for the last fifteen minutes. For example:1.24
.mappings.total_count
(ormtc
,mappingsTotalCount
): The number of mappings, including runtime and object fields.mappings.total_estimated_overhead_in_bytes
(ormteo
,mappingsTotalEstimatedOverheadInBytes
): The estimated heap overhead, in bytes, of mappings on this node, which allows for 1KiB of heap for every mapped field.master
(orm
): Indicates whether the node is the elected master node. Returned values include*
(elected master) and-
(not elected master).merges.current
(ormc
,mergesCurrent
): The number of current merge operations.merges.current_docs
(ormcd
,mergesCurrentDocs
): The number of current merging documents.merges.current_size
(ormcs
,mergesCurrentSize
): The size of current merges. For example:0b
.merges.total
(ormt
,mergesTotal
): The number of completed merge operations.merges.total_docs
(ormtd
,mergesTotalDocs
): The number of merged documents.merges.total_size
(ormts
,mergesTotalSize
): The total size of merges. For example:0b
.merges.total_time
(ormtt
,mergesTotalTime
): The time spent merging documents. For example:0s
.name
(orn
): The node name.node.role
(orr
,role
,nodeRole
): The roles of the node. Returned values includec
(cold node),d
(data node),f
(frozen node),h
(hot node),i
(ingest node),l
(machine learning node),m
(master-eligible node),r
(remote cluster client node),s
(content node),t
(transform node),v
(voting-only node),w
(warm node), and-
(coordinating node only). For example,dim
indicates a master-eligible data and ingest node.pid
(orp
): The process identifier.port
(orpo
): The bound transport port number.query_cache.memory_size
(orqcm
,queryCacheMemory
): The used query cache memory. For example:0b
.query_cache.evictions
(orqce
,queryCacheEvictions
): The number of query cache evictions.query_cache.hit_count
(orqchc
,queryCacheHitCount
): The query cache hit count.query_cache.miss_count
(orqcmc
,queryCacheMissCount
): The query cache miss count.ram.current
(orrc
,ramCurrent
): The used total memory. For example:513.4mb
.ram.max
(orrm
,ramMax
): The total memory. For example:2.9gb
.ram.percent
(orrp
,ramPercent
): The used percentage of the total operating system memory. This reflects all processes running on the operating system instead of only Elasticsearch and is not guaranteed to correlate to its performance.refresh.total
(orrto
,refreshTotal
): The number of refresh operations.refresh.time
(orrti
,refreshTime
): The time spent in refresh operations. For example:91ms
.request_cache.memory_size
(orrcm
,requestCacheMemory
): The used request cache memory. For example:0b
.request_cache.evictions
(orrce
,requestCacheEvictions
): The number of request cache evictions.request_cache.hit_count
(orrchc
,requestCacheHitCount
): The request cache hit count.request_cache.miss_count
(orrcmc
,requestCacheMissCount
): The request cache miss count.script.compilations
(orscrcc
,scriptCompilations
): The number of total script compilations.script.cache_evictions
(orscrce
,scriptCacheEvictions
): The number of total compiled scripts evicted from cache.search.fetch_current
(orsfc
,searchFetchCurrent
): The number of current fetch phase operations.search.fetch_time
(orsfti
,searchFetchTime
): The time spent in fetch phase. For example:37ms
.search.fetch_total
(orsfto
,searchFetchTotal
): The number of fetch operations.search.open_contexts
(orso
,searchOpenContexts
): The number of open search contexts.search.query_current
(orsqc
,searchQueryCurrent
): The number of current query phase operations.search.query_time
(orsqti
,searchQueryTime
): The time spent in query phase. For example:43ms
.search.query_total
(orsqto
,searchQueryTotal
): The number of query operations.search.scroll_current
(orscc
,searchScrollCurrent
): The number of open scroll contexts.search.scroll_time
(orscti
,searchScrollTime
): The amount of time scroll contexts were held open. For example:2m
.search.scroll_total
(orscto
,searchScrollTotal
): The number of completed scroll contexts.segments.count
(orsc
,segmentsCount
): The number of segments.segments.fixed_bitset_memory
(orsfbm
,fixedBitsetMemory
): The memory used by fixed bit sets for nested object field types and type filters for types referred in join fields. For example:1.0kb
.segments.index_writer_memory
(orsiwm
,segmentsIndexWriterMemory
): The memory used by the index writer. For example:18mb
.segments.memory
(orsm
,segmentsMemory
): The memory used by segments. For example:1.4kb
.segments.version_map_memory
(orsvmm
,segmentsVersionMapMemory
): The memory used by the version map. For example:1.0kb
.shard_stats.total_count
(orsstc
,shards
,shardStatsTotalCount
): The number of shards assigned.suggest.current
(orsuc
,suggestCurrent
): The number of current suggest operations.suggest.time
(orsuti
,suggestTime
): The time spent in suggest operations.suggest.total
(orsuto
,suggestTotal
): The number of suggest operations.uptime
(oru
): The amount of node uptime. For example:17.3m
.version
(orv
): The Elasticsearch version. For example:9.0.0
.
-
s
string | array[string] A comma-separated list of column names or aliases that determines the sort order. Sorting defaults to ascending and can be changed by setting
:asc
or:desc
as a suffix to the column name. -
master_timeout
string The period to wait for a connection to the master node.
Values are
-1
or0
. -
time
string The unit used to display time values.
Values are
nanos
,micros
,ms
,s
,m
,h
, ord
.
GET /_cat/nodes?v=true&format=json
curl \
--request GET 'http://api.example.com/_cat/nodes' \
--header "Authorization: $API_KEY"
[
{
"ip": "127.0.0.1",
"heap.percent": "65",
"ram.percent": "99",
"cpu": "42",
"load_1m": "3.07",
"load_5m": null,
"load_15m": null,
"node.role": "cdfhilmrstw",
"master": "*",
"name": "mJw06l1"
}
]
[
{
"id": "veJR",
"ip": "127.0.0.1",
"port": "59938",
"v": "9.0.0",
"m": "*"
}
]
Get segment information
Get low-level information about the Lucene segments in index shards. For data streams, the API returns information about the backing indices. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the index segments API.
Path parameters
-
index
string | array[string] Required A comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (
*
). To target all data streams and indices, omit this parameter or use*
or_all
.
Query parameters
-
bytes
string The unit used to display byte values.
Values are
b
,kb
,mb
,gb
,tb
, orpb
. -
h
string | array[string] List of columns to appear in the response. Supports simple wildcards.
-
s
string | array[string] List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting
:asc
or:desc
as a suffix to the column name. -
local
boolean If
true
, the request computes the list of selected nodes from the local cluster state. Iffalse
the list of selected nodes are computed from the cluster state of the master node. In both cases the coordinating node will send requests for further information to each selected node. -
master_timeout
string Period to wait for a connection to the master node.
Values are
-1
or0
.
GET /_cat/segments?v=true&format=json
curl \
--request GET 'http://api.example.com/_cat/segments/{index}' \
--header "Authorization: $API_KEY"
[
{
"index": "test",
"shard": "0",
"prirep": "p",
"ip": "127.0.0.1",
"segment": "_0",
"generation": "0",
"docs.count": "1",
"docs.deleted": "0",
"size": "3kb",
"size.memory": "0",
"committed": "false",
"searchable": "true",
"version": "9.12.0",
"compound": "true"
},
{
"index": "test1",
"shard": "0",
"prirep": "p",
"ip": "127.0.0.1",
"segment": "_0",
"generation": "0",
"docs.count": "1",
"docs.deleted": "0",
"size": "3kb",
"size.memory": "0",
"committed": "false",
"searchable": "true",
"version": "9.12.0",
"compound": "true"
}
]
Get snapshot information
Added in 2.1.0
Get information about the snapshots stored in one or more repositories. A snapshot is a backup of an index or running Elasticsearch cluster. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the get snapshot API.
Path parameters
-
repository
string | array[string] Required A comma-separated list of snapshot repositories used to limit the request. Accepts wildcard expressions.
_all
returns all repositories. If any repository fails during the request, Elasticsearch returns an error.
Query parameters
-
h
string | array[string] List of columns to appear in the response. Supports simple wildcards.
-
s
string | array[string] List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting
:asc
or:desc
as a suffix to the column name. -
master_timeout
string Period to wait for a connection to the master node.
Values are
-1
or0
. -
time
string Unit used to display time values.
Values are
nanos
,micros
,ms
,s
,m
,h
, ord
.
GET /_cat/snapshots/repo1?v=true&s=id&format=json
curl \
--request GET 'http://api.example.com/_cat/snapshots/{repository}' \
--header "Authorization: $API_KEY"
[
{
"id": "snap1",
"repository": "repo1",
"status": "FAILED",
"start_epoch": "1445616705",
"start_time": "18:11:45",
"end_epoch": "1445616978",
"end_time": "18:16:18",
"duration": "4.6m",
"indices": "1",
"successful_shards": "4",
"failed_shards": "1",
"total_shards": "5"
},
{
"id": "snap2",
"repository": "repo1",
"status": "SUCCESS",
"start_epoch": "1445634298",
"start_time": "23:04:58",
"end_epoch": "1445634672",
"end_time": "23:11:12",
"duration": "6.2m",
"indices": "2",
"successful_shards": "10",
"failed_shards": "0",
"total_shards": "10"
}
]
Explain the shard allocations
Added in 5.0.0
Get explanations for shard allocations in the cluster. For unassigned shards, it provides an explanation for why the shard is unassigned. For assigned shards, it provides an explanation for why the shard is remaining on its current node and has not moved or rebalanced to another node. This API can be very useful when attempting to diagnose why a shard is unassigned or why a shard continues to remain on its current node when you might expect otherwise.
Query parameters
-
include_disk_info
boolean If true, returns information about disk usage and shard sizes.
-
include_yes_decisions
boolean If true, returns YES decisions in explanation.
-
master_timeout
string Period to wait for a connection to the master node.
Values are
-1
or0
.
Body
-
current_node
string Specifies the node ID or the name of the node to only explain a shard that is currently located on the specified node.
-
index
string -
primary
boolean If true, returns explanation for the primary shard for the given shard ID.
-
shard
number Specifies the ID of the shard that you would like an explanation for.
GET _cluster/allocation/explain
{
"index": "my-index-000001",
"shard": 0,
"primary": false,
"current_node": "my-node"
}
curl \
--request GET 'http://api.example.com/_cluster/allocation/explain' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"index\": \"my-index-000001\",\n \"shard\": 0,\n \"primary\": false,\n \"current_node\": \"my-node\"\n}"'
{
"index": "my-index-000001",
"shard": 0,
"primary": false,
"current_node": "my-node"
}
{
"index" : "my-index-000001",
"shard" : 0,
"primary" : true,
"current_state" : "unassigned",
"unassigned_info" : {
"reason" : "INDEX_CREATED",
"at" : "2017-01-04T18:08:16.600Z",
"last_allocation_status" : "no"
},
"can_allocate" : "no",
"allocate_explanation" : "Elasticsearch isn't allowed to allocate this shard to any of the nodes in the cluster. Choose a node to which you expect this shard to be allocated, find this node in the node-by-node explanation, and address the reasons which prevent Elasticsearch from allocating this shard there.",
"node_allocation_decisions" : [
{
"node_id" : "8qt2rY-pT6KNZB3-hGfLnw",
"node_name" : "node-0",
"transport_address" : "127.0.0.1:9401",
"roles" : ["data", "data_cold", "data_content", "data_frozen", "data_hot", "data_warm", "ingest", "master", "ml", "remote_cluster_client", "transform"],
"node_attributes" : {},
"node_decision" : "no",
"weight_ranking" : 1,
"deciders" : [
{
"decider" : "filter",
"decision" : "NO",
"explanation" : "node does not match index setting [index.routing.allocation.include] filters [_name:\"nonexistent_node\"]"
}
]
}
]
}
{
"index" : "my-index-000001",
"shard" : 0,
"primary" : true,
"current_state" : "unassigned",
"unassigned_info" : {
"at" : "2017-01-04T18:03:28.464Z",
"failed shard on node [mEKjwwzLT1yJVb8UxT6anw]: failed recovery, failure RecoveryFailedException",
"reason": "ALLOCATION_FAILED",
"failed_allocation_attempts": 5,
"last_allocation_status": "no",
},
"can_allocate": "no",
"allocate_explanation": "cannot allocate because allocation is not permitted to any of the nodes",
"node_allocation_decisions" : [
{
"node_id" : "3sULLVJrRneSg0EfBB-2Ew",
"node_name" : "node_t0",
"transport_address" : "127.0.0.1:9400",
"roles" : ["data_content", "data_hot"],
"node_decision" : "no",
"store" : {
"matching_size" : "4.2kb",
"matching_size_in_bytes" : 4325
},
"deciders" : [
{
"decider": "max_retry",
"decision" : "NO",
"explanation": "shard has exceeded the maximum number of retries [5] on failed allocation attempts - manually call [POST /_cluster/reroute?retry_failed] to retry, [unassigned_info[[reason=ALLOCATION_FAILED], at[2024-07-30T21:04:12.166Z], failed_attempts[5], failed_nodes[[mEKjwwzLT1yJVb8UxT6anw]], delayed=false, details[failed shard on node [mEKjwwzLT1yJVb8UxT6anw]: failed recovery, failure RecoveryFailedException], allocation_status[deciders_no]]]"
}
]
}
]
}
Get remote cluster information
Added in 6.1.0
Get information about configured remote clusters. The API returns connection and endpoint information keyed by the configured remote cluster alias.
This API returns information that reflects current state on the local cluster.
The connected
field does not necessarily reflect whether a remote cluster is down or unavailable, only whether there is currently an open connection to it.
Elasticsearch does not spontaneously try to reconnect to a disconnected remote cluster.
To trigger a reconnection, attempt a cross-cluster search, ES|QL cross-cluster search, or try the resolve cluster endpoint.
curl \
--request GET 'http://api.example.com/_remote/info' \
--header "Authorization: $API_KEY"
Reroute the cluster
Added in 5.0.0
Manually change the allocation of individual shards in the cluster. For example, a shard can be moved from one node to another explicitly, an allocation can be canceled, and an unassigned shard can be explicitly allocated to a specific node.
It is important to note that after processing any reroute commands Elasticsearch will perform rebalancing as normal (respecting the values of settings such as cluster.routing.rebalance.enable
) in order to remain in a balanced state.
For example, if the requested allocation includes moving a shard from node1 to node2 then this may cause a shard to be moved from node2 back to node1 to even things out.
The cluster can be set to disable allocations using the cluster.routing.allocation.enable
setting.
If allocations are disabled then the only allocations that will be performed are explicit ones given using the reroute command, and consequent allocations due to rebalancing.
The cluster will attempt to allocate a shard a maximum of index.allocation.max_retries
times in a row (defaults to 5
), before giving up and leaving the shard unallocated.
This scenario can be caused by structural problems such as having an analyzer which refers to a stopwords file which doesn’t exist on all nodes.
Once the problem has been corrected, allocation can be manually retried by calling the reroute API with the ?retry_failed
URI query parameter, which will attempt a single retry round for these shards.
Query parameters
-
dry_run
boolean If true, then the request simulates the operation. It will calculate the result of applying the commands to the current cluster state and return the resulting cluster state after the commands (and rebalancing) have been applied; it will not actually perform the requested changes.
-
explain
boolean If true, then the response contains an explanation of why the commands can or cannot run.
-
metric
string | array[string] Limits the information returned to the specified metrics.
-
retry_failed
boolean If true, then retries allocation of shards that are blocked due to too many subsequent allocation failures.
-
master_timeout
string Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1
or0
. -
timeout
string Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1
or0
.
POST /_cluster/reroute?metric=none
{
"commands": [
{
"move": {
"index": "test", "shard": 0,
"from_node": "node1", "to_node": "node2"
}
},
{
"allocate_replica": {
"index": "test", "shard": 1,
"node": "node3"
}
}
]
}
curl \
--request POST 'http://api.example.com/_cluster/reroute' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"commands\": [\n {\n \"move\": {\n \"index\": \"test\", \"shard\": 0,\n \"from_node\": \"node1\", \"to_node\": \"node2\"\n }\n },\n {\n \"allocate_replica\": {\n \"index\": \"test\", \"shard\": 1,\n \"node\": \"node3\"\n }\n }\n ]\n}"'
{
"commands": [
{
"move": {
"index": "test", "shard": 0,
"from_node": "node1", "to_node": "node2"
}
},
{
"allocate_replica": {
"index": "test", "shard": 1,
"node": "node3"
}
}
]
}
Get the cluster state
Added in 1.3.0
Get comprehensive information about the state of the cluster.
The cluster state is an internal data structure which keeps track of a variety of information needed by every node, including the identity and attributes of the other nodes in the cluster; cluster-wide settings; index metadata, including the mapping and settings for each index; the location and status of every shard copy in the cluster.
The elected master node ensures that every node in the cluster has a copy of the same cluster state. This API lets you retrieve a representation of this internal state for debugging or diagnostic purposes. You may need to consult the Elasticsearch source code to determine the precise meaning of the response.
By default the API will route requests to the elected master node since this node is the authoritative source of cluster states.
You can also retrieve the cluster state held on the node handling the API request by adding the ?local=true
query parameter.
Elasticsearch may need to expend significant effort to compute a response to this API in larger clusters, and the response may comprise a very large quantity of data. If you use this API repeatedly, your cluster may become unstable.
WARNING: The response is a representation of an internal data structure. Its format is not subject to the same compatibility guarantees as other more stable APIs and may change from version to version. Do not query this API using external monitoring tools. Instead, obtain the information you require using other more stable cluster APIs.
Path parameters
-
metric
string | array[string] Required Limit the information returned to the specified metrics
Query parameters
-
allow_no_indices
boolean Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes
_all
string or when no indices have been specified) -
expand_wildcards
string | array[string] Whether to expand wildcard expression to concrete indices that are open, closed or both.
Supported values include:
all
: Match any data stream or index, including hidden ones.open
: Match open, non-hidden indices. Also matches any non-hidden data stream.closed
: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden
: Match hidden data streams and hidden indices. Must be combined withopen
,closed
, orboth
.none
: Wildcard expressions are not accepted.
Values are
all
,open
,closed
,hidden
, ornone
. -
flat_settings
boolean Return settings in flat format (default: false)
-
local
boolean Return local information, do not retrieve the state from master node (default: false)
-
master_timeout
string Specify timeout for connection to master
Values are
-1
or0
. -
wait_for_metadata_version
number Wait for the metadata version to be equal or greater than the specified metadata version
-
wait_for_timeout
string The maximum time to wait for wait_for_metadata_version before timing out
Values are
-1
or0
.
curl \
--request GET 'http://api.example.com/_cluster/state/{metric}' \
--header "Authorization: $API_KEY"
Get the cluster state
Added in 1.3.0
Get comprehensive information about the state of the cluster.
The cluster state is an internal data structure which keeps track of a variety of information needed by every node, including the identity and attributes of the other nodes in the cluster; cluster-wide settings; index metadata, including the mapping and settings for each index; the location and status of every shard copy in the cluster.
The elected master node ensures that every node in the cluster has a copy of the same cluster state. This API lets you retrieve a representation of this internal state for debugging or diagnostic purposes. You may need to consult the Elasticsearch source code to determine the precise meaning of the response.
By default the API will route requests to the elected master node since this node is the authoritative source of cluster states.
You can also retrieve the cluster state held on the node handling the API request by adding the ?local=true
query parameter.
Elasticsearch may need to expend significant effort to compute a response to this API in larger clusters, and the response may comprise a very large quantity of data. If you use this API repeatedly, your cluster may become unstable.
WARNING: The response is a representation of an internal data structure. Its format is not subject to the same compatibility guarantees as other more stable APIs and may change from version to version. Do not query this API using external monitoring tools. Instead, obtain the information you require using other more stable cluster APIs.
Query parameters
-
allow_no_indices
boolean Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes
_all
string or when no indices have been specified) -
expand_wildcards
string | array[string] Whether to expand wildcard expression to concrete indices that are open, closed or both.
Supported values include:
all
: Match any data stream or index, including hidden ones.open
: Match open, non-hidden indices. Also matches any non-hidden data stream.closed
: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden
: Match hidden data streams and hidden indices. Must be combined withopen
,closed
, orboth
.none
: Wildcard expressions are not accepted.
Values are
all
,open
,closed
,hidden
, ornone
. -
flat_settings
boolean Return settings in flat format (default: false)
-
local
boolean Return local information, do not retrieve the state from master node (default: false)
-
master_timeout
string Specify timeout for connection to master
Values are
-1
or0
. -
wait_for_metadata_version
number Wait for the metadata version to be equal or greater than the specified metadata version
-
wait_for_timeout
string The maximum time to wait for wait_for_metadata_version before timing out
Values are
-1
or0
.
curl \
--request GET 'http://api.example.com/_cluster/state/{metric}/{index}' \
--header "Authorization: $API_KEY"
Activate the connector draft filter
Technical preview
Activates the valid draft filtering for a connector.
Path parameters
-
connector_id
string Required The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_filtering/_activate' \
--header "Authorization: $API_KEY"
Update the connector draft filtering validation
Technical preview
Update the draft filtering validation info for a connector.
Path parameters
-
connector_id
string Required The unique identifier of the connector to be updated
Body
Required
-
validation
object Required
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_filtering/_validation' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"validation":{"errors":[{"ids":["string"],"messages":["string"]}],"state":"edited"}}'
Update the connector is_native flag
Beta
Path parameters
-
connector_id
string Required The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_native' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"is_native":true}'
Downsample an index
Technical preview
Aggregate a time series (TSDS) index and store pre-computed statistical summaries (min
, max
, sum
, value_count
and avg
) for each metric field grouped by a configured time interval.
For example, a TSDS index that contains metrics sampled every 10 seconds can be downsampled to an hourly index.
All documents within an hour interval are summarized and stored as a single document in the downsample index.
NOTE: Only indices in a time series data stream are supported.
Neither field nor document level security can be defined on the source index.
The source index must be read only (index.blocks.write: true
).
Path parameters
-
index
string Required Name of the time series index to downsample.
-
target_index
string Required Name of the index to create.
Body
Required
-
fixed_interval
string Required A date histogram interval. Similar to
Duration
with additional units:w
(week),M
(month),q
(quarter) andy
(year)
POST /my-time-series-index/_downsample/my-downsampled-time-series-index
{
"fixed_interval": "1d"
}
curl \
--request POST 'http://api.example.com/{index}/_downsample/{target_index}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"fixed_interval\": \"1d\"\n}"'
{
"fixed_interval": "1d"
}
Create a new document in the index
Added in 5.0.0
You can index a new JSON document with the /<target>/_doc/
or /<target>/_create/<_id>
APIs
Using _create
guarantees that the document is indexed only if it does not already exist.
It returns a 409 response when a document with a same ID already exists in the index.
To update an existing document, you must use the /<target>/_doc/
API.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:
- To add a document using the
PUT /<target>/_create/<_id>
orPOST /<target>/_create/<_id>
request formats, you must have thecreate_doc
,create
,index
, orwrite
index privilege. - To automatically create a data stream or index with this API request, you must have the
auto_configure
,create_index
, ormanage
index privilege.
Automatic data stream creation requires a matching index template with data stream enabled.
Automatically create data streams and indices
If the request's target doesn't exist and matches an index template with a data_stream
definition, the index operation automatically creates the data stream.
If the target doesn't exist and doesn't match a data stream template, the operation automatically creates the index and applies any matching index templates.
NOTE: Elasticsearch includes several built-in index templates. To avoid naming collisions with these templates, refer to index pattern documentation.
If no mapping exists, the index operation creates a dynamic mapping. By default, new fields and objects are automatically added to the mapping if needed.
Automatic index creation is controlled by the action.auto_create_index
setting.
If it is true
, any index can be created automatically.
You can modify this setting to explicitly allow or block automatic creation of indices that match specified patterns or set it to false
to turn off automatic index creation entirely.
Specify a comma-separated list of patterns you want to allow or prefix each pattern with +
or -
to indicate whether it should be allowed or blocked.
When a list is specified, the default behaviour is to disallow.
NOTE: The action.auto_create_index
setting affects the automatic creation of indices only.
It does not affect the creation of data streams.
Routing
By default, shard placement — or routing — is controlled by using a hash of the document's ID value.
For more explicit control, the value fed into the hash function used by the router can be directly specified on a per-operation basis using the routing
parameter.
When setting up explicit mapping, you can also use the _routing
field to direct the index operation to extract the routing value from the document itself.
This does come at the (very minimal) cost of an additional document parsing pass.
If the _routing
mapping is defined and set to be required, the index operation will fail if no routing value is provided or extracted.
NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing
setting enabled in the template.
Distributed
The index operation is directed to the primary shard based on its route and performed on the actual node containing this shard. After the primary shard completes the operation, if needed, the update is distributed to applicable replicas.
Active shards
To improve the resiliency of writes to the system, indexing operations can be configured to wait for a certain number of active shard copies before proceeding with the operation.
If the requisite number of active shard copies are not available, then the write operation must wait and retry, until either the requisite shard copies have started or a timeout occurs.
By default, write operations only wait for the primary shards to be active before proceeding (that is to say wait_for_active_shards
is 1
).
This default can be overridden in the index settings dynamically by setting index.write.wait_for_active_shards
.
To alter this behavior per operation, use the wait_for_active_shards request
parameter.
Valid values are all or any positive integer up to the total number of configured copies per shard in the index (which is number_of_replicas
+1).
Specifying a negative value or a number greater than the number of shard copies will throw an error.
For example, suppose you have a cluster of three nodes, A, B, and C and you create an index index with the number of replicas set to 3 (resulting in 4 shard copies, one more copy than there are nodes).
If you attempt an indexing operation, by default the operation will only ensure the primary copy of each shard is available before proceeding.
This means that even if B and C went down and A hosted the primary shard copies, the indexing operation would still proceed with only one copy of the data.
If wait_for_active_shards
is set on the request to 3
(and all three nodes are up), the indexing operation will require 3 active shard copies before proceeding.
This requirement should be met because there are 3 active nodes in the cluster, each one holding a copy of the shard.
However, if you set wait_for_active_shards
to all
(or to 4
, which is the same in this situation), the indexing operation will not proceed as you do not have all 4 copies of each shard active in the index.
The operation will timeout unless a new node is brought up in the cluster to host the fourth copy of the shard.
It is important to note that this setting greatly reduces the chances of the write operation not writing to the requisite number of shard copies, but it does not completely eliminate the possibility, because this check occurs before the write operation starts.
After the write operation is underway, it is still possible for replication to fail on any number of shard copies but still succeed on the primary.
The _shards
section of the API response reveals the number of shard copies on which replication succeeded and failed.
Path parameters
-
index
string Required The name of the data stream or index to target. If the target doesn't exist and matches the name or wildcard (
*
) pattern of an index template with adata_stream
definition, this request creates the data stream. If the target doesn't exist and doesn’t match a data stream template, this request creates the index. -
id
string Required A unique identifier for the document. To automatically generate a document ID, use the
POST /<target>/_doc/
request format.
Query parameters
-
if_primary_term
number Only perform the operation if the document has this primary term.
-
if_seq_no
number Only perform the operation if the document has this sequence number.
-
include_source_on_error
boolean True or false if to include the document source in the error message in case of parsing errors.
-
op_type
string Set to
create
to only index the document if it does not already exist (put if absent). If a document with the specified_id
already exists, the indexing operation will fail. The behavior is the same as using the<index>/_create
endpoint. If a document ID is specified, this paramater defaults toindex
. Otherwise, it defaults tocreate
. If the request targets a data stream, anop_type
ofcreate
is required.Supported values include:
index
: Overwrite any documents that already exist.create
: Only index documents that do not already exist.
Values are
index
orcreate
. -
pipeline
string The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, setting the value to
_none
turns off the default ingest pipeline for this request. If a final pipeline is configured, it will always run regardless of the value of this parameter. -
refresh
string If
true
, Elasticsearch refreshes the affected shards to make this operation visible to search. Ifwait_for
, it waits for a refresh to make this operation visible to search. Iffalse
, it does nothing with refreshes.Values are
true
,false
, orwait_for
. -
require_alias
boolean If
true
, the destination must be an index alias. -
require_data_stream
boolean If
true
, the request's actions must target a data stream (existing or to be created). -
routing
string A custom value that is used to route operations to a specific shard.
-
timeout
string The period the request waits for the following operations: automatic index creation, dynamic mapping updates, waiting for active shards. Elasticsearch waits for at least the specified timeout period before failing. The actual wait time could be longer, particularly when multiple waits occur.
This parameter is useful for situations where the primary shard assigned to perform the operation might not be available when the operation runs. Some reasons for this might be that the primary shard is currently recovering from a gateway or undergoing relocation. By default, the operation will wait on the primary shard to become available for at least 1 minute before failing and responding with an error. The actual wait time could be longer, particularly when multiple waits occur.
Values are
-1
or0
. -
version
number The explicit version number for concurrency control. It must be a non-negative long number.
-
version_type
string The version type.
Supported values include:
internal
: Use internal versioning that starts at 1 and increments with each update or delete.external
: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte
: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document. NOTE: Theexternal_gte
version type is meant for special use cases and should be used with care. If used incorrectly, it can result in loss of data.force
: This option is deprecated because it can cause primary and replica shards to diverge.
Values are
internal
,external
,external_gte
, orforce
. -
wait_for_active_shards
number | string The number of shard copies that must be active before proceeding with the operation. You can set it to
all
or any positive integer up to the total number of shards in the index (number_of_replicas+1
). The default value of1
means it waits for each primary shard to be active.Values are
all
orindex-setting
.
PUT my-index-000001/_create/1
{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}
curl \
--request PUT 'http://api.example.com/{index}/_create/{id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"@timestamp\": \"2099-11-15T13:12:00\",\n \"message\": \"GET /search HTTP/1.1 200 1070000\",\n \"user\": {\n \"id\": \"kimchy\"\n }\n}"'
{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}
Create a new document in the index
Added in 5.0.0
You can index a new JSON document with the /<target>/_doc/
or /<target>/_create/<_id>
APIs
Using _create
guarantees that the document is indexed only if it does not already exist.
It returns a 409 response when a document with a same ID already exists in the index.
To update an existing document, you must use the /<target>/_doc/
API.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:
- To add a document using the
PUT /<target>/_create/<_id>
orPOST /<target>/_create/<_id>
request formats, you must have thecreate_doc
,create
,index
, orwrite
index privilege. - To automatically create a data stream or index with this API request, you must have the
auto_configure
,create_index
, ormanage
index privilege.
Automatic data stream creation requires a matching index template with data stream enabled.
Automatically create data streams and indices
If the request's target doesn't exist and matches an index template with a data_stream
definition, the index operation automatically creates the data stream.
If the target doesn't exist and doesn't match a data stream template, the operation automatically creates the index and applies any matching index templates.
NOTE: Elasticsearch includes several built-in index templates. To avoid naming collisions with these templates, refer to index pattern documentation.
If no mapping exists, the index operation creates a dynamic mapping. By default, new fields and objects are automatically added to the mapping if needed.
Automatic index creation is controlled by the action.auto_create_index
setting.
If it is true
, any index can be created automatically.
You can modify this setting to explicitly allow or block automatic creation of indices that match specified patterns or set it to false
to turn off automatic index creation entirely.
Specify a comma-separated list of patterns you want to allow or prefix each pattern with +
or -
to indicate whether it should be allowed or blocked.
When a list is specified, the default behaviour is to disallow.
NOTE: The action.auto_create_index
setting affects the automatic creation of indices only.
It does not affect the creation of data streams.
Routing
By default, shard placement — or routing — is controlled by using a hash of the document's ID value.
For more explicit control, the value fed into the hash function used by the router can be directly specified on a per-operation basis using the routing
parameter.
When setting up explicit mapping, you can also use the _routing
field to direct the index operation to extract the routing value from the document itself.
This does come at the (very minimal) cost of an additional document parsing pass.
If the _routing
mapping is defined and set to be required, the index operation will fail if no routing value is provided or extracted.
NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing
setting enabled in the template.
Distributed
The index operation is directed to the primary shard based on its route and performed on the actual node containing this shard. After the primary shard completes the operation, if needed, the update is distributed to applicable replicas.
Active shards
To improve the resiliency of writes to the system, indexing operations can be configured to wait for a certain number of active shard copies before proceeding with the operation.
If the requisite number of active shard copies are not available, then the write operation must wait and retry, until either the requisite shard copies have started or a timeout occurs.
By default, write operations only wait for the primary shards to be active before proceeding (that is to say wait_for_active_shards
is 1
).
This default can be overridden in the index settings dynamically by setting index.write.wait_for_active_shards
.
To alter this behavior per operation, use the wait_for_active_shards request
parameter.
Valid values are all or any positive integer up to the total number of configured copies per shard in the index (which is number_of_replicas
+1).
Specifying a negative value or a number greater than the number of shard copies will throw an error.
For example, suppose you have a cluster of three nodes, A, B, and C and you create an index index with the number of replicas set to 3 (resulting in 4 shard copies, one more copy than there are nodes).
If you attempt an indexing operation, by default the operation will only ensure the primary copy of each shard is available before proceeding.
This means that even if B and C went down and A hosted the primary shard copies, the indexing operation would still proceed with only one copy of the data.
If wait_for_active_shards
is set on the request to 3
(and all three nodes are up), the indexing operation will require 3 active shard copies before proceeding.
This requirement should be met because there are 3 active nodes in the cluster, each one holding a copy of the shard.
However, if you set wait_for_active_shards
to all
(or to 4
, which is the same in this situation), the indexing operation will not proceed as you do not have all 4 copies of each shard active in the index.
The operation will timeout unless a new node is brought up in the cluster to host the fourth copy of the shard.
It is important to note that this setting greatly reduces the chances of the write operation not writing to the requisite number of shard copies, but it does not completely eliminate the possibility, because this check occurs before the write operation starts.
After the write operation is underway, it is still possible for replication to fail on any number of shard copies but still succeed on the primary.
The _shards
section of the API response reveals the number of shard copies on which replication succeeded and failed.
Path parameters
-
index
string Required The name of the data stream or index to target. If the target doesn't exist and matches the name or wildcard (
*
) pattern of an index template with adata_stream
definition, this request creates the data stream. If the target doesn't exist and doesn’t match a data stream template, this request creates the index. -
id
string Required A unique identifier for the document. To automatically generate a document ID, use the
POST /<target>/_doc/
request format.
Query parameters
-
if_primary_term
number Only perform the operation if the document has this primary term.
-
if_seq_no
number Only perform the operation if the document has this sequence number.
-
include_source_on_error
boolean True or false if to include the document source in the error message in case of parsing errors.
-
op_type
string Set to
create
to only index the document if it does not already exist (put if absent). If a document with the specified_id
already exists, the indexing operation will fail. The behavior is the same as using the<index>/_create
endpoint. If a document ID is specified, this paramater defaults toindex
. Otherwise, it defaults tocreate
. If the request targets a data stream, anop_type
ofcreate
is required.Supported values include:
index
: Overwrite any documents that already exist.create
: Only index documents that do not already exist.
Values are
index
orcreate
. -
pipeline
string The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, setting the value to
_none
turns off the default ingest pipeline for this request. If a final pipeline is configured, it will always run regardless of the value of this parameter. -
refresh
string If
true
, Elasticsearch refreshes the affected shards to make this operation visible to search. Ifwait_for
, it waits for a refresh to make this operation visible to search. Iffalse
, it does nothing with refreshes.Values are
true
,false
, orwait_for
. -
require_alias
boolean If
true
, the destination must be an index alias. -
require_data_stream
boolean If
true
, the request's actions must target a data stream (existing or to be created). -
routing
string A custom value that is used to route operations to a specific shard.
-
timeout
string The period the request waits for the following operations: automatic index creation, dynamic mapping updates, waiting for active shards. Elasticsearch waits for at least the specified timeout period before failing. The actual wait time could be longer, particularly when multiple waits occur.
This parameter is useful for situations where the primary shard assigned to perform the operation might not be available when the operation runs. Some reasons for this might be that the primary shard is currently recovering from a gateway or undergoing relocation. By default, the operation will wait on the primary shard to become available for at least 1 minute before failing and responding with an error. The actual wait time could be longer, particularly when multiple waits occur.
Values are
-1
or0
. -
version
number The explicit version number for concurrency control. It must be a non-negative long number.
-
version_type
string The version type.
Supported values include:
internal
: Use internal versioning that starts at 1 and increments with each update or delete.external
: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte
: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document. NOTE: Theexternal_gte
version type is meant for special use cases and should be used with care. If used incorrectly, it can result in loss of data.force
: This option is deprecated because it can cause primary and replica shards to diverge.
Values are
internal
,external
,external_gte
, orforce
. -
wait_for_active_shards
number | string The number of shard copies that must be active before proceeding with the operation. You can set it to
all
or any positive integer up to the total number of shards in the index (number_of_replicas+1
). The default value of1
means it waits for each primary shard to be active.Values are
all
orindex-setting
.
PUT my-index-000001/_create/1
{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}
curl \
--request POST 'http://api.example.com/{index}/_create/{id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"@timestamp\": \"2099-11-15T13:12:00\",\n \"message\": \"GET /search HTTP/1.1 200 1070000\",\n \"user\": {\n \"id\": \"kimchy\"\n }\n}"'
{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}
Delete a document
Remove a JSON document from the specified index.
NOTE: You cannot send deletion requests directly to a data stream. To delete a document in a data stream, you must target the backing index containing the document.
Optimistic concurrency control
Delete operations can be made conditional and only be performed if the last modification to the document was assigned the sequence number and primary term specified by the if_seq_no
and if_primary_term
parameters.
If a mismatch is detected, the operation will result in a VersionConflictException
and a status code of 409
.
Versioning
Each document indexed is versioned.
When deleting a document, the version can be specified to make sure the relevant document you are trying to delete is actually being deleted and it has not changed in the meantime.
Every write operation run on a document, deletes included, causes its version to be incremented.
The version number of a deleted document remains available for a short time after deletion to allow for control of concurrent operations.
The length of time for which a deleted document's version remains available is determined by the index.gc_deletes
index setting.
Routing
If routing is used during indexing, the routing value also needs to be specified to delete a document.
If the _routing
mapping is set to required
and no routing value is specified, the delete API throws a RoutingMissingException
and rejects the request.
For example:
DELETE /my-index-000001/_doc/1?routing=shard-1
This request deletes the document with ID 1, but it is routed based on the user. The document is not deleted if the correct routing is not specified.
Distributed
The delete operation gets hashed into a specific shard ID. It then gets redirected into the primary shard within that ID group and replicated (if needed) to shard replicas within that ID group.
Query parameters
-
if_primary_term
number Only perform the operation if the document has this primary term.
-
if_seq_no
number Only perform the operation if the document has this sequence number.
-
refresh
string If
true
, Elasticsearch refreshes the affected shards to make this operation visible to search. Ifwait_for
, it waits for a refresh to make this operation visible to search. Iffalse
, it does nothing with refreshes.Values are
true
,false
, orwait_for
. -
routing
string A custom value used to route operations to a specific shard.
-
timeout
string The period to wait for active shards.
This parameter is useful for situations where the primary shard assigned to perform the delete operation might not be available when the delete operation runs. Some reasons for this might be that the primary shard is currently recovering from a store or undergoing relocation. By default, the delete operation will wait on the primary shard to become available for up to 1 minute before failing and responding with an error.
Values are
-1
or0
. -
version
number An explicit version number for concurrency control. It must match the current version of the document for the request to succeed.
-
version_type
string The version type.
Supported values include:
internal
: Use internal versioning that starts at 1 and increments with each update or delete.external
: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte
: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document. NOTE: Theexternal_gte
version type is meant for special use cases and should be used with care. If used incorrectly, it can result in loss of data.force
: This option is deprecated because it can cause primary and replica shards to diverge.
Values are
internal
,external
,external_gte
, orforce
. -
wait_for_active_shards
number | string The minimum number of shard copies that must be active before proceeding with the operation. You can set it to
all
or any positive integer up to the total number of shards in the index (number_of_replicas+1
). The default value of1
means it waits for each primary shard to be active.Values are
all
orindex-setting
.
DELETE /my-index-000001/_doc/1
curl \
--request DELETE 'http://api.example.com/{index}/_doc/{id}' \
--header "Authorization: $API_KEY"
{
"_shards": {
"total": 2,
"failed": 0,
"successful": 2
},
"_index": "my-index-000001",
"_id": "1",
"_version": 2,
"_primary_term": 1,
"_seq_no": 5,
"result": "deleted"
}
Get the async EQL status
Added in 7.9.0
Get the current status for an async EQL search or a stored synchronous EQL search without returning results.
Path parameters
-
id
string Required Identifier for the search.
curl \
--request GET 'http://api.example.com/_eql/search/status/{id}' \
--header "Authorization: $API_KEY"
{
"id": "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
"is_running" : true,
"is_partial" : true,
"start_time_in_millis" : 1611690235000,
"expiration_time_in_millis" : 1611690295000
}
Get the features
Added in 7.12.0
Get a list of features that can be included in snapshots using the feature_states
field when creating a snapshot.
You can use this API to determine which feature states to include when taking a snapshot.
By default, all feature states are included in a snapshot if that snapshot includes the global state, or none if it does not.
A feature state includes one or more system indices necessary for a given feature to function. In order to ensure data integrity, all system indices that comprise a feature state are snapshotted and restored together.
The features listed by this API are a combination of built-in features and features defined by plugins. In order for a feature state to be listed in this API and recognized as a valid feature state by the create snapshot API, the plugin that defines that feature must be installed on the master node.
Query parameters
-
master_timeout
string Period to wait for a connection to the master node.
Values are
-1
or0
.
curl \
--request GET 'http://api.example.com/_features' \
--header "Authorization: $API_KEY"
{
"features": [
{
"name": "tasks",
"description": "Manages task results"
},
{
"name": "kibana",
"description": "Manages Kibana configuration and reports"
}
]
}
Run multiple Fleet searches
Technical preview
Run several Fleet searches with a single API request.
The API follows the same structure as the multi search API.
However, similar to the Fleet search API, it supports the wait_for_checkpoints
parameter.
Query parameters
-
allow_no_indices
boolean If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.
-
ccs_minimize_roundtrips
boolean If true, network roundtrips between the coordinating node and remote clusters are minimized for cross-cluster search requests.
-
expand_wildcards
string | array[string] Type of index that wildcard expressions can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supported values include:
all
: Match any data stream or index, including hidden ones.open
: Match open, non-hidden indices. Also matches any non-hidden data stream.closed
: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden
: Match hidden data streams and hidden indices. Must be combined withopen
,closed
, orboth
.none
: Wildcard expressions are not accepted.
Values are
all
,open
,closed
,hidden
, ornone
. -
ignore_throttled
boolean If true, concrete, expanded or aliased indices are ignored when frozen.
-
max_concurrent_searches
number Maximum number of concurrent searches the multi search API can execute.
-
Maximum number of concurrent shard requests that each sub-search request executes per node.
-
pre_filter_shard_size
number Defines a threshold that enforces a pre-filter roundtrip to prefilter search shards based on query rewriting if the number of shards the search request expands to exceeds the threshold. This filter roundtrip can limit the number of shards significantly if for instance a shard can not match any documents based on its rewrite method i.e., if date filters are mandatory to match but the shard bounds and the query are disjoint.
-
search_type
string Indicates whether global term and document frequencies should be used when scoring returned documents.
Supported values include:
query_then_fetch
: Documents are scored using local term and document frequencies for the shard. This is usually faster but less accurate.dfs_query_then_fetch
: Documents are scored using global term and document frequencies across all shards. This is usually slower but more accurate.
Values are
query_then_fetch
ordfs_query_then_fetch
. -
rest_total_hits_as_int
boolean If true, hits.total are returned as an integer in the response. Defaults to false, which returns an object.
-
typed_keys
boolean Specifies whether aggregation and suggester names should be prefixed by their respective types in the response.
-
wait_for_checkpoints
array[number] A comma separated list of checkpoints. When configured, the search API will only be executed on a shard after the relevant checkpoint has become visible for search. Defaults to an empty list which will cause Elasticsearch to immediately execute the search.
-
allow_partial_search_results
boolean If true, returns partial results if there are shard request timeouts or shard failures. If false, returns an error with no partial results. Defaults to the configured cluster setting
search.default_allow_partial_results
, which is true by default.
Body
object
Required
-
allow_no_indices
boolean -
expand_wildcards
string | array[string] -
index
string | array[string] -
preference
string -
request_cache
boolean -
routing
string -
search_type
string Values are
query_then_fetch
ordfs_query_then_fetch
. -
ccs_minimize_roundtrips
boolean -
allow_partial_search_results
boolean -
ignore_throttled
boolean
curl \
--request POST 'http://api.example.com/_fleet/_fleet_msearch' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '[{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"index":"string","preference":"string","request_cache":true,"routing":"string","search_type":"query_then_fetch","ccs_minimize_roundtrips":true,"allow_partial_search_results":true,"ignore_throttled":true}]'
Index
Index APIs enable you to manage individual indices, index settings, aliases, mappings, and index templates.
Create or update a component template
Added in 7.8.0
Component templates are building blocks for constructing index templates that specify index mappings, settings, and aliases.
An index template can be composed of multiple component templates.
To use a component template, specify it in an index template’s composed_of
list.
Component templates are only applied to new data streams and indices as part of a matching index template.
Settings and mappings specified directly in the index template or the create index request override any settings or mappings specified in a component template.
Component templates are only used during index creation. For data streams, this includes data stream creation and the creation of a stream’s backing indices. Changes to component templates do not affect existing indices, including a stream’s backing indices.
You can use C-style /* *\/
block comments in component templates.
You can include comments anywhere in the request body except before the opening curly bracket.
Applying component templates
You cannot directly apply a component template to a data stream or index.
To be applied, a component template must be included in an index template's composed_of
list.
Path parameters
-
name
string Required Name of the component template to create. Elasticsearch includes the following built-in component templates:
logs-mappings
;logs-settings
;metrics-mappings
;metrics-settings
;synthetics-mapping
;synthetics-settings
. Elastic Agent uses these templates to configure backing indices for its data streams. If you use Elastic Agent and want to overwrite one of these templates, set theversion
for your replacement template higher than the current version. If you don’t use Elastic Agent and want to disable all built-in component and index templates, setstack.templates.enabled
tofalse
using the cluster update settings API.
Query parameters
-
create
boolean If
true
, this request cannot replace or update existing component templates. -
master_timeout
string Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1
or0
.
Body
Required
-
template
object Required -
version
number -
_meta
object -
deprecated
boolean Marks this index template as deprecated. When creating or updating a non-deprecated index template that uses deprecated components, Elasticsearch will emit a deprecation warning.
PUT _component_template/template_1
{
"template": null,
"settings": {
"number_of_shards": 1
},
"mappings": {
"_source": {
"enabled": false
},
"properties": {
"host_name": {
"type": "keyword"
},
"created_at": {
"type": "date",
"format": "EEE MMM dd HH:mm:ss Z yyyy"
}
}
}
}
curl \
--request POST 'http://api.example.com/_component_template/{name}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"template\": null,\n \"settings\": {\n \"number_of_shards\": 1\n },\n \"mappings\": {\n \"_source\": {\n \"enabled\": false\n },\n \"properties\": {\n \"host_name\": {\n \"type\": \"keyword\"\n },\n \"created_at\": {\n \"type\": \"date\",\n \"format\": \"EEE MMM dd HH:mm:ss Z yyyy\"\n }\n }\n }\n}"'
{
"template": null,
"settings": {
"number_of_shards": 1
},
"mappings": {
"_source": {
"enabled": false
},
"properties": {
"host_name": {
"type": "keyword"
},
"created_at": {
"type": "date",
"format": "EEE MMM dd HH:mm:ss Z yyyy"
}
}
}
}
{
"template": null,
"settings": {
"number_of_shards": 1
},
"aliases": {
"alias1": {},
"alias2": {
"filter": {
"term": {
"user.id": "kimchy"
}
},
"routing": "shard-1"
},
"{index}-alias": {}
}
}
Get tokens from text analysis
The analyze API performs analysis on a text string and returns the resulting tokens.
Generating excessive amount of tokens may cause a node to run out of memory.
The index.analyze.max_token_count
setting enables you to limit the number of tokens that can be produced.
If more than this limit of tokens gets generated, an error occurs.
The _analyze
endpoint without a specified index will always use 10000
as its limit.
Query parameters
-
index
string Index used to derive the analyzer. If specified, the
analyzer
or field parameter overrides this value. If no index is specified or the index does not have a default analyzer, the analyze API uses the standard analyzer.
Body
-
analyzer
string The name of the analyzer that should be applied to the provided
text
. This could be a built-in analyzer, or an analyzer that’s been configured in the index. -
attributes
array[string] Array of token attributes used to filter the output of the
explain
parameter. -
char_filter
array Array of character filters used to preprocess characters before the tokenizer.
External documentation -
explain
boolean If
true
, the response includes token attributes and additional details. -
field
string Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
-
filter
array Array of token filters used to apply after the tokenizer.
External documentation -
normalizer
string Normalizer to use to convert text into a single token.
text
string | array[string]
GET /_analyze
{
"analyzer": "standard",
"text": "this is a test"
}
curl \
--request GET 'http://api.example.com/_analyze' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"analyzer\": \"standard\",\n \"text\": \"this is a test\"\n}"'
{
"analyzer": "standard",
"text": "this is a test"
}
{
"analyzer": "standard",
"text": [
"this is a test",
"the second text"
]
}
{
"tokenizer": "keyword",
"filter": [
"lowercase"
],
"char_filter": [
"html_strip"
],
"text": "this is a <b>test</b>"
}
{
"tokenizer": "whitespace",
"filter": [
"lowercase",
{
"type": "stop",
"stopwords": [
"a",
"is",
"this"
]
}
],
"text": "this is a test"
}
{
"field": "obj1.field1",
"text": "this is a test"
}
{
"normalizer": "my_normalizer",
"text": "BaR"
}
{
"tokenizer": "standard",
"filter": [
"snowball"
],
"text": "detailed output",
"explain": true,
"attributes": [
"keyword"
]
}
{
"detail": {
"custom_analyzer": true,
"charfilters": [],
"tokenizer": {
"name": "standard",
"tokens": [
{
"token": "detailed",
"start_offset": 0,
"end_offset": 8,
"type": "<ALPHANUM>",
"position": 0
},
{
"token": "output",
"start_offset": 9,
"end_offset": 15,
"type": "<ALPHANUM>",
"position": 1
}
]
},
"tokenfilters": [
{
"name": "snowball",
"tokens": [
{
"token": "detail",
"start_offset": 0,
"end_offset": 8,
"type": "<ALPHANUM>",
"position": 0,
"keyword": false
},
{
"token": "output",
"start_offset": 9,
"end_offset": 15,
"type": "<ALPHANUM>",
"position": 1,
"keyword": false
}
]
}
]
}
}
Get index templates
Added in 7.9.0
Get information about one or more index templates.
Path parameters
-
name
string Required Comma-separated list of index template names used to limit the request. Wildcard (*) expressions are supported.
Query parameters
-
local
boolean If true, the request retrieves information from the local node only. Defaults to false, which means information is retrieved from the master node.
-
flat_settings
boolean If true, returns settings in flat format.
-
master_timeout
string Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1
or0
. -
include_defaults
boolean If true, returns all relevant default configurations for the index template.
curl \
--request GET 'http://api.example.com/_index_template/{name}' \
--header "Authorization: $API_KEY"
Get mapping definitions
For data streams, the API retrieves mappings for the stream’s backing indices.
Query parameters
-
allow_no_indices
boolean If
false
, the request returns an error if any wildcard expression, index alias, or_all
value targets only missing or closed indices. This behavior applies even if the request targets other open indices. -
expand_wildcards
string | array[string] Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as
open,hidden
. Valid values are:all
,open
,closed
,hidden
,none
.Supported values include:
all
: Match any data stream or index, including hidden ones.open
: Match open, non-hidden indices. Also matches any non-hidden data stream.closed
: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden
: Match hidden data streams and hidden indices. Must be combined withopen
,closed
, orboth
.none
: Wildcard expressions are not accepted.
Values are
all
,open
,closed
,hidden
, ornone
. -
local
boolean Deprecated If
true
, the request retrieves information from the local node only. -
master_timeout
string Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1
or0
.
curl \
--request GET 'http://api.example.com/_mapping' \
--header "Authorization: $API_KEY"
Get mapping definitions
For data streams, the API retrieves mappings for the stream’s backing indices.
Path parameters
-
index
string | array[string] Required Comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (
*
). To target all data streams and indices, omit this parameter or use*
or_all
.
Query parameters
-
allow_no_indices
boolean If
false
, the request returns an error if any wildcard expression, index alias, or_all
value targets only missing or closed indices. This behavior applies even if the request targets other open indices. -
expand_wildcards
string | array[string] Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as
open,hidden
. Valid values are:all
,open
,closed
,hidden
,none
.Supported values include:
all
: Match any data stream or index, including hidden ones.open
: Match open, non-hidden indices. Also matches any non-hidden data stream.closed
: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden
: Match hidden data streams and hidden indices. Must be combined withopen
,closed
, orboth
.none
: Wildcard expressions are not accepted.
Values are
all
,open
,closed
,hidden
, ornone
. -
local
boolean Deprecated If
true
, the request retrieves information from the local node only. -
master_timeout
string Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1
or0
.
curl \
--request GET 'http://api.example.com/{index}/_mapping' \
--header "Authorization: $API_KEY"
Update index settings
Changes dynamic index settings in real time. For data streams, index setting changes are applied to all backing indices by default.
To revert a setting to the default value, use a null value.
The list of per-index settings that can be updated dynamically on live indices can be found in index settings documentation.
To preserve existing settings from being updated, set the preserve_existing
parameter to true
.
There are multiple valid ways to represent index settings in the request body. You can specify only the setting, for example:
{
"number_of_replicas": 1
}
Or you can use an index
setting object:
{
"index": {
"number_of_replicas": 1
}
}
Or you can use dot annotation:
{
"index.number_of_replicas": 1
}
Or you can embed any of the aforementioned options in a settings
object. For example:
{
"settings": {
"index": {
"number_of_replicas": 1
}
}
}
NOTE: You can only define new analyzers on closed indices. To add an analyzer, you must close the index, define the analyzer, and reopen the index. You cannot close the write index of a data stream. To update the analyzer for a data stream's write index and future backing indices, update the analyzer in the index template used by the stream. Then roll over the data stream to apply the new analyzer to the stream's write index and future backing indices. This affects searches and any new data added to the stream after the rollover. However, it does not affect the data stream's backing indices or their existing data. To change the analyzer for existing backing indices, you must create a new data stream and reindex your data into it.
Query parameters
-
allow_no_indices
boolean If
false
, the request returns an error if any wildcard expression, index alias, or_all
value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targetingfoo*,bar*
returns an error if an index starts withfoo
but no index starts withbar
. -
expand_wildcards
string | array[string] Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as
open,hidden
.Supported values include:
all
: Match any data stream or index, including hidden ones.open
: Match open, non-hidden indices. Also matches any non-hidden data stream.closed
: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden
: Match hidden data streams and hidden indices. Must be combined withopen
,closed
, orboth
.none
: Wildcard expressions are not accepted.
Values are
all
,open
,closed
,hidden
, ornone
. -
flat_settings
boolean If
true
, returns settings in flat format. -
master_timeout
string Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1
or0
. -
preserve_existing
boolean If
true
, existing index settings remain unchanged. -
reopen
boolean Whether to close and reopen the index to apply non-dynamic settings. If set to
true
the indices to which the settings are being applied will be closed temporarily and then reopened in order to apply the changes. -
timeout
string Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1
or0
.
Body
Required
PUT /my-index-000001/_settings
{
"index" : {
"number_of_replicas" : 2
}
}
curl \
--request PUT 'http://api.example.com/_settings' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"index\" : {\n \"number_of_replicas\" : 2\n }\n}"'
{
"index" : {
"number_of_replicas" : 2
}
}
{
"index" : {
"refresh_interval" : null
}
}
{
"analysis" : {
"analyzer":{
"content":{
"type":"custom",
"tokenizer":"whitespace"
}
}
}
}
POST /my-index-000001/_open
Get index shard stores
Get store information about replica shards in one or more indices. For data streams, the API retrieves store information for the stream's backing indices.
The index shard stores API returns the following information:
- The node on which each replica shard exists.
- The allocation ID for each replica shard.
- A unique ID for each replica shard.
- Any errors encountered while opening the shard index or from an earlier failure.
By default, the API returns store information only for primary shards that are unassigned or have one or more unassigned replica shards.
Path parameters
-
index
string | array[string] Required List of data streams, indices, and aliases used to limit the request.
Query parameters
-
allow_no_indices
boolean If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.
-
expand_wildcards
string | array[string] Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supported values include:
all
: Match any data stream or index, including hidden ones.open
: Match open, non-hidden indices. Also matches any non-hidden data stream.closed
: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden
: Match hidden data streams and hidden indices. Must be combined withopen
,closed
, orboth
.none
: Wildcard expressions are not accepted.
Values are
all
,open
,closed
,hidden
, ornone
. -
status
string | array[string] List of shard health statuses used to limit the request.
Supported values include:
green
: The primary shard and all replica shards are assigned.yellow
: One or more replica shards are unassigned.red
: The primary shard is unassigned.all
: Return all shards, regardless of health status.
Values are
green
,yellow
,red
, orall
.
GET /_shard_stores?status=green
curl \
--request GET 'http://api.example.com/{index}/_shard_stores' \
--header "Authorization: $API_KEY"
{
"indices": {
"my-index-000001": {
"shards": {
"0": {
"stores": [
{
"sPa3OgxLSYGvQ4oPs-Tajw": {
"name": "node_t0",
"ephemeral_id": "9NlXRFGCT1m8tkvYCMK-8A",
"transport_address": "local[1]",
"external_id": "node_t0",
"attributes": {},
"roles": [],
"version": "8.10.0",
"min_index_version": 7000099,
"max_index_version": 8100099
},
"allocation_id": "2iNySv_OQVePRX-yaRH_lQ",
"allocation": "primary",
"store_exception": {}
}
]
}
}
}
}
}
Simulate an index
Added in 7.9.0
Get the index configuration that would be applied to the specified index from an existing index template.
Path parameters
-
name
string Required Name of the index to simulate
Query parameters
-
create
boolean Whether the index template we optionally defined in the body should only be dry-run added if new or can also replace an existing one
-
cause
string User defined reason for dry-run creating the new template for simulation purposes
-
master_timeout
string Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1
or0
. -
include_defaults
boolean If true, returns all relevant default configurations for the index template.
POST /_index_template/_simulate_index/my-index-000001
curl \
--request POST 'http://api.example.com/_index_template/_simulate_index/{name}' \
--header "Authorization: $API_KEY"
{
"template" : {
"settings" : {
"index" : {
"number_of_shards" : "2",
"number_of_replicas" : "0",
"routing" : {
"allocation" : {
"include" : {
"_tier_preference" : "data_content"
}
}
}
}
},
"mappings" : {
"properties" : {
"@timestamp" : {
"type" : "date"
}
}
},
"aliases" : { }
},
"overlapping" : [
{
"name" : "template_1",
"index_patterns" : [
"my-index-*"
]
}
]
}
Query parameters
-
allow_no_indices
boolean If
false
, the request returns an error if any wildcard expression, index alias, or_all
value targets only missing or closed indices. This behavior applies even if the request targets other open indices. -
all_shards
boolean If
true
, the validation is executed on all shards instead of one random shard per index. -
analyzer
string Analyzer to use for the query string. This parameter can only be used when the
q
query string parameter is specified. -
analyze_wildcard
boolean If
true
, wildcard and prefix queries are analyzed. -
default_operator
string The default operator for query string query:
AND
orOR
.Values are
and
,AND
,or
, orOR
. -
df
string Field to use as default where no field prefix is given in the query string. This parameter can only be used when the
q
query string parameter is specified. -
expand_wildcards
string | array[string] Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as
open,hidden
. Valid values are:all
,open
,closed
,hidden
,none
.Supported values include:
all
: Match any data stream or index, including hidden ones.open
: Match open, non-hidden indices. Also matches any non-hidden data stream.closed
: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden
: Match hidden data streams and hidden indices. Must be combined withopen
,closed
, orboth
.none
: Wildcard expressions are not accepted.
Values are
all
,open
,closed
,hidden
, ornone
. -
explain
boolean If
true
, the response returns detailed information if an error has occurred. -
lenient
boolean If
true
, format-based query failures (such as providing text to a numeric field) in the query string will be ignored. -
rewrite
boolean If
true
, returns a more detailed explanation showing the actual Lucene query that will be executed. -
q
string Query in the Lucene query string syntax.
Body
-
query
object An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
External documentation
curl \
--request POST 'http://api.example.com/_validate/query' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"query":{}}'
Path parameters
-
index
string | array[string] Required Comma-separated list of data streams, indices, and aliases to search. Supports wildcards (
*
). To search all data streams or indices, omit this parameter or use*
or_all
.
Query parameters
-
allow_no_indices
boolean If
false
, the request returns an error if any wildcard expression, index alias, or_all
value targets only missing or closed indices. This behavior applies even if the request targets other open indices. -
all_shards
boolean If
true
, the validation is executed on all shards instead of one random shard per index. -
analyzer
string Analyzer to use for the query string. This parameter can only be used when the
q
query string parameter is specified. -
analyze_wildcard
boolean If
true
, wildcard and prefix queries are analyzed. -
default_operator
string The default operator for query string query:
AND
orOR
.Values are
and
,AND
,or
, orOR
. -
df
string Field to use as default where no field prefix is given in the query string. This parameter can only be used when the
q
query string parameter is specified. -
expand_wildcards
string | array[string] Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as
open,hidden
. Valid values are:all
,open
,closed
,hidden
,none
.Supported values include:
all
: Match any data stream or index, including hidden ones.open
: Match open, non-hidden indices. Also matches any non-hidden data stream.closed
: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden
: Match hidden data streams and hidden indices. Must be combined withopen
,closed
, orboth
.none
: Wildcard expressions are not accepted.
Values are
all
,open
,closed
,hidden
, ornone
. -
explain
boolean If
true
, the response returns detailed information if an error has occurred. -
lenient
boolean If
true
, format-based query failures (such as providing text to a numeric field) in the query string will be ignored. -
rewrite
boolean If
true
, returns a more detailed explanation showing the actual Lucene query that will be executed. -
q
string Query in the Lucene query string syntax.
Body
-
query
object An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
External documentation
curl \
--request POST 'http://api.example.com/{index}/_validate/query' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"query":{}}'
Migrate to data tiers routing
Added in 7.14.0
Switch the indices, ILM policies, and legacy, composable, and component templates from using custom node attributes and attribute-based allocation filters to using data tiers. Optionally, delete one legacy index template. Using node roles enables ILM to automatically move the indices between data tiers.
Migrating away from custom node attributes routing can be manually performed. This API provides an automated way of performing three out of the four manual steps listed in the migration guide:
- Stop setting the custom hot attribute on new indices.
- Remove custom allocation settings from existing ILM policies.
- Replace custom allocation settings from existing indices with the corresponding tier preference.
ILM must be stopped before performing the migration.
Use the stop ILM and get ILM status APIs to wait until the reported operation mode is STOPPED
.
Query parameters
-
dry_run
boolean If true, simulates the migration from node attributes based allocation filters to data tiers, but does not perform the migration. This provides a way to retrieve the indices and ILM policies that need to be migrated.
-
master_timeout
string The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error. It can also be set to
-1
to indicate that the request should never timeout.Values are
-1
or0
.
Body
-
legacy_template_to_delete
string -
node_attribute
string
POST /_ilm/migrate_to_data_tiers
{
"legacy_template_to_delete": "global-template",
"node_attribute": "custom_attribute_name"
}
curl \
--request POST 'http://api.example.com/_ilm/migrate_to_data_tiers' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"legacy_template_to_delete\": \"global-template\",\n \"node_attribute\": \"custom_attribute_name\"\n}"'
{
"legacy_template_to_delete": "global-template",
"node_attribute": "custom_attribute_name"
}
{
"dry_run": false,
"removed_legacy_template":"global-template",
"migrated_ilm_policies":["policy_with_allocate_action"],
"migrated_indices":["warm-index-to-migrate-000001"],
"migrated_legacy_templates":["a-legacy-template"],
"migrated_composable_templates":["a-composable-template"],
"migrated_component_templates":["a-component-template"]
}
Start the ILM plugin
Added in 6.6.0
Start the index lifecycle management plugin if it is currently stopped. ILM is started automatically when the cluster is formed. Restarting ILM is necessary only when it has been stopped using the stop ILM API.
Query parameters
-
master_timeout
string Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1
or0
. -
timeout
string Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1
or0
.
curl \
--request POST 'http://api.example.com/_ilm/start' \
--header "Authorization: $API_KEY"
{
"acknowledged": true
}
Perform inference on the service
Added in 8.11.0
This API enables you to use machine learning models to perform specific tasks on data that you provide as an input. It returns a response with the results of the tasks. The inference endpoint you use can perform one specific task that has been defined when the endpoint was created with the create inference API.
For details about using this API with a service, such as Amazon Bedrock, Anthropic, or HuggingFace, refer to the service-specific documentation.
The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.
Path parameters
-
inference_id
string Required The unique identifier for the inference endpoint.
Query parameters
-
timeout
string The amount of time to wait for the inference request to complete.
Values are
-1
or0
.
Body
-
query
string The query input, which is required only for the
rerank
task. It is not required for other tasks. input
string | array[string] Required The text on which you want to perform the inference task. It can be a single string or an array.
Inference endpoints for the
completion
task type currently only support a single string as input.-
task_settings
object
curl \
--request POST 'http://api.example.com/_inference/{inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"query":"string","input":"string","task_settings":{}}'
Perform inference on the service
Added in 8.11.0
This API enables you to use machine learning models to perform specific tasks on data that you provide as an input. It returns a response with the results of the tasks. The inference endpoint you use can perform one specific task that has been defined when the endpoint was created with the create inference API.
For details about using this API with a service, such as Amazon Bedrock, Anthropic, or HuggingFace, refer to the service-specific documentation.
The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.
Path parameters
-
task_type
string Required The type of inference task that the model performs.
Values are
sparse_embedding
,text_embedding
,rerank
,completion
, orchat_completion
. -
inference_id
string Required The unique identifier for the inference endpoint.
Query parameters
-
timeout
string The amount of time to wait for the inference request to complete.
Values are
-1
or0
.
Body
-
query
string The query input, which is required only for the
rerank
task. It is not required for other tasks. input
string | array[string] Required The text on which you want to perform the inference task. It can be a single string or an array.
Inference endpoints for the
completion
task type currently only support a single string as input.-
task_settings
object
curl \
--request POST 'http://api.example.com/_inference/{task_type}/{inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"query":"string","input":"string","task_settings":{}}'
Create an Azure OpenAI inference endpoint
Added in 8.14.0
Create an inference endpoint to perform an inference task with the azureopenai
service.
The list of chat completion models that you can choose from in your Azure OpenAI deployment include:
The list of embeddings models that you can choose from in your deployment can be found in the Azure models documentation.
Path parameters
-
task_type
string Required The type of the inference task that the model will perform. NOTE: The
chat_completion
task type only supports streaming and only through the _stream API.Values are
completion
ortext_embedding
. -
azureopenai_inference_id
string Required The unique identifier of the inference endpoint.
Body
-
chunking_settings
object -
service
string Required Value is
azureopenai
. -
service_settings
object Required -
task_settings
object
PUT _inference/text_embedding/azure_openai_embeddings
{
"service": "azureopenai",
"service_settings": {
"api_key": "Api-Key",
"resource_name": "Resource-name",
"deployment_id": "Deployment-id",
"api_version": "2024-02-01"
}
}
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{azureopenai_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"azureopenai\",\n \"service_settings\": {\n \"api_key\": \"Api-Key\",\n \"resource_name\": \"Resource-name\",\n \"deployment_id\": \"Deployment-id\",\n \"api_version\": \"2024-02-01\"\n }\n}"'
{
"service": "azureopenai",
"service_settings": {
"api_key": "Api-Key",
"resource_name": "Resource-name",
"deployment_id": "Deployment-id",
"api_version": "2024-02-01"
}
}
{
"service": "azureopenai",
"service_settings": {
"api_key": "Api-Key",
"resource_name": "Resource-name",
"deployment_id": "Deployment-id",
"api_version": "2024-02-01"
}
}
Create a Mistral inference endpoint
Added in 8.15.0
Creates an inference endpoint to perform an inference task with the mistral
service.
Path parameters
-
task_type
string Required The task type. The only valid task type for the model to perform is
text_embedding
.Value is
text_embedding
. -
mistral_inference_id
string Required The unique identifier of the inference endpoint.
Body
-
chunking_settings
object -
service
string Required Value is
mistral
. -
service_settings
object Required
PUT _inference/text_embedding/mistral-embeddings-test
{
"service": "mistral",
"service_settings": {
"api_key": "Mistral-API-Key",
"model": "mistral-embed"
}
}
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{mistral_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"mistral\",\n \"service_settings\": {\n \"api_key\": \"Mistral-API-Key\",\n \"model\": \"mistral-embed\" \n }\n}"'
{
"service": "mistral",
"service_settings": {
"api_key": "Mistral-API-Key",
"model": "mistral-embed"
}
}
Get GeoIP database configurations
Added in 8.15.0
Get information about one or more IP geolocation database configurations.
Path parameters
-
id
string | array[string] Required A comma-separated list of database configuration IDs to retrieve. Wildcard (
*
) expressions are supported. To get all database configurations, omit this parameter or use*
.
curl \
--request GET 'http://api.example.com/_ingest/geoip/database/{id}' \
--header "Authorization: $API_KEY"
Create or update a pipeline
Added in 5.0.0
Changes made using this API take effect immediately.
Path parameters
-
id
string Required ID of the ingest pipeline to create or update.
Query parameters
-
master_timeout
string Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1
or0
. -
timeout
string Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1
or0
. -
if_version
number Required version for optimistic concurrency control for pipeline updates
Body
Required
-
_meta
object -
description
string Description of the ingest pipeline.
-
on_failure
array[object] Processors to run immediately after a processor failure. Each processor supports a processor-level
on_failure
value. If a processor without anon_failure
value fails, Elasticsearch uses this pipeline-level parameter as a fallback. The processors in this parameter run sequentially in the order specified. Elasticsearch will not attempt to run the pipeline's remaining processors. -
processors
array[object] Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.
-
version
number -
deprecated
boolean Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.
PUT _ingest/pipeline/my-pipeline-id
{
"description" : "My optional pipeline description",
"processors" : [
{
"set" : {
"description" : "My optional processor description",
"field": "my-keyword-field",
"value": "foo"
}
}
]
}
curl \
--request PUT 'http://api.example.com/_ingest/pipeline/{id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"description\" : \"My optional pipeline description\",\n \"processors\" : [\n {\n \"set\" : {\n \"description\" : \"My optional processor description\",\n \"field\": \"my-keyword-field\",\n \"value\": \"foo\"\n }\n }\n ]\n}"'
{
"description" : "My optional pipeline description",
"processors" : [
{
"set" : {
"description" : "My optional processor description",
"field": "my-keyword-field",
"value": "foo"
}
}
]
}
{
"description" : "My optional pipeline description",
"processors" : [
{
"set" : {
"description" : "My optional processor description",
"field": "my-keyword-field",
"value": "foo"
}
}
],
"_meta": {
"reason": "set my-keyword-field to foo",
"serialization": {
"class": "MyPipeline",
"id": 10
}
}
}
Get pipelines
Added in 5.0.0
Get information about one or more ingest pipelines. This API returns a local reference of the pipeline.
Query parameters
-
master_timeout
string Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1
or0
. -
summary
boolean Return pipelines without their definitions (default: false)
curl \
--request GET 'http://api.example.com/_ingest/pipeline' \
--header "Authorization: $API_KEY"
{
"my-pipeline-id" : {
"description" : "describe pipeline",
"version" : 123,
"processors" : [
{
"set" : {
"field" : "foo",
"value" : "bar"
}
}
]
}
}
Run a grok processor
Added in 6.1.0
Extract structured fields out of a single text field within a document. You must choose which field to extract matched fields from, as well as the grok pattern you expect will match. A grok pattern is like a regular expression that supports aliased expressions that can be reused.
curl \
--request GET 'http://api.example.com/_ingest/processor/grok' \
--header "Authorization: $API_KEY"
Logstash
Logstash APIs enable you to manage pipelines that are used by Logstash Central Management.
Create a calendar
Added in 6.2.0
Path parameters
-
calendar_id
string Required A string that uniquely identifies a calendar.
Body
-
job_ids
array[string] An array of anomaly detection job identifiers.
-
description
string A description of the calendar.
curl \
--request PUT 'http://api.example.com/_ml/calendars/{calendar_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"job_ids":["string"],"description":"string"}'
Add anomaly detection job to calendar
Added in 6.2.0
Path parameters
-
calendar_id
string Required A string that uniquely identifies a calendar.
-
job_id
string | array[string] Required An identifier for the anomaly detection jobs. It can be a job identifier, a group name, or a comma-separated list of jobs or groups.
curl \
--request PUT 'http://api.example.com/_ml/calendars/{calendar_id}/jobs/{job_id}' \
--header "Authorization: $API_KEY"
Get anomaly detection job results for buckets
Added in 5.4.0
The API presents a chronological view of the records, grouped by bucket.
Query parameters
-
anomaly_score
number Returns buckets with anomaly scores greater or equal than this value.
-
desc
boolean If
true
, the buckets are sorted in descending order. -
end
string | number Returns buckets with timestamps earlier than this time.
-1
means it is unset and results are not limited to specific timestamps. -
exclude_interim
boolean If
true
, the output excludes interim results. -
expand
boolean If true, the output includes anomaly records.
-
from
number Skips the specified number of buckets.
-
size
number Specifies the maximum number of buckets to obtain.
-
sort
string Specifies the sort field for the requested buckets.
-
start
string | number Returns buckets with timestamps after this time.
-1
means it is unset and results are not limited to specific timestamps.
Body
-
anomaly_score
number Refer to the description for the
anomaly_score
query parameter. -
desc
boolean Refer to the description for the
desc
query parameter. end
string | number A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
-
exclude_interim
boolean Refer to the description for the
exclude_interim
query parameter. -
expand
boolean Refer to the description for the
expand
query parameter. -
page
object -
sort
string Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
start
string | number A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
curl \
--request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/results/buckets/{timestamp}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"anomaly_score":42.0,"desc":true,"":"string","exclude_interim":true,"expand":true,"page":{"from":42.0,"size":42.0},"sort":"string"}'
Get calendar configuration info
Added in 6.2.0
curl \
--request POST 'http://api.example.com/_ml/calendars' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"page":{"from":42.0,"size":42.0}}'
Get anomaly detection job results for categories
Added in 5.4.0
Path parameters
-
job_id
string Required Identifier for the anomaly detection job.
Query parameters
-
from
number Skips the specified number of categories.
-
partition_field_value
string Only return categories for the specified partition.
-
size
number Specifies the maximum number of categories to obtain.
curl \
--request GET 'http://api.example.com/_ml/anomaly_detectors/{job_id}/results/categories' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"page":{"from":42.0,"size":42.0}}'
Get anomaly detection job stats
Added in 5.5.0
Path parameters
-
job_id
string Required Identifier for the anomaly detection job. It can be a job identifier, a group name, a comma-separated list of jobs, or a wildcard expression. If you do not specify one of these options, the API returns information for all anomaly detection jobs.
Query parameters
-
allow_no_match
boolean Specifies what to do when the request:
- Contains wildcard expressions and there are no jobs that match.
- Contains the _all string or no identifiers and there are no matches.
- Contains wildcard expressions and there are only partial matches.
If
true
, the API returns an emptyjobs
array when there are no matches and the subset of results when there are partial matches. Iffalse
, the API returns a404
status code when there are no matches or only partial matches.
curl \
--request GET 'http://api.example.com/_ml/anomaly_detectors/{job_id}/_stats' \
--header "Authorization: $API_KEY"
Open anomaly detection jobs
Added in 5.4.0
An anomaly detection job must be opened to be ready to receive and analyze data. It can be opened and closed multiple times throughout its lifecycle. When you open a new job, it starts with an empty model. When you open an existing job, the most recent model state is automatically loaded. The job is ready to resume its analysis from where it left off, once new data is received.
Path parameters
-
job_id
string Required Identifier for the anomaly detection job.
Query parameters
-
timeout
string Controls the time to wait until a job has opened.
Values are
-1
or0
.
curl \
--request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/_open' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"timeout\": \"35m\"\n}"'
{
"timeout": "35m"
}
{
"opened": true,
"node": "node-1"
}
Preview a datafeed
Added in 5.4.0
This API returns the first "page" of search results from a datafeed. You can preview an existing datafeed or provide configuration details for a datafeed and anomaly detection job in the API. The preview shows the structure of the data that will be passed to the anomaly detection engine. IMPORTANT: When Elasticsearch security features are enabled, the preview uses the credentials of the user that called the API. However, when the datafeed starts it uses the roles of the last user that created or updated the datafeed. To get a preview that accurately reflects the behavior of the datafeed, use the appropriate credentials. You can also use secondary authorization headers to supply the credentials.
Path parameters
-
datafeed_id
string Required A numerical character string that uniquely identifies the datafeed. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters. NOTE: If you use this path parameter, you cannot provide datafeed or anomaly detection job configuration details in the request body.
Body
-
datafeed_config
object -
job_config
object
curl \
--request POST 'http://api.example.com/_ml/datafeeds/{datafeed_id}/_preview' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"datafeed_config":{"aggregations":{},"chunking_config":{"mode":"auto","time_span":"string"},"datafeed_id":"string","delayed_data_check_config":{"check_window":"string","enabled":true},"frequency":"string","indices":"string","indices_options":{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"ignore_throttled":true},"job_id":"string","max_empty_searches":42.0,"query":{},"query_delay":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"script_fields":{"additionalProperty1":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true},"additionalProperty2":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true}},"scroll_size":42.0},"job_config":{"allow_lazy_open":true,"analysis_config":{"bucket_span":"string","":"string","categorization_field_name":"string","categorization_filters":["string"],"detectors":[{"by_field_name":"string","custom_rules":[{"actions":["skip_result"],"conditions":[{}],"scope":{}}],"detector_description":"string","detector_index":42.0,"exclude_frequent":"all","field_name":"string","function":"string","over_field_name":"string","partition_field_name":"string","use_null":true}],"influencers":["string"],"latency":"string","model_prune_window":"string","multivariate_by_fields":true,"per_partition_categorization":{"enabled":true,"stop_on_warn":true},"summary_count_field_name":"string"},"analysis_limits":{"categorization_examples_limit":42.0,"":42.0},"background_persist_interval":"string","custom_settings":{},"daily_model_snapshot_retention_after_days":42.0,"data_description":{"format":"string","time_field":"string","time_format":"string","field_delimiter":"string"},"datafeed_config":{"aggregations":{},"chunking_config":{"mode":"auto","time_span":"string"},"datafeed_id":"string","delayed_data_check_config":{"check_window":"string","enabled":true},"frequency":"string","indices":"string","indices_options":{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"ignore_throttled":true},"job_id":"string","max_empty_searches":42.0,"query":{},"query_delay":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"script_fields":{"additionalProperty1":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true},"additionalProperty2":{"script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true}},"scroll_size":42.0},"description":"string","groups":["string"],"job_id":"string","job_type":"string","model_plot_config":{"annotations_enabled":true,"enabled":true,"terms":"string"},"model_snapshot_retention_days":42.0,"renormalization_window_days":42.0,"results_index_name":"string","results_retention_days":42.0}}'
Revert to a snapshot
Added in 5.4.0
The machine learning features react quickly to anomalous input, learning new behaviors in data. Highly anomalous input increases the variance in the models whilst the system learns whether this is a new step-change in behavior or a one-off event. In the case where this anomalous input is known to be a one-off, then it might be appropriate to reset the model state to a time before this event. For example, you might consider reverting to a saved snapshot after Black Friday or a critical system failure.
Path parameters
-
job_id
string Required Identifier for the anomaly detection job.
-
snapshot_id
string Required You can specify
empty
as the . Reverting to the empty snapshot means the anomaly detection job starts learning a new model from scratch when it is started.
Query parameters
-
delete_intervening_results
boolean If true, deletes the results in the time period between the latest results and the time of the reverted snapshot. It also resets the model to accept records for this time period. If you choose not to delete intervening results when reverting a snapshot, the job will not accept input data that is older than the current time. If you want to resend data, then delete the intervening results.
Body
-
delete_intervening_results
boolean Refer to the description for the
delete_intervening_results
query parameter.
curl \
--request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}/_revert' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"delete_intervening_results":true}'
Evaluate data frame analytics
Added in 7.3.0
The API packages together commonly used evaluation metrics for various types of machine learning features. This has been designed for use on indexes created by data frame analytics. Evaluation requires both a ground truth field and an analytics result field to be present.
Body
Required
-
evaluation
object Required -
index
string Required -
query
object An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
External documentation
POST _ml/data_frame/_evaluate
{
"index": "animal_classification",
"evaluation": {
"classification": {
"actual_field": "animal_class",
"predicted_field": "ml.animal_class_prediction",
"metrics": {
"multiclass_confusion_matrix": {}
}
}
}
}
curl \
--request POST 'http://api.example.com/_ml/data_frame/_evaluate' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"index\": \"animal_classification\",\n \"evaluation\": {\n \"classification\": {\n \"actual_field\": \"animal_class\",\n \"predicted_field\": \"ml.animal_class_prediction\",\n \"metrics\": {\n \"multiclass_confusion_matrix\": {}\n }\n }\n }\n}"'
{
"index": "animal_classification",
"evaluation": {
"classification": {
"actual_field": "animal_class",
"predicted_field": "ml.animal_class_prediction",
"metrics": {
"multiclass_confusion_matrix": {}
}
}
}
}
{
"index": "animal_classification",
"evaluation": {
"classification": {
"actual_field": "animal_class",
"metrics": {
"auc_roc": {
"class_name": "dog"
}
}
}
}
}
{
"index": "my_analytics_dest_index",
"evaluation": {
"outlier_detection": {
"actual_field": "is_outlier",
"predicted_probability_field": "ml.outlier_score"
}
}
}
{
"index": "house_price_predictions",
"query": {
"bool": {
"filter": [
{
"term": {
"ml.is_training": false
}
}
]
}
},
"evaluation": {
"regression": {
"actual_field": "price",
"predicted_field": "ml.price_prediction",
"metrics": {
"r_squared": {},
"mse": {},
"msle": {
"offset": 10
},
"huber": {
"delta": 1.5
}
}
}
}
}
{
"index": "house_price_predictions",
"query": {
"term": {
"ml.is_training": {
"value": true
}
}
},
"evaluation": {
"regression": {
"actual_field": "price",
"predicted_field": "ml.price_prediction",
"metrics": {
"r_squared": {},
"mse": {},
"msle": {},
"huber": {}
}
}
}
}
{
"classification": {
"multiclass_confusion_matrix": {
"confusion_matrix": [
{
"actual_class": "cat",
"actual_class_doc_count": 12,
"predicted_classes": [
{
"predicted_class": "cat",
"count": 12
},
{
"predicted_class": "dog",
"count": 0
}
],
"other_predicted_class_doc_count": 0
},
{
"actual_class": "dog",
"actual_class_doc_count": 11,
"predicted_classes": [
{
"predicted_class": "dog",
"count": 7
},
{
"predicted_class": "cat",
"count": 4
}
],
"other_predicted_class_doc_count": 0
}
],
"other_actual_class_count": 0
}
}
}
{
"classification": {
"auc_roc": {
"value": 0.8941788639536681
}
}
}
{
"outlier_detection": {
"auc_roc": {
"value": 0.9258475774641445
},
"confusion_matrix": {
"0.25": {
"tp": 5,
"fp": 9,
"tn": 204,
"fn": 5
},
"0.5": {
"tp": 1,
"fp": 5,
"tn": 208,
"fn": 9
},
"0.75": {
"tp": 0,
"fp": 4,
"tn": 209,
"fn": 10
}
},
"precision": {
"0.25": 0.35714285714285715,
"0.5": 0.16666666666666666,
"0.75": 0
},
"recall": {
"0.25": 0.5,
"0.5": 0.1,
"0.75": 0
}
}
}
Preview features used by data frame analytics
Added in 7.13.0
Preview the extracted features used by a data frame analytics config.
Path parameters
-
id
string Required Identifier for the data frame analytics job.
curl \
--request GET 'http://api.example.com/_ml/data_frame/analytics/{id}/_preview' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"config":{"source":{"index":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"_source":{"includes":["string"],"excludes":["string"]},"query":{}},"analysis":{"classification":{"alpha":42.0,"dependent_variable":"string","downsample_factor":42.0,"early_stopping_enabled":true,"eta":42.0,"eta_growth_rate_per_tree":42.0,"feature_bag_fraction":42.0,"feature_processors":[{"frequency_encoding":{},"multi_encoding":{},"n_gram_encoding":{},"one_hot_encoding":{},"target_mean_encoding":{}}],"gamma":42.0,"lambda":42.0,"max_optimization_rounds_per_hyperparameter":42.0,"max_trees":42.0,"num_top_feature_importance_values":42.0,"prediction_field_name":"string","randomize_seed":42.0,"soft_tree_depth_limit":42.0,"soft_tree_depth_tolerance":42.0,"":"string","class_assignment_objective":"string","num_top_classes":42.0},"outlier_detection":{"compute_feature_influence":true,"feature_influence_threshold":42.0,"method":"string","n_neighbors":42.0,"outlier_fraction":42.0,"standardization_enabled":true},"regression":{"alpha":42.0,"dependent_variable":"string","downsample_factor":42.0,"early_stopping_enabled":true,"eta":42.0,"eta_growth_rate_per_tree":42.0,"feature_bag_fraction":42.0,"feature_processors":[{"frequency_encoding":{},"multi_encoding":{},"n_gram_encoding":{},"one_hot_encoding":{},"target_mean_encoding":{}}],"gamma":42.0,"lambda":42.0,"max_optimization_rounds_per_hyperparameter":42.0,"max_trees":42.0,"num_top_feature_importance_values":42.0,"prediction_field_name":"string","randomize_seed":42.0,"soft_tree_depth_limit":42.0,"soft_tree_depth_tolerance":42.0,"":"string","loss_function":"string","loss_function_parameter":42.0}},"model_memory_limit":"string","max_num_threads":42.0,"analyzed_fields":{"includes":["string"],"excludes":["string"]}}}'
Create part of a trained model definition
Added in 8.0.0
Body
Required
-
definition
string Required The definition part for the model. Must be a base64 encoded string.
-
total_definition_length
number Required The total uncompressed definition length in bytes. Not base64 encoded.
-
total_parts
number Required The total number of parts that will be uploaded. Must be greater than 0.
curl \
--request PUT 'http://api.example.com/_ml/trained_models/{model_id}/definition/{part}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"definition":"string","total_definition_length":42.0,"total_parts":42.0}'
Create a trained model vocabulary
Added in 8.0.0
This API is supported only for natural language processing (NLP) models.
The vocabulary is stored in the index as described in inference_config.*.vocabulary
of the trained model definition.
Path parameters
-
model_id
string Required The unique identifier of the trained model.
Body
Required
-
vocabulary
array[string] Required The model vocabulary, which must not be empty.
-
merges
array[string] The optional model merges if required by the tokenizer.
-
scores
array[number] The optional vocabulary value scores if required by the tokenizer.
curl \
--request PUT 'http://api.example.com/_ml/trained_models/{model_id}/vocabulary' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"vocabulary":["string"],"merges":["string"],"scores":[42.0]}'
Get the shutdown status
Added in 7.13.0
Get information about nodes that are ready to be shut down, have shut down preparations still in progress, or have stalled. The API returns status information for each part of the shut down process.
NOTE: This feature is designed for indirect use by Elasticsearch Service, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.
If the operator privileges feature is enabled, you must be an operator to use this API.
Query parameters
-
master_timeout
string Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Values are
nanos
,micros
,ms
,s
,m
,h
, ord
.
GET /_nodes/USpTGYaBSIKbgSUJR2Z9lg/shutdown
curl \
--request GET 'http://api.example.com/_nodes/shutdown' \
--header "Authorization: $API_KEY"
{
"nodes": [
{
"node_id": "USpTGYaBSIKbgSUJR2Z9lg",
"type": "RESTART",
"reason": "Demonstrating how the node shutdown API works",
"shutdown_startedmillis": 1624406108685,
"allocation_delay": "10m",
"status": "COMPLETE",
"shard_migration": {
"status": "COMPLETE",
"shard_migrations_remaining": 0,
"explanation": "no shard relocation is necessary for a node restart"
},
"persistent_tasks": {
"status": "COMPLETE"
},
"plugins": {
"status": "COMPLETE"
}
}
]
}
Create or update a query ruleset
Added in 8.10.0
There is a limit of 100 rules per ruleset.
This limit can be increased by using the xpack.applications.rules.max_rules_per_ruleset
cluster setting.
IMPORTANT: Due to limitations within pinned queries, you can only select documents using ids
or docs
, but cannot use both in single rule.
It is advised to use one or the other in query rulesets, to avoid errors.
Additionally, pinned queries have a maximum limit of 100 pinned hits.
If multiple matching rules pin more than 100 documents, only the first 100 documents are pinned in the order they are specified in the ruleset.
Path parameters
-
ruleset_id
string Required The unique identifier of the query ruleset to be created or updated.
PUT _query_rules/my-ruleset
{
"rules": [
{
"rule_id": "my-rule1",
"type": "pinned",
"criteria": [
{
"type": "contains",
"metadata": "user_query",
"values": [ "pugs", "puggles" ]
},
{
"type": "exact",
"metadata": "user_country",
"values": [ "us" ]
}
],
"actions": {
"ids": [
"id1",
"id2"
]
}
},
{
"rule_id": "my-rule2",
"type": "pinned",
"criteria": [
{
"type": "fuzzy",
"metadata": "user_query",
"values": [ "rescue dogs" ]
}
],
"actions": {
"docs": [
{
"_index": "index1",
"_id": "id3"
},
{
"_index": "index2",
"_id": "id4"
}
]
}
}
]
}
curl \
--request PUT 'http://api.example.com/_query_rules/{ruleset_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"rules\": [\n {\n \"rule_id\": \"my-rule1\",\n \"type\": \"pinned\",\n \"criteria\": [\n {\n \"type\": \"contains\",\n \"metadata\": \"user_query\",\n \"values\": [ \"pugs\", \"puggles\" ]\n },\n {\n \"type\": \"exact\",\n \"metadata\": \"user_country\",\n \"values\": [ \"us\" ]\n }\n ],\n \"actions\": {\n \"ids\": [\n \"id1\",\n \"id2\"\n ]\n }\n },\n {\n \"rule_id\": \"my-rule2\",\n \"type\": \"pinned\",\n \"criteria\": [\n {\n \"type\": \"fuzzy\",\n \"metadata\": \"user_query\",\n \"values\": [ \"rescue dogs\" ]\n }\n ],\n \"actions\": {\n \"docs\": [\n {\n \"_index\": \"index1\",\n \"_id\": \"id3\"\n },\n {\n \"_index\": \"index2\",\n \"_id\": \"id4\"\n }\n ]\n }\n }\n ]\n}"'
{
"rules": [
{
"rule_id": "my-rule1",
"type": "pinned",
"criteria": [
{
"type": "contains",
"metadata": "user_query",
"values": [ "pugs", "puggles" ]
},
{
"type": "exact",
"metadata": "user_country",
"values": [ "us" ]
}
],
"actions": {
"ids": [
"id1",
"id2"
]
}
},
{
"rule_id": "my-rule2",
"type": "pinned",
"criteria": [
{
"type": "fuzzy",
"metadata": "user_query",
"values": [ "rescue dogs" ]
}
],
"actions": {
"docs": [
{
"_index": "index1",
"_id": "id3"
},
{
"_index": "index2",
"_id": "id4"
}
]
}
}
]
}
Get rollup job information
Deprecated
Technical preview
Get the configuration, stats, and status of rollup jobs.
NOTE: This API returns only active (both STARTED
and STOPPED
) jobs.
If a job was created, ran for a while, then was deleted, the API does not return any details about it.
For details about a historical rollup job, the rollup capabilities API may be more useful.
GET _rollup/job/sensor
curl \
--request GET 'http://api.example.com/_rollup/job' \
--header "Authorization: $API_KEY"
{
"jobs": [
{
"config": {
"id": "sensor",
"index_pattern": "sensor-*",
"rollup_index": "sensor_rollup",
"cron": "*/30 * * * * ?",
"groups": {
"date_histogram": {
"fixed_interval": "1h",
"delay": "7d",
"field": "timestamp",
"time_zone": "UTC"
},
"terms": {
"fields": [
"node"
]
}
},
"metrics": [
{
"field": "temperature",
"metrics": [
"min",
"max",
"sum"
]
},
{
"field": "voltage",
"metrics": [
"avg"
]
}
],
"timeout": "20s",
"page_size": 1000
},
"status": {
"job_state": "stopped"
},
"stats": {
"pages_processed": 0,
"documents_processed": 0,
"rollups_indexed": 0,
"trigger_count": 0,
"index_failures": 0,
"index_time_in_ms": 0,
"index_total": 0,
"search_failures": 0,
"search_time_in_ms": 0,
"search_total": 0,
"processing_time_in_ms": 0,
"processing_total": 0
}
}
]
}
Search rolled-up data
Deprecated
Technical preview
The rollup search endpoint is needed because, internally, rolled-up documents utilize a different document structure than the original data. It rewrites standard Query DSL into a format that matches the rollup documents then takes the response and rewrites it back to what a client would expect given the original query.
The request body supports a subset of features from the regular search API. The following functionality is not available:
size
: Because rollups work on pre-aggregated data, no search hits can be returned and so size must be set to zero or omitted entirely.
highlighter
, suggestors
, post_filter
, profile
, explain
: These are similarly disallowed.
Searching both historical rollup and non-rollup data
The rollup search API has the capability to search across both "live" non-rollup data and the aggregated rollup data. This is done by simply adding the live indices to the URI. For example:
GET sensor-1,sensor_rollup/_rollup_search
{
"size": 0,
"aggregations": {
"max_temperature": {
"max": {
"field": "temperature"
}
}
}
}
The rollup search endpoint does two things when the search runs:
- The original request is sent to the non-rollup index unaltered.
- A rewritten version of the original request is sent to the rollup index.
When the two responses are received, the endpoint rewrites the rollup response and merges the two together. During the merging process, if there is any overlap in buckets between the two responses, the buckets from the non-rollup index are used.
Path parameters
-
index
string | array[string] Required A comma-separated list of data streams and indices used to limit the request. This parameter has the following rules:
- At least one data stream, index, or wildcard expression must be specified. This target can include a rollup or non-rollup index. For data streams, the stream's backing indices can only serve as non-rollup indices. Omitting the parameter or using
_all
are not permitted. - Multiple non-rollup indices may be specified.
- Only one rollup index may be specified. If more than one are supplied, an exception occurs.
- Wildcard expressions (
*
) may be used. If they match more than one rollup index, an exception occurs. However, you can use an expression to match multiple non-rollup indices or data streams.
- At least one data stream, index, or wildcard expression must be specified. This target can include a rollup or non-rollup index. For data streams, the stream's backing indices can only serve as non-rollup indices. Omitting the parameter or using
Query parameters
-
rest_total_hits_as_int
boolean Indicates whether hits.total should be rendered as an integer or an object in the rest search response
-
typed_keys
boolean Specify whether aggregation and suggester names should be prefixed by their respective types in the response
Body
Required
-
aggregations
object Specifies aggregations.
External documentation -
query
object An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
External documentation -
size
number Must be zero if set, as rollups work on pre-aggregated data.
GET /sensor_rollup/_rollup_search
{
"size": 0,
"aggregations": {
"max_temperature": {
"max": {
"field": "temperature"
}
}
}
}
curl \
--request POST 'http://api.example.com/{index}/_rollup_search' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"size\": 0,\n \"aggregations\": {\n \"max_temperature\": {\n \"max\": {\n \"field\": \"temperature\"\n }\n }\n }\n}"'
{
"size": 0,
"aggregations": {
"max_temperature": {
"max": {
"field": "temperature"
}
}
}
}
{
"took" : 102,
"timed_out" : false,
"terminated_early" : false,
"_shards" : {} ,
"hits" : {
"total" : {
"value": 0,
"relation": "eq"
},
"max_score" : 0.0,
"hits" : [ ]
},
"aggregations" : {
"max_temperature" : {
"value" : 202.0
}
}
}
Create or update a script or search template
Creates or updates a stored script or search template.
Path parameters
-
id
string Required The identifier for the stored script or search template. It must be unique within the cluster.
Query parameters
-
context
string The context in which the script or search template should run. To prevent errors, the API immediately compiles the script or template in this context. If you specify both this and the
<context>
path parameter, the API uses the request path parameter. -
master_timeout
string The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error. It can also be set to
-1
to indicate that the request should never timeout.Values are
-1
or0
. -
timeout
string The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error. It can also be set to
-1
to indicate that the request should never timeout.Values are
-1
or0
.
PUT _scripts/my-search-template
{
"script": {
"lang": "mustache",
"source": {
"query": {
"match": {
"message": "{{query_string}}"
}
},
"from": "{{from}}",
"size": "{{size}}"
}
}
}
curl \
--request PUT 'http://api.example.com/_scripts/{id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"script\": {\n \"lang\": \"mustache\",\n \"source\": {\n \"query\": {\n \"match\": {\n \"message\": \"{{query_string}}\"\n }\n },\n \"from\": \"{{from}}\",\n \"size\": \"{{size}}\"\n }\n }\n}"'
{
"script": {
"lang": "mustache",
"source": {
"query": {
"match": {
"message": "{{query_string}}"
}
},
"from": "{{from}}",
"size": "{{size}}"
}
}
}
{
"script": {
"lang": "painless",
"source": "Math.log(_score * 2) + params['my_modifier']"
}
}
Delete an async search
Added in 7.7.0
If the asynchronous search is still running, it is cancelled.
Otherwise, the saved search results are deleted.
If the Elasticsearch security features are enabled, the deletion of a specific async search is restricted to: the authenticated user that submitted the original search request; users that have the cancel_task
cluster privilege.
Path parameters
-
id
string Required A unique identifier for the async search.
curl \
--request DELETE 'http://api.example.com/_async_search/{id}' \
--header "Authorization: $API_KEY"
Close a point in time
Added in 7.10.0
A point in time must be opened explicitly before being used in search requests.
The keep_alive
parameter tells Elasticsearch how long it should persist.
A point in time is automatically closed when the keep_alive
period has elapsed.
However, keeping points in time has a cost; close them as soon as they are no longer required for search requests.
DELETE /_pit
{
"id": "46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA=="
}
curl \
--request DELETE 'http://api.example.com/_pit' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"id\": \"46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA==\"\n}"'
{
"id": "46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA=="
}
{
"succeeded": true,
"num_freed": 3
}
Evaluate ranked search results
Added in 6.2.0
Evaluate the quality of ranked search results over a set of typical search queries.
Path parameters
-
index
string | array[string] Required A comma-separated list of data streams, indices, and index aliases used to limit the request. Wildcard (
*
) expressions are supported. To target all data streams and indices in a cluster, omit this parameter or use_all
or*
.
Query parameters
-
allow_no_indices
boolean If
false
, the request returns an error if any wildcard expression, index alias, or_all
value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targetingfoo*,bar*
returns an error if an index starts withfoo
but no index starts withbar
. -
expand_wildcards
string | array[string] Whether to expand wildcard expression to concrete indices that are open, closed or both.
Supported values include:
all
: Match any data stream or index, including hidden ones.open
: Match open, non-hidden indices. Also matches any non-hidden data stream.closed
: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden
: Match hidden data streams and hidden indices. Must be combined withopen
,closed
, orboth
.none
: Wildcard expressions are not accepted.
Values are
all
,open
,closed
,hidden
, ornone
. -
search_type
string Search operation type
curl \
--request GET 'http://api.example.com/{index}/_rank_eval' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"requests":[{"id":"string","request":{"query":{},"size":42.0},"ratings":[{"_id":"string","_index":"string","rating":42.0}],"template_id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}}}],"metric":{"precision":{"k":42.0,"relevant_rating_threshold":42.0,"ignore_unlabeled":true},"recall":{"k":42.0,"relevant_rating_threshold":42.0},"mean_reciprocal_rank":{"k":42.0,"relevant_rating_threshold":42.0},"dcg":{"k":42.0,"normalize":true},"expected_reciprocal_rank":{"k":42.0,"maximum_relevance":42.0}}}'
Run a search
Get search hits that match the query defined in the request.
You can provide search queries using the q
query string parameter or the request body.
If both are specified, only the query parameter is used.
If the Elasticsearch security features are enabled, you must have the read index privilege for the target data stream, index, or alias. For cross-cluster search, refer to the documentation about configuring CCS privileges.
To search a point in time (PIT) for an alias, you must have the read
index privilege for the alias's data streams or indices.
Search slicing
When paging through a large number of documents, it can be helpful to split the search into multiple slices to consume them independently with the slice
and pit
properties.
By default the splitting is done first on the shards, then locally on each shard.
The local splitting partitions the shard into contiguous ranges based on Lucene document IDs.
For instance if the number of shards is equal to 2 and you request 4 slices, the slices 0 and 2 are assigned to the first shard and the slices 1 and 3 are assigned to the second shard.
IMPORTANT: The same point-in-time ID should be used for all slices. If different PIT IDs are used, slices can overlap and miss documents. This situation can occur because the splitting criterion is based on Lucene document IDs, which are not stable across changes to the index.
Query parameters
-
allow_no_indices
boolean If
false
, the request returns an error if any wildcard expression, index alias, or_all
value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targetingfoo*,bar*
returns an error if an index starts withfoo
but no index starts withbar
. -
allow_partial_search_results
boolean If
true
and there are shard request timeouts or shard failures, the request returns partial results. Iffalse
, it returns an error with no partial results.To override the default behavior, you can set the
search.default_allow_partial_results
cluster setting tofalse
. -
analyzer
string The analyzer to use for the query string. This parameter can be used only when the
q
query string parameter is specified. -
analyze_wildcard
boolean If
true
, wildcard and prefix queries are analyzed. This parameter can be used only when theq
query string parameter is specified. -
batched_reduce_size
number The number of shard results that should be reduced at once on the coordinating node. If the potential number of shards in the request can be large, this value should be used as a protection mechanism to reduce the memory overhead per search request.
-
ccs_minimize_roundtrips
boolean If
true
, network round-trips between the coordinating node and the remote clusters are minimized when running cross-cluster search (CCS) requests. -
default_operator
string The default operator for the query string query:
AND
orOR
. This parameter can be used only when theq
query string parameter is specified.Values are
and
,AND
,or
, orOR
. -
df
string The field to use as a default when no field prefix is given in the query string. This parameter can be used only when the
q
query string parameter is specified. -
docvalue_fields
string | array[string] A comma-separated list of fields to return as the docvalue representation of a field for each hit.
-
expand_wildcards
string | array[string] The type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. It supports comma-separated values such as
open,hidden
.Supported values include:
all
: Match any data stream or index, including hidden ones.open
: Match open, non-hidden indices. Also matches any non-hidden data stream.closed
: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden
: Match hidden data streams and hidden indices. Must be combined withopen
,closed
, orboth
.none
: Wildcard expressions are not accepted.
Values are
all
,open
,closed
,hidden
, ornone
. -
explain
boolean If
true
, the request returns detailed information about score computation as part of a hit. -
ignore_throttled
boolean Deprecated If
true
, concrete, expanded or aliased indices will be ignored when frozen. -
include_named_queries_score
boolean If
true
, the response includes the score contribution from any named queries.This functionality reruns each named query on every hit in a search response. Typically, this adds a small overhead to a request. However, using computationally expensive named queries on a large number of hits may add significant overhead.
-
lenient
boolean If
true
, format-based query failures (such as providing text to a numeric field) in the query string will be ignored. This parameter can be used only when theq
query string parameter is specified. -
The number of concurrent shard requests per node that the search runs concurrently. This value should be used to limit the impact of the search on the cluster in order to limit the number of concurrent shard requests.
-
preference
string The nodes and shards used for the search. By default, Elasticsearch selects from eligible nodes and shards using adaptive replica selection, accounting for allocation awareness. Valid values are:
_only_local
to run the search only on shards on the local node._local
to, if possible, run the search on shards on the local node, or if not, select shards using the default method._only_nodes:<node-id>,<node-id>
to run the search on only the specified nodes IDs. If suitable shards exist on more than one selected node, use shards on those nodes using the default method. If none of the specified nodes are available, select shards from any available node using the default method._prefer_nodes:<node-id>,<node-id>
to if possible, run the search on the specified nodes IDs. If not, select shards using the default method._shards:<shard>,<shard>
to run the search only on the specified shards. You can combine this value with otherpreference
values. However, the_shards
value must come first. For example:_shards:2,3|_local
.<custom-string>
(any string that does not start with_
) to route searches with the same<custom-string>
to the same shards in the same order.
-
pre_filter_shard_size
number A threshold that enforces a pre-filter roundtrip to prefilter search shards based on query rewriting if the number of shards the search request expands to exceeds the threshold. This filter roundtrip can limit the number of shards significantly if for instance a shard can not match any documents based on its rewrite method (if date filters are mandatory to match but the shard bounds and the query are disjoint). When unspecified, the pre-filter phase is executed if any of these conditions is met:
- The request targets more than 128 shards.
- The request targets one or more read-only index.
- The primary sort of the query targets an indexed field.
-
request_cache
boolean If
true
, the caching of search results is enabled for requests wheresize
is0
. It defaults to index level settings. -
routing
string A custom value that is used to route operations to a specific shard.
-
scroll
string The period to retain the search context for scrolling. By default, this value cannot exceed
1d
(24 hours). You can change this limit by using thesearch.max_keep_alive
cluster-level setting.Values are
-1
or0
. -
search_type
string Indicates how distributed term frequencies are calculated for relevance scoring.
Supported values include:
query_then_fetch
: Documents are scored using local term and document frequencies for the shard. This is usually faster but less accurate.dfs_query_then_fetch
: Documents are scored using global term and document frequencies across all shards. This is usually slower but more accurate.
Values are
query_then_fetch
ordfs_query_then_fetch
. -
stats
array[string] Specific
tag
of the request for logging and statistical purposes. -
stored_fields
string | array[string] A comma-separated list of stored fields to return as part of a hit. If no fields are specified, no stored fields are included in the response. If this field is specified, the
_source
parameter defaults tofalse
. You can pass_source: true
to return both source fields and stored fields in the search response. -
suggest_field
string The field to use for suggestions.
-
suggest_mode
string The suggest mode. This parameter can be used only when the
suggest_field
andsuggest_text
query string parameters are specified.Supported values include:
missing
: Only generate suggestions for terms that are not in the shard.popular
: Only suggest terms that occur in more docs on the shard than the original term.always
: Suggest any matching suggestions based on terms in the suggest text.
Values are
missing
,popular
, oralways
. -
suggest_size
number The number of suggestions to return. This parameter can be used only when the
suggest_field
andsuggest_text
query string parameters are specified. -
suggest_text
string The source text for which the suggestions should be returned. This parameter can be used only when the
suggest_field
andsuggest_text
query string parameters are specified. -
terminate_after
number The maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting.
IMPORTANT: Use with caution. Elasticsearch applies this parameter to each shard handling the request. When possible, let Elasticsearch perform early termination automatically. Avoid specifying this parameter for requests that target data streams with backing indices across multiple data tiers. If set to
0
(default), the query does not terminate early. -
timeout
string The period of time to wait for a response from each shard. If no response is received before the timeout expires, the request fails and returns an error. It defaults to no timeout.
Values are
-1
or0
. -
track_total_hits
boolean | number The number of hits matching the query to count accurately. If
true
, the exact number of hits is returned at the cost of some performance. Iffalse
, the response does not include the total number of hits matching the query. -
track_scores
boolean If
true
, the request calculates and returns document scores, even if the scores are not used for sorting. -
typed_keys
boolean If
true
, aggregation and suggester names are be prefixed by their respective types in the response. -
rest_total_hits_as_int
boolean Indicates whether
hits.total
should be rendered as an integer or an object in the rest search response. -
version
boolean If
true
, the request returns the document version as part of a hit. -
_source
boolean | string | array[string] The source fields that are returned for matching documents. These fields are returned in the
hits._source
property of the search response. Valid values are:true
to return the entire document source.false
to not return the document source.<string>
to return the source fields that are specified as a comma-separated list that supports wildcard (*
) patterns.
-
_source_excludes
string | array[string] A comma-separated list of source fields to exclude from the response. You can also use this parameter to exclude fields from the subset specified in
_source_includes
query parameter. If the_source
parameter isfalse
, this parameter is ignored. -
_source_includes
string | array[string] A comma-separated list of source fields to include in the response. If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the
_source_excludes
query parameter. If the_source
parameter isfalse
, this parameter is ignored. -
seq_no_primary_term
boolean If
true
, the request returns the sequence number and primary term of the last modification of each hit. -
q
string A query in the Lucene query string syntax. Query parameter searches do not support the full Elasticsearch Query DSL but are handy for testing.
IMPORTANT: This parameter overrides the query parameter in the request body. If both parameters are specified, documents matching the query request body parameter are not returned.
-
size
number The number of hits to return. By default, you cannot page through more than 10,000 hits using the
from
andsize
parameters. To page through more hits, use thesearch_after
parameter. -
from
number The starting document offset, which must be non-negative. By default, you cannot page through more than 10,000 hits using the
from
andsize
parameters. To page through more hits, use thesearch_after
parameter. -
sort
string | array[string] A comma-separated list of
<field>:<direction>
pairs.
Body
-
aggregations
object Defines the aggregations that are run as part of the search request.
External documentation -
collapse
object External documentation -
explain
boolean If
true
, the request returns detailed information about score computation as part of a hit. -
ext
object Configuration of search extensions defined by Elasticsearch plugins.
-
from
number The starting document offset, which must be non-negative. By default, you cannot page through more than 10,000 hits using the
from
andsize
parameters. To page through more hits, use thesearch_after
parameter. -
highlight
object -
track_total_hits
boolean | number Number of hits matching the query to count accurately. If true, the exact number of hits is returned at the cost of some performance. If false, the response does not include the total number of hits matching the query. Defaults to 10,000 hits.
-
indices_boost
array[object] Boost the
_score
of documents from specified indices. The boost value is the factor by which scores are multiplied. A boost value greater than1.0
increases the score. A boost value between0
and1.0
decreases the score.External documentation -
docvalue_fields
array[object] An array of wildcard (
*
) field patterns. The request returns doc values for field names matching these patterns in thehits.fields
property of the response.External documentation knn
object | array[object] The approximate kNN search to run.
-
rank
object -
min_score
number The minimum
_score
for matching documents. Documents with a lower_score
are not included in search results and results collected by aggregations. -
post_filter
object An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
External documentation -
profile
boolean Set to
true
to return detailed timing information about the execution of individual components in a search request. NOTE: This is a debugging tool and adds significant overhead to search execution. -
query
object An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
External documentation rescore
object | array[object] Can be used to improve precision by reordering just the top (for example 100 - 500) documents returned by the
query
andpost_filter
phases.-
retriever
object -
script_fields
object Retrieve a script evaluation (based on different fields) for each hit.
-
search_after
array[number | string | boolean | null] A field value.
-
size
number The number of hits to return, which must not be negative. By default, you cannot page through more than 10,000 hits using the
from
andsize
parameters. To page through more hits, use thesearch_after
property. -
slice
object _source
boolean | object Defines how to fetch a source. Fetching can be disabled entirely, or the source can be filtered.
-
fields
array[object] An array of wildcard (
*
) field patterns. The request returns values for field names matching these patterns in thehits.fields
property of the response. -
suggest
object -
terminate_after
number The maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting.
IMPORTANT: Use with caution. Elasticsearch applies this property to each shard handling the request. When possible, let Elasticsearch perform early termination automatically. Avoid specifying this property for requests that target data streams with backing indices across multiple data tiers.
If set to
0
(default), the query does not terminate early. -
timeout
string The period of time to wait for a response from each shard. If no response is received before the timeout expires, the request fails and returns an error. Defaults to no timeout.
-
track_scores
boolean If
true
, calculate and return document scores, even if the scores are not used for sorting. -
version
boolean If
true
, the request returns the document version as part of a hit. -
seq_no_primary_term
boolean If
true
, the request returns sequence number and primary term of the last modification of each hit.External documentation -
stored_fields
string | array[string] -
pit
object -
runtime_mappings
object -
stats
array[string] The stats groups to associate with the search. Each group maintains a statistics aggregation for its associated searches. You can retrieve these stats using the indices stats API.
GET /my-index-000001/_search?from=40&size=20
{
"query": {
"term": {
"user.id": "kimchy"
}
}
}
curl \
--request POST 'http://api.example.com/_search' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"query\": {\n \"term\": {\n \"user.id\": \"kimchy\"\n }\n }\n}"'
{
"query": {
"term": {
"user.id": "kimchy"
}
}
}
{
"size": 100,
"query": {
"match" : {
"title" : "elasticsearch"
}
},
"pit": {
"id": "46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA==",
"keep_alive": "1m"
}
}
{
"slice": {
"id": 0,
"max": 2
},
"query": {
"match": {
"message": "foo"
}
},
"pit": {
"id": "46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA=="
}
}
{
"took": 5,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 20,
"relation": "eq"
},
"max_score": 1.3862942,
"hits": [
{
"_index": "my-index-000001",
"_id": "0",
"_score": 1.3862942,
"_source": {
"@timestamp": "2099-11-15T14:12:12",
"http": {
"request": {
"method": "get"
},
"response": {
"status_code": 200,
"bytes": 1070000
},
"version": "1.1"
},
"source": {
"ip": "127.0.0.1"
},
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}
}
]
}
}
Get terms in an index
Added in 7.14.0
Discover terms that match a partial string in an index. This API is designed for low-latency look-ups used in auto-complete scenarios.
The terms enum API may return terms from deleted documents. Deleted documents are initially only marked as deleted. It is not until their segments are merged that documents are actually deleted. Until that happens, the terms enum API will return terms from these documents.
Path parameters
-
index
string Required A comma-separated list of data streams, indices, and index aliases to search. Wildcard (
*
) expressions are supported. To search all data streams or indices, omit this parameter or use*
or_all
.
Body
-
field
string Required Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
-
size
number The number of matching terms to return.
-
timeout
string A duration. Units can be
nanos
,micros
,ms
(milliseconds),s
(seconds),m
(minutes),h
(hours) andd
(days). Also accepts "0" without a unit and "-1" to indicate an unspecified value. -
case_insensitive
boolean When
true
, the provided search string is matched against index terms without case sensitivity. -
index_filter
object An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
External documentation -
string
string The string to match at the start of indexed terms. If it is not provided, all terms in the field are considered.
The prefix string cannot be larger than the largest possible keyword value, which is Lucene's term byte-length limit of 32766.
-
search_after
string The string after which terms in the index should be returned. It allows for a form of pagination if the last result from one request is passed as the
search_after
parameter for a subsequent request.
POST stackoverflow/_terms_enum
{
"field" : "tags",
"string" : "kiba"
}
curl \
--request POST 'http://api.example.com/{index}/_terms_enum' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"field\" : \"tags\",\n \"string\" : \"kiba\"\n}"'
{
"field" : "tags",
"string" : "kiba"
}
{
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"terms": [
"kibana"
],
"complete" : true
}
Get cache statistics
Technical preview
Get statistics about the shared cache for partially mounted indices.
Query parameters
-
master_timeout
string A duration. Units can be
nanos
,micros
,ms
(milliseconds),s
(seconds),m
(minutes),h
(hours) andd
(days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.Values are
-1
or0
.
GET /_searchable_snapshots/cache/stats
curl \
--request GET 'http://api.example.com/_searchable_snapshots/cache/stats' \
--header "Authorization: $API_KEY"
{
"nodes" : {
"eerrtBMtQEisohZzxBLUSw" : {
"shared_cache" : {
"reads" : 6051,
"bytes_read_in_bytes" : 5448829,
"writes" : 37,
"bytes_written_in_bytes" : 1208320,
"evictions" : 5,
"num_regions" : 65536,
"size_in_bytes" : 1099511627776,
"region_size_in_bytes" : 16777216
}
}
}
}
Clear the privileges cache
Added in 7.9.0
Evict privileges from the native application privilege cache. The cache is also automatically cleared for applications that have their privileges updated.
Path parameters
-
application
string Required A comma-separated list of applications. To clear all applications, use an asterism (
*
). It does not support other wildcard patterns.
curl \
--request POST 'http://api.example.com/_security/privilege/{application}/_clear_cache' \
--header "Authorization: $API_KEY"
Create an API key
Added in 6.7.0
Create an API key for access without requiring basic authentication.
IMPORTANT: If the credential that is used to authenticate this request is an API key, the derived API key cannot have any privileges. If you specify privileges, the API returns an error.
A successful request returns a JSON structure that contains the API key, its unique id, and its name. If applicable, it also returns expiration information for the API key in milliseconds.
NOTE: By default, API keys never expire. You can specify expiration information when you create the API keys.
The API keys are created by the Elasticsearch API key service, which is automatically enabled. To configure or turn off the API key service, refer to API key service setting documentation.
Query parameters
-
refresh
string If
true
(the default) then refresh the affected shards to make this operation visible to search, ifwait_for
then wait for a refresh to make this operation visible to search, iffalse
then do nothing with refreshes.Values are
true
,false
, orwait_for
.
Body
Required
-
expiration
string A duration. Units can be
nanos
,micros
,ms
(milliseconds),s
(seconds),m
(minutes),h
(hours) andd
(days). Also accepts "0" without a unit and "-1" to indicate an unspecified value. -
name
string -
role_descriptors
object An array of role descriptors for this API key. When it is not specified or it is an empty array, the API key will have a point in time snapshot of permissions of the authenticated user. If you supply role descriptors, the resultant permissions are an intersection of API keys permissions and the authenticated user's permissions thereby limiting the access scope for API keys. The structure of role descriptor is the same as the request for the create role API. For more details, refer to the create or update roles API.
NOTE: Due to the way in which this permission intersection is calculated, it is not possible to create an API key that is a child of another API key, unless the derived key is created without any privileges. In this case, you must explicitly specify a role descriptor with no privileges. The derived API key can be used for authentication; it will not have authority to call Elasticsearch APIs.
External documentation -
metadata
object
POST /_security/api_key
{
"name": "my-api-key",
"expiration": "1d",
"role_descriptors": {
"role-a": {
"cluster": ["all"],
"indices": [
{
"names": ["index-a*"],
"privileges": ["read"]
}
]
},
"role-b": {
"cluster": ["all"],
"indices": [
{
"names": ["index-b*"],
"privileges": ["all"]
}
]
}
},
"metadata": {
"application": "my-application",
"environment": {
"level": 1,
"trusted": true,
"tags": ["dev", "staging"]
}
}
}
curl \
--request PUT 'http://api.example.com/_security/api_key' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"name\": \"my-api-key\",\n \"expiration\": \"1d\", \n \"role_descriptors\": { \n \"role-a\": {\n \"cluster\": [\"all\"],\n \"indices\": [\n {\n \"names\": [\"index-a*\"],\n \"privileges\": [\"read\"]\n }\n ]\n },\n \"role-b\": {\n \"cluster\": [\"all\"],\n \"indices\": [\n {\n \"names\": [\"index-b*\"],\n \"privileges\": [\"all\"]\n }\n ]\n }\n },\n \"metadata\": {\n \"application\": \"my-application\",\n \"environment\": {\n \"level\": 1,\n \"trusted\": true,\n \"tags\": [\"dev\", \"staging\"]\n }\n }\n}"'
{
"name": "my-api-key",
"expiration": "1d",
"role_descriptors": {
"role-a": {
"cluster": ["all"],
"indices": [
{
"names": ["index-a*"],
"privileges": ["read"]
}
]
},
"role-b": {
"cluster": ["all"],
"indices": [
{
"names": ["index-b*"],
"privileges": ["all"]
}
]
}
},
"metadata": {
"application": "my-application",
"environment": {
"level": 1,
"trusted": true,
"tags": ["dev", "staging"]
}
}
}
{
"id": "VuaCfGcBCdbkQm-e5aOx",
"name": "my-api-key",
"expiration": 1544068612110,
"api_key": "ui2lp2axTNmsyakw9tvNnw",
"encoded": "VnVhQ2ZHY0JDZGJrUW0tZTVhT3g6dWkybHAyYXhUTm1zeWFrdzl0dk5udw=="
}
Get roles
Get roles in the native realm. The role management APIs are generally the preferred way to manage roles, rather than using file-based role management. The get roles API cannot retrieve roles that are defined in roles files.
Path parameters
-
name
string | array[string] Required The name of the role. You can specify multiple roles as a comma-separated list. If you do not specify this parameter, the API returns information about all roles.
GET /_security/role/my_admin_role
curl \
--request GET 'http://api.example.com/_security/role/{name}' \
--header "Authorization: $API_KEY"
{
"my_admin_role": {
"description": "Grants full access to all management features within the cluster.",
"cluster" : [ "all" ],
"indices" : [
{
"names" : [ "index1", "index2" ],
"privileges" : [ "all" ],
"allow_restricted_indices" : false,
"field_security" : {
"grant" : [ "title", "body" ]}
}
],
"applications" : [ ],
"run_as" : [ "other_user" ],
"metadata" : {
"version" : 1
},
"transient_metadata": {
"enabled": true
}
}
}
Enroll Kibana
Added in 8.0.0
Enable a Kibana instance to configure itself for communication with a secured Elasticsearch cluster.
NOTE: This API is currently intended for internal use only by Kibana. Kibana uses this API internally to configure itself for communications with an Elasticsearch cluster that already has security features enabled.
GET /_security/enroll/kibana
curl \
--request GET 'http://api.example.com/_security/enroll/kibana' \
--header "Authorization: $API_KEY"
{
"token" : {
"name" : "enroll-process-token-1629123923000",
"value": "AAEAAWVsYXN0aWM...vZmxlZXQtc2VydmVyL3Rva2VuMTo3TFdaSDZ"
},
"http_ca" : "MIIJlAIBAzVoGCSqGSIb3...vsDfsA3UZBAjEPfhubpQysAICAA=",
}
Get a user profile
Added in 8.2.0
Get a user's profile using the unique profile ID.
NOTE: The user profile feature is designed only for use by Kibana and Elastic's Observability, Enterprise Search, and Elastic Security solutions. Individual users and external applications should not call this API directly. Elastic reserves the right to change or remove this feature in future releases without prior notice.
Path parameters
-
uid
string | array[string] Required A unique identifier for the user profile.
Query parameters
-
data
string | array[string] A comma-separated list of filters for the
data
field of the profile document. To return all content usedata=*
. To return a subset of content usedata=<key>
to retrieve content nested under the specified<key>
. By default returns nodata
content.
GET /_security/profile/u_79HkWkwmnBH5gqFKwoxggWPjEBOur1zLPXQPEl1VBW0_0
curl \
--request GET 'http://api.example.com/_security/profile/{uid}' \
--header "Authorization: $API_KEY"
{
"profiles": [
{
"uid": "u_79HkWkwmnBH5gqFKwoxggWPjEBOur1zLPXQPEl1VBW0_0",
"enabled": true,
"last_synchronized": 1642650651037,
"user": {
"username": "jacknich",
"roles": [
"admin", "other_role1"
],
"realm_name": "native",
"full_name": "Jack Nicholson",
"email": "jacknich@example.com"
},
"labels": {
"direction": "north"
},
"data": {},
"_doc": {
"_primary_term": 88,
"_seq_no": 66
}
}
]
}
{
"profiles": [
{
"uid": "u_79HkWkwmnBH5gqFKwoxggWPjEBOur1zLPXQPEl1VBW0_0",
"enabled": true,
"last_synchronized": 1642650651037,
"user": {
"username": "jacknich",
"roles": [
"admin", "other_role1"
],
"realm_name": "native",
"full_name": "Jack Nicholson",
"email": "jacknich@example.com"
},
"labels": {
"direction": "north"
},
"data": {
"app1": {
"key1": "value1"
}
},
"_doc": {
"_primary_term": 88,
"_seq_no": 66
}
}
]
}
{
"profiles": [],
"errors": {
"count": 1,
"details": {
"u_FmxQt3gr1BBH5wpnz9HkouPj3Q710XkOgg1PWkwLPBW_5": {
"type": "resource_not_found_exception",
"reason": "profile document not found"
}
}
}
}
Check user privileges
Added in 6.4.0
Determine whether the specified user has a specified list of privileges. All users can use this API, but only to determine their own privileges. To check the privileges of other users, you must use the run as feature.
Body
Required
-
application
array[object] -
cluster
array[string] A list of the cluster privileges that you want to check.
-
index
array[object]
GET /_security/user/_has_privileges
{
"cluster": [ "monitor", "manage" ],
"index" : [
{
"names": [ "suppliers", "products" ],
"privileges": [ "read" ]
},
{
"names": [ "inventory" ],
"privileges" : [ "read", "write" ]
}
],
"application": [
{
"application": "inventory_manager",
"privileges" : [ "read", "data:write/inventory" ],
"resources" : [ "product/1852563" ]
}
]
}
curl \
--request GET 'http://api.example.com/_security/user/_has_privileges' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"cluster\": [ \"monitor\", \"manage\" ],\n \"index\" : [\n {\n \"names\": [ \"suppliers\", \"products\" ],\n \"privileges\": [ \"read\" ]\n },\n {\n \"names\": [ \"inventory\" ],\n \"privileges\" : [ \"read\", \"write\" ]\n }\n ],\n \"application\": [\n {\n \"application\": \"inventory_manager\",\n \"privileges\" : [ \"read\", \"data:write/inventory\" ],\n \"resources\" : [ \"product/1852563\" ]\n }\n ]\n}"'
{
"cluster": [ "monitor", "manage" ],
"index" : [
{
"names": [ "suppliers", "products" ],
"privileges": [ "read" ]
},
{
"names": [ "inventory" ],
"privileges" : [ "read", "write" ]
}
],
"application": [
{
"application": "inventory_manager",
"privileges" : [ "read", "data:write/inventory" ],
"resources" : [ "product/1852563" ]
}
]
}
{
"username": "rdeniro",
"has_all_requested" : false,
"cluster" : {
"monitor" : true,
"manage" : false
},
"index" : {
"suppliers" : {
"read" : true
},
"products" : {
"read" : true
},
"inventory" : {
"read" : true,
"write" : false
}
},
"application" : {
"inventory_manager" : {
"product/1852563" : {
"read": false,
"data:write/inventory": false
}
}
}
}
Logout of OpenID Connect
Invalidate an access token and a refresh token that were generated as a response to the /_security/oidc/authenticate
API.
If the OpenID Connect authentication realm in Elasticsearch is accordingly configured, the response to this call will contain a URI pointing to the end session endpoint of the OpenID Connect Provider in order to perform single logout.
Elasticsearch exposes all the necessary OpenID Connect related functionality with the OpenID Connect APIs. These APIs are used internally by Kibana in order to provide OpenID Connect based authentication, but can also be used by other, custom web applications or other clients.
Body
Required
-
token
string Required The access token to be invalidated.
-
refresh_token
string The refresh token to be invalidated.
POST /_security/oidc/logout
{
"token" : "dGhpcyBpcyBub3QgYSByZWFsIHRva2VuIGJ1dCBpdCBpcyBvbmx5IHRlc3QgZGF0YS4gZG8gbm90IHRyeSB0byByZWFkIHRva2VuIQ==",
"refresh_token": "vLBPvmAB6KvwvJZr27cS"
}
curl \
--request POST 'http://api.example.com/_security/oidc/logout' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"token\" : \"dGhpcyBpcyBub3QgYSByZWFsIHRva2VuIGJ1dCBpdCBpcyBvbmx5IHRlc3QgZGF0YS4gZG8gbm90IHRyeSB0byByZWFkIHRva2VuIQ==\",\n \"refresh_token\": \"vLBPvmAB6KvwvJZr27cS\"\n}"'
{
"token" : "dGhpcyBpcyBub3QgYSByZWFsIHRva2VuIGJ1dCBpdCBpcyBvbmx5IHRlc3QgZGF0YS4gZG8gbm90IHRyeSB0byByZWFkIHRva2VuIQ==",
"refresh_token": "vLBPvmAB6KvwvJZr27cS"
}
{
"redirect" : "https://op-provider.org/logout?id_token_hint=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c&post_logout_redirect_uri=http%3A%2F%2Foidc-kibana.elastic.co%2Floggedout&state=lGYK0EcSLjqH6pkT5EVZjC6eIW5YCGgywj2sxROO"
}
Update user profile data
Added in 8.2.0
Update specific data for the user profile that is associated with a unique ID.
NOTE: The user profile feature is designed only for use by Kibana and Elastic's Observability, Enterprise Search, and Elastic Security solutions. Individual users and external applications should not call this API directly. Elastic reserves the right to change or remove this feature in future releases without prior notice.
To use this API, you must have one of the following privileges:
- The
manage_user_profile
cluster privilege. - The
update_profile_data
global privilege for the namespaces that are referenced in the request.
This API updates the labels
and data
fields of an existing user profile document with JSON objects.
New keys and their values are added to the profile document and conflicting keys are replaced by data that's included in the request.
For both labels and data, content is namespaced by the top-level fields.
The update_profile_data
global privilege grants privileges for updating only the allowed namespaces.
Path parameters
-
uid
string Required A unique identifier for the user profile.
Query parameters
-
if_seq_no
number Only perform the operation if the document has this sequence number.
-
if_primary_term
number Only perform the operation if the document has this primary term.
-
refresh
string If 'true', Elasticsearch refreshes the affected shards to make this operation visible to search. If 'wait_for', it waits for a refresh to make this operation visible to search. If 'false', nothing is done with refreshes.
Values are
true
,false
, orwait_for
.
Body
Required
-
labels
object Searchable data that you want to associate with the user profile. This field supports a nested data structure. Within the labels object, top-level keys cannot begin with an underscore (
_
) or contain a period (.
). -
data
object Non-searchable data that you want to associate with the user profile. This field supports a nested data structure. Within the
data
object, top-level keys cannot begin with an underscore (_
) or contain a period (.
). The data object is not searchable, but can be retrieved with the get user profile API.
POST /_security/profile/u_P_0BMHgaOK3p7k-PFWUCbw9dQ-UFjt01oWJ_Dp2PmPc_0/_data
{
"labels": {
"direction": "east"
},
"data": {
"app1": {
"theme": "default"
}
}
}
curl \
--request PUT 'http://api.example.com/_security/profile/{uid}/_data' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"labels\": {\n \"direction\": \"east\"\n },\n \"data\": {\n \"app1\": {\n \"theme\": \"default\"\n }\n }\n}"'
{
"labels": {
"direction": "east"
},
"data": {
"app1": {
"theme": "default"
}
}
}
{
"acknowledged": true
}
Snapshot and restore
Snapshot and restore APIs enable you to set up snapshot repositories, manage snapshot backups, and restore snapshots to a running cluster.
Create a snapshot
Added in 0.0.0
Take a snapshot of a cluster or of data streams and indices.
Path parameters
-
repository
string Required The name of the repository for the snapshot.
-
snapshot
string Required The name of the snapshot. It supportes date math. It must be unique in the repository.
Query parameters
-
master_timeout
string The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1
or0
. -
wait_for_completion
boolean If
true
, the request returns a response when the snapshot is complete. Iffalse
, the request returns a response when the snapshot initializes.
Body
-
expand_wildcards
string | array[string] -
feature_states
array[string] The feature states to include in the snapshot. Each feature state includes one or more system indices containing related data. You can view a list of eligible features using the get features API.
If
include_global_state
istrue
, all current feature states are included by default. Ifinclude_global_state
isfalse
, no feature states are included by default.Note that specifying an empty array will result in the default behavior. To exclude all feature states, regardless of the
include_global_state
value, specify an array with only the valuenone
(["none"]
). -
include_global_state
boolean If
true
, the current cluster state is included in the snapshot. The cluster state includes persistent cluster settings, composable index templates, legacy index templates, ingest pipelines, and ILM policies. It also includes data stored in system indices, such as Watches and task records (configurable viafeature_states
). -
indices
string | array[string] -
metadata
object -
partial
boolean If
true
, it enables you to restore a partial snapshot of indices with unavailable shards. Only shards that were successfully included in the snapshot will be restored. All missing shards will be recreated as empty.If
false
, the entire restore operation will fail if one or more indices included in the snapshot do not have all primary shards available.
PUT /_snapshot/my_repository/snapshot_2?wait_for_completion=true
{
"indices": "index_1,index_2",
"ignore_unavailable": true,
"include_global_state": false,
"metadata": {
"taken_by": "user123",
"taken_because": "backup before upgrading"
}
}
curl \
--request PUT 'http://api.example.com/_snapshot/{repository}/{snapshot}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"indices\": \"index_1,index_2\",\n \"ignore_unavailable\": true,\n \"include_global_state\": false,\n \"metadata\": {\n \"taken_by\": \"user123\",\n \"taken_because\": \"backup before upgrading\"\n }\n}"'
{
"indices": "index_1,index_2",
"ignore_unavailable": true,
"include_global_state": false,
"metadata": {
"taken_by": "user123",
"taken_because": "backup before upgrading"
}
}
{
"snapshot": {
"snapshot": "snapshot_2",
"uuid": "vdRctLCxSketdKb54xw67g",
"repository": "my_repository",
"version_id": <version_id>,
"version": <version>,
"indices": [],
"data_streams": [],
"feature_states": [],
"include_global_state": false,
"metadata": {
"taken_by": "user123",
"taken_because": "backup before upgrading"
},
"state": "SUCCESS",
"start_time": "2020-06-25T14:00:28.850Z",
"start_time_in_millis": 1593093628850,
"end_time": "2020-06-25T14:00:28.850Z",
"end_time_in_millis": 1593094752018,
"duration_in_millis": 0,
"failures": [],
"shards": {
"total": 0,
"failed": 0,
"successful": 0
}
}
}
Verify the repository integrity
Technical preview
Verify the integrity of the contents of a snapshot repository.
This API enables you to perform a comprehensive check of the contents of a repository, looking for any anomalies in its data or metadata which might prevent you from restoring snapshots from the repository or which might cause future snapshot create or delete operations to fail.
If you suspect the integrity of the contents of one of your snapshot repositories, cease all write activity to this repository immediately, set its read_only
option to true
, and use this API to verify its integrity.
Until you do so:
- It may not be possible to restore some snapshots from this repository.
- Searchable snapshots may report errors when searched or may have unassigned shards.
- Taking snapshots into this repository may fail or may appear to succeed but have created a snapshot which cannot be restored.
- Deleting snapshots from this repository may fail or may appear to succeed but leave the underlying data on disk.
- Continuing to write to the repository while it is in an invalid state may causing additional damage to its contents.
If the API finds any problems with the integrity of the contents of your repository, Elasticsearch will not be able to repair the damage. The only way to bring the repository back into a fully working state after its contents have been damaged is by restoring its contents from a repository backup which was taken before the damage occurred. You must also identify what caused the damage and take action to prevent it from happening again.
If you cannot restore a repository backup, register a new repository and use this for all future snapshot operations. In some cases it may be possible to recover some of the contents of a damaged repository, either by restoring as many of its snapshots as needed and taking new snapshots of the restored data, or by using the reindex API to copy data from any searchable snapshots mounted from the damaged repository.
Avoid all operations which write to the repository while the verify repository integrity API is running. If something changes the repository contents while an integrity verification is running then Elasticsearch may incorrectly report having detected some anomalies in its contents due to the concurrent writes. It may also incorrectly fail to report some anomalies that the concurrent writes prevented it from detecting.
NOTE: This API is intended for exploratory use by humans. You should expect the request parameters and the response format to vary in future versions.
NOTE: This API may not work correctly in a mixed-version cluster.
The default values for the parameters of this API are designed to limit the impact of the integrity verification on other activities in your cluster.
For instance, by default it will only use at most half of the snapshot_meta
threads to verify the integrity of each snapshot, allowing other snapshot operations to use the other half of this thread pool.
If you modify these parameters to speed up the verification process, you risk disrupting other snapshot-related operations in your cluster.
For large repositories, consider setting up a separate single-node Elasticsearch cluster just for running the integrity verification API.
The response exposes implementation details of the analysis which may change from version to version. The response body format is therefore not considered stable and may be different in newer versions.
Path parameters
-
repository
string | array[string] Required The name of the snapshot repository.
Query parameters
-
blob_thread_pool_concurrency
number If
verify_blob_contents
istrue
, this parameter specifies how many blobs to verify at once. -
The maximum number of index snapshots to verify concurrently within each index verification.
-
The number of indices to verify concurrently. The default behavior is to use the entire
snapshot_meta
thread pool. -
max_bytes_per_sec
string If
verify_blob_contents
istrue
, this parameter specifies the maximum amount of data that Elasticsearch will read from the repository every second. -
max_failed_shard_snapshots
number The number of shard snapshot failures to track during integrity verification, in order to avoid excessive resource usage. If your repository contains more than this number of shard snapshot failures, the verification will fail.
-
meta_thread_pool_concurrency
number The maximum number of snapshot metadata operations to run concurrently. The default behavior is to use at most half of the
snapshot_meta
thread pool at once. -
The number of snapshots to verify concurrently. The default behavior is to use at most half of the
snapshot_meta
thread pool at once. -
verify_blob_contents
boolean Indicates whether to verify the checksum of every data blob in the repository. If this feature is enabled, Elasticsearch will read the entire repository contents, which may be extremely slow and expensive.
curl \
--request POST 'http://api.example.com/_snapshot/{repository}/_verify_integrity' \
--header "Authorization: $API_KEY"
Run a policy
Added in 7.4.0
Immediately create a snapshot according to the snapshot lifecycle policy without waiting for the scheduled time. The snapshot policy is normally applied according to its schedule, but you might want to manually run a policy before performing an upgrade or other maintenance.
Path parameters
-
policy_id
string Required The id of the snapshot lifecycle policy to be executed
Query parameters
-
master_timeout
string The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1
or0
. -
timeout
string The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1
or0
.
POST /_slm/policy/daily-snapshots/_execute
curl \
--request PUT 'http://api.example.com/_slm/policy/{policy_id}/_execute' \
--header "Authorization: $API_KEY"
{
"snapshot_name": "daily-snap-2019.04.24-gwrqoo2xtea3q57vvg0uea"
}
Get the snapshot lifecycle management status
Added in 7.6.0
Query parameters
-
master_timeout
string The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error. To indicate that the request should never timeout, set it to
-1
.Values are
-1
or0
. -
timeout
string The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error. To indicate that the request should never timeout, set it to
-1
.Values are
-1
or0
.
GET _slm/status
curl \
--request GET 'http://api.example.com/_slm/status' \
--header "Authorization: $API_KEY"
{
"operation_mode": "RUNNING"
}
Query parameters
-
format
string The format for the response. You can also specify a format using the
Accept
HTTP header. If you specify both this parameter and theAccept
HTTP header, this parameter takes precedence.Values are
csv
,json
,tsv
,txt
,yaml
,cbor
, orsmile
.
Body
Required
-
allow_partial_search_results
boolean If
true
, the response has partial results when there are shard request timeouts or shard failures. Iffalse
, the API returns an error with no partial results. -
catalog
string The default catalog (cluster) for queries. If unspecified, the queries execute on the data in the local cluster only.
-
columnar
boolean If
true
, the results are in a columnar fashion: one row represents all the values of a certain column from the current page of results. The API supports this parameter only for CBOR, JSON, SMILE, and YAML responses.External documentation -
cursor
string The cursor used to retrieve a set of paginated results. If you specify a cursor, the API only uses the
columnar
andtime_zone
request body parameters. It ignores other request body parameters. -
fetch_size
number The maximum number of rows (or entries) to return in one response.
-
field_multi_value_leniency
boolean If
false
, the API returns an exception when encountering multiple values for a field. Iftrue
, the API is lenient and returns the first value from the array with no guarantee of consistent results. -
filter
object An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
External documentation -
index_using_frozen
boolean If
true
, the search can run on frozen indices. -
keep_alive
string A duration. Units can be
nanos
,micros
,ms
(milliseconds),s
(seconds),m
(minutes),h
(hours) andd
(days). Also accepts "0" without a unit and "-1" to indicate an unspecified value. -
keep_on_completion
boolean If
true
, Elasticsearch stores synchronous searches if you also specify thewait_for_completion_timeout
parameter. Iffalse
, Elasticsearch only stores async searches that don't finish before thewait_for_completion_timeout
. -
page_timeout
string A duration. Units can be
nanos
,micros
,ms
(milliseconds),s
(seconds),m
(minutes),h
(hours) andd
(days). Also accepts "0" without a unit and "-1" to indicate an unspecified value. -
params
object The values for parameters in the query.
-
query
string The SQL query to run.
External documentation -
request_timeout
string A duration. Units can be
nanos
,micros
,ms
(milliseconds),s
(seconds),m
(minutes),h
(hours) andd
(days). Also accepts "0" without a unit and "-1" to indicate an unspecified value. -
runtime_mappings
object -
time_zone
string -
wait_for_completion_timeout
string A duration. Units can be
nanos
,micros
,ms
(milliseconds),s
(seconds),m
(minutes),h
(hours) andd
(days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
POST _sql?format=txt
{
"query": "SELECT * FROM library ORDER BY page_count DESC LIMIT 5"
}
curl \
--request POST 'http://api.example.com/_sql' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"query\": \"SELECT * FROM library ORDER BY page_count DESC LIMIT 5\"\n}"'
{
"query": "SELECT * FROM library ORDER BY page_count DESC LIMIT 5"
}
Get a synonym rule
Added in 8.10.0
Get a synonym rule from a synonym set.
GET _synonyms/my-synonyms-set/test-1
curl \
--request GET 'http://api.example.com/_synonyms/{set_id}/{rule_id}' \
--header "Authorization: $API_KEY"
{
"id": "test-1",
"synonyms": "hello, hi"
}
Test a Grok pattern
Added in 8.13.0
Test a Grok pattern on one or more lines of text. The API indicates whether the lines match the pattern together with the offsets and lengths of the matched substrings.
Query parameters
-
ecs_compatibility
string The mode of compatibility with ECS compliant Grok patterns. Use this parameter to specify whether to use ECS Grok patterns instead of legacy ones when the structure finder creates a Grok pattern. Valid values are
disabled
andv1
.
Body
Required
-
grok_pattern
string Required -
text
array[string] Required The lines of text to run the Grok pattern on.
GET _text_structure/test_grok_pattern
{
"grok_pattern": "Hello %{WORD:first_name} %{WORD:last_name}",
"text": [
"Hello John Doe",
"this does not match"
]
}
curl \
--request GET 'http://api.example.com/_text_structure/test_grok_pattern' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"grok_pattern\": \"Hello %{WORD:first_name} %{WORD:last_name}\",\n \"text\": [\n \"Hello John Doe\",\n \"this does not match\"\n ]\n}"'
{
"grok_pattern": "Hello %{WORD:first_name} %{WORD:last_name}",
"text": [
"Hello John Doe",
"this does not match"
]
}
{
"matches": [
{
"matched": true,
"fields": {
"first_name": [
{
"match": "John",
"offset": 6,
"length": 4
}
],
"last_name": [
{
"match": "Doe",
"offset": 11,
"length": 3
}
]
}
},
{
"matched": false
}
]
}
Preview a transform
Added in 7.2.0
Generates a preview of the results that you will get when you create a transform with the same configuration.
It returns a maximum of 100 results. The calculations are based on all the current data in the source index. It also generates a list of mappings and settings for the destination index. These values are determined based on the field types of the source index and the transform aggregations.
Path parameters
-
transform_id
string Required Identifier for the transform to preview. If you specify this path parameter, you cannot provide transform configuration details in the request body.
Query parameters
-
timeout
string Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1
or0
.
Body
-
dest
object -
description
string Free text description of the transform.
-
frequency
string A duration. Units can be
nanos
,micros
,ms
(milliseconds),s
(seconds),m
(minutes),h
(hours) andd
(days). Also accepts "0" without a unit and "-1" to indicate an unspecified value. -
pivot
object -
source
object -
settings
object -
sync
object -
retention_policy
object -
latest
object
POST _transform/_preview
{
"source": {
"index": "kibana_sample_data_ecommerce"
},
"pivot": {
"group_by": {
"customer_id": {
"terms": {
"field": "customer_id",
"missing_bucket": true
}
}
},
"aggregations": {
"max_price": {
"max": {
"field": "taxful_total_price"
}
}
}
}
}
curl \
--request GET 'http://api.example.com/_transform/{transform_id}/_preview' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"source\": {\n \"index\": \"kibana_sample_data_ecommerce\"\n },\n \"pivot\": {\n \"group_by\": {\n \"customer_id\": {\n \"terms\": {\n \"field\": \"customer_id\",\n \"missing_bucket\": true\n }\n }\n },\n \"aggregations\": {\n \"max_price\": {\n \"max\": {\n \"field\": \"taxful_total_price\"\n }\n }\n }\n }\n}"'
{
"source": {
"index": "kibana_sample_data_ecommerce"
},
"pivot": {
"group_by": {
"customer_id": {
"terms": {
"field": "customer_id",
"missing_bucket": true
}
}
},
"aggregations": {
"max_price": {
"max": {
"field": "taxful_total_price"
}
}
}
}
}
{
"preview": [
{
"max_price": 171,
"customer_id": "10"
},
{
"max_price": 233,
"customer_id": "11"
},
{
"max_price": 200,
"customer_id": "12"
},
{
"max_price": 301,
"customer_id": "13"
},
{
"max_price": 176,
"customer_id": "14"
},
{
"max_price": 2250,
"customer_id": "15"
},
{
"max_price": 170,
"customer_id": "16"
},
{
"max_price": 243,
"customer_id": "17"
},
{
"max_price": 154,
"customer_id": "18"
},
{
"max_price": 393,
"customer_id": "19"
},
{
"max_price": 165,
"customer_id": "20"
},
{
"max_price": 115,
"customer_id": "21"
},
{
"max_price": 192,
"customer_id": "22"
},
{
"max_price": 169,
"customer_id": "23"
},
{
"max_price": 230,
"customer_id": "24"
},
{
"max_price": 278,
"customer_id": "25"
},
{
"max_price": 200,
"customer_id": "26"
},
{
"max_price": 344,
"customer_id": "27"
},
{
"max_price": 175,
"customer_id": "28"
},
{
"max_price": 177,
"customer_id": "29"
},
{
"max_price": 190,
"customer_id": "30"
},
{
"max_price": 190,
"customer_id": "31"
},
{
"max_price": 205,
"customer_id": "32"
},
{
"max_price": 215,
"customer_id": "33"
},
{
"max_price": 270,
"customer_id": "34"
},
{
"max_price": 184,
"customer_id": "36"
},
{
"max_price": 222,
"customer_id": "37"
},
{
"max_price": 370,
"customer_id": "38"
},
{
"max_price": 240,
"customer_id": "39"
},
{
"max_price": 230,
"customer_id": "4"
},
{
"max_price": 229,
"customer_id": "41"
},
{
"max_price": 190,
"customer_id": "42"
},
{
"max_price": 150,
"customer_id": "43"
},
{
"max_price": 175,
"customer_id": "44"
},
{
"max_price": 190,
"customer_id": "45"
},
{
"max_price": 150,
"customer_id": "46"
},
{
"max_price": 310,
"customer_id": "48"
},
{
"max_price": 223,
"customer_id": "49"
},
{
"max_price": 283,
"customer_id": "5"
},
{
"max_price": 185,
"customer_id": "50"
},
{
"max_price": 190,
"customer_id": "51"
},
{
"max_price": 333,
"customer_id": "52"
},
{
"max_price": 165,
"customer_id": "6"
},
{
"max_price": 144,
"customer_id": "7"
},
{
"max_price": 198,
"customer_id": "8"
},
{
"max_price": 210,
"customer_id": "9"
}
],
"generated_dest_index": {
"mappings": {
"_meta": {
"_transform": {
"transform": "transform-preview",
"version": {
"created": "10.0.0"
},
"creation_date_in_millis": 1712948905889
},
"created_by": "transform"
},
"properties": {
"max_price": {
"type": "half_float"
},
"customer_id": {
"type": "keyword"
}
}
},
"settings": {
"index": {
"number_of_shards": "1",
"auto_expand_replicas": "0-1"
}
},
"aliases": {}
}
}
Schedule a transform to start now
Added in 8.7.0
Instantly run a transform to process data.
If you run this API, the transform will process the new data instantly,
without waiting for the configured frequency interval. After the API is called,
the transform will be processed again at now + frequency
unless the API
is called again in the meantime.
Path parameters
-
transform_id
string Required Identifier for the transform.
Query parameters
-
timeout
string Controls the time to wait for the scheduling to take place
Values are
-1
or0
.
curl \
--request POST 'http://api.example.com/_transform/{transform_id}/_schedule_now' \
--header "Authorization: $API_KEY"
{
"acknowledged": true
}
Activate a watch
A watch can be either active or inactive.
Path parameters
-
watch_id
string Required The watch identifier.
curl \
--request PUT 'http://api.example.com/_watcher/watch/{watch_id}/_activate' \
--header "Authorization: $API_KEY"
Deactivate a watch
A watch can be either active or inactive.
Path parameters
-
watch_id
string Required The watch identifier.
curl \
--request PUT 'http://api.example.com/_watcher/watch/{watch_id}/_deactivate' \
--header "Authorization: $API_KEY"
Get Watcher statistics
Added in 5.5.0
This API always returns basic metrics. You retrieve more metrics by using the metric parameter.
Path parameters
-
metric
string | array[string] Required Defines which additional metrics are included in the response.
Supported values include:
_all
(orall
),queued_watches
,current_watches
,pending_watches
Values are
_all
,all
,queued_watches
,current_watches
, orpending_watches
.
Query parameters
-
emit_stacktraces
boolean Defines whether stack traces are generated for each watch that is running.
-
metric
string | array[string] Defines which additional metrics are included in the response.
Supported values include:
_all
(orall
),queued_watches
,current_watches
,pending_watches
Values are
_all
,all
,queued_watches
,current_watches
, orpending_watches
.
GET _watcher/stats
curl \
--request GET 'http://api.example.com/_watcher/stats/{metric}' \
--header "Authorization: $API_KEY"
{
"watcher_state": "started",
"watch_count": 1,
"execution_thread_pool": {
"size": 1000,
"max_size": 1
}
}
{
"watcher_state": "started",
"watch_count": 2,
"execution_thread_pool": {
"queue_size": 1000,
"max_size": 20
},
"current_watches": [
{
"watch_id": "slow_condition",
"watch_record_id": "slow_condition_3-2015-05-13T07:42:32.179Z",
"triggered_time": "2015-05-12T11:53:51.800Z",
"execution_time": "2015-05-13T07:42:32.179Z",
"execution_phase": "condition"
}
]
}
{
"watcher_state": "started",
"watch_count": 10,
"execution_thread_pool": {
"queue_size": 1000,
"max_size": 20
},
"queued_watches": [
{
"watch_id": "slow_condition4",
"watch_record_id": "slow_condition4_223-2015-05-21T11:59:59.811Z",
"triggered_time": "2015-05-21T11:59:59.811Z",
"execution_time": "2015-05-21T11:59:59.811Z"
}
]
}