Get node statistics
Get statistics for nodes in a cluster. By default, all stats are returned. You can limit the returned information by using metrics.
Path parameters
-
metric
string | array[string] Required Limit the information returned to the specified metrics
Query parameters
-
completion_fields
string | array[string] Comma-separated list or wildcard expressions of fields to include in fielddata and suggest statistics.
-
fielddata_fields
string | array[string] Comma-separated list or wildcard expressions of fields to include in fielddata statistics.
-
fields
string | array[string] Comma-separated list or wildcard expressions of fields to include in the statistics.
-
groups
boolean Comma-separated list of search groups to include in the search statistics.
-
include_segment_file_sizes
boolean If true, the call reports the aggregated disk usage of each one of the Lucene index files (only applies if segment stats are requested).
-
level
string Indicates whether statistics are aggregated at the cluster, index, or shard level.
Values are
cluster,indices, orshards. -
timeout
string Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1or0. -
types
array[string] A comma-separated list of document types for the indexing index metric.
-
include_unloaded_segments
boolean If
true, the response includes information from segments that are not loaded into memory.
curl \
--request GET 'http://api.example.com/_nodes/stats/{metric}' \
--header "Authorization: $API_KEY"
Check in a connector sync job
Technical preview
Check in a connector sync job and set the last_seen field to the current time before updating it in the internal index.
To sync data using self-managed connectors, you need to deploy the Elastic connector service on your own infrastructure. This service runs automatically on Elastic Cloud for Elastic managed connectors.
Path parameters
-
connector_sync_job_id
string Required The unique identifier of the connector sync job to be checked in.
curl \
--request PUT 'http://api.example.com/_connector/_sync_job/{connector_sync_job_id}/_check_in' \
--header "Authorization: $API_KEY"Get multiple term vectors
Get multiple term vectors with a single request.
You can specify existing documents by index and ID or provide artificial documents in the body of the request.
You can specify the index in the request body or request URI.
The response contains a docs array with all the fetched termvectors.
Each element has the structure provided by the termvectors API.
Artificial documents
You can also use mtermvectors to generate term vectors for artificial documents provided in the body of the request.
The mapping used is determined by the specified _index.
Path parameters
-
index
string Required The name of the index that contains the documents.
Query parameters
-
ids
array[string] A comma-separated list of documents ids. You must define ids as parameter or set "ids" or "docs" in the request body
-
fields
string | array[string] A comma-separated list or wildcard expressions of fields to include in the statistics. It is used as the default list unless a specific field list is provided in the
completion_fieldsorfielddata_fieldsparameters. -
field_statistics
boolean If
true, the response includes the document count, sum of document frequencies, and sum of total term frequencies. -
offsets
boolean If
true, the response includes term offsets. -
payloads
boolean If
true, the response includes term payloads. -
positions
boolean If
true, the response includes term positions. -
preference
string The node or shard the operation should be performed on. It is random by default.
-
realtime
boolean If true, the request is real-time as opposed to near-real-time.
-
routing
string A custom value used to route operations to a specific shard.
-
term_statistics
boolean If true, the response includes term frequency and document frequency.
-
version
number If
true, returns the document version as part of a hit. -
version_type
string The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document. NOTE: Theexternal_gteversion type is meant for special use cases and should be used with care. If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.
Values are
internal,external,external_gte, orforce.
curl \
--request GET 'http://api.example.com/{index}/_mtermvectors' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"docs\": [\n {\n \"_id\": \"2\",\n \"fields\": [\n \"message\"\n ],\n \"term_statistics\": true\n },\n {\n \"_id\": \"1\"\n }\n ]\n}"'{
"docs": [
{
"_id": "2",
"fields": [
"message"
],
"term_statistics": true
},
{
"_id": "1"
}
]
}{
"ids": [ "1", "2" ],
"fields": [
"message"
],
"term_statistics": true
}{
"docs": [
{
"_index": "my-index-000001",
"doc" : {
"message" : "test test test"
}
},
{
"_index": "my-index-000001",
"doc" : {
"message" : "Another test ..."
}
}
]
}
Update documents
Added in 2.4.0
Updates documents that match the specified query. If no query is specified, performs an update on every document in the data stream or index without modifying the source, which is useful for picking up mapping changes.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or alias:
readindexorwrite
You can specify the query criteria in the request URI or the request body using the same syntax as the search API.
When you submit an update by query request, Elasticsearch gets a snapshot of the data stream or index when it begins processing the request and updates matching documents using internal versioning.
When the versions match, the document is updated and the version number is incremented.
If a document changes between the time that the snapshot is taken and the update operation is processed, it results in a version conflict and the operation fails.
You can opt to count version conflicts instead of halting and returning by setting conflicts to proceed.
Note that if you opt to count version conflicts, the operation could attempt to update more documents from the source than max_docs until it has successfully updated max_docs documents or it has gone through every document in the source query.
NOTE: Documents with a version equal to 0 cannot be updated using update by query because internal versioning does not support 0 as a valid version number.
While processing an update by query request, Elasticsearch performs multiple search requests sequentially to find all of the matching documents. A bulk update request is performed for each batch of matching documents. Any query or update failures cause the update by query request to fail and the failures are shown in the response. Any update requests that completed successfully still stick, they are not rolled back.
Throttling update requests
To control the rate at which update by query issues batches of update operations, you can set requests_per_second to any positive decimal number.
This pads each batch with a wait time to throttle the rate.
Set requests_per_second to -1 to turn off throttling.
Throttling uses a wait time between batches so that the internal scroll requests can be given a timeout that takes the request padding into account.
The padding time is the difference between the batch size divided by the requests_per_second and the time spent writing.
By default the batch size is 1000, so if requests_per_second is set to 500:
target_time = 1000 / 500 per second = 2 seconds
wait_time = target_time - write_time = 2 seconds - .5 seconds = 1.5 seconds
Since the batch is issued as a single _bulk request, large batch sizes cause Elasticsearch to create many requests and wait before starting the next set. This is "bursty" instead of "smooth".
Slicing
Update by query supports sliced scroll to parallelize the update process. This can improve efficiency and provide a convenient way to break the request down into smaller parts.
Setting slices to auto chooses a reasonable number for most data streams and indices.
This setting will use one slice per shard, up to a certain limit.
If there are multiple source data streams or indices, it will choose the number of slices based on the index or backing index with the smallest number of shards.
Adding slices to _update_by_query just automates the manual process of creating sub-requests, which means it has some quirks:
- You can see these requests in the tasks APIs. These sub-requests are "child" tasks of the task for the request with slices.
- Fetching the status of the task for the request with
slicesonly contains the status of completed slices. - These sub-requests are individually addressable for things like cancellation and rethrottling.
- Rethrottling the request with
sliceswill rethrottle the unfinished sub-request proportionally. - Canceling the request with slices will cancel each sub-request.
- Due to the nature of slices each sub-request won't get a perfectly even portion of the documents. All documents will be addressed, but some slices may be larger than others. Expect larger slices to have a more even distribution.
- Parameters like
requests_per_secondandmax_docson a request with slices are distributed proportionally to each sub-request. Combine that with the point above about distribution being uneven and you should conclude that usingmax_docswithslicesmight not result in exactlymax_docsdocuments being updated. - Each sub-request gets a slightly different snapshot of the source data stream or index though these are all taken at approximately the same time.
If you're slicing manually or otherwise tuning automatic slicing, keep in mind that:
- Query performance is most efficient when the number of slices is equal to the number of shards in the index or backing index. If that number is large (for example, 500), choose a lower number as too many slices hurts performance. Setting slices higher than the number of shards generally does not improve efficiency and adds overhead.
- Update performance scales linearly across available resources with the number of slices.
Whether query or update performance dominates the runtime depends on the documents being reindexed and cluster resources.
Update the document source
Update by query supports scripts to update the document source.
As with the update API, you can set ctx.op to change the operation that is performed.
Set ctx.op = "noop" if your script decides that it doesn't have to make any changes.
The update by query operation skips updating the document and increments the noop counter.
Set ctx.op = "delete" if your script decides that the document should be deleted.
The update by query operation deletes the document and increments the deleted counter.
Update by query supports only index, noop, and delete.
Setting ctx.op to anything else is an error.
Setting any other field in ctx is an error.
This API enables you to only modify the source of matching documents; you cannot move them.
Path parameters
-
index
string | array[string] Required A comma-separated list of data streams, indices, and aliases to search. It supports wildcards (
*). To search all data streams or indices, omit this parameter or use*or_all.
Query parameters
-
allow_no_indices
boolean If
false, the request returns an error if any wildcard expression, index alias, or_allvalue targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targetingfoo*,bar*returns an error if an index starts withfoobut no index starts withbar. -
analyzer
string The analyzer to use for the query string. This parameter can be used only when the
qquery string parameter is specified. -
analyze_wildcard
boolean If
true, wildcard and prefix queries are analyzed. This parameter can be used only when theqquery string parameter is specified. -
conflicts
string The preferred behavior when update by query hits version conflicts:
abortorproceed.Supported values include:
abort: Stop reindexing if there are conflicts.proceed: Continue reindexing even if there are conflicts.
Values are
abortorproceed. -
default_operator
string The default operator for query string query:
ANDorOR. This parameter can be used only when theqquery string parameter is specified.Values are
and,AND,or, orOR. -
df
string The field to use as default where no field prefix is given in the query string. This parameter can be used only when the
qquery string parameter is specified. -
expand_wildcards
string | array[string] The type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. It supports comma-separated values, such as
open,hidden. Valid values are:all,open,closed,hidden,none.Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined withopen,closed, orboth.none: Wildcard expressions are not accepted.
Values are
all,open,closed,hidden, ornone. -
from
number Skips the specified number of documents.
-
lenient
boolean If
true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored. This parameter can be used only when theqquery string parameter is specified. -
max_docs
number The maximum number of documents to process. It defaults to all documents. When set to a value less then or equal to
scroll_sizethen a scroll will not be used to retrieve the results for the operation. -
pipeline
string The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, then setting the value to
_nonedisables the default ingest pipeline for this request. If a final pipeline is configured it will always run, regardless of the value of this parameter. -
preference
string The node or shard the operation should be performed on. It is random by default.
-
q
string A query in the Lucene query string syntax.
-
refresh
boolean If
true, Elasticsearch refreshes affected shards to make the operation visible to search after the request completes. This is different than the update API'srefreshparameter, which causes just the shard that received the request to be refreshed. -
request_cache
boolean If
true, the request cache is used for this request. It defaults to the index-level setting. -
requests_per_second
number The throttle for this request in sub-requests per second.
-
routing
string A custom value used to route operations to a specific shard.
-
scroll
string The period to retain the search context for scrolling.
Values are
-1or0. -
scroll_size
number The size of the scroll request that powers the operation.
-
search_timeout
string An explicit timeout for each search request. By default, there is no timeout.
Values are
-1or0. -
search_type
string The type of the search operation. Available options include
query_then_fetchanddfs_query_then_fetch.Supported values include:
query_then_fetch: Documents are scored using local term and document frequencies for the shard. This is usually faster but less accurate.dfs_query_then_fetch: Documents are scored using global term and document frequencies across all shards. This is usually slower but more accurate.
Values are
query_then_fetchordfs_query_then_fetch. -
slices
number | string The number of slices this task should be divided into.
Value is
auto. -
sort
array[string] A comma-separated list of : pairs.
-
stats
array[string] The specific
tagof the request for logging and statistical purposes. -
terminate_after
number The maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting.
IMPORTANT: Use with caution. Elasticsearch applies this parameter to each shard handling the request. When possible, let Elasticsearch perform early termination automatically. Avoid specifying this parameter for requests that target data streams with backing indices across multiple data tiers.
-
timeout
string The period each update request waits for the following operations: dynamic mapping updates, waiting for active shards. By default, it is one minute. This guarantees Elasticsearch waits for at least the timeout before failing. The actual wait time could be longer, particularly when multiple waits occur.
Values are
-1or0. -
version
boolean If
true, returns the document version as part of a hit. -
version_type
boolean Should the document increment the version number (internal) on hit or not (reindex)
-
wait_for_active_shards
number | string The number of shard copies that must be active before proceeding with the operation. Set to
allor any positive integer up to the total number of shards in the index (number_of_replicas+1). Thetimeoutparameter controls how long each write request waits for unavailable shards to become available. Both work exactly the way they work in the bulk API.Values are
allorindex-setting. -
wait_for_completion
boolean If
true, the request blocks until the operation is complete. Iffalse, Elasticsearch performs some preflight checks, launches the request, and returns a task ID that you can use to cancel or get the status of the task. Elasticsearch creates a record of this task as a document at.tasks/task/${taskId}.
curl \
--request POST 'http://api.example.com/{index}/_update_by_query' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"query\": { \n \"term\": {\n \"user.id\": \"kimchy\"\n }\n }\n}"'{
"query": {
"term": {
"user.id": "kimchy"
}
}
}{
"script": {
"source": "ctx._source.count++",
"lang": "painless"
},
"query": {
"term": {
"user.id": "kimchy"
}
}
}{
"slice": {
"id": 0,
"max": 2
},
"script": {
"source": "ctx._source['extra'] = 'test'"
}
}{
"script": {
"source": "ctx._source['extra'] = 'test'"
}
}
Check component templates
Added in 7.8.0
Returns information about whether a particular component template exists.
Path parameters
-
name
string | array[string] Required Comma-separated list of component template names used to limit the request. Wildcard (*) expressions are supported.
Query parameters
-
master_timeout
string Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1or0. -
local
boolean If true, the request retrieves information from the local node only. Defaults to false, which means information is retrieved from the master node.
curl \
--request HEAD 'http://api.example.com/_component_template/{name}' \
--header "Authorization: $API_KEY"
Get index statistics
Added in 1.3.0
For data streams, the API retrieves statistics for the stream's backing indices.
By default, the returned statistics are index-level with primaries and total aggregations.
primaries are the values for only the primary shards.
total are the accumulated values for both primary and replica shards.
To get shard-level statistics, set the level parameter to shards.
NOTE: When moving to another node, the shard-level statistics for a shard are cleared. Although the shard is no longer part of the node, that node retains any node-level statistics to which the shard contributed.
Path parameters
-
index
string | array[string] Required A comma-separated list of index names; use
_allor empty string to perform the operation on all indices
Query parameters
-
completion_fields
string | array[string] Comma-separated list or wildcard expressions of fields to include in fielddata and suggest statistics.
-
expand_wildcards
string | array[string] Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as
open,hidden.Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined withopen,closed, orboth.none: Wildcard expressions are not accepted.
Values are
all,open,closed,hidden, ornone. -
fielddata_fields
string | array[string] Comma-separated list or wildcard expressions of fields to include in fielddata statistics.
-
fields
string | array[string] Comma-separated list or wildcard expressions of fields to include in the statistics.
-
forbid_closed_indices
boolean If true, statistics are not collected from closed indices.
-
groups
string | array[string] Comma-separated list of search groups to include in the search statistics.
-
include_segment_file_sizes
boolean If true, the call reports the aggregated disk usage of each one of the Lucene index files (only applies if segment stats are requested).
-
include_unloaded_segments
boolean If true, the response includes information from segments that are not loaded into memory.
-
level
string Indicates whether statistics are aggregated at the cluster, index, or shard level.
Values are
cluster,indices, orshards.
curl \
--request GET 'http://api.example.com/{index}/_stats' \
--header "Authorization: $API_KEY"Query parameters
-
allow_no_indices
boolean If
false, the request returns an error if any wildcard expression, index alias, or_allvalue targets only missing or closed indices. This behavior applies even if the request targets other open indices. -
all_shards
boolean If
true, the validation is executed on all shards instead of one random shard per index. -
analyzer
string Analyzer to use for the query string. This parameter can only be used when the
qquery string parameter is specified. -
analyze_wildcard
boolean If
true, wildcard and prefix queries are analyzed. -
default_operator
string The default operator for query string query:
ANDorOR.Values are
and,AND,or, orOR. -
df
string Field to use as default where no field prefix is given in the query string. This parameter can only be used when the
qquery string parameter is specified. -
expand_wildcards
string | array[string] Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as
open,hidden. Valid values are:all,open,closed,hidden,none.Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined withopen,closed, orboth.none: Wildcard expressions are not accepted.
Values are
all,open,closed,hidden, ornone. -
explain
boolean If
true, the response returns detailed information if an error has occurred. -
lenient
boolean If
true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored. -
rewrite
boolean If
true, returns a more detailed explanation showing the actual Lucene query that will be executed. -
q
string Query in the Lucene query string syntax.
Body
-
query
object An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
External documentation
curl \
--request POST 'http://api.example.com/_validate/query' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"query":{}}'
Create an AlibabaCloud AI Search inference endpoint
Added in 8.16.0
Create an inference endpoint to perform an inference task with the alibabacloud-ai-search service.
Path parameters
-
task_type
string Required The type of the inference task that the model will perform.
Values are
completion,rerank,space_embedding, ortext_embedding. -
alibabacloud_inference_id
string Required The unique identifier of the inference endpoint.
Body
-
chunking_settings
object -
service
string Required Value is
alibabacloud-ai-search. -
service_settings
object Required -
task_settings
object
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{alibabacloud_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"alibabacloud-ai-search\",\n \"service_settings\": {\n \"host\" : \"default-j01.platform-cn-shanghai.opensearch.aliyuncs.com\",\n \"api_key\": \"AlibabaCloud-API-Key\",\n \"service_id\": \"ops-qwen-turbo\",\n \"workspace\" : \"default\"\n }\n}"'{
"service": "alibabacloud-ai-search",
"service_settings": {
"host" : "default-j01.platform-cn-shanghai.opensearch.aliyuncs.com",
"api_key": "AlibabaCloud-API-Key",
"service_id": "ops-qwen-turbo",
"workspace" : "default"
}
}{
"service": "alibabacloud-ai-search",
"service_settings": {
"api_key": "AlibabaCloud-API-Key",
"service_id": "ops-bge-reranker-larger",
"host": "default-j01.platform-cn-shanghai.opensearch.aliyuncs.com",
"workspace": "default"
}
}{
"service": "alibabacloud-ai-search",
"service_settings": {
"api_key": "AlibabaCloud-API-Key",
"service_id": "ops-text-sparse-embedding-001",
"host": "default-j01.platform-cn-shanghai.opensearch.aliyuncs.com",
"workspace": "default"
}
}{
"service": "alibabacloud-ai-search",
"service_settings": {
"api_key": "AlibabaCloud-API-Key",
"service_id": "ops-text-embedding-001",
"host": "default-j01.platform-cn-shanghai.opensearch.aliyuncs.com",
"workspace": "default"
}
}
Delete IP geolocation database configurations
Added in 8.15.0
Path parameters
-
id
string | array[string] Required A comma-separated list of IP location database configurations.
Query parameters
-
master_timeout
string The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error. A value of
-1indicates that the request should never time out.Values are
-1or0. -
timeout
string The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error. A value of
-1indicates that the request should never time out.Values are
-1or0.
curl \
--request DELETE 'http://api.example.com/_ingest/ip_location/database/{id}' \
--header "Authorization: $API_KEY"
Simulate data ingestion
Technical preview
Run ingest pipelines against a set of provided documents, optionally with substitute pipeline definitions, to simulate ingesting data into an index.
This API is meant to be used for troubleshooting or pipeline development, as it does not actually index any data into Elasticsearch.
The API runs the default and final pipeline for that index against a set of documents provided in the body of the request. If a pipeline contains a reroute processor, it follows that reroute processor to the new index, running that index's pipelines as well the same way that a non-simulated ingest would. No data is indexed into Elasticsearch. Instead, the transformed document is returned, along with the list of pipelines that have been run and the name of the index where the document would have been indexed if this were not a simulation. The transformed document is validated against the mappings that would apply to this index, and any validation error is reported in the result.
This API differs from the simulate pipeline API in that you specify a single pipeline for that API, and it runs only that one pipeline. The simulate pipeline API is more useful for developing a single pipeline, while the simulate ingest API is more useful for troubleshooting the interaction of the various pipelines that get applied when ingesting into an index.
By default, the pipeline definitions that are currently in the system are used. However, you can supply substitute pipeline definitions in the body of the request. These will be used in place of the pipeline definitions that are already in the system. This can be used to replace existing pipeline definitions or to create new ones. The pipeline substitutions are used only within this request.
Query parameters
-
pipeline
string The pipeline to use as the default pipeline. This value can be used to override the default pipeline of the index.
Body
Required
-
docs
array[object] Required Sample documents to test in the pipeline.
-
A map of component template names to substitute component template definition objects.
-
index_template_substitutions
object A map of index template names to substitute index template definition objects.
-
mapping_addition
object -
pipeline_substitutions
object Pipelines to test. If you don’t specify the
pipelinerequest path parameter, this parameter is required. If you specify both this and the request path parameter, the API only uses the request path parameter.
curl \
--request GET 'http://api.example.com/_ingest/_simulate' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"docs\": [\n {\n \"_id\": 123,\n \"_index\": \"my-index\",\n \"_source\": {\n \"foo\": \"bar\"\n }\n },\n {\n \"_id\": 456,\n \"_index\": \"my-index\",\n \"_source\": {\n \"foo\": \"rab\"\n }\n }\n ]\n}"'{
"docs": [
{
"_id": 123,
"_index": "my-index",
"_source": {
"foo": "bar"
}
},
{
"_id": 456,
"_index": "my-index",
"_source": {
"foo": "rab"
}
}
]
}{
"docs": [
{
"_index": "my-index",
"_id": 123,
"_source": {
"foo": "bar"
}
},
{
"_index": "my-index",
"_id": 456,
"_source": {
"foo": "rab"
}
}
],
"pipeline_substitutions": {
"my-pipeline": {
"processors": [
{
"uppercase": {
"field": "foo"
}
}
]
}
}
}{
"docs": [
{
"_index": "my-index",
"_id": "123",
"_source": {
"foo": "foo"
}
},
{
"_index": "my-index",
"_id": "456",
"_source": {
"bar": "rab"
}
}
],
"component_template_substitutions": {
"my-mappings_template": {
"template": {
"mappings": {
"dynamic": "strict",
"properties": {
"foo": {
"type": "keyword"
},
"bar": {
"type": "keyword"
}
}
}
}
}
}
}{
"docs": [
{
"_id": "id",
"_index": "my-index",
"_source": {
"foo": "bar"
}
},
{
"_id": "id",
"_index": "my-index",
"_source": {
"foo": "rab"
}
}
],
"pipeline_substitutions": {
"my-pipeline": {
"processors": [
{
"set": {
"field": "field3",
"value": "value3"
}
}
]
}
},
"component_template_substitutions": {
"my-component-template": {
"template": {
"mappings": {
"dynamic": true,
"properties": {
"field3": {
"type": "keyword"
}
}
},
"settings": {
"index": {
"default_pipeline": "my-pipeline"
}
}
}
}
},
"index_template_substitutions": {
"my-index-template": {
"index_patterns": [
"my-index-*"
],
"composed_of": [
"component_template_1",
"component_template_2"
]
}
},
"mapping_addition": {
"dynamic": "strict",
"properties": {
"foo": {
"type": "keyword"
}
}
}
}{
"docs": [
{
"doc": null,
"_id": 123,
"_index": "my-index",
"_version": -3,
"_source": {
"field1": "value1",
"field2": "value2",
"foo": "bar"
},
"executed_pipelines": [
"my-pipeline",
"my-final-pipeline"
]
},
{
"doc": null,
"_id": 456,
"_index": "my-index",
"_version": "-3,",
"_source": {
"field1": "value1",
"field2": "value2",
"foo": "rab"
},
"executed_pipelines": [
"my-pipeline",
"my-final-pipeline"
]
}
]
}{
"docs": [
{
"doc": null,
"_id": 123,
"_index": "my-index",
"_version": -3,
"_source": {
"field2": "value2",
"foo": "BAR"
},
"executed_pipelines": [
"my-pipeline",
"my-final-pipeline"
]
},
{
"doc": null,
"_id": 456,
"_index": "my-index",
"_version": -3,
"_source": {
"field2": "value2",
"foo": "RAB"
},
"executed_pipelines": [
"my-pipeline",
"my-final-pipeline"
]
}
]
}{
"docs": [
{
"doc": {
"_id": "123",
"_index": "my-index",
"_version": -3,
"_source": {
"foo": "foo"
},
"executed_pipelines": []
}
},
{
"doc": {
"_id": "456",
"_index": "my-index",
"_version": -3,
"_source": {
"bar": "rab"
},
"executed_pipelines": []
}
}
]
}
Force buffered data to be processed
Deprecated
Added in 5.4.0
The flush jobs API is only applicable when sending data for analysis using the post data API. Depending on the content of the buffer, then it might additionally calculate new results. Both flush and close operations are similar, however the flush is more efficient if you are expecting to send more data for analysis. When flushing, the job remains open and is available to continue analyzing data. A close operation additionally prunes and persists the model state to disk and the job must be opened again before analyzing further data.
Path parameters
-
job_id
string Required Identifier for the anomaly detection job.
Query parameters
-
advance_time
string | number Specifies to advance to a particular time value. Results are generated and the model is updated for data from the specified time interval.
-
calc_interim
boolean If true, calculates the interim results for the most recent bucket or all buckets within the latency period.
-
end
string | number When used in conjunction with
calc_interimandstart, specifies the range of buckets on which to calculate interim results. -
skip_time
string | number Specifies to skip to a particular time value. Results are not generated and the model is not updated for data from the specified time interval.
-
start
string | number When used in conjunction with
calc_interim, specifies the range of buckets on which to calculate interim results.
Body
advance_time
string | number A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
-
calc_interim
boolean Refer to the description for the
calc_interimquery parameter. end
string | number A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
skip_time
string | number A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
start
string | number A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
curl \
--request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/_flush' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"":"string","calc_interim":true}'
Get anomaly detection job results for buckets
Added in 5.4.0
The API presents a chronological view of the records, grouped by bucket.
Path parameters
-
job_id
string Required Identifier for the anomaly detection job.
Query parameters
-
anomaly_score
number Returns buckets with anomaly scores greater or equal than this value.
-
desc
boolean If
true, the buckets are sorted in descending order. -
end
string | number Returns buckets with timestamps earlier than this time.
-1means it is unset and results are not limited to specific timestamps. -
exclude_interim
boolean If
true, the output excludes interim results. -
expand
boolean If true, the output includes anomaly records.
-
from
number Skips the specified number of buckets.
-
size
number Specifies the maximum number of buckets to obtain.
-
sort
string Specifies the sort field for the requested buckets.
-
start
string | number Returns buckets with timestamps after this time.
-1means it is unset and results are not limited to specific timestamps.
Body
-
anomaly_score
number Refer to the description for the
anomaly_scorequery parameter. -
desc
boolean Refer to the description for the
descquery parameter. end
string | number A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
-
exclude_interim
boolean Refer to the description for the
exclude_interimquery parameter. -
expand
boolean Refer to the description for the
expandquery parameter. -
page
object -
sort
string Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
start
string | number A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
curl \
--request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/results/buckets' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"anomaly_score":42.0,"desc":true,"":"string","exclude_interim":true,"expand":true,"page":{"from":42.0,"size":42.0},"sort":"string"}'
Delete a data frame analytics job
Added in 7.3.0
Path parameters
-
id
string Required Identifier for the data frame analytics job.
curl \
--request DELETE 'http://api.example.com/_ml/data_frame/analytics/{id}' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}
Create an index from a source index
Technical preview
Copy the mappings and settings from the source index to a destination index while allowing request settings and mappings to override the source values.
Body
Required
-
mappings_override
object -
settings_override
object -
remove_index_blocks
boolean If index blocks should be removed when creating destination index (optional)
curl \
--request PUT 'http://api.example.com/_create_from/{source}/{dest}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"mappings_override":{"all_field":{"analyzer":"string","enabled":true,"omit_norms":true,"search_analyzer":"string","similarity":"string","store":true,"store_term_vector_offsets":true,"store_term_vector_payloads":true,"store_term_vector_positions":true,"store_term_vectors":true},"date_detection":true,"dynamic":"strict","dynamic_date_formats":["string"],"dynamic_templates":[{}],"_field_names":{"enabled":true},"index_field":{"enabled":true},"_meta":{"additionalProperty1":{},"additionalProperty2":{}},"numeric_detection":true,"properties":{},"_routing":{"required":true},"_size":{"enabled":true},"_source":{"compress":true,"compress_threshold":"string","enabled":true,"excludes":["string"],"includes":["string"],"mode":"disabled"},"runtime":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"enabled":true,"subobjects":"true","_data_stream_timestamp":{"enabled":true}},"settings_override":{"index":{},"mode":"string","routing_path":"string","soft_deletes":{"enabled":true,"retention_lease":{"period":"string"}},"sort":{"field":"string","order":"asc","mode":"min","missing":"_last"},"number_of_shards":42.0,"number_of_replicas":42.0,"number_of_routing_shards":42.0,"check_on_startup":"true","codec":"string","":"string","load_fixed_bitset_filters_eagerly":true,"hidden":true,"auto_expand_replicas":"string","merge":{"scheduler":{"":42.0}},"search":{"idle":{"after":"string"},"slowlog":{"level":"string","source":42.0,"reformat":true,"threshold":{"query":{"warn":"string","info":"string","debug":"string","trace":"string"},"fetch":{"warn":"string","info":"string","debug":"string","trace":"string"}}}},"refresh_interval":"string","max_result_window":42.0,"max_inner_result_window":42.0,"max_rescore_window":42.0,"max_docvalue_fields_search":42.0,"max_script_fields":42.0,"max_ngram_diff":42.0,"max_shingle_diff":42.0,"blocks":{"":true},"max_refresh_listeners":42.0,"analyze":{"":42.0},"highlight":{"max_analyzed_offset":42.0},"max_terms_count":42.0,"max_regex_length":42.0,"routing":{"allocation":{"enable":"all","include":{"_tier_preference":"string","_id":"string"},"initial_recovery":{"_id":"string"},"disk":{"threshold_enabled":true}},"rebalance":{"enable":"all"}},"gc_deletes":"string","default_pipeline":"string","final_pipeline":"string","lifecycle":{"name":"string","":true,"origination_date":42.0,"parse_origination_date":true,"step":{"wait_time_threshold":"string"},"rollover_alias":"string","prefer_ilm":true},"provided_name":"string","uuid":"string","version":{"created":"string","created_string":"string"},"verified_before_close":true,"format":"string","max_slices_per_scroll":42.0,"translog":{"sync_interval":"string","durability":"request","":42.0,"retention":{"":42.0,"age":"string"}},"query_string":{"":true},"priority":42.0,"top_metrics_max_size":42.0,"analysis":{"analyzer":{},"char_filter":{},"filter":{},"normalizer":{},"tokenizer":{}},"settings":{},"time_series":{"":"string"},"queries":{"cache":{"enabled":true}},"similarity":{},"mapping":{"coerce":true,"total_fields":{"limit":42.0,"ignore_dynamic_beyond_limit":true},"depth":{"limit":42.0},"nested_fields":{"limit":42.0},"nested_objects":{"limit":42.0},"field_name_length":{"limit":42.0},"dimension_fields":{"limit":42.0},"source":{"mode":"disabled"},"ignore_malformed":true},"indexing.slowlog":{"level":"string","source":42.0,"reformat":true,"threshold":{"index":{"warn":"string","info":"string","debug":"string","trace":"string"}}},"indexing_pressure":{"memory":{"limit":42.0}},"store":{"":"fs","allow_mmap":true}},"remove_index_blocks":true}'
Create an index from a source index
Technical preview
Copy the mappings and settings from the source index to a destination index while allowing request settings and mappings to override the source values.
Body
Required
-
mappings_override
object -
settings_override
object -
remove_index_blocks
boolean If index blocks should be removed when creating destination index (optional)
curl \
--request POST 'http://api.example.com/_create_from/{source}/{dest}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"mappings_override":{"all_field":{"analyzer":"string","enabled":true,"omit_norms":true,"search_analyzer":"string","similarity":"string","store":true,"store_term_vector_offsets":true,"store_term_vector_payloads":true,"store_term_vector_positions":true,"store_term_vectors":true},"date_detection":true,"dynamic":"strict","dynamic_date_formats":["string"],"dynamic_templates":[{}],"_field_names":{"enabled":true},"index_field":{"enabled":true},"_meta":{"additionalProperty1":{},"additionalProperty2":{}},"numeric_detection":true,"properties":{},"_routing":{"required":true},"_size":{"enabled":true},"_source":{"compress":true,"compress_threshold":"string","enabled":true,"excludes":["string"],"includes":["string"],"mode":"disabled"},"runtime":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"":"painless","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"enabled":true,"subobjects":"true","_data_stream_timestamp":{"enabled":true}},"settings_override":{"index":{},"mode":"string","routing_path":"string","soft_deletes":{"enabled":true,"retention_lease":{"period":"string"}},"sort":{"field":"string","order":"asc","mode":"min","missing":"_last"},"number_of_shards":42.0,"number_of_replicas":42.0,"number_of_routing_shards":42.0,"check_on_startup":"true","codec":"string","":"string","load_fixed_bitset_filters_eagerly":true,"hidden":true,"auto_expand_replicas":"string","merge":{"scheduler":{"":42.0}},"search":{"idle":{"after":"string"},"slowlog":{"level":"string","source":42.0,"reformat":true,"threshold":{"query":{"warn":"string","info":"string","debug":"string","trace":"string"},"fetch":{"warn":"string","info":"string","debug":"string","trace":"string"}}}},"refresh_interval":"string","max_result_window":42.0,"max_inner_result_window":42.0,"max_rescore_window":42.0,"max_docvalue_fields_search":42.0,"max_script_fields":42.0,"max_ngram_diff":42.0,"max_shingle_diff":42.0,"blocks":{"":true},"max_refresh_listeners":42.0,"analyze":{"":42.0},"highlight":{"max_analyzed_offset":42.0},"max_terms_count":42.0,"max_regex_length":42.0,"routing":{"allocation":{"enable":"all","include":{"_tier_preference":"string","_id":"string"},"initial_recovery":{"_id":"string"},"disk":{"threshold_enabled":true}},"rebalance":{"enable":"all"}},"gc_deletes":"string","default_pipeline":"string","final_pipeline":"string","lifecycle":{"name":"string","":true,"origination_date":42.0,"parse_origination_date":true,"step":{"wait_time_threshold":"string"},"rollover_alias":"string","prefer_ilm":true},"provided_name":"string","uuid":"string","version":{"created":"string","created_string":"string"},"verified_before_close":true,"format":"string","max_slices_per_scroll":42.0,"translog":{"sync_interval":"string","durability":"request","":42.0,"retention":{"":42.0,"age":"string"}},"query_string":{"":true},"priority":42.0,"top_metrics_max_size":42.0,"analysis":{"analyzer":{},"char_filter":{},"filter":{},"normalizer":{},"tokenizer":{}},"settings":{},"time_series":{"":"string"},"queries":{"cache":{"enabled":true}},"similarity":{},"mapping":{"coerce":true,"total_fields":{"limit":42.0,"ignore_dynamic_beyond_limit":true},"depth":{"limit":42.0},"nested_fields":{"limit":42.0},"nested_objects":{"limit":42.0},"field_name_length":{"limit":42.0},"dimension_fields":{"limit":42.0},"source":{"mode":"disabled"},"ignore_malformed":true},"indexing.slowlog":{"level":"string","source":42.0,"reformat":true,"threshold":{"index":{"warn":"string","info":"string","debug":"string","trace":"string"}}},"indexing_pressure":{"memory":{"limit":42.0}},"store":{"":"fs","allow_mmap":true}},"remove_index_blocks":true}'
Get the migration reindexing status
Technical preview
Get the status of a migration reindex attempt for a data stream or index.
Path parameters
-
index
string | array[string] Required The index or data stream name.
curl \
--request GET 'http://api.example.com/_migration/reindex/{index}/_status' \
--header "Authorization: $API_KEY"Render a search template
Render a search template as a search request body.
Path parameters
-
id
string Required The ID of the search template to render. If no
sourceis specified, this or theidrequest body parameter is required.
curl \
--request POST 'http://api.example.com/_render/template/{id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"id\": \"my-search-template\",\n \"params\": {\n \"query_string\": \"hello world\",\n \"from\": 20,\n \"size\": 10\n }\n}"'{
"id": "my-search-template",
"params": {
"query_string": "hello world",
"from": 20,
"size": 10
}
}
Search a vector tile
Added in 7.15.0
Search a vector tile for geospatial values. Before using this API, you should be familiar with the Mapbox vector tile specification. The API returns results as a binary mapbox vector tile.
Internally, Elasticsearch translates a vector tile search API request into a search containing:
- A
geo_bounding_boxquery on the<field>. The query uses the<zoom>/<x>/<y>tile as a bounding box. - A
geotile_gridorgeohex_gridaggregation on the<field>. Thegrid_aggparameter determines the aggregation type. The aggregation uses the<zoom>/<x>/<y>tile as a bounding box. - Optionally, a
geo_boundsaggregation on the<field>. The search only includes this aggregation if theexact_boundsparameter istrue. - If the optional parameter
with_labelsistrue, the internal search will include a dynamic runtime field that calls thegetLabelPositionfunction of the geometry doc value. This enables the generation of new point features containing suggested geometry labels, so that, for example, multi-polygons will have only one label.
For example, Elasticsearch may translate a vector tile search API request with a grid_agg argument of geotile and an exact_bounds argument of true into the following search
GET my-index/_search
{
"size": 10000,
"query": {
"geo_bounding_box": {
"my-geo-field": {
"top_left": {
"lat": -40.979898069620134,
"lon": -45
},
"bottom_right": {
"lat": -66.51326044311186,
"lon": 0
}
}
}
},
"aggregations": {
"grid": {
"geotile_grid": {
"field": "my-geo-field",
"precision": 11,
"size": 65536,
"bounds": {
"top_left": {
"lat": -40.979898069620134,
"lon": -45
},
"bottom_right": {
"lat": -66.51326044311186,
"lon": 0
}
}
}
},
"bounds": {
"geo_bounds": {
"field": "my-geo-field",
"wrap_longitude": false
}
}
}
}
The API returns results as a binary Mapbox vector tile. Mapbox vector tiles are encoded as Google Protobufs (PBF). By default, the tile contains three layers:
- A
hitslayer containing a feature for each<field>value matching thegeo_bounding_boxquery. - An
aggslayer containing a feature for each cell of thegeotile_gridorgeohex_grid. The layer only contains features for cells with matching data. - A meta layer containing:
- A feature containing a bounding box. By default, this is the bounding box of the tile.
- Value ranges for any sub-aggregations on the
geotile_gridorgeohex_grid. - Metadata for the search.
The API only returns features that can display at its zoom level. For example, if a polygon feature has no area at its zoom level, the API omits it. The API returns errors as UTF-8 encoded JSON.
IMPORTANT: You can specify several options for this API as either a query parameter or request body parameter. If you specify both parameters, the query parameter takes precedence.
Grid precision for geotile
For a grid_agg of geotile, you can use cells in the aggs layer as tiles for lower zoom levels.
grid_precision represents the additional zoom levels available through these cells. The final precision is computed by as follows: <zoom> + grid_precision.
For example, if <zoom> is 7 and grid_precision is 8, then the geotile_grid aggregation will use a precision of 15.
The maximum final precision is 29.
The grid_precision also determines the number of cells for the grid as follows: (2^grid_precision) x (2^grid_precision).
For example, a value of 8 divides the tile into a grid of 256 x 256 cells.
The aggs layer only contains features for cells with matching data.
Grid precision for geohex
For a grid_agg of geohex, Elasticsearch uses <zoom> and grid_precision to calculate a final precision as follows: <zoom> + grid_precision.
This precision determines the H3 resolution of the hexagonal cells produced by the geohex aggregation.
The following table maps the H3 resolution for each precision.
For example, if <zoom> is 3 and grid_precision is 3, the precision is 6.
At a precision of 6, hexagonal cells have an H3 resolution of 2.
If <zoom> is 3 and grid_precision is 4, the precision is 7.
At a precision of 7, hexagonal cells have an H3 resolution of 3.
| Precision | Unique tile bins | H3 resolution | Unique hex bins | Ratio |
|---|---|---|---|---|
| 1 | 4 | 0 | 122 | 30.5 |
| 2 | 16 | 0 | 122 | 7.625 |
| 3 | 64 | 1 | 842 | 13.15625 |
| 4 | 256 | 1 | 842 | 3.2890625 |
| 5 | 1024 | 2 | 5882 | 5.744140625 |
| 6 | 4096 | 2 | 5882 | 1.436035156 |
| 7 | 16384 | 3 | 41162 | 2.512329102 |
| 8 | 65536 | 3 | 41162 | 0.6280822754 |
| 9 | 262144 | 4 | 288122 | 1.099098206 |
| 10 | 1048576 | 4 | 288122 | 0.2747745514 |
| 11 | 4194304 | 5 | 2016842 | 0.4808526039 |
| 12 | 16777216 | 6 | 14117882 | 0.8414913416 |
| 13 | 67108864 | 6 | 14117882 | 0.2103728354 |
| 14 | 268435456 | 7 | 98825162 | 0.3681524172 |
| 15 | 1073741824 | 8 | 691776122 | 0.644266719 |
| 16 | 4294967296 | 8 | 691776122 | 0.1610666797 |
| 17 | 17179869184 | 9 | 4842432842 | 0.2818666889 |
| 18 | 68719476736 | 10 | 33897029882 | 0.4932667053 |
| 19 | 274877906944 | 11 | 237279209162 | 0.8632167343 |
| 20 | 1099511627776 | 11 | 237279209162 | 0.2158041836 |
| 21 | 4398046511104 | 12 | 1660954464122 | 0.3776573213 |
| 22 | 17592186044416 | 13 | 11626681248842 | 0.6609003122 |
| 23 | 70368744177664 | 13 | 11626681248842 | 0.165225078 |
| 24 | 281474976710656 | 14 | 81386768741882 | 0.2891438866 |
| 25 | 1125899906842620 | 15 | 569707381193162 | 0.5060018015 |
| 26 | 4503599627370500 | 15 | 569707381193162 | 0.1265004504 |
| 27 | 18014398509482000 | 15 | 569707381193162 | 0.03162511259 |
| 28 | 72057594037927900 | 15 | 569707381193162 | 0.007906278149 |
| 29 | 288230376151712000 | 15 | 569707381193162 | 0.001976569537 |
Hexagonal cells don't align perfectly on a vector tile. Some cells may intersect more than one vector tile. To compute the H3 resolution for each precision, Elasticsearch compares the average density of hexagonal bins at each resolution with the average density of tile bins at each zoom level. Elasticsearch uses the H3 resolution that is closest to the corresponding geotile density.
Path parameters
-
index
string | array[string] Required Comma-separated list of data streams, indices, or aliases to search
-
field
string Required Field containing geospatial data to return
-
zoom
number Required Zoom level for the vector tile to search
-
x
number Required X coordinate for the vector tile to search
-
y
number Required Y coordinate for the vector tile to search
Query parameters
-
exact_bounds
boolean If
false, the meta layer's feature is the bounding box of the tile. If true, the meta layer's feature is a bounding box resulting from a geo_bounds aggregation. The aggregation runs on values that intersect the // tile with wrap_longitude set to false. The resulting bounding box may be larger than the vector tile. -
extent
number The size, in pixels, of a side of the tile. Vector tiles are square with equal sides.
-
grid_agg
string Aggregation used to create a grid for
field.Values are
geotileorgeohex. -
grid_precision
number Additional zoom levels available through the aggs layer. For example, if is 7 and grid_precision is 8, you can zoom in up to level 15. Accepts 0-8. If 0, results don't include the aggs layer.
-
grid_type
string Determines the geometry type for features in the aggs layer. In the aggs layer, each feature represents a geotile_grid cell. If 'grid' each feature is a Polygon of the cells bounding box. If 'point' each feature is a Point that is the centroid of the cell.
Values are
grid,point, orcentroid. -
size
number Maximum number of features to return in the hits layer. Accepts 0-10000. If 0, results don't include the hits layer.
-
with_labels
boolean If
true, the hits and aggs layers will contain additional point features representing suggested label positions for the original features.PointandMultiPointfeatures will have one of the points selected.PolygonandMultiPolygonfeatures will have a single point generated, either the centroid, if it is within the polygon, or another point within the polygon selected from the sorted triangle-tree.LineStringfeatures will likewise provide a roughly central point selected from the triangle-tree.- The aggregation results will provide one central point for each aggregation bucket.
All attributes from the original features will also be copied to the new label features. In addition, the new features will be distinguishable using the tag
_mvt_label_position.
Body
-
aggs
object Sub-aggregations for the geotile_grid.
It supports the following aggregation types:
avgboxplotcardinalityextended statsmaxmedian absolute deviationminpercentilepercentile-rankstatssumvalue count
The aggregation names can't start with
_mvt_. The_mvt_prefix is reserved for internal aggregations. -
buffer
number The size, in pixels, of a clipping buffer outside the tile. This allows renderers to avoid outline artifacts from geometries that extend past the extent of the tile.
-
exact_bounds
boolean If
false, the meta layer's feature is the bounding box of the tile. Iftrue, the meta layer's feature is a bounding box resulting from ageo_boundsaggregation. The aggregation runs on values that intersect the<zoom>/<x>/<y>tile withwrap_longitudeset tofalse. The resulting bounding box may be larger than the vector tile. -
extent
number The size, in pixels, of a side of the tile. Vector tiles are square with equal sides.
-
fields
string | array[string] -
grid_agg
string Values are
geotileorgeohex. -
grid_precision
number Additional zoom levels available through the aggs layer. For example, if
<zoom>is7andgrid_precisionis8, you can zoom in up to level 15. Accepts 0-8. If 0, results don't include the aggs layer. -
grid_type
string Values are
grid,point, orcentroid. -
query
object An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
External documentation -
runtime_mappings
object -
size
number The maximum number of features to return in the hits layer. Accepts 0-10000. If 0, results don't include the hits layer.
-
track_total_hits
boolean | number Number of hits matching the query to count accurately. If true, the exact number of hits is returned at the cost of some performance. If false, the response does not include the total number of hits matching the query. Defaults to 10,000 hits.
-
with_labels
boolean If
true, the hits and aggs layers will contain additional point features representing suggested label positions for the original features.PointandMultiPointfeatures will have one of the points selected.PolygonandMultiPolygonfeatures will have a single point generated, either the centroid, if it is within the polygon, or another point within the polygon selected from the sorted triangle-tree.LineStringfeatures will likewise provide a roughly central point selected from the triangle-tree.- The aggregation results will provide one central point for each aggregation bucket.
All attributes from the original features will also be copied to the new label features. In addition, the new features will be distinguishable using the tag
_mvt_label_position.
curl \
--request GET 'http://api.example.com/{index}/_mvt/{field}/{zoom}/{x}/{y}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"grid_agg\": \"geotile\",\n \"grid_precision\": 2,\n \"fields\": [\n \"name\",\n \"price\"\n ],\n \"query\": {\n \"term\": {\n \"included\": true\n }\n },\n \"aggs\": {\n \"min_price\": {\n \"min\": {\n \"field\": \"price\"\n }\n },\n \"max_price\": {\n \"max\": {\n \"field\": \"price\"\n }\n },\n \"avg_price\": {\n \"avg\": {\n \"field\": \"price\"\n }\n }\n }\n}"'{
"grid_agg": "geotile",
"grid_precision": 2,
"fields": [
"name",
"price"
],
"query": {
"term": {
"included": true
}
},
"aggs": {
"min_price": {
"min": {
"field": "price"
}
},
"max_price": {
"max": {
"field": "price"
}
},
"avg_price": {
"avg": {
"field": "price"
}
}
}
}{
"hits": {
"extent": 4096,
"version": 2,
"features": [
{
"geometry": {
"type": "Point",
"coordinates": [
3208,
3864
]
},
"properties": {
"_id": "1",
"_index": "museums",
"name": "NEMO Science Museum",
"price": 1750
},
"type": 1
},
{
"geometry": {
"type": "Point",
"coordinates": [
3429,
3496
]
},
"properties": {
"_id": "3",
"_index": "museums",
"name": "Nederlands Scheepvaartmuseum",
"price": 1650
},
"type": 1
},
{
"geometry": {
"type": "Point",
"coordinates": [
3429,
3496
]
},
"properties": {
"_id": "4",
"_index": "museums",
"name": "Amsterdam Centre for Architecture",
"price": 0
},
"type": 1
}
]
},
"aggs": {
"extent": 4096,
"version": 2,
"features": [
{
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
3072,
3072
],
[
4096,
3072
],
[
4096,
4096
],
[
3072,
4096
],
[
3072,
3072
]
]
]
},
"properties": {
"_count": 3,
"max_price.value": 1750.0,
"min_price.value": 0.0,
"avg_price.value": 1133.3333333333333
},
"type": 3
}
]
},
"meta": {
"extent": 4096,
"version": 2,
"features": [
{
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
0,
0
],
[
4096,
0
],
[
4096,
4096
],
[
0,
4096
],
[
0,
0
]
]
]
},
"properties": {
"_shards.failed": 0,
"_shards.skipped": 0,
"_shards.successful": 1,
"_shards.total": 1,
"aggregations._count.avg": 3.0,
"aggregations._count.count": 1,
"aggregations._count.max": 3.0,
"aggregations._count.min": 3.0,
"aggregations._count.sum": 3.0,
"aggregations.avg_price.avg": 1133.3333333333333,
"aggregations.avg_price.count": 1,
"aggregations.avg_price.max": 1133.3333333333333,
"aggregations.avg_price.min": 1133.3333333333333,
"aggregations.avg_price.sum": 1133.3333333333333,
"aggregations.max_price.avg": 1750.0,
"aggregations.max_price.count": 1,
"aggregations.max_price.max": 1750.0,
"aggregations.max_price.min": 1750.0,
"aggregations.max_price.sum": 1750.0,
"aggregations.min_price.avg": 0.0,
"aggregations.min_price.count": 1,
"aggregations.min_price.max": 0.0,
"aggregations.min_price.min": 0.0,
"aggregations.min_price.sum": 0.0,
"hits.max_score": 0.0,
"hits.total.relation": "eq",
"hits.total.value": 3,
"timed_out": false,
"took": 2
},
"type": 3
}
]
}
}
Render a search application query
Technical preview
Generate an Elasticsearch query using the specified query parameters and the search template associated with the search application or a default template if none is specified.
If a parameter used in the search template is not specified in params, the parameter's default value will be used.
The API returns the specific Elasticsearch query that would be generated and run by calling the search application search API.
You must have read privileges on the backing alias of the search application.
Path parameters
-
name
string Required The name of the search application to render teh query for.
curl \
--request POST 'http://api.example.com/_application/search_application/{name}/_render_query' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"params\": {\n \"query_string\": \"my first query\",\n \"text_fields\": [\n {\n \"name\": \"title\",\n \"boost\": 5\n },\n {\n \"name\": \"description\",\n \"boost\": 1\n }\n ]\n }\n}"'{
"params": {
"query_string": "my first query",
"text_fields": [
{
"name": "title",
"boost": 5
},
{
"name": "description",
"boost": 1
}
]
}
}{
"from": 0,
"size": 10,
"query": {
"multi_match": {
"query": "my first query",
"fields": [
"description^1.0",
"title^5.0"
]
}
},
"explain": false
}
Get service accounts
Added in 7.13.0
Get a list of service accounts that match the provided path parameters.
NOTE: Currently, only the elastic/fleet-server service account is available.
curl \
--request GET 'http://api.example.com/_security/service' \
--header "Authorization: $API_KEY"{
"elastic/fleet-server": {
"role_descriptor": {
"cluster": [
"monitor",
"manage_own_api_key",
"read_fleet_secrets"
],
"indices": [
{
"names": [
"logs-*",
"metrics-*",
"traces-*",
".logs-endpoint.diagnostic.collection-*",
".logs-endpoint.action.responses-*",
".logs-endpoint.heartbeat-*"
],
"privileges": [
"write",
"create_index",
"auto_configure"
],
"allow_restricted_indices": false
},
{
"names": [
"profiling-*"
],
"privileges": [
"read",
"write"
],
"allow_restricted_indices": false
},
{
"names": [
"traces-apm.sampled-*"
],
"privileges": [
"read",
"monitor",
"maintenance"
],
"allow_restricted_indices": false
},
{
"names": [
".fleet-secrets*"
],
"privileges": [
"read"
],
"allow_restricted_indices": true
},
{
"names": [
".fleet-actions*"
],
"privileges": [
"read",
"write",
"monitor",
"create_index",
"auto_configure",
"maintenance"
],
"allow_restricted_indices": true
},
{
"names": [
".fleet-agents*"
],
"privileges": [
"read",
"write",
"monitor",
"create_index",
"auto_configure",
"maintenance"
],
"allow_restricted_indices": true
},
{
"names": [
".fleet-artifacts*"
],
"privileges": [
"read",
"write",
"monitor",
"create_index",
"auto_configure",
"maintenance"
],
"allow_restricted_indices": true
},
{
"names": [
".fleet-enrollment-api-keys*"
],
"privileges": [
"read",
"write",
"monitor",
"create_index",
"auto_configure",
"maintenance"
],
"allow_restricted_indices": true
},
{
"names": [
".fleet-policies*"
],
"privileges": [
"read",
"write",
"monitor",
"create_index",
"auto_configure",
"maintenance"
],
"allow_restricted_indices": true
},
{
"names": [
".fleet-policies-leader*"
],
"privileges": [
"read",
"write",
"monitor",
"create_index",
"auto_configure",
"maintenance"
],
"allow_restricted_indices": true
},
{
"names": [
".fleet-servers*"
],
"privileges": [
"read",
"write",
"monitor",
"create_index",
"auto_configure",
"maintenance"
],
"allow_restricted_indices": true
},
{
"names": [
".fleet-fileds*"
],
"privileges": [
"read",
"write",
"monitor",
"create_index",
"auto_configure",
"maintenance"
],
"allow_restricted_indices": true
},
{
"names": [
"synthetics-*"
],
"privileges": [
"read",
"write",
"create_index",
"auto_configure"
],
"allow_restricted_indices": false
}
],
"applications": [
{
"application": "kibana-*",
"privileges": [
"reserved_fleet-setup"
],
"resources": [
"*"
]
}
],
"run_as": [],
"metadata": {},
"transient_metadata": {
"enabled": true
}
}
}
}
Logout of SAML completely
Added in 7.14.0
Verifies the logout response sent from the SAML IdP.
NOTE: This API is intended for use by custom web applications other than Kibana. If you are using Kibana, refer to the documentation for configuring SAML single-sign-on on the Elastic Stack.
The SAML IdP may send a logout response back to the SP after handling the SP-initiated SAML Single Logout. This API verifies the response by ensuring the content is relevant and validating its signature. An empty response is returned if the verification process is successful. The response can be sent by the IdP with either the HTTP-Redirect or the HTTP-Post binding. The caller of this API must prepare the request accordingly so that this API can handle either of them.
Body
Required
-
realm
string Required The name of the SAML realm in Elasticsearch for which the configuration is used to verify the logout response.
ids
string | array[string] Required -
query_string
string If the SAML IdP sends the logout response with the HTTP-Redirect binding, this field must be set to the query string of the redirect URI.
-
content
string If the SAML IdP sends the logout response with the HTTP-Post binding, this field must be set to the value of the SAMLResponse form parameter from the logout response.
curl \
--request POST 'http://api.example.com/_security/saml/complete_logout' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"realm\": \"saml1\",\n \"ids\": [ \"_1c368075e0b3...\" ],\n \"query_string\": \"SAMLResponse=fZHLasMwEEVbfb1bf...\u0026SigAlg=http%3A%2F%2Fwww.w3.org%2F2000%2F09%2Fxmldsig%23rsa-sha1\u0026Signature=CuCmFn%2BLqnaZGZJqK...\"\n}"'{
"realm": "saml1",
"ids": [ "_1c368075e0b3..." ],
"query_string": "SAMLResponse=fZHLasMwEEVbfb1bf...&SigAlg=http%3A%2F%2Fwww.w3.org%2F2000%2F09%2Fxmldsig%23rsa-sha1&Signature=CuCmFn%2BLqnaZGZJqK..."
}{
"realm": "saml1",
"ids": [ "_1c368075e0b3..." ],
"content": "PHNhbWxwOkxvZ291dFJlc3BvbnNlIHhtbG5zOnNhbWxwPSJ1cm46..."
}
Get snapshot information
Added in 0.0.0
NOTE: The after parameter and next field enable you to iterate through snapshots with some consistency guarantees regarding concurrent creation or deletion of snapshots.
It is guaranteed that any snapshot that exists at the beginning of the iteration and is not concurrently deleted will be seen during the iteration.
Snapshots concurrently created may be seen during an iteration.
Path parameters
-
repository
string Required A comma-separated list of snapshot repository names used to limit the request. Wildcard (
*) expressions are supported. -
snapshot
string | array[string] Required A comma-separated list of snapshot names to retrieve Wildcards (
*) are supported.- To get information about all snapshots in a registered repository, use a wildcard (
*) or_all. - To get information about any snapshots that are currently running, use
_current.
- To get information about all snapshots in a registered repository, use a wildcard (
Query parameters
-
after
string An offset identifier to start pagination from as returned by the next field in the response body.
-
from_sort_value
string The value of the current sort column at which to start retrieval. It can be a string
snapshot-or a repository name when sorting by snapshot or repository name. It can be a millisecond time value or a number when sorting byindex-or shard count. -
index_details
boolean If
true, the response includes additional information about each index in the snapshot comprising the number of shards in the index, the total size of the index in bytes, and the maximum number of segments per shard in the index. The default isfalse, meaning that this information is omitted. -
index_names
boolean If
true, the response includes the name of each index in each snapshot. -
include_repository
boolean If
true, the response includes the repository name in each snapshot. -
master_timeout
string The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Values are
-1or0. -
order
string The sort order. Valid values are
ascfor ascending anddescfor descending order. The default behavior is ascending order.Supported values include:
asc: Ascending (smallest to largest)desc: Descending (largest to smallest)
Values are
ascordesc. -
offset
number Numeric offset to start pagination from based on the snapshots matching this request. Using a non-zero value for this parameter is mutually exclusive with using the after parameter. Defaults to 0.
-
size
number The maximum number of snapshots to return. The default is 0, which means to return all that match the request without limit.
-
slm_policy_filter
string Filter snapshots by a comma-separated list of snapshot lifecycle management (SLM) policy names that snapshots belong to.
You can use wildcards (
*) and combinations of wildcards followed by exclude patterns starting with-. For example, the pattern*,-policy-a-\*will return all snapshots except for those that were created by an SLM policy with a name starting withpolicy-a-. Note that the wildcard pattern*matches all snapshots created by an SLM policy but not those snapshots that were not created by an SLM policy. To include snapshots that were not created by an SLM policy, you can use the special pattern_nonethat will match all snapshots without an SLM policy. -
sort
string The sort order for the result. The default behavior is sorting by snapshot start time stamp.
Values are
start_time,duration,name,index_count,repository,shard_count, orfailed_shard_count. -
verbose
boolean If
true, returns additional information about each snapshot such as the version of Elasticsearch which took the snapshot, the start and end times of the snapshot, and the number of shards snapshotted.NOTE: The parameters
size,order,after,from_sort_value,offset,slm_policy_filter, andsortare not supported when you setverbose=falseand the sort order for requests withverbose=falseis undefined.
curl \
--request GET 'http://api.example.com/_snapshot/{repository}/{snapshot}' \
--header "Authorization: $API_KEY"{
"snapshots": [
{
"snapshot": "snapshot_1",
"uuid": "dKb54xw67gvdRctLCxSket",
"repository": "my_repository",
"version_id": <version_id>,
"version": <version>,
"indices": [],
"data_streams": [],
"feature_states": [],
"include_global_state": true,
"state": "SUCCESS",
"start_time": "2020-07-06T21:55:18.128Z",
"start_time_in_millis": 1593093628849,
"end_time": "2020-07-06T21:55:18.129Z",
"end_time_in_millis": 1593093628850,
"duration_in_millis": 1,
"failures": [],
"shards": {
"total": 0,
"failed": 0,
"successful": 0
}
},
{
"snapshot": "snapshot_2",
"uuid": "vdRctLCxSketdKb54xw67g",
"repository": "my_repository",
"version_id": <version_id>,
"version": <version>,
"indices": [],
"data_streams": [],
"feature_states": [],
"include_global_state": true,
"state": "SUCCESS",
"start_time": "2020-07-06T21:55:18.130Z",
"start_time_in_millis": 1593093628851,
"end_time": "2020-07-06T21:55:18.130Z",
"end_time_in_millis": 1593093628851,
"duration_in_millis": 0,
"failures": [],
"shards": {
"total": 0,
"failed": 0,
"successful": 0
}
},
{
"snapshot": "snapshot_3",
"uuid": "dRctdKb54xw67gvLCxSket",
"repository": "my_repository",
"version_id": <version_id>,
"version": <version>,
"indices": [],
"data_streams": [],
"feature_states": [],
"include_global_state": true,
"state": "SUCCESS",
"start_time": "2020-07-06T21:55:18.131Z",
"start_time_in_millis": 1593093628852,
"end_time": "2020-07-06T21:55:18.135Z",
"end_time_in_millis": 1593093628856,
"duration_in_millis": 4,
"failures": [],
"shards": {
"total": 0,
"failed": 0,
"successful": 0
}
}
],
"total": 3,
"remaining": 0
}Snapshot lifecycle management
Snapshot lifecycle management (SLM) APIs enable you to set up policies to automatically take snapshots and control how long they are retained.
Find the structure of text messages
Find the structure of a list of text messages. The messages must contain data that is suitable to be ingested into Elasticsearch.
This API provides a starting point for ingesting data into Elasticsearch in a format that is suitable for subsequent use with other Elastic Stack functionality. Use this API rather than the find text structure API if your input text has already been split up into separate messages by some other process.
The response from the API contains:
- Sample messages.
- Statistics that reveal the most common values for all fields detected within the text and basic numeric statistics for numeric fields.
- Information about the structure of the text, which is useful when you write ingest configurations to index it or similarly formatted text. Appropriate mappings for an Elasticsearch index, which you could use to ingest the text.
All this information can be calculated by the structure finder with no guidance. However, you can optionally override some of the decisions about the text structure by specifying one or more query parameters.
If the structure finder produces unexpected results, specify the explain query parameter and an explanation will appear in the response.
It helps determine why the returned structure was chosen.
Query parameters
-
column_names
string If the format is
delimited, you can specify the column names in a comma-separated list. If this parameter is not specified, the structure finder uses the column names from the header row of the text. If the text does not have a header role, columns are named "column1", "column2", "column3", for example. -
delimiter
string If you the format is
delimited, you can specify the character used to delimit the values in each row. Only a single character is supported; the delimiter cannot have multiple characters. By default, the API considers the following possibilities: comma, tab, semi-colon, and pipe (|). In this default scenario, all rows must have the same number of fields for the delimited format to be detected. If you specify a delimiter, up to 10% of the rows can have a different number of columns than the first row. -
ecs_compatibility
string The mode of compatibility with ECS compliant Grok patterns. Use this parameter to specify whether to use ECS Grok patterns instead of legacy ones when the structure finder creates a Grok pattern. This setting primarily has an impact when a whole message Grok pattern such as
%{CATALINALOG}matches the input. If the structure finder identifies a common structure but has no idea of meaning then generic field names such aspath,ipaddress,field1, andfield2are used in thegrok_patternoutput, with the intention that a user who knows the meanings rename these fields before using it.Values are
disabledorv1. -
explain
boolean If this parameter is set to true, the response includes a field named
explanation, which is an array of strings that indicate how the structure finder produced its result. -
format
string The high level structure of the text. By default, the API chooses the format. In this default scenario, all rows must have the same number of fields for a delimited format to be detected. If the format is
delimitedand the delimiter is not set, however, the API tolerates up to 5% of rows that have a different number of columns than the first row.Values are
delimited,ndjson,semi_structured_text, orxml. -
grok_pattern
string If the format is
semi_structured_text, you can specify a Grok pattern that is used to extract fields from every message in the text. The name of the timestamp field in the Grok pattern must match what is specified in thetimestamp_fieldparameter. If that parameter is not specified, the name of the timestamp field in the Grok pattern must match "timestamp". Ifgrok_patternis not specified, the structure finder creates a Grok pattern. -
quote
string If the format is
delimited, you can specify the character used to quote the values in each row if they contain newlines or the delimiter character. Only a single character is supported. If this parameter is not specified, the default value is a double quote ("). If your delimited text format does not use quoting, a workaround is to set this argument to a character that does not appear anywhere in the sample. -
should_trim_fields
boolean If the format is
delimited, you can specify whether values between delimiters should have whitespace trimmed from them. If this parameter is not specified and the delimiter is pipe (|), the default value is true. Otherwise, the default value isfalse. -
timeout
string The maximum amount of time that the structure analysis can take. If the analysis is still running when the timeout expires, it will be stopped.
Values are
-1or0. -
timestamp_field
string The name of the field that contains the primary timestamp of each record in the text. In particular, if the text was ingested into an index, this is the field that would be used to populate the
@timestampfield.If the format is
semi_structured_text, this field must match the name of the appropriate extraction in thegrok_pattern. Therefore, for semi-structured text, it is best not to specify this parameter unlessgrok_patternis also specified.For structured text, if you specify this parameter, the field must exist within the text.
If this parameter is not specified, the structure finder makes a decision about which field (if any) is the primary timestamp field. For structured text, it is not compulsory to have a timestamp in the text.
-
timestamp_format
string The Java time format of the timestamp field in the text. Only a subset of Java time format letter groups are supported:
adddEEEEEEEHHHhMMMMMMMMMMmmssXXXXXyyyyyyzzz
Additionally
Sletter groups (fractional seconds) of length one to nine are supported providing they occur afterssand are separated from thessby a period (.), comma (,), or colon (:). Spacing and punctuation is also permitted with the exception a question mark (?), newline, and carriage return, together with literal text enclosed in single quotes. For example,MM/dd HH.mm.ss,SSSSSS 'in' yyyyis a valid override format.One valuable use case for this parameter is when the format is semi-structured text, there are multiple timestamp formats in the text, and you know which format corresponds to the primary timestamp, but you do not want to specify the full
grok_pattern. Another is when the timestamp format is one that the structure finder does not consider by default.If this parameter is not specified, the structure finder chooses the best format from a built-in set.
If the special value
nullis specified, the structure finder will not look for a primary timestamp in the text. When the format is semi-structured text, this will result in the structure finder treating the text as single-line messages.
curl \
--request GET 'http://api.example.com/_text_structure/find_message_structure' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"messages\": [\n \"[2024-03-05T10:52:36,256][INFO ][o.a.l.u.VectorUtilPanamaProvider] [laptop] Java vector incubator API enabled; uses preferredBitSize=128\",\n \"[2024-03-05T10:52:41,038][INFO ][o.e.p.PluginsService ] [laptop] loaded module [repository-url]\",\n \"[2024-03-05T10:52:41,042][INFO ][o.e.p.PluginsService ] [laptop] loaded module [rest-root]\",\n \"[2024-03-05T10:52:41,043][INFO ][o.e.p.PluginsService ] [laptop] loaded module [x-pack-core]\",\n \"[2024-03-05T10:52:41,043][INFO ][o.e.p.PluginsService ] [laptop] loaded module [x-pack-redact]\",\n \"[2024-03-05T10:52:41,043][INFO ][o.e.p.PluginsService ] [laptop] loaded module [ingest-user-agent]\",\n \"[2024-03-05T10:52:41,044][INFO ][o.e.p.PluginsService ] [laptop] loaded module [x-pack-monitoring]\",\n \"[2024-03-05T10:52:41,044][INFO ][o.e.p.PluginsService ] [laptop] loaded module [repository-s3]\",\n \"[2024-03-05T10:52:41,044][INFO ][o.e.p.PluginsService ] [laptop] loaded module [x-pack-analytics]\",\n \"[2024-03-05T10:52:41,044][INFO ][o.e.p.PluginsService ] [laptop] loaded module [x-pack-ent-search]\",\n \"[2024-03-05T10:52:41,044][INFO ][o.e.p.PluginsService ] [laptop] loaded module [x-pack-autoscaling]\",\n \"[2024-03-05T10:52:41,044][INFO ][o.e.p.PluginsService ] [laptop] loaded module [lang-painless]]\",\n \"[2024-03-05T10:52:41,059][INFO ][o.e.p.PluginsService ] [laptop] loaded module [lang-expression]\",\n \"[2024-03-05T10:52:41,059][INFO ][o.e.p.PluginsService ] [laptop] loaded module [x-pack-eql]\",\n \"[2024-03-05T10:52:43,291][INFO ][o.e.e.NodeEnvironment ] [laptop] heap size [16gb], compressed ordinary object pointers [true]\",\n \"[2024-03-05T10:52:46,098][INFO ][o.e.x.s.Security ] [laptop] Security is enabled\",\n \"[2024-03-05T10:52:47,227][INFO ][o.e.x.p.ProfilingPlugin ] [laptop] Profiling is enabled\",\n \"[2024-03-05T10:52:47,259][INFO ][o.e.x.p.ProfilingPlugin ] [laptop] profiling index templates will not be installed or reinstalled\",\n \"[2024-03-05T10:52:47,755][INFO ][o.e.i.r.RecoverySettings ] [laptop] using rate limit [40mb] with [default=40mb, read=0b, write=0b, max=0b]\",\n \"[2024-03-05T10:52:47,787][INFO ][o.e.d.DiscoveryModule ] [laptop] using discovery type [multi-node] and seed hosts providers [settings]\",\n \"[2024-03-05T10:52:49,188][INFO ][o.e.n.Node ] [laptop] initialized\",\n \"[2024-03-05T10:52:49,199][INFO ][o.e.n.Node ] [laptop] starting ...\"\n ]\n}"'{
"messages": [
"[2024-03-05T10:52:36,256][INFO ][o.a.l.u.VectorUtilPanamaProvider] [laptop] Java vector incubator API enabled; uses preferredBitSize=128",
"[2024-03-05T10:52:41,038][INFO ][o.e.p.PluginsService ] [laptop] loaded module [repository-url]",
"[2024-03-05T10:52:41,042][INFO ][o.e.p.PluginsService ] [laptop] loaded module [rest-root]",
"[2024-03-05T10:52:41,043][INFO ][o.e.p.PluginsService ] [laptop] loaded module [x-pack-core]",
"[2024-03-05T10:52:41,043][INFO ][o.e.p.PluginsService ] [laptop] loaded module [x-pack-redact]",
"[2024-03-05T10:52:41,043][INFO ][o.e.p.PluginsService ] [laptop] loaded module [ingest-user-agent]",
"[2024-03-05T10:52:41,044][INFO ][o.e.p.PluginsService ] [laptop] loaded module [x-pack-monitoring]",
"[2024-03-05T10:52:41,044][INFO ][o.e.p.PluginsService ] [laptop] loaded module [repository-s3]",
"[2024-03-05T10:52:41,044][INFO ][o.e.p.PluginsService ] [laptop] loaded module [x-pack-analytics]",
"[2024-03-05T10:52:41,044][INFO ][o.e.p.PluginsService ] [laptop] loaded module [x-pack-ent-search]",
"[2024-03-05T10:52:41,044][INFO ][o.e.p.PluginsService ] [laptop] loaded module [x-pack-autoscaling]",
"[2024-03-05T10:52:41,044][INFO ][o.e.p.PluginsService ] [laptop] loaded module [lang-painless]]",
"[2024-03-05T10:52:41,059][INFO ][o.e.p.PluginsService ] [laptop] loaded module [lang-expression]",
"[2024-03-05T10:52:41,059][INFO ][o.e.p.PluginsService ] [laptop] loaded module [x-pack-eql]",
"[2024-03-05T10:52:43,291][INFO ][o.e.e.NodeEnvironment ] [laptop] heap size [16gb], compressed ordinary object pointers [true]",
"[2024-03-05T10:52:46,098][INFO ][o.e.x.s.Security ] [laptop] Security is enabled",
"[2024-03-05T10:52:47,227][INFO ][o.e.x.p.ProfilingPlugin ] [laptop] Profiling is enabled",
"[2024-03-05T10:52:47,259][INFO ][o.e.x.p.ProfilingPlugin ] [laptop] profiling index templates will not be installed or reinstalled",
"[2024-03-05T10:52:47,755][INFO ][o.e.i.r.RecoverySettings ] [laptop] using rate limit [40mb] with [default=40mb, read=0b, write=0b, max=0b]",
"[2024-03-05T10:52:47,787][INFO ][o.e.d.DiscoveryModule ] [laptop] using discovery type [multi-node] and seed hosts providers [settings]",
"[2024-03-05T10:52:49,188][INFO ][o.e.n.Node ] [laptop] initialized",
"[2024-03-05T10:52:49,199][INFO ][o.e.n.Node ] [laptop] starting ..."
]
}{
"num_lines_analyzed" : 22,
"num_messages_analyzed" : 22,
"sample_start" : "[2024-03-05T10:52:36,256][INFO ][o.a.l.u.VectorUtilPanamaProvider] [laptop] Java vector incubator API enabled; uses preferredBitSize=128\n[2024-03-05T10:52:41,038][INFO ][o.e.p.PluginsService ] [laptop] loaded module [repository-url]\n",
"charset" : "UTF-8",
"format" : "semi_structured_text",
"multiline_start_pattern" : "^\\[\\b\\d{4}-\\d{2}-\\d{2}[T ]\\d{2}:\\d{2}",
"grok_pattern" : "\\[%{TIMESTAMP_ISO8601:timestamp}\\]\\[%{LOGLEVEL:loglevel} \\]\\[.*",
"ecs_compatibility" : "disabled",
"timestamp_field" : "timestamp",
"joda_timestamp_formats" : [
"ISO8601"
],
"java_timestamp_formats" : [
"ISO8601"
],
"need_client_timezone" : true,
"mappings" : {
"properties" : {
"@timestamp" : {
"type" : "date"
},
"loglevel" : {
"type" : "keyword"
},
"message" : {
"type" : "text"
}
}
},
"ingest_pipeline" : {
"description" : "Ingest pipeline created by text structure finder",
"processors" : [
{
"grok" : {
"field" : "message",
"patterns" : [
"\\[%{TIMESTAMP_ISO8601:timestamp}\\]\\[%{LOGLEVEL:loglevel} \\]\\[.*"
],
"ecs_compatibility" : "disabled"
}
},
{
"date" : {
"field" : "timestamp",
"timezone" : "{{ event.timezone }}",
"formats" : [
"ISO8601"
]
}
},
{
"remove" : {
"field" : "timestamp"
}
}
]
},
"field_stats" : {
"loglevel" : {
"count" : 22,
"cardinality" : 1,
"top_hits" : [
{
"value" : "INFO",
"count" : 22
}
]
},
"message" : {
"count" : 22,
"cardinality" : 22,
"top_hits" : [
{
"value" : "[2024-03-05T10:52:36,256][INFO ][o.a.l.u.VectorUtilPanamaProvider] [laptop] Java vector incubator API enabled; uses preferredBitSize=128",
"count" : 1
},
{
"value" : "[2024-03-05T10:52:41,038][INFO ][o.e.p.PluginsService ] [laptop] loaded module [repository-url]",
"count" : 1
},
{
"value" : "[2024-03-05T10:52:41,042][INFO ][o.e.p.PluginsService ] [laptop] loaded module [rest-root]",
"count" : 1
},
{
"value" : "[2024-03-05T10:52:41,043][INFO ][o.e.p.PluginsService ] [laptop] loaded module [ingest-user-agent]",
"count" : 1
},
{
"value" : "[2024-03-05T10:52:41,043][INFO ][o.e.p.PluginsService ] [laptop] loaded module [x-pack-core]",
"count" : 1
},
{
"value" : "[2024-03-05T10:52:41,043][INFO ][o.e.p.PluginsService ] [laptop] loaded module [x-pack-redact]",
"count" : 1
},
{
"value" : "[2024-03-05T10:52:41,044][INFO ][o.e.p.PluginsService ] [laptop] loaded module [lang-painless]]",
"count" : 1
},
{
"value" : "[2024-03-05T10:52:41,044][INFO ][o.e.p.PluginsService ] [laptop] loaded module [repository-s3]",
"count" : 1
},
{
"value" : "[2024-03-05T10:52:41,044][INFO ][o.e.p.PluginsService ] [laptop] loaded module [x-pack-analytics]",
"count" : 1
},
{
"value" : "[2024-03-05T10:52:41,044][INFO ][o.e.p.PluginsService ] [laptop] loaded module [x-pack-autoscaling]",
"count" : 1
}
]
},
"timestamp" : {
"count" : 22,
"cardinality" : 14,
"earliest" : "2024-03-05T10:52:36,256",
"latest" : "2024-03-05T10:52:49,199",
"top_hits" : [
{
"value" : "2024-03-05T10:52:41,044",
"count" : 6
},
{
"value" : "2024-03-05T10:52:41,043",
"count" : 3
},
{
"value" : "2024-03-05T10:52:41,059",
"count" : 2
},
{
"value" : "2024-03-05T10:52:36,256",
"count" : 1
},
{
"value" : "2024-03-05T10:52:41,038",
"count" : 1
},
{
"value" : "2024-03-05T10:52:41,042",
"count" : 1
},
{
"value" : "2024-03-05T10:52:43,291",
"count" : 1
},
{
"value" : "2024-03-05T10:52:46,098",
"count" : 1
},
{
"value" : "2024-03-05T10:52:47,227",
"count" : 1
},
{
"value" : "2024-03-05T10:52:47,259",
"count" : 1
}
]
}
}
}Deactivate a watch
A watch can be either active or inactive.
Path parameters
-
watch_id
string Required The watch identifier.
curl \
--request POST 'http://api.example.com/_watcher/watch/{watch_id}/_deactivate' \
--header "Authorization: $API_KEY"Run a watch
This API can be used to force execution of the watch outside of its triggering logic or to simulate the watch execution for debugging purposes.
For testing and debugging purposes, you also have fine-grained control on how the watch runs. You can run the watch without running all of its actions or alternatively by simulating them. You can also force execution by ignoring the watch condition and control whether a watch record would be written to the watch history after it runs.
You can use the run watch API to run watches that are not yet registered by specifying the watch definition inline. This serves as great tool for testing and debugging your watches prior to adding them to Watcher.
When Elasticsearch security features are enabled on your cluster, watches are run with the privileges of the user that stored the watches.
If your user is allowed to read index a, but not index b, then the exact same set of rules will apply during execution of a watch.
When using the run watch API, the authorization data of the user that called the API will be used as a base, instead of the information who stored the watch.
Query parameters
-
debug
boolean Defines whether the watch runs in debug mode.
Body
-
action_modes
object Determines how to handle the watch actions as part of the watch execution.
-
alternative_input
object When present, the watch uses this object as a payload instead of executing its own input.
-
ignore_condition
boolean When set to
true, the watch execution uses the always condition. This can also be specified as an HTTP parameter. -
record_execution
boolean When set to
true, the watch record representing the watch execution result is persisted to the.watcher-historyindex for the current time. In addition, the status of the watch is updated, possibly throttling subsequent runs. This can also be specified as an HTTP parameter. -
simulated_actions
object -
trigger_data
object -
watch
object
curl \
--request POST 'http://api.example.com/_watcher/watch/_execute' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"trigger_data\" : { \n \"triggered_time\" : \"now\",\n \"scheduled_time\" : \"now\"\n },\n \"alternative_input\" : { \n \"foo\" : \"bar\"\n },\n \"ignore_condition\" : true, \n \"action_modes\" : {\n \"my-action\" : \"force_simulate\" \n },\n \"record_execution\" : true \n}"'{
"trigger_data" : {
"triggered_time" : "now",
"scheduled_time" : "now"
},
"alternative_input" : {
"foo" : "bar"
},
"ignore_condition" : true,
"action_modes" : {
"my-action" : "force_simulate"
},
"record_execution" : true
}{
"action_modes" : {
"action1" : "force_simulate",
"action2" : "skip"
}
}{
"watch" : {
"trigger" : { "schedule" : { "interval" : "10s" } },
"input" : {
"search" : {
"request" : {
"indices" : [ "logs" ],
"body" : {
"query" : {
"match" : { "message": "error" }
}
}
}
}
},
"condition" : {
"compare" : { "ctx.payload.hits.total" : { "gt" : 0 }}
},
"actions" : {
"log_error" : {
"logging" : {
"text" : "Found {{ctx.payload.hits.total}} errors in the logs"
}
}
}
}
}{
"_id": "my_watch_0-2015-06-02T23:17:55.124Z",
"watch_record": {
"@timestamp": "2015-06-02T23:17:55.124Z",
"watch_id": "my_watch",
"node": "my_node",
"messages": [],
"trigger_event": {
"type": "manual",
"triggered_time": "2015-06-02T23:17:55.124Z",
"manual": {
"schedule": {
"scheduled_time": "2015-06-02T23:17:55.124Z"
}
}
},
"state": "executed",
"status": {
"version": 1,
"execution_state": "executed",
"state": {
"active": true,
"timestamp": "2015-06-02T23:17:55.111Z"
},
"last_checked": "2015-06-02T23:17:55.124Z",
"last_met_condition": "2015-06-02T23:17:55.124Z",
"actions": {
"test_index": {
"ack": {
"timestamp": "2015-06-02T23:17:55.124Z",
"state": "ackable"
},
"last_execution": {
"timestamp": "2015-06-02T23:17:55.124Z",
"successful": true
},
"last_successful_execution": {
"timestamp": "2015-06-02T23:17:55.124Z",
"successful": true
}
}
}
},
"input": {
"simple": {
"payload": {
"send": "yes"
}
}
},
"condition": {
"always": {}
},
"result": {
"execution_time": "2015-06-02T23:17:55.124Z",
"execution_duration": 12608,
"input": {
"type": "simple",
"payload": {
"foo": "bar"
},
"status": "success"
},
"condition": {
"type": "always",
"met": true,
"status": "success"
},
"actions": [
{
"id": "test_index",
"index": {
"response": {
"index": "test",
"version": 1,
"created": true,
"result": "created",
"id": "AVSHKzPa9zx62AzUzFXY"
}
},
"status": "success",
"type": "index"
}
]
},
"user": "test_admin"
}
}