Forget a follower
Added in 6.7.0
Remove the cross-cluster replication follower retention leases from the leader.
A following index takes out retention leases on its leader index. These leases are used to increase the likelihood that the shards of the leader index retain the history of operations that the shards of the following index need to run replication. When a follower index is converted to a regular index by the unfollow API (either by directly calling the API or by index lifecycle management tasks), these leases are removed. However, removal of the leases can fail, for example when the remote cluster containing the leader index is unavailable. While the leases will eventually expire on their own, their extended existence can cause the leader index to hold more history than necessary and prevent index lifecycle management from performing some operations on the leader index. This API exists to enable manually removing the leases when the unfollow API is unable to do so.
NOTE: This API does not stop replication by a following index. If you use this API with a follower index that is still actively following, the following index will add back retention leases on the leader. The only purpose of this API is to handle the case of failure to remove the following retention leases after the unfollow API is invoked.
Path parameters
-
index
string Required the name of the leader index for which specified follower retention leases should be removed
Query parameters
-
timeout
string Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
Body
Required
-
follower_cluster
string -
follower_index
string -
follower_index_uuid
string -
leader_remote_cluster
string
curl \
--request POST 'http://api.example.com/{index}/_ccr/forget_follower' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"follower_cluster\" : \"\u003cfollower_cluster\u003e\",\n \"follower_index\" : \"\u003cfollower_index\u003e\",\n \"follower_index_uuid\" : \"\u003cfollower_index_uuid\u003e\",\n \"leader_remote_cluster\" : \"\u003cleader_remote_cluster\u003e\"\n}"'{
"follower_cluster" : "<follower_cluster>",
"follower_index" : "<follower_index>",
"follower_index_uuid" : "<follower_index_uuid>",
"leader_remote_cluster" : "<leader_remote_cluster>"
}{
"_shards" : {
"total" : 1,
"successful" : 1,
"failed" : 0,
"failures" : [ ]
}
}
Pause an auto-follow pattern
Added in 7.5.0
Pause a cross-cluster replication auto-follow pattern. When the API returns, the auto-follow pattern is inactive. New indices that are created on the remote cluster and match the auto-follow patterns are ignored.
You can resume auto-following with the resume auto-follow pattern API. When it resumes, the auto-follow pattern is active again and automatically configures follower indices for newly created indices on the remote cluster that match its patterns. Remote indices that were created while the pattern was paused will also be followed, unless they have been deleted or closed in the interim.
Path parameters
-
name
string Required The name of the auto-follow pattern to pause.
Query parameters
-
master_timeout
string The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. It can also be set to
-1to indicate that the request should never timeout.
curl \
--request POST 'http://api.example.com/_ccr/auto_follow/{name}/pause' \
--header "Authorization: $API_KEY"{
"acknowledged" : true
}
Unfollow an index
Added in 6.5.0
Convert a cross-cluster replication follower index to a regular index. The API stops the following task associated with a follower index and removes index metadata and settings associated with cross-cluster replication. The follower index must be paused and closed before you call the unfollow API.
Currently cross-cluster replication does not support converting an existing regular index to a follower index. Converting a follower index to a regular index is an irreversible operation.
Path parameters
-
index
string Required The name of the follower index.
Query parameters
-
master_timeout
string The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. It can also be set to
-1to indicate that the request should never timeout.
curl \
--request POST 'http://api.example.com/{index}/_ccr/unfollow' \
--header "Authorization: $API_KEY"{
"acknowledged" : true
}Get index information
Get information about one or more indices. For data streams, the API returns information about the stream’s backing indices.
Path parameters
-
index
string | array[string] Required Comma-separated list of data streams, indices, and index aliases used to limit the request. Wildcard expressions (*) are supported.
Query parameters
-
allow_no_indices
boolean If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.
-
expand_wildcards
string | array[string] Type of index that wildcard expressions can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined withopen,closed, orboth.none: Wildcard expressions are not accepted.
-
flat_settings
boolean If true, returns settings in flat format.
-
include_defaults
boolean If true, return all default settings in the response.
-
local
boolean If true, the request retrieves information from the local node only. Defaults to false, which means information is retrieved from the master node.
-
master_timeout
string Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
-
features
string | array[string] Return only information on specified index features
Supported values include:
aliases,mappings,settings
curl \
--request GET 'http://api.example.com/{index}' \
--header "Authorization: $API_KEY"Refresh an index
A refresh makes recent operations performed on one or more indices available for search. For data streams, the API runs the refresh operation on the stream’s backing indices.
By default, Elasticsearch periodically refreshes indices every second, but only on indices that have received one search request or more in the last 30 seconds.
You can change this default interval with the index.refresh_interval setting.
Refresh requests are synchronous and do not return a response until the refresh operation completes.
Refreshes are resource-intensive. To ensure good cluster performance, it's recommended to wait for Elasticsearch's periodic refresh rather than performing an explicit refresh when possible.
If your application workflow indexes documents and then runs a search to retrieve the indexed document, it's recommended to use the index API's refresh=wait_for query parameter option.
This option ensures the indexing operation waits for a periodic refresh before running the search.
Query parameters
-
allow_no_indices
boolean If
false, the request returns an error if any wildcard expression, index alias, or_allvalue targets only missing or closed indices. This behavior applies even if the request targets other open indices. -
expand_wildcards
string | array[string] Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as
open,hidden. Valid values are:all,open,closed,hidden,none.Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined withopen,closed, orboth.none: Wildcard expressions are not accepted.
curl \
--request POST 'http://api.example.com/_refresh' \
--header "Authorization: $API_KEY"
Simulate data ingestion
Technical preview
Run ingest pipelines against a set of provided documents, optionally with substitute pipeline definitions, to simulate ingesting data into an index.
This API is meant to be used for troubleshooting or pipeline development, as it does not actually index any data into Elasticsearch.
The API runs the default and final pipeline for that index against a set of documents provided in the body of the request. If a pipeline contains a reroute processor, it follows that reroute processor to the new index, running that index's pipelines as well the same way that a non-simulated ingest would. No data is indexed into Elasticsearch. Instead, the transformed document is returned, along with the list of pipelines that have been run and the name of the index where the document would have been indexed if this were not a simulation. The transformed document is validated against the mappings that would apply to this index, and any validation error is reported in the result.
This API differs from the simulate pipeline API in that you specify a single pipeline for that API, and it runs only that one pipeline. The simulate pipeline API is more useful for developing a single pipeline, while the simulate ingest API is more useful for troubleshooting the interaction of the various pipelines that get applied when ingesting into an index.
By default, the pipeline definitions that are currently in the system are used. However, you can supply substitute pipeline definitions in the body of the request. These will be used in place of the pipeline definitions that are already in the system. This can be used to replace existing pipeline definitions or to create new ones. The pipeline substitutions are used only within this request.
Query parameters
-
pipeline
string The pipeline to use as the default pipeline. This value can be used to override the default pipeline of the index.
Body
Required
-
docs
array[object] Required Sample documents to test in the pipeline.
-
A map of component template names to substitute component template definition objects.
-
index_template_substitutions
object A map of index template names to substitute index template definition objects.
-
mapping_addition
object -
pipeline_substitutions
object Pipelines to test. If you don’t specify the
pipelinerequest path parameter, this parameter is required. If you specify both this and the request path parameter, the API only uses the request path parameter.
curl \
--request GET 'http://api.example.com/_ingest/_simulate' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"docs\": [\n {\n \"_id\": 123,\n \"_index\": \"my-index\",\n \"_source\": {\n \"foo\": \"bar\"\n }\n },\n {\n \"_id\": 456,\n \"_index\": \"my-index\",\n \"_source\": {\n \"foo\": \"rab\"\n }\n }\n ]\n}"'{
"docs": [
{
"_id": 123,
"_index": "my-index",
"_source": {
"foo": "bar"
}
},
{
"_id": 456,
"_index": "my-index",
"_source": {
"foo": "rab"
}
}
]
}{
"docs": [
{
"_index": "my-index",
"_id": 123,
"_source": {
"foo": "bar"
}
},
{
"_index": "my-index",
"_id": 456,
"_source": {
"foo": "rab"
}
}
],
"pipeline_substitutions": {
"my-pipeline": {
"processors": [
{
"uppercase": {
"field": "foo"
}
}
]
}
}
}{
"docs": [
{
"_index": "my-index",
"_id": "123",
"_source": {
"foo": "foo"
}
},
{
"_index": "my-index",
"_id": "456",
"_source": {
"bar": "rab"
}
}
],
"component_template_substitutions": {
"my-mappings_template": {
"template": {
"mappings": {
"dynamic": "strict",
"properties": {
"foo": {
"type": "keyword"
},
"bar": {
"type": "keyword"
}
}
}
}
}
}
}{
"docs": [
{
"_id": "id",
"_index": "my-index",
"_source": {
"foo": "bar"
}
},
{
"_id": "id",
"_index": "my-index",
"_source": {
"foo": "rab"
}
}
],
"pipeline_substitutions": {
"my-pipeline": {
"processors": [
{
"set": {
"field": "field3",
"value": "value3"
}
}
]
}
},
"component_template_substitutions": {
"my-component-template": {
"template": {
"mappings": {
"dynamic": true,
"properties": {
"field3": {
"type": "keyword"
}
}
},
"settings": {
"index": {
"default_pipeline": "my-pipeline"
}
}
}
}
},
"index_template_substitutions": {
"my-index-template": {
"index_patterns": [
"my-index-*"
],
"composed_of": [
"component_template_1",
"component_template_2"
]
}
},
"mapping_addition": {
"dynamic": "strict",
"properties": {
"foo": {
"type": "keyword"
}
}
}
}{
"docs": [
{
"doc": null,
"_id": 123,
"_index": "my-index",
"_version": -3,
"_source": {
"field1": "value1",
"field2": "value2",
"foo": "bar"
},
"executed_pipelines": [
"my-pipeline",
"my-final-pipeline"
]
},
{
"doc": null,
"_id": 456,
"_index": "my-index",
"_version": "-3,",
"_source": {
"field1": "value1",
"field2": "value2",
"foo": "rab"
},
"executed_pipelines": [
"my-pipeline",
"my-final-pipeline"
]
}
]
}{
"docs": [
{
"doc": null,
"_id": 123,
"_index": "my-index",
"_version": -3,
"_source": {
"field2": "value2",
"foo": "BAR"
},
"executed_pipelines": [
"my-pipeline",
"my-final-pipeline"
]
},
{
"doc": null,
"_id": 456,
"_index": "my-index",
"_version": -3,
"_source": {
"field2": "value2",
"foo": "RAB"
},
"executed_pipelines": [
"my-pipeline",
"my-final-pipeline"
]
}
]
}{
"docs": [
{
"doc": {
"_id": "123",
"_index": "my-index",
"_version": -3,
"_source": {
"foo": "foo"
},
"executed_pipelines": []
}
},
{
"doc": {
"_id": "456",
"_index": "my-index",
"_version": -3,
"_source": {
"bar": "rab"
},
"executed_pipelines": []
}
}
]
}Path parameters
-
id
string Required The identifier for the stored script or search template.
Query parameters
-
master_timeout
string The period to wait for the master node. If the master node is not available before the timeout expires, the request fails and returns an error. It can also be set to
-1to indicate that the request should never timeout.
curl \
--request GET 'http://api.example.com/_scripts/{id}' \
--header "Authorization: $API_KEY"