Get trained models Generally available; Added in 7.7.0

GET /_cat/ml/trained_models/{model_id}
Api key auth Basic auth Bearer auth

All methods and paths for this operation:

GET /_cat/ml/trained_models

GET /_cat/ml/trained_models/{model_id}

Get configuration and usage information about inference trained models.

IMPORTANT: CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get trained models statistics API.

Required authorization

  • Cluster privileges: monitor_ml

Path parameters

  • model_id string Required

    A unique identifier for the trained model.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request: contains wildcard expressions and there are no models that match; contains the _all string or no identifiers and there are no matches; contains wildcard expressions and there are only partial matches. If true, the API returns an empty array when there are no matches and the subset of results when there are partial matches. If false, the API returns a 404 status code when there are no matches or only partial matches.

  • bytes string

    The unit used to display byte values.

    Values are b, kb, mb, gb, tb, or pb.

  • h string | array[string]

    A comma-separated list of column names to display.

    Supported values include:

    • create_time (or ct): The time when the trained model was created.
    • created_by (or c, createdBy): Information on the creator of the trained model.
    • data_frame_analytics_id (or df, dataFrameAnalytics, dfid): Identifier for the data frame analytics job that created the model. Only displayed if it is still available.
    • description (or d): The description of the trained model.
    • heap_size (or hs, modelHeapSize): The estimated heap size to keep the trained model in memory.
    • id: Identifier for the trained model.
    • ingest.count (or ic, ingestCount): The total number of documents that are processed by the model.
    • ingest.current (or icurr, ingestCurrent): The total number of document that are currently being handled by the trained model.
    • ingest.failed (or if, ingestFailed): The total number of failed ingest attempts with the trained model.
    • ingest.pipelines (or ip, ingestPipelines): The total number of ingest pipelines that are referencing the trained model.
    • ingest.time (or it, ingestTime): The total time that is spent processing documents with the trained model.
    • license (or l): The license level of the trained model.
    • operations (or o, modelOperations): The estimated number of operations to use the trained model. This number helps measuring the computational complexity of the model.
    • version (or v): The Elasticsearch version number in which the trained model was created.
  • s string | array[string]

    A comma-separated list of column names or aliases used to sort the response.

    Supported values include:

    • create_time (or ct): The time when the trained model was created.
    • created_by (or c, createdBy): Information on the creator of the trained model.
    • data_frame_analytics_id (or df, dataFrameAnalytics, dfid): Identifier for the data frame analytics job that created the model. Only displayed if it is still available.
    • description (or d): The description of the trained model.
    • heap_size (or hs, modelHeapSize): The estimated heap size to keep the trained model in memory.
    • id: Identifier for the trained model.
    • ingest.count (or ic, ingestCount): The total number of documents that are processed by the model.
    • ingest.current (or icurr, ingestCurrent): The total number of document that are currently being handled by the trained model.
    • ingest.failed (or if, ingestFailed): The total number of failed ingest attempts with the trained model.
    • ingest.pipelines (or ip, ingestPipelines): The total number of ingest pipelines that are referencing the trained model.
    • ingest.time (or it, ingestTime): The total time that is spent processing documents with the trained model.
    • license (or l): The license level of the trained model.
    • operations (or o, modelOperations): The estimated number of operations to use the trained model. This number helps measuring the computational complexity of the model.
    • version (or v): The Elasticsearch version number in which the trained model was created.
  • from number

    Skips the specified number of transforms.

  • size number

    The maximum number of transforms to display.

  • time string

    Unit used to display time values.

    Values are nanos, micros, ms, s, m, h, or d.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • id string

      The model identifier.

    • created_by string

      Information about the creator of the model.

    • heap_size number | string

      The estimated heap size to keep the model in memory.

      One of:
      number-1 number string-2 string

      The estimated heap size to keep the model in memory.

    • operations string

      The estimated number of operations to use the model. This number helps to measure the computational complexity of the model.

    • license string

      The license level of the model.

    • create_time string | number

      The time the model was created.

      One of:
      string-1 string UnitMillis number

      The time the model was created.

    • version string

      The version of Elasticsearch when the model was created.

    • description string

      A description of the model.

    • ingest.pipelines string

      The number of pipelines that are referencing the model.

    • ingest.count string

      The total number of documents that are processed by the model.

    • ingest.time string

      The total time spent processing documents with thie model.

    • ingest.current string

      The total number of documents that are currently being handled by the model.

    • ingest.failed string

      The total number of failed ingest attempts with the model.

    • data_frame.id string

      The identifier for the data frame analytics job that created the model. Only displayed if the job is still available.

    • data_frame.create_time string

      The time the data frame analytics job was created.

    • data_frame.source_index string

      The source index used to train in the data frame analysis.

    • data_frame.analysis string

      The analysis used by the data frame to build the model.

    • type string Generally available; Added in 8.0.0
GET /_cat/ml/trained_models/{model_id} 
GET _cat/ml/trained_models?v=true&format=json

resp = client.cat.ml_trained_models(
    v=True,
    format="json",
)
const response = await client.cat.mlTrainedModels({
  v: "true",
  format: "json",
});
response = client.cat.ml_trained_models(
  v: "true",
  format: "json"
)
$resp = $client->cat()->mlTrainedModels([
    "v" => "true",
    "format" => "json",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_cat/ml/trained_models?v=true&format=json"
client.cat().mlTrainedModels();
Response examples (200)
A successful response from `GET _cat/ml/trained_models?v=true&format=json`. 
[
  {
    "id": "ddddd-1580216177138",
    "heap_size": "0b",
    "operations": "196",
    "create_time": "2025-03-25T00:01:38.662Z",
    "type": "pytorch",
    "ingest.pipelines": "0",
    "data_frame.id": "__none__"
  },
  {
    "id": "lang_ident_model_1",
    "heap_size": "1mb",
    "operations": "39629",
    "create_time": "2019-12-05T12:28:34.594Z",
    "type": "lang_ident",
    "ingest.pipelines": "0",
    "data_frame.id": "__none__"
  }
]

Get node attribute information Generally available

GET /_cat/nodeattrs
Api key auth Basic auth Bearer auth

Get information about custom node attributes. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the nodes info API.

Required authorization

  • Cluster privileges: monitor

Query parameters

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

  • local boolean

    If true, the request computes the list of selected nodes from the local cluster state. If false the list of selected nodes are computed from the cluster state of the master node. In both cases the coordinating node will send requests for further information to each selected node.

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • node string

      The node name.

    • id string

      The unique node identifier.

    • pid string

      The process identifier.

    • host string

      The host name.

    • ip string

      The IP address.

    • port string

      The bound transport port.

    • attr string

      The attribute name.

    • value string

      The attribute value.
GET /_cat/nodeattrs 
GET /_cat/nodeattrs?v=true&format=json

resp = client.cat.nodeattrs(
    v=True,
    format="json",
)
const response = await client.cat.nodeattrs({
  v: "true",
  format: "json",
});
response = client.cat.nodeattrs(
  v: "true",
  format: "json"
)
$resp = $client->cat()->nodeattrs([
    "v" => "true",
    "format" => "json",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_cat/nodeattrs?v=true&format=json"
client.cat().nodeattrs();
Response examples (200)
A successful response from `GET /_cat/nodeattrs?v=true&format=json`. The `node`, `host`, and `ip` columns provide basic information about each node. The `attr` and `value` columns return custom node attributes, one per line. 
[
  {
    "node": "node-0",
    "host": "127.0.0.1",
    "ip": "127.0.0.1",
    "attr": "testattr",
    "value": "test"
  }
]
A successful response from `GET /_cat/nodeattrs?v=true&h=name,pid,attr,value`. It returns the `name`, `pid`, `attr`, and `value` columns. 
[
  {
    "name": "node-0",
    "pid": "19566",
    "attr": "testattr",
    "value": "test"
  }
]

Get transform information Generally available; Added in 7.7.0

GET /_cat/transforms/{transform_id}
Api key auth Basic auth Bearer auth

All methods and paths for this operation:

GET /_cat/transforms

GET /_cat/transforms/{transform_id}

Get configuration and usage information about transforms.

CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get transform statistics API.

Required authorization

  • Cluster privileges: monitor_transform

Path parameters

  • transform_id string Required

    A transform identifier or a wildcard expression. If you do not specify one of these options, the API returns information for all transforms.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request: contains wildcard expressions and there are no transforms that match; contains the _all string or no identifiers and there are no matches; contains wildcard expressions and there are only partial matches. If true, it returns an empty transforms array when there are no matches and the subset of results when there are partial matches. If false, the request returns a 404 status code when there are no matches or only partial matches.

  • from number

    Skips the specified number of transforms.

  • h string | array[string]

    Comma-separated list of column names to display.

    Supported values include:

    • changes_last_detection_time (or cldt): The timestamp when changes were last detected in the source indices.
    • checkpoint (or cp): The sequence number for the checkpoint.
    • checkpoint_duration_time_exp_avg (or cdtea, checkpointTimeExpAvg): Exponential moving average of the duration of the checkpoint, in milliseconds.
    • checkpoint_progress (or c, checkpointProgress): The progress of the next checkpoint that is currently in progress.
    • create_time (or ct, createTime): The time the transform was created.
    • delete_time (or dtime): The amount of time spent deleting, in milliseconds.
    • description (or d): The description of the transform.
    • dest_index (or di, destIndex): The destination index for the transform. The mappings of the destination index are deduced based on the source fields when possible. If alternate mappings are required, use the Create index API prior to starting the transform.
    • documents_deleted (or docd): The number of documents that have been deleted from the destination index due to the retention policy for this transform.
    • documents_indexed (or doci): The number of documents that have been indexed into the destination index for the transform.
    • docs_per_second (or dps): Specifies a limit on the number of input documents per second. This setting throttles the transform by adding a wait time between search requests. The default value is null, which disables throttling.
    • documents_processed (or docp): The number of documents that have been processed from the source index of the transform.
    • frequency (or f): The interval between checks for changes in the source indices when the transform is running continuously. Also determines the retry interval in the event of transient failures while the transform is searching or indexing. The minimum value is 1s and the maximum is 1h. The default value is 1m.
    • id: Identifier for the transform.
    • index_failure (or if): The number of indexing failures.
    • index_time (or itime): The amount of time spent indexing, in milliseconds.
    • index_total (or it): The number of index operations.
    • indexed_documents_exp_avg (or idea): Exponential moving average of the number of new documents that have been indexed.
    • last_search_time (or lst, lastSearchTime): The timestamp of the last search in the source indices. This field is only shown if the transform is running.
    • max_page_search_size (or mpsz): Defines the initial page size to use for the composite aggregation for each checkpoint. If circuit breaker exceptions occur, the page size is dynamically adjusted to a lower value. The minimum value is 10 and the maximum is 65,536. The default value is 500.
    • pages_processed (or pp): The number of search or bulk index operations processed. Documents are processed in batches instead of individually.
    • pipeline (or p): The unique identifier for an ingest pipeline.
    • processed_documents_exp_avg (or pdea): Exponential moving average of the number of documents that have been processed.
    • processing_time (or pt): The amount of time spent processing results, in milliseconds.
    • reason (or r): If a transform has a failed state, this property provides details about the reason for the failure.
    • search_failure (or sf): The number of search failures.
    • search_time (or stime): The amount of time spent searching, in milliseconds.
    • search_total (or st): The number of search operations on the source index for the transform.
    • source_index (or si, sourceIndex): The source indices for the transform. It can be a single index, an index pattern (for example, "my-index-*"), an array of indices (for example, ["my-index-000001", "my-index-000002"]), or an array of index patterns (for example, ["my-index-*", "my-other-index-*"]. For remote indices use the syntax "remote_name:index_name". If any indices are in remote clusters then the master node and at least one transform node must have the remote_cluster_client node role.

    • state (or s): The status of the transform, which can be one of the following values:

      • aborting: The transform is aborting.
      • failed: The transform failed. For more information about the failure, check the reason field.
      • indexing: The transform is actively processing data and creating new documents.
      • started: The transform is running but not actively indexing data.
      • stopped: The transform is stopped.
      • stopping: The transform is stopping.

    • transform_type (or tt): Indicates the type of transform: batch or continuous.

    • trigger_count (or tc): The number of times the transform has been triggered by the scheduler. For example, the scheduler triggers the transform indexer to check for updates or ingest new data at an interval specified in the frequency property.

    • version (or v): The version of Elasticsearch that existed on the node when the transform was created.

  • s string | array[string]

    Comma-separated list of column names or column aliases used to sort the response.

    Supported values include:

    • changes_last_detection_time (or cldt): The timestamp when changes were last detected in the source indices.
    • checkpoint (or cp): The sequence number for the checkpoint.
    • checkpoint_duration_time_exp_avg (or cdtea, checkpointTimeExpAvg): Exponential moving average of the duration of the checkpoint, in milliseconds.
    • checkpoint_progress (or c, checkpointProgress): The progress of the next checkpoint that is currently in progress.
    • create_time (or ct, createTime): The time the transform was created.
    • delete_time (or dtime): The amount of time spent deleting, in milliseconds.
    • description (or d): The description of the transform.
    • dest_index (or di, destIndex): The destination index for the transform. The mappings of the destination index are deduced based on the source fields when possible. If alternate mappings are required, use the Create index API prior to starting the transform.
    • documents_deleted (or docd): The number of documents that have been deleted from the destination index due to the retention policy for this transform.
    • documents_indexed (or doci): The number of documents that have been indexed into the destination index for the transform.
    • docs_per_second (or dps): Specifies a limit on the number of input documents per second. This setting throttles the transform by adding a wait time between search requests. The default value is null, which disables throttling.
    • documents_processed (or docp): The number of documents that have been processed from the source index of the transform.
    • frequency (or f): The interval between checks for changes in the source indices when the transform is running continuously. Also determines the retry interval in the event of transient failures while the transform is searching or indexing. The minimum value is 1s and the maximum is 1h. The default value is 1m.
    • id: Identifier for the transform.
    • index_failure (or if): The number of indexing failures.
    • index_time (or itime): The amount of time spent indexing, in milliseconds.
    • index_total (or it): The number of index operations.
    • indexed_documents_exp_avg (or idea): Exponential moving average of the number of new documents that have been indexed.
    • last_search_time (or lst, lastSearchTime): The timestamp of the last search in the source indices. This field is only shown if the transform is running.
    • max_page_search_size (or mpsz): Defines the initial page size to use for the composite aggregation for each checkpoint. If circuit breaker exceptions occur, the page size is dynamically adjusted to a lower value. The minimum value is 10 and the maximum is 65,536. The default value is 500.
    • pages_processed (or pp): The number of search or bulk index operations processed. Documents are processed in batches instead of individually.
    • pipeline (or p): The unique identifier for an ingest pipeline.
    • processed_documents_exp_avg (or pdea): Exponential moving average of the number of documents that have been processed.
    • processing_time (or pt): The amount of time spent processing results, in milliseconds.
    • reason (or r): If a transform has a failed state, this property provides details about the reason for the failure.
    • search_failure (or sf): The number of search failures.
    • search_time (or stime): The amount of time spent searching, in milliseconds.
    • search_total (or st): The number of search operations on the source index for the transform.
    • source_index (or si, sourceIndex): The source indices for the transform. It can be a single index, an index pattern (for example, "my-index-*"), an array of indices (for example, ["my-index-000001", "my-index-000002"]), or an array of index patterns (for example, ["my-index-*", "my-other-index-*"]. For remote indices use the syntax "remote_name:index_name". If any indices are in remote clusters then the master node and at least one transform node must have the remote_cluster_client node role.

    • state (or s): The status of the transform, which can be one of the following values:

      • aborting: The transform is aborting.
      • failed: The transform failed. For more information about the failure, check the reason field.
      • indexing: The transform is actively processing data and creating new documents.
      • started: The transform is running but not actively indexing data.
      • stopped: The transform is stopped.
      • stopping: The transform is stopping.

    • transform_type (or tt): Indicates the type of transform: batch or continuous.

    • trigger_count (or tc): The number of times the transform has been triggered by the scheduler. For example, the scheduler triggers the transform indexer to check for updates or ingest new data at an interval specified in the frequency property.

    • version (or v): The version of Elasticsearch that existed on the node when the transform was created.

  • time string

    The unit used to display time values.

    Values are nanos, micros, ms, s, m, h, or d.

  • size number

    The maximum number of transforms to obtain.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • id string

      The transform identifier.

    • state string

      The status of the transform. Returned values include: aborting: The transform is aborting. failed: The transform failed. For more information about the failure, check thereasonfield. indexing: The transform is actively processing data and creating new documents. started: The transform is running but not actively indexing data. stopped: The transform is stopped. stopping`: The transform is stopping.

    • checkpoint string

      The sequence number for the checkpoint.

    • documents_processed string

      The number of documents that have been processed from the source index of the transform.

    • checkpoint_progress string | null

      The progress of the next checkpoint that is currently in progress.

      One of:
      string-1 string string-2 string | null

    • last_search_time string | null

      The timestamp of the last search in the source indices. This field is shown only if the transform is running.

      One of:
      string-1 string string-2 string | null

    • changes_last_detection_time string | null

      The timestamp when changes were last detected in the source indices.

      One of:
      string-1 string string-2 string | null
    • create_time string

      The time the transform was created.

    • version string

      The version of Elasticsearch that existed on the node when the transform was created.

    • source_index string

      The source indices for the transform.

    • dest_index string

      The destination index for the transform.

    • pipeline string

      The unique identifier for the ingest pipeline.

    • description string

      The description of the transform.

    • transform_type string

      The type of transform: batch or continuous.

    • frequency string

      The interval between checks for changes in the source indices when the transform is running continuously.

    • max_page_search_size string

      The initial page size that is used for the composite aggregation for each checkpoint.

    • docs_per_second string

      The number of input documents per second.

    • reason string

      If a transform has a failed state, these details describe the reason for failure.

    • search_total string

      The total number of search operations on the source index for the transform.

    • search_failure string

      The total number of search failures.

    • search_time string

      The total amount of search time, in milliseconds.

    • index_total string

      The total number of index operations done by the transform.

    • index_failure string

      The total number of indexing failures.

    • index_time string

      The total time spent indexing documents, in milliseconds.

    • documents_indexed string

      The number of documents that have been indexed into the destination index for the transform.

    • delete_time string

      The total time spent deleting documents, in milliseconds.

    • documents_deleted string

      The number of documents deleted from the destination index due to the retention policy for the transform.

    • trigger_count string

      The number of times the transform has been triggered by the scheduler. For example, the scheduler triggers the transform indexer to check for updates or ingest new data at an interval specified in the frequency property.

    • pages_processed string

      The number of search or bulk index operations processed. Documents are processed in batches instead of individually.

    • processing_time string

      The total time spent processing results, in milliseconds.

    • checkpoint_duration_time_exp_avg string

      The exponential moving average of the duration of the checkpoint, in milliseconds.

    • indexed_documents_exp_avg string

      The exponential moving average of the number of new documents that have been indexed.

    • processed_documents_exp_avg string

      The exponential moving average of the number of documents that have been processed.
GET /_cat/transforms/{transform_id} 
GET /_cat/transforms?v=true&format=json

resp = client.cat.transforms(
    v=True,
    format="json",
)
const response = await client.cat.transforms({
  v: "true",
  format: "json",
});
response = client.cat.transforms(
  v: "true",
  format: "json"
)
$resp = $client->cat()->transforms([
    "v" => "true",
    "format" => "json",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_cat/transforms?v=true&format=json"
client.cat().transforms();
Response examples (200)
A successful response from `GET /_cat/transforms?v=true&format=json`. 
[
  {
    "id" : "ecommerce_transform",
    "state" : "started",
    "checkpoint" : "1",
    "documents_processed" : "705",
    "checkpoint_progress" : "100.00",
    "changes_last_detection_time" : null
  }
]

Get cluster-wide settings Generally available

GET /_cluster/settings
Api key auth Basic auth Bearer auth

By default, it returns only settings that have been explicitly defined.

Required authorization

  • Cluster privileges: monitor
External documentation

Query parameters

  • flat_settings boolean

    If true, returns settings in flat format.

  • include_defaults boolean

    If true, also returns default values for all other cluster settings, reflecting the values in the elasticsearch.yml file of one of the nodes in the cluster. If the nodes in your cluster do not all have the same values in their elasticsearch.yml config files then the values returned by this API may vary from invocation to invocation and may not reflect the values that Elasticsearch uses in all situations. Use the GET _nodes/settings API to fetch the settings for each individual node in your cluster.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • persistent object Required

      The settings that persist after the cluster restarts.

      Hide persistent attribute Show persistent attribute object
      • * object Additional properties
    • transient object Required

      The settings that do not persist after the cluster restarts.

      Hide transient attribute Show transient attribute object
      • * object Additional properties
    • defaults object

      The default setting values.

      Hide defaults attribute Show defaults attribute object
      • * object Additional properties
GET /_cluster/settings 
GET /_cluster/settings?filter_path=persistent.cluster.remote

resp = client.cluster.get_settings(
    filter_path="persistent.cluster.remote",
)
const response = await client.cluster.getSettings({
  filter_path: "persistent.cluster.remote",
});
response = client.cluster.get_settings(
  filter_path: "persistent.cluster.remote"
)
$resp = $client->cluster()->getSettings([
    "filter_path" => "persistent.cluster.remote",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_cluster/settings?filter_path=persistent.cluster.remote"

Clear the archived repositories metering Technical preview; Added in 7.16.0

DELETE /_nodes/{node_id}/_repositories_metering/{max_archive_version}
Api key auth Basic auth Bearer auth

Clear the archived repositories metering information in the cluster.

Required authorization

  • Cluster privileges: monitor,manage

Path parameters

  • node_id string | array[string] Required

    Comma-separated list of node IDs or names used to limit returned information.

  • max_archive_version number Required

    Specifies the maximum archive_version to be cleared from the archive.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object

      Contains statistics about the number of nodes selected by the request’s node filters.

      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide failures attributes Show failures attributes object
        • type string Required

          The type of error

        • reason
        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by
        • root_cause array[object]
        • suppressed array[object]
      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string Required

      Name of the cluster. Based on the cluster.name setting.

    • nodes object Required

      Contains repositories metering information for the nodes selected by the request.

      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • repository_name string Required

          Repository name.

        • repository_type string Required

          Repository type.

        • repository_location object Required

          Represents an unique location within the repository.

          Hide repository_location attributes Show repository_location attributes object
          • base_path string Required
          • container string

            Container name (Azure)

          • bucket string

            Bucket name (GCP, S3)

        • repository_ephemeral_id string Required

          An identifier that changes every time the repository is updated.

        • Time unit for milliseconds

        • Time unit for milliseconds

        • archived boolean Required

          A flag that tells whether or not this object has been archived. When a repository is closed or updated the repository metering information is archived and kept for a certain period of time. This allows retrieving the repository metering information of previous repository instantiations.

        • cluster_version number

          The cluster state version when this object was archived, this field can be used as a logical timestamp to delete all the archived metrics up to an observed version. This field is only present for archived repository metering information objects. The main purpose of this field is to avoid possible race conditions during repository metering information deletions, i.e. deleting archived repositories metering information that we haven’t observed yet.

        • request_counts object Required

          An object with the number of request performed against the repository grouped by request type.

          Hide request_counts attributes Show request_counts attributes object
          • GetBlobProperties number

            Number of Get Blob Properties requests (Azure)

          • GetBlob number

            Number of Get Blob requests (Azure)

          • ListBlobs number

            Number of List Blobs requests (Azure)

          • PutBlob number

            Number of Put Blob requests (Azure)

          • PutBlock number

            Number of Put Block (Azure)

          • PutBlockList number

            Number of Put Block List requests

          • GetObject number

            Number of get object requests (GCP, S3)

          • ListObjects number

            Number of list objects requests (GCP, S3)

          • InsertObject number

            Number of insert object requests, including simple, multipart and resumable uploads. Resumable uploads can perform multiple http requests to insert a single object but they are considered as a single request since they are billed as an individual operation. (GCP)

          • PutObject number

            Number of PutObject requests (S3)

          • PutMultipartObject number

            Number of Multipart requests, including CreateMultipartUpload, UploadPart and CompleteMultipartUpload requests (S3)
DELETE /_nodes/{node_id}/_repositories_metering/{max_archive_version} 
curl \
 --request DELETE 'http://api.example.com/_nodes/{node_id}/_repositories_metering/{max_archive_version}' \
 --header "Authorization: $API_KEY"

Activate the connector draft filter Technical preview; Added in 8.12.0

PUT /_connector/{connector_id}/_filtering/_activate
Api key auth Basic auth Bearer auth

Activates the valid draft filtering for a connector.

Path parameters

  • connector_id string Required

    The unique identifier of the connector to be updated

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.
PUT /_connector/{connector_id}/_filtering/_activate 
curl \
 --request PUT 'http://api.example.com/_connector/{connector_id}/_filtering/_activate' \
 --header "Authorization: $API_KEY"

Create or update auto-follow patterns Generally available; Added in 6.5.0

PUT /_ccr/auto_follow/{name}
Api key auth Basic auth Bearer auth

Create a collection of cross-cluster replication auto-follow patterns for a remote cluster. Newly created indices on the remote cluster that match any of the patterns are automatically configured as follower indices. Indices on the remote cluster that were created before the auto-follow pattern was created will not be auto-followed even if they match the pattern.

This API can also be used to update auto-follow patterns. NOTE: Follower indices that were configured automatically before updating an auto-follow pattern will remain unchanged even if they do not match against the new patterns.

External documentation

Path parameters

  • name string Required

    The name of the collection of auto-follow patterns.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

application/json

Body Required

  • remote_cluster string Required

    The remote cluster containing the leader indices to match against.

  • follow_index_pattern string

    The name of follower index. The template {{leader_index}} can be used to derive the name of the follower index from the name of the leader index. When following a data stream, use {{leader_index}}; CCR does not support changes to the names of a follower data stream’s backing indices.

  • leader_index_patterns array[string]

    An array of simple index patterns to match against indices in the remote cluster specified by the remote_cluster field.

  • leader_index_exclusion_patterns array[string]

    An array of simple index patterns that can be used to exclude indices from being auto-followed. Indices in the remote cluster whose names are matching one or more leader_index_patterns and one or more leader_index_exclusion_patterns won’t be followed.

  • max_outstanding_read_requests number

    The maximum number of outstanding reads requests from the remote cluster.

    Default value is 12.

  • settings object

    Settings to override from the leader index. Note that certain settings can not be overrode (e.g., index.number_of_shards).

    Hide settings attribute Show settings attribute object
    • * object Additional properties
  • max_outstanding_write_requests number

    The maximum number of outstanding reads requests from the remote cluster.

    Default value is 9.

  • read_poll_timeout string

    The maximum time to wait for new operations on the remote cluster when the follower index is synchronized with the leader index. When the timeout has elapsed, the poll for operations will return to the follower so that it can update some statistics. Then the follower will immediately attempt to read from the leader again.

  • max_read_request_operation_count number

    The maximum number of operations to pull per read from the remote cluster.

    Default value is 5120.

  • max_read_request_size number | string

    The maximum size in bytes of per read of a batch of operations pulled from the remote cluster.

    One of:
    number-1 number string-2 string

    The maximum size in bytes of per read of a batch of operations pulled from the remote cluster.

  • max_retry_delay string

    The maximum time to wait before retrying an operation that failed exceptionally. An exponential backoff strategy is employed when retrying.

  • max_write_buffer_count number

    The maximum number of operations that can be queued for writing. When this limit is reached, reads from the remote cluster will be deferred until the number of queued operations goes below the limit.

    Default value is 2147483647.

  • max_write_buffer_size number | string

    The maximum total bytes of operations that can be queued for writing. When this limit is reached, reads from the remote cluster will be deferred until the total bytes of queued operations goes below the limit.

    One of:
    number-1 number string-2 string

    The maximum total bytes of operations that can be queued for writing. When this limit is reached, reads from the remote cluster will be deferred until the total bytes of queued operations goes below the limit.

  • max_write_request_operation_count number

    The maximum number of operations per bulk write request executed on the follower.

    Default value is 5120.

  • max_write_request_size number | string

    The maximum total bytes of operations per bulk write request executed on the follower.

    One of:
    number-1 number string-2 string

    The maximum total bytes of operations per bulk write request executed on the follower.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
PUT /_ccr/auto_follow/{name} 
PUT /_ccr/auto_follow/my_auto_follow_pattern
{
  "remote_cluster" : "remote_cluster",
  "leader_index_patterns" :
  [
    "leader_index*"
  ],
  "follow_index_pattern" : "{{leader_index}}-follower",
  "settings": {
    "index.number_of_replicas": 0
  },
  "max_read_request_operation_count" : 1024,
  "max_outstanding_read_requests" : 16,
  "max_read_request_size" : "1024k",
  "max_write_request_operation_count" : 32768,
  "max_write_request_size" : "16k",
  "max_outstanding_write_requests" : 8,
  "max_write_buffer_count" : 512,
  "max_write_buffer_size" : "512k",
  "max_retry_delay" : "10s",
  "read_poll_timeout" : "30s"
}
resp = client.ccr.put_auto_follow_pattern(
    name="my_auto_follow_pattern",
    remote_cluster="remote_cluster",
    leader_index_patterns=[
        "leader_index*"
    ],
    follow_index_pattern="{{leader_index}}-follower",
    settings={
        "index.number_of_replicas": 0
    },
    max_read_request_operation_count=1024,
    max_outstanding_read_requests=16,
    max_read_request_size="1024k",
    max_write_request_operation_count=32768,
    max_write_request_size="16k",
    max_outstanding_write_requests=8,
    max_write_buffer_count=512,
    max_write_buffer_size="512k",
    max_retry_delay="10s",
    read_poll_timeout="30s",
)
const response = await client.ccr.putAutoFollowPattern({
  name: "my_auto_follow_pattern",
  remote_cluster: "remote_cluster",
  leader_index_patterns: ["leader_index*"],
  follow_index_pattern: "{{leader_index}}-follower",
  settings: {
    "index.number_of_replicas": 0,
  },
  max_read_request_operation_count: 1024,
  max_outstanding_read_requests: 16,
  max_read_request_size: "1024k",
  max_write_request_operation_count: 32768,
  max_write_request_size: "16k",
  max_outstanding_write_requests: 8,
  max_write_buffer_count: 512,
  max_write_buffer_size: "512k",
  max_retry_delay: "10s",
  read_poll_timeout: "30s",
});
response = client.ccr.put_auto_follow_pattern(
  name: "my_auto_follow_pattern",
  body: {
    "remote_cluster": "remote_cluster",
    "leader_index_patterns": [
      "leader_index*"
    ],
    "follow_index_pattern": "{{leader_index}}-follower",
    "settings": {
      "index.number_of_replicas": 0
    },
    "max_read_request_operation_count": 1024,
    "max_outstanding_read_requests": 16,
    "max_read_request_size": "1024k",
    "max_write_request_operation_count": 32768,
    "max_write_request_size": "16k",
    "max_outstanding_write_requests": 8,
    "max_write_buffer_count": 512,
    "max_write_buffer_size": "512k",
    "max_retry_delay": "10s",
    "read_poll_timeout": "30s"
  }
)
$resp = $client->ccr()->putAutoFollowPattern([
    "name" => "my_auto_follow_pattern",
    "body" => [
        "remote_cluster" => "remote_cluster",
        "leader_index_patterns" => array(
            "leader_index*",
        ),
        "follow_index_pattern" => "{{leader_index}}-follower",
        "settings" => [
            "index.number_of_replicas" => 0,
        ],
        "max_read_request_operation_count" => 1024,
        "max_outstanding_read_requests" => 16,
        "max_read_request_size" => "1024k",
        "max_write_request_operation_count" => 32768,
        "max_write_request_size" => "16k",
        "max_outstanding_write_requests" => 8,
        "max_write_buffer_count" => 512,
        "max_write_buffer_size" => "512k",
        "max_retry_delay" => "10s",
        "read_poll_timeout" => "30s",
    ],
]);
curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"remote_cluster":"remote_cluster","leader_index_patterns":["leader_index*"],"follow_index_pattern":"{{leader_index}}-follower","settings":{"index.number_of_replicas":0},"max_read_request_operation_count":1024,"max_outstanding_read_requests":16,"max_read_request_size":"1024k","max_write_request_operation_count":32768,"max_write_request_size":"16k","max_outstanding_write_requests":8,"max_write_buffer_count":512,"max_write_buffer_size":"512k","max_retry_delay":"10s","read_poll_timeout":"30s"}' "$ELASTICSEARCH_URL/_ccr/auto_follow/my_auto_follow_pattern"
client.ccr().putAutoFollowPattern(p -> p
    .followIndexPattern("{{leader_index}}-follower")
    .leaderIndexPatterns("leader_index*")
    .maxOutstandingReadRequests(16)
    .maxOutstandingWriteRequests(8)
    .maxReadRequestOperationCount(1024)
    .maxReadRequestSize("1024k")
    .maxRetryDelay(m -> m
        .time("10s")
    )
    .maxWriteBufferCount(512)
    .maxWriteBufferSize("512k")
    .maxWriteRequestOperationCount(32768)
    .maxWriteRequestSize("16k")
    .name("my_auto_follow_pattern")
    .readPollTimeout(r -> r
        .time("30s")
    )
    .remoteCluster("remote_cluster")
    .settings("index.number_of_replicas", JsonData.fromJson("0"))
);
Request example
Run `PUT /_ccr/auto_follow/my_auto_follow_pattern` to creates an auto-follow pattern. 
{
  "remote_cluster" : "remote_cluster",
  "leader_index_patterns" :
  [
    "leader_index*"
  ],
  "follow_index_pattern" : "{{leader_index}}-follower",
  "settings": {
    "index.number_of_replicas": 0
  },
  "max_read_request_operation_count" : 1024,
  "max_outstanding_read_requests" : 16,
  "max_read_request_size" : "1024k",
  "max_write_request_operation_count" : 32768,
  "max_write_request_size" : "16k",
  "max_outstanding_write_requests" : 8,
  "max_write_buffer_count" : 512,
  "max_write_buffer_size" : "512k",
  "max_retry_delay" : "10s",
  "read_poll_timeout" : "30s"
}
Response examples (200)
A successful response for creating an auto-follow pattern. 
{
  "acknowledged": true
}

Get follower stats Generally available; Added in 6.5.0

GET /{index}/_ccr/stats
Api key auth Basic auth Bearer auth

Get cross-cluster replication follower stats. The API returns shard-level stats about the "following tasks" associated with each shard for the specified indices.

Required authorization

  • Cluster privileges: monitor
External documentation

Path parameters

  • index string | array[string] Required

    A comma-delimited list of index patterns.

Query parameters

  • timeout string

    The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • indices array[object] Required

      An array of follower index statistics.

      Hide indices attributes Show indices attributes object
      • index string Required

        The name of the follower index.

      • shards array[object] Required

        An array of shard-level following task statistics.

        Hide shards attributes Show shards attributes object
        • bytes_read number Required

          The total of transferred bytes read from the leader. This is only an estimate and does not account for compression if enabled.

        • failed_read_requests number Required

          The number of failed reads.

        • failed_write_requests number Required

          The number of failed bulk write requests on the follower.

        • fatal_exception object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • follower_aliases_version number Required

          The index aliases version the follower is synced up to.

        • follower_global_checkpoint number Required

          The current global checkpoint on the follower. The difference between the leader_global_checkpoint and the follower_global_checkpoint is an indication of how much the follower is lagging the leader.

        • follower_index string Required

          The name of the follower index.

        • follower_mapping_version number Required

          The mapping version the follower is synced up to.

        • follower_max_seq_no number Required

          The current maximum sequence number on the follower.

        • follower_settings_version number Required

          The index settings version the follower is synced up to.

        • last_requested_seq_no number Required

          The starting sequence number of the last batch of operations requested from the leader.

        • leader_global_checkpoint number Required

          The current global checkpoint on the leader known to the follower task.

        • leader_index string Required

          The name of the index in the leader cluster being followed.

        • leader_max_seq_no number Required

          The current maximum sequence number on the leader known to the follower task.

        • operations_read number Required

          The total number of operations read from the leader.

        • operations_written number Required

          The number of operations written on the follower.

        • outstanding_read_requests number Required

          The number of active read requests from the follower.

        • outstanding_write_requests number Required

          The number of active bulk write requests on the follower.

        • read_exceptions array[object] Required

          An array of objects representing failed reads.

        • remote_cluster string Required

          The remote cluster containing the leader index.

        • shard_id number Required

          The numerical shard ID, with values from 0 to one less than the number of replicas.

        • successful_read_requests number Required

          The number of successful fetches.

        • successful_write_requests number Required

          The number of bulk write requests run on the follower.

        • time_since_last_read string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • time_since_last_read_millis
        • total_read_remote_exec_time string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • total_read_remote_exec_time_millis
        • total_read_time string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • total_read_time_millis
        • total_write_time string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • total_write_time_millis
        • write_buffer_operation_count number Required

          The number of write operations queued on the follower.

        • write_buffer_size_in_bytes
GET /{index}/_ccr/stats 
GET /follower_index/_ccr/stats

resp = client.ccr.follow_stats(
    index="follower_index",
)
const response = await client.ccr.followStats({
  index: "follower_index",
});
response = client.ccr.follow_stats(
  index: "follower_index"
)
$resp = $client->ccr()->followStats([
    "index" => "follower_index",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/follower_index/_ccr/stats"
client.ccr().followStats(f -> f
    .index("follower_index")
);
Response examples (200)
A successful response from `GET /follower_index/_ccr/stats`, which retrieves follower stats. 
{
  "indices" : [
    {
      "index" : "follower_index",
      "total_global_checkpoint_lag" : 256,
      "shards" : [
        {
          "remote_cluster" : "remote_cluster",
          "leader_index" : "leader_index",
          "follower_index" : "follower_index",
          "shard_id" : 0,
          "leader_global_checkpoint" : 1024,
          "leader_max_seq_no" : 1536,
          "follower_global_checkpoint" : 768,
          "follower_max_seq_no" : 896,
          "last_requested_seq_no" : 897,
          "outstanding_read_requests" : 8,
          "outstanding_write_requests" : 2,
          "write_buffer_operation_count" : 64,
          "follower_mapping_version" : 4,
          "follower_settings_version" : 2,
          "follower_aliases_version" : 8,
          "total_read_time_millis" : 32768,
          "total_read_remote_exec_time_millis" : 16384,
          "successful_read_requests" : 32,
          "failed_read_requests" : 0,
          "operations_read" : 896,
          "bytes_read" : 32768,
          "total_write_time_millis" : 16384,
          "write_buffer_size_in_bytes" : 1536,
          "successful_write_requests" : 16,
          "failed_write_requests" : 0,
          "operations_written" : 832,
          "read_exceptions" : [ ],
          "time_since_last_read_millis" : 8
        }
      ]
    }
  ]
}

Resume an auto-follow pattern Generally available; Added in 7.5.0

POST /_ccr/auto_follow/{name}/resume
Api key auth Basic auth Bearer auth

Resume a cross-cluster replication auto-follow pattern that was paused. The auto-follow pattern will resume configuring following indices for newly created indices that match its patterns on the remote cluster. Remote indices created while the pattern was paused will also be followed unless they have been deleted or closed in the interim.

Required authorization

  • Cluster privileges: manage_ccr
External documentation

Path parameters

  • name string Required

    The name of the auto-follow pattern to resume.

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
POST /_ccr/auto_follow/{name}/resume 
POST /_ccr/auto_follow/my_auto_follow_pattern/resume

resp = client.ccr.resume_auto_follow_pattern(
    name="my_auto_follow_pattern",
)
const response = await client.ccr.resumeAutoFollowPattern({
  name: "my_auto_follow_pattern",
});
response = client.ccr.resume_auto_follow_pattern(
  name: "my_auto_follow_pattern"
)
$resp = $client->ccr()->resumeAutoFollowPattern([
    "name" => "my_auto_follow_pattern",
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_ccr/auto_follow/my_auto_follow_pattern/resume"
client.ccr().resumeAutoFollowPattern(r -> r
    .name("my_auto_follow_pattern")
);
Response examples (200)
A successful response `POST /_ccr/auto_follow/my_auto_follow_pattern/resume`, which resumes an auto-follow pattern. 
{
  "acknowledged" : true
}

Promote a data stream Generally available; Added in 7.9.0

POST /_data_stream/_promote/{name}
Api key auth Basic auth Bearer auth

Promote a data stream from a replicated data stream managed by cross-cluster replication (CCR) to a regular data stream.

With CCR auto following, a data stream from a remote cluster can be replicated to the local cluster. These data streams can't be rolled over in the local cluster. These replicated data streams roll over only if the upstream data stream rolls over. In the event that the remote cluster is no longer available, the data stream in the local cluster can be promoted to a regular data stream, which allows these data streams to be rolled over in the local cluster.

NOTE: When promoting a data stream, ensure the local cluster has a data stream enabled index template that matches the data stream. If this is missing, the data stream will not be able to roll over until a matching index template is created. This will affect the lifecycle management of the data stream and interfere with the data stream size and retention.

Path parameters

  • name string Required

    The name of the data stream

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
POST /_data_stream/_promote/{name} 
POST /_data_stream/_promote/my-data-stream

resp = client.indices.promote_data_stream(
    name="my-data-stream",
)
const response = await client.indices.promoteDataStream({
  name: "my-data-stream",
});
response = client.indices.promote_data_stream(
  name: "my-data-stream"
)
$resp = $client->indices()->promoteDataStream([
    "name" => "my-data-stream",
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_data_stream/_promote/my-data-stream"
client.indices().promoteDataStream(p -> p
    .name("my-data-stream")
);

Document

Learn more about reading and writing documents

Bulk index or delete documents Generally available

PUT /{index}/_bulk
Api key auth Basic auth Bearer auth

All methods and paths for this operation:

POST /_bulk

PUT /_bulk
POST /{index}/_bulk
PUT /{index}/_bulk

Perform multiple index, create, delete, and update actions in a single request. This reduces overhead and can greatly increase indexing speed.

If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:

  • To use the create action, you must have the create_doc, create, index, or write index privilege. Data streams support only the create action.
  • To use the index action, you must have the create, index, or write index privilege.
  • To use the delete action, you must have the delete or write index privilege.
  • To use the update action, you must have the index or write index privilege.
  • To automatically create a data stream or index with a bulk API request, you must have the auto_configure, create_index, or manage index privilege.
  • To make the result of a bulk operation visible to search using the refresh parameter, you must have the maintenance or manage index privilege.

Automatic data stream creation requires a matching index template with data stream enabled.

The actions are specified in the request body using a newline delimited JSON (NDJSON) structure:

action_and_meta_data\n
optional_source\n
action_and_meta_data\n
optional_source\n
....
action_and_meta_data\n
optional_source\n

The index and create actions expect a source on the next line and have the same semantics as the op_type parameter in the standard index API. A create action fails if a document with the same ID already exists in the target An index action adds or replaces a document as necessary.

NOTE: Data streams support only the create action. To update or delete a document in a data stream, you must target the backing index containing the document.

An update action expects that the partial doc, upsert, and script and its options are specified on the next line.

A delete action does not expect a source on the next line and has the same semantics as the standard delete API.

NOTE: The final line of data must end with a newline character (\n). Each newline character may be preceded by a carriage return (\r). When sending NDJSON data to the _bulk endpoint, use a Content-Type header of application/json or application/x-ndjson. Because this format uses literal newline characters (\n) as delimiters, make sure that the JSON actions and sources are not pretty printed.

If you provide a target in the request path, it is used for any actions that don't explicitly specify an _index argument.

A note on the format: the idea here is to make processing as fast as possible. As some of the actions are redirected to other shards on other nodes, only action_meta_data is parsed on the receiving node side.

Client libraries using this protocol should try and strive to do something similar on the client side, and reduce buffering as much as possible.

There is no "correct" number of actions to perform in a single bulk request. Experiment with different settings to find the optimal size for your particular workload. Note that Elasticsearch limits the maximum size of a HTTP request to 100mb by default so clients must ensure that no request exceeds this size. It is not possible to index a single document that exceeds the size limit, so you must pre-process any such documents into smaller pieces before sending them to Elasticsearch. For instance, split documents into pages or chapters before indexing them, or store raw binary data in a system outside Elasticsearch and replace the raw data with a link to the external system in the documents that you send to Elasticsearch.

Client suppport for bulk requests

Some of the officially supported clients provide helpers to assist with bulk requests and reindexing:

  • Go: Check out esutil.BulkIndexer
  • Perl: Check out Search::Elasticsearch::Client::5_0::Bulk and Search::Elasticsearch::Client::5_0::Scroll
  • Python: Check out elasticsearch.helpers.*
  • JavaScript: Check out client.helpers.*
  • .NET: Check out BulkAllObservable
  • PHP: Check out bulk indexing.
  • Ruby: Check out Elasticsearch::Helpers::BulkHelper

Submitting bulk requests with cURL

If you're providing text file input to curl, you must use the --data-binary flag instead of plain -d. The latter doesn't preserve newlines. For example:

$ cat requests
{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
$ curl -s -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/_bulk --data-binary "@requests"; echo
{"took":7, "errors": false, "items":[{"index":{"_index":"test","_id":"1","_version":1,"result":"created","forced_refresh":false}}]}

Optimistic concurrency control

Each index and delete action within a bulk API call may include the if_seq_no and if_primary_term parameters in their respective action and meta data lines. The if_seq_no and if_primary_term parameters control how operations are run, based on the last modification to existing documents. See Optimistic concurrency control for more details.

Versioning

Each bulk item can include the version value using the version field. It automatically follows the behavior of the index or delete operation based on the _version mapping. It also support the version_type.

Routing

Each bulk item can include the routing value using the routing field. It automatically follows the behavior of the index or delete operation based on the _routing mapping.

NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.

Wait for active shards

When making bulk calls, you can set the wait_for_active_shards parameter to require a minimum number of shard copies to be active before starting to process the bulk request.

Refresh

Control when the changes made by this request are visible to search.

NOTE: Only the shards that receive the bulk request will be affected by refresh. Imagine a _bulk?refresh=wait_for request with three documents in it that happen to be routed to different shards in an index with five shards. The request will only wait for those three shards to refresh. The other two shards that make up the index do not participate in the _bulk request at all.

Path parameters

  • index string Required

    The name of the data stream, index, or index alias to perform bulk actions on.

Query parameters

  • include_source_on_error boolean

    True or false if to include the document source in the error message in case of parsing errors.

  • list_executed_pipelines boolean

    If true, the response will include the ingest pipelines that were run for each index or create.

  • pipeline string

    The pipeline identifier to use to preprocess incoming documents. If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request. If a final pipeline is configured, it will always run regardless of the value of this parameter.

  • refresh string

    If true, Elasticsearch refreshes the affected shards to make this operation visible to search. If wait_for, wait for a refresh to make this operation visible to search. If false, do nothing with refreshes. Valid values: true, false, wait_for.

    Values are true, false, or wait_for.

  • routing string

    A custom value that is used to route operations to a specific shard.

  • _source boolean | string | array[string]

    Indicates whether to return the _source field (true or false) or contains a list of fields to return.

  • _source_excludes string | array[string]

    A comma-separated list of source fields to exclude from the response. You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter. If the _source parameter is false, this parameter is ignored.

  • _source_includes string | array[string]

    A comma-separated list of source fields to include in the response. If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the _source_excludes query parameter. If the _source parameter is false, this parameter is ignored.

  • timeout string

    The period each action waits for the following operations: automatic index creation, dynamic mapping updates, and waiting for active shards. The default is 1m (one minute), which guarantees Elasticsearch waits for at least the timeout before failing. The actual wait time could be longer, particularly when multiple waits occur.

    Values are -1 or 0.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). The default is 1, which waits for each primary shard to be active.

    Values are all or index-setting.

  • require_alias boolean

    If true, the request's actions must target an index alias.

  • require_data_stream boolean

    If true, the request's actions must target a data stream (existing or to be created).

application/json

Body object Required

One of:
OperationContainer object UpdateAction object object-3 object
  • index object

    Index the specified document. If the document exists, it replaces the document and increments the version. The following line must contain the source data to be indexed.

    Hide index attributes Show index attributes object
    • if_primary_term number
    • dynamic_templates object

      A map from the full name of fields to the name of dynamic templates. It defaults to an empty map. If a name matches a dynamic template, that template will be applied regardless of other match predicates defined in the template. If a field is already defined in the mapping, then this parameter won't be used.

    • pipeline string

      The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request. If a final pipeline is configured, it will always run regardless of the value of this parameter.

    • require_alias boolean

      If true, the request's actions must target an index alias.

      Default value is false.

  • create object

    Index the specified document if it does not already exist. The following line must contain the source data to be indexed.

    Hide create attributes Show create attributes object
    • if_primary_term number
    • dynamic_templates object

      A map from the full name of fields to the name of dynamic templates. It defaults to an empty map. If a name matches a dynamic template, that template will be applied regardless of other match predicates defined in the template. If a field is already defined in the mapping, then this parameter won't be used.

    • pipeline string

      The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request. If a final pipeline is configured, it will always run regardless of the value of this parameter.

    • require_alias boolean

      If true, the request's actions must target an index alias.

      Default value is false.

  • update object

    Perform a partial document update. The following line must contain the partial document and update options.

    Hide update attributes Show update attributes object
    • _id string

      The document ID.

    • _index string

      The name of the index or index alias to perform the action on.

    • routing string

      A custom value used to route operations to a specific shard.

    • if_primary_term number
    • if_seq_no number
    • version number
    • version_type string

      Supported values include:

      • internal: Use internal versioning that starts at 1 and increments with each update or delete.
      • external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.
      • external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document. NOTE: The external_gte version type is meant for special use cases and should be used with care. If used incorrectly, it can result in loss of data.
      • force: This option is deprecated because it can cause primary and replica shards to diverge.

      Values are internal, external, external_gte, or force.

    • require_alias boolean

      If true, the request's actions must target an index alias.

      Default value is false.

    • retry_on_conflict number

      The number of times an update should be retried in the case of a version conflict.

  • delete object

    Remove the specified document from the index.

    Hide delete attributes Show delete attributes object
    • _id string

      The document ID.

    • _index string

      The name of the index or index alias to perform the action on.

    • routing string

      A custom value used to route operations to a specific shard.

    • if_primary_term number
    • if_seq_no number
    • version number
    • version_type string

      Supported values include:

      • internal: Use internal versioning that starts at 1 and increments with each update or delete.
      • external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.
      • external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document. NOTE: The external_gte version type is meant for special use cases and should be used with care. If used incorrectly, it can result in loss of data.
      • force: This option is deprecated because it can cause primary and replica shards to diverge.

      Values are internal, external, external_gte, or force.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • errors boolean Required

      If true, one or more of the operations in the bulk request did not complete successfully.

    • items array[object] Required

      The result of each operation in the bulk request, in the order they were submitted.

      Hide items attribute Show items attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object

        • _id string | null

          The document ID associated with the operation.

          One of:
          string-1 string string-2 string | null
        • _index string Required

          The name of the index associated with the operation. If the operation targeted a data stream, this is the backing index into which the document was written.

        • status number Required

          The HTTP status code returned for the operation.

        • failure_store string

          Values are not_applicable_or_unknown, used, not_enabled, or failed.

        • error object

          Additional information about the failed operation. The property is returned only for failed operations.

          Hide error attributes Show error attributes object
          • type string Required

            The type of error

          • reason
          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • root_cause array[object]
          • suppressed array[object]
        • _primary_term number

          The primary term assigned to the document for the operation. This property is returned only for successful operations.

        • result string

          The result of the operation. Successful values are created, deleted, and updated.

        • _seq_no number

          The sequence number assigned to the document for the operation. Sequence numbers are used to ensure an older version of a document doesn't overwrite a newer version.

        • _shards object

          Shard information for the operation.

          Hide _shards attribute Show _shards attribute object
          • failures array[object]
        • _version number

          The document version associated with the operation. The document version is incremented each time the document is updated. This property is returned only for successful actions.

        • forced_refresh boolean
        • get object
          Hide get attributes Show get attributes object
          • fields object
          • found boolean Required
          • _primary_term number
          • _source object
    • took number Required

      The length of time, in milliseconds, it took to process the bulk request.

    • ingest_took number
PUT /{index}/_bulk 
POST _bulk
{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test", "_id" : "2" } }
{ "create" : { "_index" : "test", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_index" : "test"} }
{ "doc" : {"field2" : "value2"} }
resp = client.bulk(
    operations=[
        {
            "index": {
                "_index": "test",
                "_id": "1"
            }
        },
        {
            "field1": "value1"
        },
        {
            "delete": {
                "_index": "test",
                "_id": "2"
            }
        },
        {
            "create": {
                "_index": "test",
                "_id": "3"
            }
        },
        {
            "field1": "value3"
        },
        {
            "update": {
                "_id": "1",
                "_index": "test"
            }
        },
        {
            "doc": {
                "field2": "value2"
            }
        }
    ],
)
const response = await client.bulk({
  operations: [
    {
      index: {
        _index: "test",
        _id: "1",
      },
    },
    {
      field1: "value1",
    },
    {
      delete: {
        _index: "test",
        _id: "2",
      },
    },
    {
      create: {
        _index: "test",
        _id: "3",
      },
    },
    {
      field1: "value3",
    },
    {
      update: {
        _id: "1",
        _index: "test",
      },
    },
    {
      doc: {
        field2: "value2",
      },
    },
  ],
});
response = client.bulk(
  body: [
    {
      "index": {
        "_index": "test",
        "_id": "1"
      }
    },
    {
      "field1": "value1"
    },
    {
      "delete": {
        "_index": "test",
        "_id": "2"
      }
    },
    {
      "create": {
        "_index": "test",
        "_id": "3"
      }
    },
    {
      "field1": "value3"
    },
    {
      "update": {
        "_id": "1",
        "_index": "test"
      }
    },
    {
      "doc": {
        "field2": "value2"
      }
    }
  ]
)
$resp = $client->bulk([
    "body" => array(
        [
            "index" => [
                "_index" => "test",
                "_id" => "1",
            ],
        ],
        [
            "field1" => "value1",
        ],
        [
            "delete" => [
                "_index" => "test",
                "_id" => "2",
            ],
        ],
        [
            "create" => [
                "_index" => "test",
                "_id" => "3",
            ],
        ],
        [
            "field1" => "value3",
        ],
        [
            "update" => [
                "_id" => "1",
                "_index" => "test",
            ],
        ],
        [
            "doc" => [
                "field2" => "value2",
            ],
        ],
    ),
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '[{"index":{"_index":"test","_id":"1"}},{"field1":"value1"},{"delete":{"_index":"test","_id":"2"}},{"create":{"_index":"test","_id":"3"}},{"field1":"value3"},{"update":{"_id":"1","_index":"test"}},{"doc":{"field2":"value2"}}]' "$ELASTICSEARCH_URL/_bulk"
Request examples
Run `POST _bulk` to perform multiple operations. 
{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test", "_id" : "2" } }
{ "create" : { "_index" : "test", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_index" : "test"} }
{ "doc" : {"field2" : "value2"} }
When you run `POST _bulk` and use the `update` action, you can use `retry_on_conflict` as a field in the action itself (not in the extra payload line) to specify how many times an update should be retried in the case of a version conflict. 
{ "update" : {"_id" : "1", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"} }
{ "update" : { "_id" : "0", "_index" : "index1", "retry_on_conflict" : 3} }
{ "script" : { "source": "ctx._source.counter += params.param1", "lang" : "painless", "params" : {"param1" : 1}}, "upsert" : {"counter" : 1}}
{ "update" : {"_id" : "2", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"}, "doc_as_upsert" : true }
{ "update" : {"_id" : "3", "_index" : "index1", "_source" : true} }
{ "doc" : {"field" : "value"} }
{ "update" : {"_id" : "4", "_index" : "index1"} }
{ "doc" : {"field" : "value"}, "_source": true}
To return only information about failed operations, run `POST /_bulk?filter_path=items.*.error`. 
{ "update": {"_id": "5", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "update": {"_id": "6", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "create": {"_id": "7", "_index": "index1"} }
{ "my_field": "foo" }
Run `POST /_bulk` to perform a bulk request that consists of index and create actions with the `dynamic_templates` parameter. The bulk request creates two new fields `work_location` and `home_location` with type `geo_point` according to the `dynamic_templates` parameter. However, the `raw_location` field is created using default dynamic mapping rules, as a text field in that case since it is supplied as a string in the JSON document. 
{ "index" : { "_index" : "my_index", "_id" : "1", "dynamic_templates": {"work_location": "geo_point"}} }
{ "field" : "value1", "work_location": "41.12,-71.34", "raw_location": "41.12,-71.34"}
{ "create" : { "_index" : "my_index", "_id" : "2", "dynamic_templates": {"home_location": "geo_point"}} }
{ "field" : "value2", "home_location": "41.12,-71.34"}
Response examples (200) 
{
   "took": 30,
   "errors": false,
   "items": [
      {
         "index": {
            "_index": "test",
            "_id": "1",
            "_version": 1,
            "result": "created",
            "_shards": {
               "total": 2,
               "successful": 1,
               "failed": 0
            },
            "status": 201,
            "_seq_no" : 0,
            "_primary_term": 1
         }
      },
      {
         "delete": {
            "_index": "test",
            "_id": "2",
            "_version": 1,
            "result": "not_found",
            "_shards": {
               "total": 2,
               "successful": 1,
               "failed": 0
            },
            "status": 404,
            "_seq_no" : 1,
            "_primary_term" : 2
         }
      },
      {
         "create": {
            "_index": "test",
            "_id": "3",
            "_version": 1,
            "result": "created",
            "_shards": {
               "total": 2,
               "successful": 1,
               "failed": 0
            },
            "status": 201,
            "_seq_no" : 2,
            "_primary_term" : 3
         }
      },
      {
         "update": {
            "_index": "test",
            "_id": "1",
            "_version": 2,
            "result": "updated",
            "_shards": {
                "total": 2,
                "successful": 1,
                "failed": 0
            },
            "status": 200,
            "_seq_no" : 3,
            "_primary_term" : 4
         }
      }
   ]
}
If you run `POST /_bulk` with operations that update non-existent documents, the operations cannot complete successfully. The API returns a response with an `errors` property value `true`. The response also includes an error object for any failed operations. The error object contains additional information about the failure, such as the error type and reason. 
{
  "took": 486,
  "errors": true,
  "items": [
    {
      "update": {
        "_index": "index1",
        "_id": "5",
        "status": 404,
        "error": {
          "type": "document_missing_exception",
          "reason": "[5]: document missing",
          "index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
          "shard": "0",
          "index": "index1"
        }
      }
    },
    {
      "update": {
        "_index": "index1",
        "_id": "6",
        "status": 404,
        "error": {
          "type": "document_missing_exception",
          "reason": "[6]: document missing",
          "index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
          "shard": "0",
          "index": "index1"
        }
      }
    },
    {
      "create": {
        "_index": "index1",
        "_id": "7",
        "_version": 1,
        "result": "created",
        "_shards": {
          "total": 2,
          "successful": 1,
          "failed": 0
        },
        "_seq_no": 0,
        "_primary_term": 1,
        "status": 201
      }
    }
  ]
}
An example response from `POST /_bulk?filter_path=items.*.error`, which returns only information about failed operations. 
{
  "items": [
    {
      "update": {
        "error": {
          "type": "document_missing_exception",
          "reason": "[5]: document missing",
          "index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
          "shard": "0",
          "index": "index1"
        }
      }
    },
    {
      "update": {
        "error": {
          "type": "document_missing_exception",
          "reason": "[6]: document missing",
          "index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
          "shard": "0",
          "index": "index1"
        }
      }
    }
  ]
}

Update documents Generally available; Added in 2.4.0

POST /{index}/_update_by_query
Api key auth Basic auth Bearer auth

Updates documents that match the specified query. If no query is specified, performs an update on every document in the data stream or index without modifying the source, which is useful for picking up mapping changes.

If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or alias:

  • read
  • index or write

You can specify the query criteria in the request URI or the request body using the same syntax as the search API.

When you submit an update by query request, Elasticsearch gets a snapshot of the data stream or index when it begins processing the request and updates matching documents using internal versioning. When the versions match, the document is updated and the version number is incremented. If a document changes between the time that the snapshot is taken and the update operation is processed, it results in a version conflict and the operation fails. You can opt to count version conflicts instead of halting and returning by setting conflicts to proceed. Note that if you opt to count version conflicts, the operation could attempt to update more documents from the source than max_docs until it has successfully updated max_docs documents or it has gone through every document in the source query.

NOTE: Documents with a version equal to 0 cannot be updated using update by query because internal versioning does not support 0 as a valid version number.

While processing an update by query request, Elasticsearch performs multiple search requests sequentially to find all of the matching documents. A bulk update request is performed for each batch of matching documents. Any query or update failures cause the update by query request to fail and the failures are shown in the response. Any update requests that completed successfully still stick, they are not rolled back.

Throttling update requests

To control the rate at which update by query issues batches of update operations, you can set requests_per_second to any positive decimal number. This pads each batch with a wait time to throttle the rate. Set requests_per_second to -1 to turn off throttling.

Throttling uses a wait time between batches so that the internal scroll requests can be given a timeout that takes the request padding into account. The padding time is the difference between the batch size divided by the requests_per_second and the time spent writing. By default the batch size is 1000, so if requests_per_second is set to 500:

target_time = 1000 / 500 per second = 2 seconds
wait_time = target_time - write_time = 2 seconds - .5 seconds = 1.5 seconds

Since the batch is issued as a single _bulk request, large batch sizes cause Elasticsearch to create many requests and wait before starting the next set. This is "bursty" instead of "smooth".

Slicing

Update by query supports sliced scroll to parallelize the update process. This can improve efficiency and provide a convenient way to break the request down into smaller parts.

Setting slices to auto chooses a reasonable number for most data streams and indices. This setting will use one slice per shard, up to a certain limit. If there are multiple source data streams or indices, it will choose the number of slices based on the index or backing index with the smallest number of shards.

Adding slices to _update_by_query just automates the manual process of creating sub-requests, which means it has some quirks:

  • You can see these requests in the tasks APIs. These sub-requests are "child" tasks of the task for the request with slices.
  • Fetching the status of the task for the request with slices only contains the status of completed slices.
  • These sub-requests are individually addressable for things like cancellation and rethrottling.
  • Rethrottling the request with slices will rethrottle the unfinished sub-request proportionally.
  • Canceling the request with slices will cancel each sub-request.
  • Due to the nature of slices each sub-request won't get a perfectly even portion of the documents. All documents will be addressed, but some slices may be larger than others. Expect larger slices to have a more even distribution.
  • Parameters like requests_per_second and max_docs on a request with slices are distributed proportionally to each sub-request. Combine that with the point above about distribution being uneven and you should conclude that using max_docs with slices might not result in exactly max_docs documents being updated.
  • Each sub-request gets a slightly different snapshot of the source data stream or index though these are all taken at approximately the same time.

If you're slicing manually or otherwise tuning automatic slicing, keep in mind that:

  • Query performance is most efficient when the number of slices is equal to the number of shards in the index or backing index. If that number is large (for example, 500), choose a lower number as too many slices hurts performance. Setting slices higher than the number of shards generally does not improve efficiency and adds overhead.
  • Update performance scales linearly across available resources with the number of slices.

Whether query or update performance dominates the runtime depends on the documents being reindexed and cluster resources.

Update the document source

Update by query supports scripts to update the document source. As with the update API, you can set ctx.op to change the operation that is performed.

Set ctx.op = "noop" if your script decides that it doesn't have to make any changes. The update by query operation skips updating the document and increments the noop counter.

Set ctx.op = "delete" if your script decides that the document should be deleted. The update by query operation deletes the document and increments the deleted counter.

Update by query supports only index, noop, and delete. Setting ctx.op to anything else is an error. Setting any other field in ctx is an error. This API enables you to only modify the source of matching documents; you cannot move them.

Required authorization

  • Index privileges: read,write

Path parameters

  • index string | array[string] Required

    A comma-separated list of data streams, indices, and aliases to search. It supports wildcards (*). To search all data streams or indices, omit this parameter or use * or _all.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • analyzer string

    The analyzer to use for the query string. This parameter can be used only when the q query string parameter is specified.

  • analyze_wildcard boolean

    If true, wildcard and prefix queries are analyzed. This parameter can be used only when the q query string parameter is specified.

  • conflicts string

    The preferred behavior when update by query hits version conflicts: abort or proceed.

    Supported values include:

    • abort: Stop reindexing if there are conflicts.
    • proceed: Continue reindexing even if there are conflicts.

    Values are abort or proceed.

  • default_operator string

    The default operator for query string query: AND or OR. This parameter can be used only when the q query string parameter is specified.

    Values are and, AND, or, or OR.

  • df string

    The field to use as default where no field prefix is given in the query string. This parameter can be used only when the q query string parameter is specified.

  • expand_wildcards string | array[string]

    The type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. It supports comma-separated values, such as open,hidden.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.

    Values are all, open, closed, hidden, or none.

  • from number

    Skips the specified number of documents.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • lenient boolean

    If true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored. This parameter can be used only when the q query string parameter is specified.

  • max_docs number

    The maximum number of documents to process. It defaults to all documents. When set to a value less then or equal to scroll_size then a scroll will not be used to retrieve the results for the operation.

  • pipeline string

    The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, then setting the value to _none disables the default ingest pipeline for this request. If a final pipeline is configured it will always run, regardless of the value of this parameter.

  • preference string

    The node or shard the operation should be performed on. It is random by default.

  • q string

    A query in the Lucene query string syntax.

  • refresh boolean

    If true, Elasticsearch refreshes affected shards to make the operation visible to search after the request completes. This is different than the update API's refresh parameter, which causes just the shard that received the request to be refreshed.

  • request_cache boolean

    If true, the request cache is used for this request. It defaults to the index-level setting.

  • requests_per_second number

    The throttle for this request in sub-requests per second.

  • routing string

    A custom value used to route operations to a specific shard.

  • scroll string

    The period to retain the search context for scrolling.

    Values are -1 or 0.

  • scroll_size number

    The size of the scroll request that powers the operation.

  • search_timeout string

    An explicit timeout for each search request. By default, there is no timeout.

    Values are -1 or 0.

  • search_type string

    The type of the search operation. Available options include query_then_fetch and dfs_query_then_fetch.

    Supported values include:

    • query_then_fetch: Documents are scored using local term and document frequencies for the shard. This is usually faster but less accurate.
    • dfs_query_then_fetch: Documents are scored using global term and document frequencies across all shards. This is usually slower but more accurate.

    Values are query_then_fetch or dfs_query_then_fetch.

  • slices number | string

    The number of slices this task should be divided into.

    Value is auto.

  • sort array[string]

    A comma-separated list of : pairs.

  • stats array[string]

    The specific tag of the request for logging and statistical purposes.

  • terminate_after number

    The maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting.

    IMPORTANT: Use with caution. Elasticsearch applies this parameter to each shard handling the request. When possible, let Elasticsearch perform early termination automatically. Avoid specifying this parameter for requests that target data streams with backing indices across multiple data tiers.

  • timeout string

    The period each update request waits for the following operations: dynamic mapping updates, waiting for active shards. By default, it is one minute. This guarantees Elasticsearch waits for at least the timeout before failing. The actual wait time could be longer, particularly when multiple waits occur.

    Values are -1 or 0.

  • version boolean

    If true, returns the document version as part of a hit.

  • version_type boolean

    Should the document increment the version number (internal) on hit or not (reindex)

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). The timeout parameter controls how long each write request waits for unavailable shards to become available. Both work exactly the way they work in the bulk API.

    Values are all or index-setting.

  • wait_for_completion boolean

    If true, the request blocks until the operation is complete. If false, Elasticsearch performs some preflight checks, launches the request, and returns a task ID that you can use to cancel or get the status of the task. Elasticsearch creates a record of this task as a document at .tasks/task/${taskId}.

application/json

Body

  • max_docs number

    The maximum number of documents to update.

  • query object

    The documents to update using the Query DSL.

    External documentation
  • script object

    The script to run to update the document source or metadata when updating.

    Hide script attributes Show script attributes object
    • source string

      The script source.

    • id string

      The id for a stored script.

    • params object

      Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

      Hide params attribute Show params attribute object
      • * object Additional properties

    • lang string

      Specifies the language the script is written in.

      Supported values include:

      • painless: Painless scripting language, purpose-built for Elasticsearch.
      • expression: Lucene’s expressions language, compiles a JavaScript expression to bytecode.
      • mustache: Mustache templated, used for templates.
      • java: Expert Java API
      Any of:
      string-1 string string-2 string

      Specifies the language the script is written in.

      Supported values include:

      • painless: Painless scripting language, purpose-built for Elasticsearch.
      • expression: Lucene’s expressions language, compiles a JavaScript expression to bytecode.
      • mustache: Mustache templated, used for templates.
      • java: Expert Java API

      Values are painless, expression, mustache, or java.

    • options object
      Hide options attribute Show options attribute object
      • * string Additional properties
  • slice object

    Slice the request manually using the provided slice ID and total number of slices.

    Hide slice attributes Show slice attributes object
    • field string

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • id string Required
    • max number Required
  • conflicts string

    The preferred behavior when update by query hits version conflicts: abort or proceed.

    Supported values include:

    • abort: Stop reindexing if there are conflicts.
    • proceed: Continue reindexing even if there are conflicts.

    Values are abort or proceed.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • batches number

      The number of scroll responses pulled back by the update by query.

    • failures array[object]

      Array of failures if there were any unrecoverable errors during the process. If this is non-empty then the request ended because of those failures. Update by query is implemented using batches. Any failure causes the entire process to end, but all failures in the current batch are collected into the array. You can use the conflicts option to prevent reindex from ending when version conflicts occur.

      Hide failures attributes Show failures attributes object
      • cause object Required

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide cause attributes Show cause attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

          One of:
          string-1 string string-2 string | null
        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • id string Required
      • index string Required
      • status number Required
    • noops number

      The number of documents that were ignored because the script used for the update by query returned a noop value for ctx.op.

    • deleted number

      The number of documents that were successfully deleted.

    • requests_per_second number

      The number of requests per second effectively run during the update by query.

    • retries object

      The number of retries attempted by update by query. bulk is the number of bulk actions retried. search is the number of search actions retried.

      Hide retries attributes Show retries attributes object
      • bulk number Required

        The number of bulk actions retried.

      • search number Required

        The number of search actions retried.

    • task string | number

      One of:
      string-1 string number-2 number
    • timed_out boolean

      If true, some requests timed out during the update by query.

    • took number

      Time unit for milliseconds

    • total number

      The number of documents that were successfully processed.

    • updated number

      The number of documents that were successfully updated.

    • version_conflicts number

      The number of version conflicts that the update by query hit.

    • throttled string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • throttled_millis number

      Time unit for milliseconds

    • throttled_until string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • throttled_until_millis number

      Time unit for milliseconds
POST /{index}/_update_by_query 
POST my-index-000001/_update_by_query?conflicts=proceed
{
  "query": { 
    "term": {
      "user.id": "kimchy"
    }
  }
}
resp = client.update_by_query(
    index="my-index-000001",
    conflicts="proceed",
    query={
        "term": {
            "user.id": "kimchy"
        }
    },
)
const response = await client.updateByQuery({
  index: "my-index-000001",
  conflicts: "proceed",
  query: {
    term: {
      "user.id": "kimchy",
    },
  },
});
response = client.update_by_query(
  index: "my-index-000001",
  conflicts: "proceed",
  body: {
    "query": {
      "term": {
        "user.id": "kimchy"
      }
    }
  }
)
$resp = $client->updateByQuery([
    "index" => "my-index-000001",
    "conflicts" => "proceed",
    "body" => [
        "query" => [
            "term" => [
                "user.id" => "kimchy",
            ],
        ],
    ],
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"query":{"term":{"user.id":"kimchy"}}}' "$ELASTICSEARCH_URL/my-index-000001/_update_by_query?conflicts=proceed"
client.updateByQuery(u -> u
    .conflicts(Conflicts.Proceed)
    .index("my-index-000001")
    .query(q -> q
        .term(t -> t
            .field("user.id")
            .value(FieldValue.of("kimchy"))
        )
    )
);
Request examples
Run `POST my-index-000001/_update_by_query?conflicts=proceed` to update documents that match a query. 
{
  "query": { 
    "term": {
      "user.id": "kimchy"
    }
  }
}
Run `POST my-index-000001/_update_by_query` with a script to update the document source. It increments the `count` field for all documents with a `user.id` of `kimchy` in `my-index-000001`. 
{
  "script": {
    "source": "ctx._source.count++",
    "lang": "painless"
  },
  "query": {
    "term": {
      "user.id": "kimchy"
    }
  }
}
Run `POST my-index-000001/_update_by_query` to slice an update by query manually. Provide a slice ID and total number of slices to each request. 
{
  "slice": {
    "id": 0,
    "max": 2
  },
  "script": {
    "source": "ctx._source['extra'] = 'test'"
  }
}
Run `POST my-index-000001/_update_by_query?refresh&slices=5` to use automatic slicing. It automatically parallelizes using sliced scroll to slice on `_id`. 
{
  "script": {
    "source": "ctx._source['extra'] = 'test'"
  }
}

Reset the features Technical preview; Added in 7.12.0

POST /_features/_reset
Api key auth Basic auth Bearer auth

Clear all of the state information stored in system indices by Elasticsearch features, including the security and machine learning indices.

WARNING: Intended for development and testing use only. Do not reset features on a production cluster.

Return a cluster to the same state as a new installation by resetting the feature state for all Elasticsearch features. This deletes all state information stored in system indices.

The response code is HTTP 200 if the state is successfully reset for all features. It is HTTP 500 if the reset operation failed for any feature.

Note that select features might provide a way to reset particular system indices. Using this API resets all features, both those that are built-in and implemented as plugins.

To list the features that will be affected, use the get features API.

IMPORTANT: The features installed on the node you submit this request to are the features that will be reset. Run on the master node if you have any doubts about which plugins are installed on individual nodes.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • features array[object] Required
      Hide features attributes Show features attributes object
      • name string Required
      • description string Required
POST /_features/_reset 
POST /_features/_reset

resp = client.features.reset_features()
const response = await client.features.resetFeatures();
response = client.features.reset_features
$resp = $client->features()->resetFeatures();
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_features/_reset"
client.features().resetFeatures(r -> r);
Response examples (200)
A successful response for clearing state information stored in system indices by Elasticsearch features. 
{
  "features" : [
    {
      "feature_name" : "security",
      "status" : "SUCCESS"
    },
    {
      "feature_name" : "tasks",
      "status" : "SUCCESS"
    }
  ]
}

Explore graph analytics Generally available

POST /{index}/_graph/explore
Api key auth Basic auth Bearer auth

All methods and paths for this operation:

GET /{index}/_graph/explore

POST /{index}/_graph/explore

Extract and summarize information about the documents and terms in an Elasticsearch data stream or index. The easiest way to understand the behavior of this API is to use the Graph UI to explore connections. An initial request to the _explore API contains a seed query that identifies the documents of interest and specifies the fields that define the vertices and connections you want to include in the graph. Subsequent requests enable you to spider out from one more vertices of interest. You can exclude vertices that have already been returned.

External documentation

Path parameters

  • index string | array[string] Required

    Name of the index.

Query parameters

  • routing string

    Custom value used to route operations to a specific shard.

  • timeout string

    Specifies the period of time to wait for a response from each shard. If no response is received before the timeout expires, the request fails and returns an error. Defaults to no timeout.

    Values are -1 or 0.

application/json

Body

  • connections object

    Specifies or more fields from which you want to extract terms that are associated with the specified vertices.

    Hide connections attributes Show connections attributes object
    • connections object

      Specifies one or more fields from which you want to extract terms that are associated with the specified vertices.

    • query object Required

      An optional guiding query that constrains the Graph API as it explores connected terms.

      External documentation
    • vertices array[object] Required

      Contains the fields you are interested in.

      Hide vertices attributes Show vertices attributes object
      • exclude array[string]

        Prevents the specified terms from being included in the results.

      • field string Required

        Identifies a field in the documents of interest.

      • include array[object]

        Identifies the terms of interest that form the starting points from which you want to spider out.

        Hide include attributes Show include attributes object
        • boost number Required
        • term string Required
      • min_doc_count number

        Specifies how many documents must contain a pair of terms before it is considered to be a useful connection. This setting acts as a certainty threshold.

        Default value is 3.

      • shard_min_doc_count number

        Controls how many documents on a particular shard have to contain a pair of terms before the connection is returned for global consideration.

        Default value is 2.

      • size number

        Specifies the maximum number of vertex terms returned for each field.

        Default value is 5.

  • controls object

    Direct the Graph API how to build the graph.

    Hide controls attributes Show controls attributes object
    • sample_diversity object

      To avoid the top-matching documents sample being dominated by a single source of results, it is sometimes necessary to request diversity in the sample. You can do this by selecting a single-value field and setting a maximum number of documents per value for that field.

      Hide sample_diversity attributes Show sample_diversity attributes object
      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • max_docs_per_value number Required
    • sample_size number

      Each hop considers a sample of the best-matching documents on each shard. Using samples improves the speed of execution and keeps exploration focused on meaningfully-connected terms. Very small values (less than 50) might not provide sufficient weight-of-evidence to identify significant connections between terms. Very large sample sizes can dilute the quality of the results and increase execution times.

      Default value is 100.

    • timeout string

      The length of time in milliseconds after which exploration will be halted and the results gathered so far are returned. This timeout is honored on a best-effort basis. Execution might overrun this timeout if, for example, a long pause is encountered while FieldData is loaded for a field.

    • use_significance boolean Required

      Filters associated terms so only those that are significantly associated with your query are included.

  • query object

    A seed query that identifies the documents of interest. Can be any valid Elasticsearch query.

    External documentation
  • vertices array[object]

    Specifies one or more fields that contain the terms you want to include in the graph as vertices.

    Hide vertices attributes Show vertices attributes object
    • exclude array[string]

      Prevents the specified terms from being included in the results.

    • field string Required

      Identifies a field in the documents of interest.

    • include array[object]

      Identifies the terms of interest that form the starting points from which you want to spider out.

      Hide include attributes Show include attributes object
      • boost number Required
      • term string Required
    • min_doc_count number

      Specifies how many documents must contain a pair of terms before it is considered to be a useful connection. This setting acts as a certainty threshold.

      Default value is 3.

    • shard_min_doc_count number

      Controls how many documents on a particular shard have to contain a pair of terms before the connection is returned for global consideration.

      Default value is 2.

    • size number

      Specifies the maximum number of vertex terms returned for each field.

      Default value is 5.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • connections array[object] Required
      Hide connections attributes Show connections attributes object
      • doc_count number Required
      • source number Required
      • target number Required
      • weight number Required
    • failures array[object] Required
      Hide failures attributes Show failures attributes object
      • index string
      • node string
      • reason object Required

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide reason attributes Show reason attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

          One of:
          string-1 string string-2 string | null
        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • shard number
      • status string
      • primary boolean
    • timed_out boolean Required
    • took number Required
    • vertices array[object] Required
      Hide vertices attributes Show vertices attributes object
      • depth number Required
      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • term string Required
      • weight number Required
POST /{index}/_graph/explore 
POST clicklogs/_graph/explore
{
  "query": {
    "match": {
      "query.raw": "midi"
    }
  },
  "vertices": [
    {
      "field": "product"
    }
  ],
  "connections": {
    "vertices": [
      {
        "field": "query.raw"
      }
    ]
  }
}
resp = client.graph.explore(
    index="clicklogs",
    query={
        "match": {
            "query.raw": "midi"
        }
    },
    vertices=[
        {
            "field": "product"
        }
    ],
    connections={
        "vertices": [
            {
                "field": "query.raw"
            }
        ]
    },
)
const response = await client.graph.explore({
  index: "clicklogs",
  query: {
    match: {
      "query.raw": "midi",
    },
  },
  vertices: [
    {
      field: "product",
    },
  ],
  connections: {
    vertices: [
      {
        field: "query.raw",
      },
    ],
  },
});
response = client.graph.explore(
  index: "clicklogs",
  body: {
    "query": {
      "match": {
        "query.raw": "midi"
      }
    },
    "vertices": [
      {
        "field": "product"
      }
    ],
    "connections": {
      "vertices": [
        {
          "field": "query.raw"
        }
      ]
    }
  }
)
$resp = $client->graph()->explore([
    "index" => "clicklogs",
    "body" => [
        "query" => [
            "match" => [
                "query.raw" => "midi",
            ],
        ],
        "vertices" => array(
            [
                "field" => "product",
            ],
        ),
        "connections" => [
            "vertices" => array(
                [
                    "field" => "query.raw",
                ],
            ),
        ],
    ],
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"query":{"match":{"query.raw":"midi"}},"vertices":[{"field":"product"}],"connections":{"vertices":[{"field":"query.raw"}]}}' "$ELASTICSEARCH_URL/clicklogs/_graph/explore"
client.graph().explore(e -> e
    .connections(c -> c
        .vertices(v -> v
            .field("query.raw")
        )
    )
    .index("clicklogs")
    .query(q -> q
        .match(m -> m
            .field("query.raw")
            .query(FieldValue.of("midi"))
        )
    )
    .vertices(v -> v
        .field("product")
    )
);
Request example
Run `POST clicklogs/_graph/explore` for a basic exploration An initial graph explore query typically begins with a query to identify strongly related terms. Seed the exploration with a query. This example is searching `clicklogs` for people who searched for the term `midi`.Identify the vertices to include in the graph. This example is looking for product codes that are significantly associated with searches for `midi`. Find the connections. This example is looking for other search terms that led people to click on the products that are associated with searches for `midi`. 
{
  "query": {
    "match": {
      "query.raw": "midi"
    }
  },
  "vertices": [
    {
      "field": "product"
    }
  ],
  "connections": {
    "vertices": [
      {
        "field": "query.raw"
      }
    ]
  }
}

Get index templates Generally available; Added in 7.9.0

GET /_index_template/{name}
Api key auth Basic auth Bearer auth

All methods and paths for this operation:

GET /_index_template

GET /_index_template/{name}

Get information about one or more index templates.

Required authorization

  • Cluster privileges: manage_index_templates

Path parameters

  • name string Required

    Comma-separated list of index template names used to limit the request. Wildcard (*) expressions are supported.

Query parameters

  • local boolean

    If true, the request retrieves information from the local node only. Defaults to false, which means information is retrieved from the master node.

  • flat_settings boolean

    If true, returns settings in flat format.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • include_defaults boolean Generally available; Added in 8.11.0

    If true, returns all relevant default configurations for the index template.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • index_templates array[object] Required
      Hide index_templates attributes Show index_templates attributes object
      • name string Required
      • index_template object Required
        Hide index_template attributes Show index_template attributes object
        • index_patterns string | array[string] Required

          Name of the index template.

        • composed_of array[string] Required

          An ordered list of component template names. Component templates are merged in the order specified, meaning that the last component template specified has the highest precedence.

        • template object

          Template to be applied. It may optionally include an aliases, mappings, or settings configuration.

        • version number

          Version number used to manage index templates externally. This number is not automatically generated by Elasticsearch.

        • priority number

          Priority to determine index template precedence when a new data stream or index is created. The index template with the highest priority is chosen. If no priority is specified the template is treated as though it is of priority 0 (lowest priority). This number is not automatically generated by Elasticsearch.

        • _meta object

          Optional user metadata about the index template. May have any contents. This map is not automatically generated by Elasticsearch.

        • allow_auto_create boolean
        • data_stream object

          If this object is included, the template is used to create data streams and their backing indices. Supports an empty object. Data streams require a matching index template with a data_stream object.

        • deprecated boolean Generally available; Added in 8.12.0

          Marks this index template as deprecated. When creating or updating a non-deprecated index template that uses deprecated components, Elasticsearch will emit a deprecation warning.

        • ignore_missing_component_templates string | array[string]

          A list of component template names that are allowed to be absent.
GET /_index_template/{name} 
GET _index_template/*?filter_path=index_templates.name,index_templates.index_template.index_patterns,index_templates.index_template.data_stream

resp = client.indices.get_index_template(
    name="*",
    filter_path="index_templates.name,index_templates.index_template.index_patterns,index_templates.index_template.data_stream",
)
const response = await client.indices.getIndexTemplate({
  name: "*",
  filter_path:
    "index_templates.name,index_templates.index_template.index_patterns,index_templates.index_template.data_stream",
});
response = client.indices.get_index_template(
  name: "*",
  filter_path: "index_templates.name,index_templates.index_template.index_patterns,index_templates.index_template.data_stream"
)
$resp = $client->indices()->getIndexTemplate([
    "name" => "*",
    "filter_path" => "index_templates.name,index_templates.index_template.index_patterns,index_templates.index_template.data_stream",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_index_template/*?filter_path=index_templates.name,index_templates.index_template.index_patterns,index_templates.index_template.data_stream"

Flush data streams or indices Generally available

GET /{index}/_flush
Api key auth Basic auth Bearer auth

All methods and paths for this operation:

POST /_flush

GET /_flush
POST /{index}/_flush
GET /{index}/_flush

Flushing a data stream or index is the process of making sure that any data that is currently only stored in the transaction log is also permanently stored in the Lucene index. When restarting, Elasticsearch replays any unflushed operations from the transaction log into the Lucene index to bring it back into the state that it was in before the restart. Elasticsearch automatically triggers flushes as needed, using heuristics that trade off the size of the unflushed transaction log against the cost of performing each flush.

After each operation has been flushed it is permanently stored in the Lucene index. This may mean that there is no need to maintain an additional copy of it in the transaction log. The transaction log is made up of multiple files, called generations, and Elasticsearch will delete any generation files when they are no longer needed, freeing up disk space.

It is also possible to trigger a flush on one or more indices using the flush API, although it is rare for users to need to call this API directly. If you call the flush API after indexing some documents then a successful response indicates that Elasticsearch has flushed all the documents that were indexed before the flush API was called.

Required authorization

  • Index privileges: maintenance

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams, indices, and aliases to flush. Supports wildcards (*). To flush all data streams and indices, omit this parameter or use * or _all.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.

    Values are all, open, closed, hidden, or none.

  • force boolean

    If true, the request forces a flush even if there are no changes to commit to the index.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • wait_if_ongoing boolean

    If true, the flush operation blocks until execution when another flush operation is running. If false, Elasticsearch returns an error if you request a flush when another flush operation is running.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • _shards object
      Hide _shards attributes Show _shards attributes object
      • failed number Required

        The number of shards the operation or search attempted to run on but failed.

      • successful number Required

        The number of shards the operation or search succeeded on.

      • total number Required

        The number of shards the operation or search will run on overall.

      • failures array[object]
        Hide failures attributes Show failures attributes object
        • index string
        • node string
        • reason object Required

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • shard number
        • status string
        • primary boolean
      • skipped number
GET /{index}/_flush 
POST /_flush

resp = client.indices.flush()
const response = await client.indices.flush();
response = client.indices.flush
$resp = $client->indices()->flush();
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_flush"
client.indices().flush(f -> f);

Get mapping definitions Generally available

GET /{index}/_mapping
Api key auth Basic auth Bearer auth

All methods and paths for this operation:

GET /_mapping

GET /{index}/_mapping

For data streams, the API retrieves mappings for the stream’s backing indices.

Required authorization

  • Index privileges: view_index_metadata

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Supported values include:

    • all: Match any data stream or index, including hidden ones.
    • open: Match open, non-hidden indices. Also matches any non-hidden data stream.
    • closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
    • hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
    • none: Wildcard expressions are not accepted.

    Values are all, open, closed, hidden, or none.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • local boolean Deprecated

    If true, the request retrieves information from the local node only.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • item object
        Hide item attributes Show item attributes object
        • all_field object
          Hide all_field attributes Show all_field attributes object
          • analyzer string Required
          • enabled boolean Required
          • omit_norms boolean Required
          • search_analyzer string Required
          • similarity string Required
          • store boolean Required
          • store_term_vector_offsets boolean Required
          • store_term_vector_payloads boolean Required
          • store_term_vector_positions boolean Required
          • store_term_vectors boolean Required
        • date_detection boolean
        • dynamic string

          Values are strict, runtime, true, or false.

        • dynamic_date_formats array[string]
        • dynamic_templates array[object]
        • _field_names object
          Hide _field_names attribute Show _field_names attribute object
          • enabled boolean Required
        • index_field object
          Hide index_field attribute Show index_field attribute object
          • enabled boolean Required
        • _meta object
          Hide _meta attribute Show _meta attribute object
          • * object Additional properties
        • numeric_detection boolean
        • properties object
        • _routing object
          Hide _routing attribute Show _routing attribute object
          • required boolean Required
        • _size object
          Hide _size attribute Show _size attribute object
          • enabled boolean Required
        • _source object
          Hide _source attributes Show _source attributes object
          • compress boolean
          • compress_threshold string
          • enabled boolean
          • excludes array[string]
          • includes array[string]
        • runtime object
          Hide runtime attribute Show runtime attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • fields object

              For type composite

            • fetch_fields array[object]

              For type lookup

            • format string

              A custom format for date type runtime fields.

        • enabled boolean
        • subobjects string

          Values are true or false.

        • _data_stream_timestamp object
          Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
          • enabled boolean Required
      • mappings object Required
        Hide mappings attributes Show mappings attributes object
        • all_field object
          Hide all_field attributes Show all_field attributes object
          • analyzer string Required
          • enabled boolean Required
          • omit_norms boolean Required
          • search_analyzer string Required
          • similarity string Required
          • store boolean Required
          • store_term_vector_offsets boolean Required
          • store_term_vector_payloads boolean Required
          • store_term_vector_positions boolean Required
          • store_term_vectors boolean Required
        • date_detection boolean
        • dynamic string

          Values are strict, runtime, true, or false.

        • dynamic_date_formats array[string]
        • dynamic_templates array[object]
        • _field_names object
          Hide _field_names attribute Show _field_names attribute object
          • enabled boolean Required
        • index_field object
          Hide index_field attribute Show index_field attribute object
          • enabled boolean Required
        • _meta object
          Hide _meta attribute Show _meta attribute object
          • * object Additional properties
        • numeric_detection boolean
        • properties object
        • _routing object
          Hide _routing attribute Show _routing attribute object
          • required boolean Required
        • _size object
          Hide _size attribute Show _size attribute object
          • enabled boolean Required
        • _source object
          Hide _source attributes Show _source attributes object
          • compress boolean
          • compress_threshold string
          • enabled boolean
          • excludes array[string]
          • includes array[string]
        • runtime object
          Hide runtime attribute Show runtime attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • fields object

              For type composite

            • fetch_fields array[object]

              For type lookup

            • format string

              A custom format for date type runtime fields.

        • enabled boolean
        • subobjects string

          Values are true or false.

        • _data_stream_timestamp object
          Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
          • enabled boolean Required
GET /{index}/_mapping 
GET /books/_mapping

resp = client.indices.get_mapping(
    index="books",
)
const response = await client.indices.getMapping({
  index: "books",
});
response = client.indices.get_mapping(
  index: "books"
)
$resp = $client->indices()->getMapping([
    "index" => "books",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/books/_mapping"
client.indices().getMapping(g -> g
    .index("books")
);

Perform chat completion inference Generally available; Added in 8.18.0

POST /_inference/chat_completion/{inference_id}/_stream
Api key auth Basic auth Bearer auth

The chat completion inference API enables real-time responses for chat completion tasks by delivering answers incrementally, reducing response times during computation. It only works with the chat_completion task type for openai and elastic inference services.

NOTE: The chat_completion task type is only available within the _stream API and only supports streaming. The Chat completion inference API and the Stream inference API differ in their response structure and capabilities. The Chat completion inference API provides more comprehensive customization options through more fields and function calling support. If you use the openai, hugging_face or the elastic service, use the Chat completion inference API.

Path parameters

  • inference_id string Required

    The inference Id

Query parameters

  • timeout string

    Specifies the amount of time to wait for the inference request to complete.

    Values are -1 or 0.

application/json

Body Required

  • messages array[object] Required

    A list of objects representing the conversation. Requests should generally only add new messages from the user (role user). The other message roles (assistant, system, or tool) should generally only be copied from the response to a previous completion request, such that the messages array is built up throughout a conversation.

    An object representing part of the conversation.

    Hide messages attributes Show messages attributes object

    • content string | array[object]

      The content of the message.

      String example:

      {
   "content": "Some string"
}

      Object example:

      {
  "content": [
      {
       "text": "Some text",
       "type": "text"
      }
   ]
}
      One of:
      string-1 string array-2 array[object]

      The content of the message.

      String example:

      {
   "content": "Some string"
}

      Object example:

      {
  "content": [
      {
       "text": "Some text",
       "type": "text"
      }
   ]
}
    • role string Required

      The role of the message author. Valid values are user, assistant, system, and tool.

    • tool_call_id string

      Only for tool role messages. The tool call that this message is responding to.

    • tool_calls array[object]

      Only for assistant role messages. The tool calls generated by the model. If it's specified, the content field is optional. Example:

      {
  "tool_calls": [
      {
          "id": "call_KcAjWtAww20AihPHphUh46Gd",
          "type": "function",
          "function": {
              "name": "get_current_weather",
              "arguments": "{\"location\":\"Boston, MA\"}"
          }
      }
  ]
}

      A tool call generated by the model.

      Hide tool_calls attributes Show tool_calls attributes object
      • id string Required

        The identifier of the tool call.

      • function object Required

        The function that the model called.

        Hide function attributes Show function attributes object
        • arguments string Required

          The arguments to call the function with in JSON format.

        • name string Required

          The name of the function to call.

      • type string Required

        The type of the tool call.

  • model string

    The ID of the model to use.

  • max_completion_tokens number

    The upper bound limit for the number of tokens that can be generated for a completion request.

  • stop array[string]

    A sequence of strings to control when the model should stop generating additional tokens.

  • temperature number

    The sampling temperature to use.

  • tool_choice string | object

    Controls which tool is called by the model. String representation: One of auto, none, or requrired. auto allows the model to choose between calling tools and generating a message. none causes the model to not call any tools. required forces the model to call one or more tools. Example (object representation):

    {
  "tool_choice": {
      "type": "function",
      "function": {
          "name": "get_current_weather"
      }
  }
}
    One of:
    string-1 string CompletionToolChoice object

    Controls which tool is called by the model. String representation: One of auto, none, or requrired. auto allows the model to choose between calling tools and generating a message. none causes the model to not call any tools. required forces the model to call one or more tools. Example (object representation):

    {
  "tool_choice": {
      "type": "function",
      "function": {
          "name": "get_current_weather"
      }
  }
}
  • tools array[object]

    A list of tools that the model can call. Example:

    {
  "tools": [
      {
          "type": "function",
          "function": {
              "name": "get_price_of_item",
              "description": "Get the current price of an item",
              "parameters": {
                  "type": "object",
                  "properties": {
                      "item": {
                          "id": "12345"
                      },
                      "unit": {
                          "type": "currency"
                      }
                  }
              }
          }
      }
  ]
}

    A list of tools that the model can call.

    Hide tools attributes Show tools attributes object
    • type string Required

      The type of tool.

    • function object Required

      The function definition.

      Hide function attributes Show function attributes object
      • description string

        A description of what the function does. This is used by the model to choose when and how to call the function.

      • name string Required

        The name of the function.

      • parameters object

        The parameters the functional accepts. This should be formatted as a JSON object.

      • strict boolean

        Whether to enable schema adherence when generating the function call.

  • top_p number

    Nucleus sampling, an alternative to sampling with temperature.

Responses

  • 200 application/json
POST /_inference/chat_completion/{inference_id}/_stream 
POST _inference/chat_completion/openai-completion/_stream
{
  "model": "gpt-4o",
  "messages": [
      {
          "role": "user",
          "content": "What is Elastic?"
      }
  ]
}
resp = client.inference.chat_completion_unified(
    inference_id="openai-completion",
    chat_completion_request={
        "model": "gpt-4o",
        "messages": [
            {
                "role": "user",
                "content": "What is Elastic?"
            }
        ]
    },
)
const response = await client.inference.chatCompletionUnified({
  inference_id: "openai-completion",
  chat_completion_request: {
    model: "gpt-4o",
    messages: [
      {
        role: "user",
        content: "What is Elastic?",
      },
    ],
  },
});
response = client.inference.chat_completion_unified(
  inference_id: "openai-completion",
  body: {
    "model": "gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": "What is Elastic?"
      }
    ]
  }
)
$resp = $client->inference()->chatCompletionUnified([
    "inference_id" => "openai-completion",
    "body" => [
        "model" => "gpt-4o",
        "messages" => array(
            [
                "role" => "user",
                "content" => "What is Elastic?",
            ],
        ),
    ],
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-4o","messages":[{"role":"user","content":"What is Elastic?"}]}' "$ELASTICSEARCH_URL/_inference/chat_completion/openai-completion/_stream"
client.inference().chatCompletionUnified(c -> c
    .inferenceId("openai-completion")
    .chatCompletionRequest(ch -> ch
        .messages(m -> m
            .content(co -> co
                .string("What is Elastic?")
            )
            .role("user")
        )
        .model("gpt-4o")
    )
);
Request examples
Run `POST _inference/chat_completion/openai-completion/_stream` to perform a chat completion on the example question with streaming. 
{
  "model": "gpt-4o",
  "messages": [
      {
          "role": "user",
          "content": "What is Elastic?"
      }
  ]
}
Run `POST _inference/chat_completion/openai-completion/_stream` to perform a chat completion using an Assistant message with `tool_calls`. 
{
  "messages": [
      {
          "role": "assistant",
          "content": "Let's find out what the weather is",
          "tool_calls": [ 
              {
                  "id": "call_KcAjWtAww20AihPHphUh46Gd",
                  "type": "function",
                  "function": {
                      "name": "get_current_weather",
                      "arguments": "{\"location\":\"Boston, MA\"}"
                  }
              }
          ]
      },
      { 
          "role": "tool",
          "content": "The weather is cold",
          "tool_call_id": "call_KcAjWtAww20AihPHphUh46Gd"
      }
  ]
}
Run `POST _inference/chat_completion/openai-completion/_stream` to perform a chat completion using a User message with `tools` and `tool_choice`. 
{
  "messages": [
      {
          "role": "user",
          "content": [
              {
                  "type": "text",
                  "text": "What's the price of a scarf?"
              }
          ]
      }
  ],
  "tools": [
      {
          "type": "function",
          "function": {
              "name": "get_current_price",
              "description": "Get the current price of a item",
              "parameters": {
                  "type": "object",
                  "properties": {
                      "item": {
                          "id": "123"
                      }
                  }
              }
          }
      }
  ],
  "tool_choice": {
      "type": "function",
      "function": {
          "name": "get_current_price"
      }
  }
}
Response examples (200)
A successful response when performing a chat completion task using a User message with `tools` and `tool_choice`. 
event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[{"delta":{"content":"","role":"assistant"},"index":0}],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk"}}

event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[{"delta":{"content":Elastic"},"index":0}],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk"}}

event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[{"delta":{"content":" is"},"index":0}],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk"}}

(...)

event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk","usage":{"completion_tokens":28,"prompt_tokens":16,"total_tokens":44}}} 

event: message
data: [DONE]

Get an inference endpoint Generally available; Added in 8.11.0

GET /_inference/{task_type}/{inference_id}
Api key auth Basic auth Bearer auth

All methods and paths for this operation:

GET /_inference

GET /_inference/{inference_id}
GET /_inference/{task_type}/{inference_id}

Path parameters

  • task_type string

    The task type

    Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.

  • inference_id string Required

    The inference Id

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • endpoints array[object] Required

      Represents an inference endpoint as returned by the GET API

      Hide endpoints attributes Show endpoints attributes object

      Represents an inference endpoint as returned by the GET API

      • chunking_settings object

        Chunking configuration object

        Hide chunking_settings attributes Show chunking_settings attributes object
        • max_chunk_size number

          The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

          Default value is 250.

        • overlap number

          The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

          Default value is 100.

        • sentence_overlap number

          The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

          Default value is 1.

        • strategy string

          The chunking strategy: sentence or word.

          Default value is sentence.

      • service string Required

        The service type

      • service_settings object Required

        Settings specific to the service

      • task_settings object

        Task settings specific to the service and task type

      • inference_id string Required

        The inference Id

      • task_type string Required

        The task type

        Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.
GET /_inference/{task_type}/{inference_id} 
GET _inference/sparse_embedding/my-elser-model

resp = client.inference.get(
    task_type="sparse_embedding",
    inference_id="my-elser-model",
)
const response = await client.inference.get({
  task_type: "sparse_embedding",
  inference_id: "my-elser-model",
});
response = client.inference.get(
  task_type: "sparse_embedding",
  inference_id: "my-elser-model"
)
$resp = $client->inference()->get([
    "task_type" => "sparse_embedding",
    "inference_id" => "my-elser-model",
]);
curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_inference/sparse_embedding/my-elser-model"
client.inference().get(g -> g
    .inferenceId("my-elser-model")
    .taskType(TaskType.SparseEmbedding)
);