Simulate pipeline APIedit

Executes an ingest pipeline against a set of provided documents.

POST /_ingest/pipeline/my-pipeline-id/_simulate
{
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "rab"
      }
    }
  ]
}

Requestedit

POST /_ingest/pipeline/<pipeline>/_simulate

GET /_ingest/pipeline/<pipeline>/_simulate

POST /_ingest/pipeline/_simulate

GET /_ingest/pipeline/_simulate

Prerequisitesedit

  • If the Elasticsearch security features are enabled, you must have the read_pipeline, manage_pipeline, manage_ingest_pipelines, or manage cluster privilege to use this API.

Descriptionedit

The simulate pipeline API executes a specific pipeline against a set of documents provided in the body of the request.

You can either specify an existing pipeline to execute against the provided documents or supply a pipeline definition in the body of the request.

Path parametersedit

<pipeline>
(Required*, string) Pipeline to test. If you don’t specify a pipeline in the request body, this parameter is required.

Query parametersedit

verbose
(Optional, Boolean) If true, the response includes output data for each processor in the executed pipeline.

Request bodyedit

pipeline

(Required*, object) Pipeline to test. If you don’t specify the <pipeline> request path parameter, this parameter is required. If you specify both this and the request path parameter, the API only uses the request path parameter.

Properties of pipeline
description
(Optional, string) Description of the ingest pipeline.
on_failure

(Optional, array of processor objects) Processors to run immediately after a processor failure.

Each processor supports a processor-level on_failure value. If a processor without an on_failure value fails, Elasticsearch uses this pipeline-level parameter as a fallback. The processors in this parameter run sequentially in the order specified. Elasticsearch will not attempt to run the pipeline’s remaining processors.

processors
(Required, array of processor objects) Processors used to preform transformations on documents before indexing. Processors run sequentially in the order specified.
version

(Optional, integer) Version number used by external systems to track ingest pipelines.

This parameter is intended for external systems only. Elasticsearch does not use or validate pipeline version numbers.

docs

(Required, array of objects) Sample documents to test in the pipeline.

Properties of docs objects
_id
(Optional, string) Unique identifier for the document. This ID must be unique within the _index.
_index
(Optional, string) Name of the index containing the document.
_routing
(Optional, string) Value used to send the document to a specific primary shard. See the _routing field.
_source
(Required, object) JSON body for the document.

Examplesedit

Specify a pipeline as a path parameteredit

POST /_ingest/pipeline/my-pipeline-id/_simulate
{
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "rab"
      }
    }
  ]
}

The API returns the following response:

{
   "docs": [
      {
         "doc": {
            "_id": "id",
            "_index": "index",
            "_source": {
               "field2": "_value",
               "foo": "bar"
            },
            "_ingest": {
               "timestamp": "2017-05-04T22:30:03.187Z"
            }
         }
      },
      {
         "doc": {
            "_id": "id",
            "_index": "index",
            "_source": {
               "field2": "_value",
               "foo": "rab"
            },
            "_ingest": {
               "timestamp": "2017-05-04T22:30:03.188Z"
            }
         }
      }
   ]
}

Specify a pipeline in the request bodyedit

POST /_ingest/pipeline/_simulate
{
  "pipeline" :
  {
    "description": "_description",
    "processors": [
      {
        "set" : {
          "field" : "field2",
          "value" : "_value"
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "rab"
      }
    }
  ]
}

The API returns the following response:

{
   "docs": [
      {
         "doc": {
            "_id": "id",
            "_index": "index",
            "_source": {
               "field2": "_value",
               "foo": "bar"
            },
            "_ingest": {
               "timestamp": "2017-05-04T22:30:03.187Z"
            }
         }
      },
      {
         "doc": {
            "_id": "id",
            "_index": "index",
            "_source": {
               "field2": "_value",
               "foo": "rab"
            },
            "_ingest": {
               "timestamp": "2017-05-04T22:30:03.188Z"
            }
         }
      }
   ]
}

View verbose resultsedit

You can use the simulate pipeline API to see how each processor affects the ingest document as it passes through the pipeline. To see the intermediate results of each processor in the simulate request, you can add the verbose parameter to the request.

POST /_ingest/pipeline/_simulate?verbose=true
{
  "pipeline" :
  {
    "description": "_description",
    "processors": [
      {
        "set" : {
          "field" : "field2",
          "value" : "_value2"
        }
      },
      {
        "set" : {
          "field" : "field3",
          "value" : "_value3"
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "rab"
      }
    }
  ]
}

The API returns the following response:

{
  "docs" : [
    {
      "processor_results" : [
        {
          "processor_type" : "set",
          "status" : "success",
          "doc" : {
            "_index" : "index",
            "_id" : "id",
            "_source" : {
              "field2" : "_value2",
              "foo" : "bar"
            },
            "_ingest" : {
              "pipeline" : "_simulate_pipeline",
              "timestamp" : "2020-07-30T01:21:24.251836Z"
            }
          }
        },
        {
          "processor_type" : "set",
          "status" : "success",
          "doc" : {
            "_index" : "index",
            "_id" : "id",
            "_source" : {
              "field3" : "_value3",
              "field2" : "_value2",
              "foo" : "bar"
            },
            "_ingest" : {
              "pipeline" : "_simulate_pipeline",
              "timestamp" : "2020-07-30T01:21:24.251836Z"
            }
          }
        }
      ]
    },
    {
      "processor_results" : [
        {
          "processor_type" : "set",
          "status" : "success",
          "doc" : {
            "_index" : "index",
            "_id" : "id",
            "_source" : {
              "field2" : "_value2",
              "foo" : "rab"
            },
            "_ingest" : {
              "pipeline" : "_simulate_pipeline",
              "timestamp" : "2020-07-30T01:21:24.251863Z"
            }
          }
        },
        {
          "processor_type" : "set",
          "status" : "success",
          "doc" : {
            "_index" : "index",
            "_id" : "id",
            "_source" : {
              "field3" : "_value3",
              "field2" : "_value2",
              "foo" : "rab"
            },
            "_ingest" : {
              "pipeline" : "_simulate_pipeline",
              "timestamp" : "2020-07-30T01:21:24.251863Z"
            }
          }
        }
      ]
    }
  ]
}