Simulate a pipeline Generally available; Added in 5.0.0

POST /_ingest/pipeline/{id}/_simulate

All methods and paths for this operation:

GET /_ingest/pipeline/_simulate

POST /_ingest/pipeline/_simulate
GET /_ingest/pipeline/{id}/_simulate
POST /_ingest/pipeline/{id}/_simulate

Run an ingest pipeline against a set of provided documents. You can either specify an existing pipeline to use with the provided documents or supply a pipeline definition in the body of the request.

Required authorization

  • Cluster privileges: read_pipeline

Path parameters

  • id string Required

    The pipeline to test. If you don't specify a pipeline in the request body, this parameter is required.

Query parameters

  • verbose boolean

    If true, the response includes output data for each processor in the executed pipeline.

application/json

Body Required

  • docs array[object] Required

    Sample documents to test in the pipeline.

    Hide docs attributes Show docs attributes object
    • _id string

      Unique identifier for the document. This ID must be unique within the _index.

    • _index string

      Name of the index containing the document.

    • _source object Required

      JSON body for the document.

  • pipeline object Additional properties

    The pipeline to test. If you don't specify the pipeline request path parameter, this parameter is required. If you specify both this and the request path parameter, the API only uses the request path parameter.

    Hide pipeline attributes Show pipeline attributes object
    • description string

      Description of the ingest pipeline.

    • on_failure array[object]

      Processors to run immediately after a processor failure.

      Hide on_failure attributes Show on_failure attributes object
      • append
      • attachment
      • bytes
      • cef
      • circle
      • community_id
      • convert
      • csv
      • date
      • date_index_name
      • dissect
      • dot_expander
      • drop
      • enrich
      • fail
      • fingerprint
      • foreach
      • ip_location
      • geo_grid
      • geoip
      • grok
      • gsub
      • html_strip
      • inference
      • join
      • json
      • kv
      • lowercase
      • network_direction
      • pipeline
      • redact
      • registered_domain
      • remove
      • rename
      • reroute
      • script
      • set
      • set_security_user
      • sort
      • split
      • terminate
      • trim
      • uppercase
      • urldecode
      • uri_parts
      • user_agent
    • processors array[object]

      Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.

      Hide processors attributes Show processors attributes object
      • append
      • attachment
      • bytes
      • cef
      • circle
      • community_id
      • convert
      • csv
      • date
      • date_index_name
      • dissect
      • dot_expander
      • drop
      • enrich
      • fail
      • fingerprint
      • foreach
      • ip_location
      • geo_grid
      • geoip
      • grok
      • gsub
      • html_strip
      • inference
      • join
      • json
      • kv
      • lowercase
      • network_direction
      • pipeline
      • redact
      • registered_domain
      • remove
      • rename
      • reroute
      • script
      • set
      • set_security_user
      • sort
      • split
      • terminate
      • trim
      • uppercase
      • urldecode
      • uri_parts
      • user_agent
    • version number

      Version number used by external systems to track ingest pipelines.

    • deprecated boolean

      Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.

      Default value is false.

    • _meta object

      Arbitrary metadata about the ingest pipeline. This map is not automatically generated by Elasticsearch.

      Hide _meta attribute Show _meta attribute object
      • * object Additional properties
    • created_date string | number Generally available; Added in 9.2.0

      Date and time when the pipeline was created. Only returned if the human query parameter is true.

      One of:

      Time unit for milliseconds

    • created_date_millis number

      Time unit for milliseconds

    • modified_date string | number Generally available; Added in 9.2.0

      Date and time when the pipeline was last modified. Only returned if the human query parameter is true.

      One of:

      Time unit for milliseconds

    • modified_date_millis number

      Time unit for milliseconds

    • field_access_pattern string

      Controls how processors in this pipeline should read and write data on a document's source.

      Values are classic or flexible.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • docs array[object] Required
      Hide docs attributes Show docs attributes object
      • doc object

        The simulated document, with optional metadata.

        Hide doc attributes Show doc attributes object
        • _routing string

          Value used to send the document to a specific primary shard.

        • _source object Required

          JSON body for the document.

      • error object

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide error attributes Show error attributes object
        • type string Required

          The type of error

        • reason
        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • root_cause array[object]
        • suppressed array[object]
      • processor_results array[object]
        Hide processor_results attributes Show processor_results attributes object
        • doc
        • tag string
        • processor_type string
        • status
        • description string
        • ignored_error
        • error
POST /_ingest/pipeline/{id}/_simulate
POST /_ingest/pipeline/_simulate
{
  "pipeline" :
  {
    "description": "_description",
    "processors": [
      {
        "set" : {
          "field" : "field2",
          "value" : "_value"
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "rab"
      }
    }
  ]
}
resp = client.ingest.simulate(
    pipeline={
        "description": "_description",
        "processors": [
            {
                "set": {
                    "field": "field2",
                    "value": "_value"
                }
            }
        ]
    },
    docs=[
        {
            "_index": "index",
            "_id": "id",
            "_source": {
                "foo": "bar"
            }
        },
        {
            "_index": "index",
            "_id": "id",
            "_source": {
                "foo": "rab"
            }
        }
    ],
)
const response = await client.ingest.simulate({
  pipeline: {
    description: "_description",
    processors: [
      {
        set: {
          field: "field2",
          value: "_value",
        },
      },
    ],
  },
  docs: [
    {
      _index: "index",
      _id: "id",
      _source: {
        foo: "bar",
      },
    },
    {
      _index: "index",
      _id: "id",
      _source: {
        foo: "rab",
      },
    },
  ],
});
response = client.ingest.simulate(
  body: {
    "pipeline": {
      "description": "_description",
      "processors": [
        {
          "set": {
            "field": "field2",
            "value": "_value"
          }
        }
      ]
    },
    "docs": [
      {
        "_index": "index",
        "_id": "id",
        "_source": {
          "foo": "bar"
        }
      },
      {
        "_index": "index",
        "_id": "id",
        "_source": {
          "foo": "rab"
        }
      }
    ]
  }
)
$resp = $client->ingest()->simulate([
    "body" => [
        "pipeline" => [
            "description" => "_description",
            "processors" => array(
                [
                    "set" => [
                        "field" => "field2",
                        "value" => "_value",
                    ],
                ],
            ),
        ],
        "docs" => array(
            [
                "_index" => "index",
                "_id" => "id",
                "_source" => [
                    "foo" => "bar",
                ],
            ],
            [
                "_index" => "index",
                "_id" => "id",
                "_source" => [
                    "foo" => "rab",
                ],
            ],
        ),
    ],
]);
curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"pipeline":{"description":"_description","processors":[{"set":{"field":"field2","value":"_value"}}]},"docs":[{"_index":"index","_id":"id","_source":{"foo":"bar"}},{"_index":"index","_id":"id","_source":{"foo":"rab"}}]}' "$ELASTICSEARCH_URL/_ingest/pipeline/_simulate"
client.ingest().simulate(s -> s
    .docs(List.of(Document.of(d -> d
            .id("id")
            .index("index")
            .source(JsonData.fromJson("{\"foo\":\"bar\"}"))
        ),Document.of(d -> d
            .id("id")
            .index("index")
            .source(JsonData.fromJson("{\"foo\":\"rab\"}"))
        )))
    .pipeline(p -> p
        .description("_description")
        .processors(pr -> pr
            .set(se -> se
                .field("field2")
                .value(JsonData.fromJson("\"_value\""))
            )
        )
    )
);
Request example
You can specify the used pipeline either in the request body or as a path parameter.
{
  "pipeline" :
  {
    "description": "_description",
    "processors": [
      {
        "set" : {
          "field" : "field2",
          "value" : "_value"
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "rab"
      }
    }
  ]
}
Response examples (200)
A successful response for running an ingest pipeline against a set of provided documents.
{
   "docs": [
      {
         "doc": {
            "_id": "id",
            "_index": "index",
            "_version": "-3",
            "_source": {
               "field2": "_value",
               "foo": "bar"
            },
            "_ingest": {
               "timestamp": "2017-05-04T22:30:03.187Z"
            }
         }
      },
      {
         "doc": {
            "_id": "id",
            "_index": "index",
            "_version": "-3",
            "_source": {
               "field2": "_value",
               "foo": "rab"
            },
            "_ingest": {
               "timestamp": "2017-05-04T22:30:03.188Z"
            }
         }
      }
   ]
}