Logstash pipelines

Create, edit, and delete your Logstash pipeline configurations.

This content applies to:

In Project settingsManagementLogstash Pipelines, you can control multiple Logstash instances and pipeline configurations.

On the Logstash side, you must enable configuration management and register Logstash to use the centrally managed pipeline configurations.

After you configure Logstash to use centralized pipeline management, you can no longer specify local pipeline configurations. The pipelines.yml file and settings such as path.config and config.string are inactive when centralized pipeline management is enabled.

Manage pipelines

  1. Configure centralized pipeline management.
  2. To add a new pipeline, go to Project settingsManagementLogstash Pipelines and click Create pipeline. Provide the following details, then click Create and deploy.
Pipeline ID
A name that uniquely identifies the pipeline. This is the ID that you used when you configured centralized pipeline management and specified a list of pipeline IDs in the xpack.management.pipeline.id setting.
A description of the pipeline configuration. This information is for your use.
The pipeline configuration. You can treat the editor like any other editor. You don't have to worry about whitespace or indentation.
Pipeline workers
The number of parallel workers used to run the filter and output stages of the pipeline.
Pipeline batch size
The maximum number of events an individual worker thread collects before executing filters and outputs.
Pipeline batch delay
Time in milliseconds to wait for each event before sending an undersized batch to pipeline workers.
Queue type
The internal queueing model for event buffering. Options are memory for in-memory queueing and persisted for disk-based acknowledged queueing.
Queue max bytes
The total capacity of the queue when persistent queues are enabled.
Queue checkpoint writes
The maximum number of events written before a checkpoint is forced when persistent queues are enabled.

To delete one or more pipelines, select their checkboxes then click Delete.

For more information about pipeline behavior, go to Centralized Pipeline Management.

On this page