Product release

Logstash 6.0.0-alpha2 released

We are pleased to announce our second alpha release for Logstash 6.0.0! You can find the release notes here or head straight to downloads to try the new features!

NOTE: Alpha releases cannot be used in production. They are for evaluation purposes only

Mo' pipelines mo' fun

We've added support for running multiple, self-contained pipelines in the same Logstash instance (on the same JVM). Logstash configurations can be complex with multiple data flows built using conditionals. This feature will help simplify that — if your data flow and processing can be separated, you can run them in an isolated fashion as separate pipelines. This separation means that a blocked output in one pipeline won’t exert backpressure in the other pipeline.

A new configuration file called pipeline.yml file has been added to define multiple pipelines that can be run on the same instance. Users can dynamically add, modify and remove pipeline configurations. This feature also provides an option to run pipelines with their own run-time settings like workers size, batch size etc. This feature isn't complete yet — the monitoring UI in x-pack currently reports metrics from a single pipeline; we're working on enhancing the UI to support multiple pipelines as well, so stay tuned for updates!

Breaking Change: glob only *.conf in conf.d folder

Currently, when Logstash is installed and set up via package managers, it loads all files found in /etc/logstash/conf.d as configuration. This can be problematic if there are non-configuration files in this folder. For example, user makes a copy of a configuration file and names it with a .bak extension which results in unexpected behavior. Starting from alpha2, Logstash will only glob files ending with .conf extension in /etc/logstash/conf.d (#6979).

Feedback

We would love for you to try 6.0.0-alpha2 and the new multi-pipeline feature! Let us know what you think!