Product release

Logstash 2.4.0 released

We are pleased to announce that Logstash 2.4 has been released. Please see the release notes for a detailed change report and you can download binaries on our products page.

New Plugin API Compatibility

While 2.4 does not have many headlining features, it's an important release that brings in key internal API changes. These changes make 2.4 compatible with plugins developed for the 5.0 API. In other words, Logstash 2.4 will be able to install most plugins that use the 5.0 API. As maintainers, this API backport eases efforts which mean we don't have to maintain 2 separate code branches for each plugin. We have a lot of plugins.

The Logstash 2.4 release enables new plugin features that are not available in the remaining 2.x series. We've recently updated a bunch of plugins to add new functionality. A popular enhancement request has been to bring in new features in Kafka's 0.9.x and 0.10.x releases like security. In 2.4, we don’t package Kafka input/output plugin versions that support newer Kafka versions (see the compatibility matrix). This is because most config options for these plugins have been changed which makes it not compatible, and we don't intend to break configurations in point releases. If you still want to get all the goodness of the latest Kafka versions in 2.4, fear not. You can simply install the plugin on top of 2.4, like so:

bin/logstash-plugin install —version 5.0.4 logstash-input-kafka 

This is just an example. Like Kafka, there are many plugins which have been updated. They'll be packaged in our upcoming 5.0 release, but if you can't wait, you can always install them on top of 2.4. Please review the changelog for individual plugins before upgrading to them.

Beats Input v3.0: Locked And Loaded

Beats input has been reimplemented using Netty, an asynchronous IO framework for Java. This rewrite for performance brings it in line with Logstash Forwarder + LS combination. In some test cases, it is faster. Go on, migrate all your LSF instances to the Filebeat goodness. I've heard Filebeat 5.0 is packed with features, so you don't want to miss out on that. As part of the Beats refactor we now only supports private keys in the PKCS8 format, you can use OpenSSL to convert them, newer version of OpenSSL will already create the key in the right format. Oh and a note for our subscription customers — LSF will reach EOL on Nov 7, 2016. 

Structured Logging

Back in the day, Logstash used to emit its logs in JSON format. Because of endless encoding issues and crashes in the logger, we had to roll it back. We've worked through most of these issues, so we're bringing structured logging back. You can tell LS to switch to JSON logging by using the --log-in-json CLI flag. Be aware that logging in JSON is verbose and may fill up disk even faster than before!

Plugin Generator

Kibana has it. Beats has it. And now Logstash has it too! We're talking about the plugin generator tool that makes it easier to bootstrap new plugins for Logstash. Previously we've recommended developers to clone/fork the examples repositories, but now you can simply do:

bin/logstash-plugin generate --type input --name xkcd --path ~/ws/elastic/plugins

This subcommand bootstraps a new plugin logstash-input-xkcd with the right directory structure and all the required files (templates) for you to start developing this plugin right away. So, go on, create that xkcd input to stream those fine comic strips to Kibana!

Feedback

We are super excited for this release of Logstash and look forward to your feedback. You can reach us at our forum, open issues in your GitHub repo or twitter(@elastic). Happy 'stashing!