May 23, 2017

Logstash Lines: Progress on Logstash modules, JRuby 9k and more

By Suyog Rao

Welcome back to The Logstash Lines! In these weekly posts, we'll share the latest happenings in the world of Logstash and its ecosystem.

Logstash modules (5.x)

We made good progress on LS modules this week. The first parts have landed on a feature branch which allows users to create modules as LS plugin gems. We've also added support to launch modules from the CLI and the ability to override configs using variables. A lot of this UX is consistent with the beats modules.

Other changes in 5.5

  • For Logstash internal metrics, we removed dependency on a ruby gem that was used to report Java/JRuby process stats. This was reimplemented directly in Java which should provide better performance and also help maintenance by bringing into LS core.
  • Persistent Queue: Fixed an issue where file descriptors were being leaked.
  • Persistent Queue: Refactored Settings classes to use a builder pattern.
  • Avro Codec: Fixed an issue where double data type was not getting serialized correctly (#12, #5).

Other changes in 6.0

  • Breaking: For rpm and deb package installs, Logstash now ignores configuration files not ending in .conf in the /etc/logstash/conf.d config directory. Users would run into issues previously since LS would load all files in this directory.

There's also plenty of infrastructure and cleanup work on-going:

Upgrade to JRuby 9k (In-progress, 6.0.0)

Logstash currently uses JRuby 1.7.26 which was EOL'd last year. This week we started an effort to upgrade to JRuby 9k for LS 6.0.0. This will bring in Ruby 2.0 support as well. This a pretty significant change and we want to make sure all our core and plugins tests pass.