02 novembre 2015

The Logstash Lines: Release Bonanza Edition!

Par Suyog Rao

Welcome back to The Logstash Lines! In these weekly posts, we’ll share the latest happenings in the world of Logstash and its ecosystem.

Its been a busy week here in Logstash land! We released 2.0.0 and 1.5.5 for #ReleaseBonanza week! Logstash 2.0 has Elasticsearch 2.0 compatibility, shutdown semantics, and some nice performance boosts across the board. Logstash 1.5.5 is a bug fix release, and the last planned one for 1.5 series. It also included Beats input, so users don’t necessarily have to upgrade to 2.0 to use Beats.

Resiliency: Persistent Queues

Yep, we’ve talked about this feature previously, but had to put it on the back burner to clear some tech debt, and to lay out some foundation. Post 2.0, we’ve renewed our efforts around this feature. First step is to rewrite Event class in pure Java so we don’t pay the serialization cost across jruby-java boundary while persisting data to disk. We made good progress refactoring existing code around this class to create a pluggable Event gem - logstash-core-event. This will allow us to have a drop-in replacement for the existing ruby Event class.

Centralized Config management

Discussed functional requirements for this project and brainstormed few design approaches.


  • We are getting ready to reach out to our lovely community to help maintain some plugins. To enable this, we’ve created a maintainer’s handbook describing our review process, versioning and labeling guidelines. Stay tuned for a detailed post!
  • Added a continuous integration badge which displays test-run results to all of the 200+ plugin repositories. You can see them on every project’s README. Oh, and every Pull Request you submit to any of Logstash’s plugin repositories is run through our continuous integration platform. This allows you to get quick feedback for code contribution!
  • Refactoring ES output plugin: This had become a bit of nightmare to maintain because we had to support multiple protocols. We are modularizing this class so we can avoid code duplication across http, transport and node protocols.
  • Win Eventlog input: We changed the existing implementation to use win32eventlog lib, which has been working well in our internal testing. We released version 3.0 of this input, so try it out and let us know! You can update to this version by:
bin/plugin install --version 3.0.0 logstash-input-eventlog

Oh, hey, one more important update – its Movember and your favorite Loggy is participating! Go on, rock the ‘stache!

Until next time!