31 October 2017

Logstash Lines: Experimenting With Bytecode Generation

By Andrew Cholakian

Welcome back to The Logstash Lines! In these weekly posts, we'll share the latest happenings in the world of Logstash and its ecosystem.

Runtime Pipeline Code Generation via Janino

I'm really excited to announce a new patch that just landed in master and will most likely be released in Logstash 6.1: native java execution with runtime bytecode generation via Janino. This is all thanks to the amazing work of Armin Braun, who's leveraged his significant expertise to deliver us this patch. You can test it out by using the special --experimental-java-execution flag on the master branch of Logstash. 

Logstash will have two ways of executing in releases with this patch:

  1. The current execution, which is very Jruby specific, crufty, and troublesome in terms of supporting pure java plugins efficiently
  2. The new pure java implementation which doesn't have a performance advantage for ruby plugins but should have some nice gains for pure java ones.

Our goal is to continue to improve the feature flagged code generation and get feedback from users willing to try it out as an experimental feature, then make it the default execution sometime in 6.x once it's been tested more in the field. If you're interested in giving it a shot it's probably best to wait till it's out of master and in a proper release behind the aforementioned feature flag as this feature is still under active development.

Notable Patches in master:

  • Buttoned some bugs for our cloud ID support for 6.0, ensuring a flawless and easy setup for cloud users in 6.0
  • Added the Logstash version info to startup.
  • Cleaned up a lot of internals related to rubygems
  • Added the ability to reference metadata fields to the ES Input