25 avril 2016

Logstash Lines: New Event API

Par Suyog Rao

Welcome back to The Logstash Lines! In these weekly posts, we'll share the latest happenings in the world of Logstash and its ecosystem.

Event API

This week, we spec'd out APIs for interacting with the Event class. Previously, there was no formal, documented API which meant that plugin developers and especially Ruby Filter users could use the internals (hash map) directly to modify data. Not a great idea! We defined setters and getters interfaces which will be used by all plugin developers. This breaking change affects almost all plugins, and we need to review/modify 200+ plugins, but this is the right thing to do in 5.0 and document properly. This will protect us against future implementation changes (e.g. plugging in Java Event implementation). See this issue for new APIs -- we would love any feedback you have! All core plugin tests are passing locally with these changes. Next up, we need to split the work to update all plugins with these new calls.

Acceptance Tests

Work continues on this feature branch to add much needed acceptance tests. Progressing well on validating plugin manager workflow, CLI switches, testing release packages on CentOS and Debian flavors. Also, working with our infra team to add these to the new Jenkins CI server.

Other Fixes/In-Progress Stuff

  • Investigating test failures in RedHat based distros triggered by JRuby 1.7.25 update. This was probably introduced by jruby-ssl update in .25 (#5179).
  • Fixed Logstash service restart does not take KILL_ON_STOP_TIMEOUT into account.
  • Fix for Event accessors cache invalidation in Java and Ruby implementation (#5132).
  • Added static analysis rake task for i18n translation usage (#5158).
  • Java Event type casting conversion optimization (#4999).
  • Elasticsearch Output: Add ingest pipeline support.
  • Elasticsearch Output: Fix bug where update index actions would not work with events with 'data' field (#4796).
  • Kafka Input: Fix for 0.9 consumer doesn't add tags or fields.

Upcoming Releases

Prepping for a 2.3.2 bug fix release this week and 5.0.0-alpha2 next week.