18 January 2019

Logstash Lines: Native support for Java plugins in Beta

By João Duarte

Welcome to Logstash Lines! With these weekly series, we're keeping you up to date with what's new in Logstash, including the latest commits and releases.

Did you know Logstash 6.6 ships with HTTP and Memcache enriching filter? Try it and let us know what you think.

Native support for Java plugins reached Beta!

This milestone irons out a lot of edges in the development experience, adds configuration validation, support for flush and codecs and introduces a "co.elastic.logstash.api" package for the core API. Feedback has been great so far and a brave user has already built an experimental plugin!

Java 11 + JRuby 9.2 support backported to 6.7

To collect more feedback on how JRuby 9.2 behaves together with Java 11 we decided to backport their support to 6.7.0.

Multiple plugin releases

This past week we did more than 15 plugin releases, mostly merged code that was sitting in github unreleased for weeks or months. We'll do a post mortem and figure out a way to automatize releases after pull requests are reviewed, merged, and tests pass.

Among the many releases, highlights are support for sliced scrolls in elasticsearch input, ability to customize the response code in http input, better conversion to boolean in mutate filter. See the full list below.

Auto detection of ILM shipping in 7.0.0 (and 6.7.0)

The ability to detect and use ILM if it is supported by the elasticsearch cluster has been added to both 7.0.0 and 6.7.0. However, only in 7.0.0 this is the default behaviour, while in 6.7.0 ILM is disabled. See more about this capability here.

All changes

Repository: elastic/Logstash

  • ASL/LIR: simplify concurrent use of AST, which is globally stateful #10415
  • Better handle malformed URIs #10414
  • Mute metrics_spec line 138 test #10412
  • Pin childprocess to 0.9 #10410
  • keystore: instances of `CharsetEncoder` are stateful and cannot be shared #10408
  • Update :branch: property to 7.0 in index.asciidoc #10404
  • Update :branch: property in 7.x index.asciidoc #10403
  • Bump version to 7.1.0 #10402
  • Add lock file to 7.0 #10401
  • Bump to 8.0.0 #10400
  • Update :branch: variable in index.asciidoc to 6.7 #10399
  • Fix building of deb and rpm artifacts #10396
  • Agent webserver: synchronise control #10393
  • X-pack Monitoring flaky specs #10392
  • Support old config options and add deprecation warnings #10390
  • Bump fpm dependency #10385
  • QA: prevent webmock from intercepting http calls to localhost #10384
  • Java 11 support - backport to 6x #10382
  • Tests: support more-precise clocks in Java 11 #10381
  • Rename config option .url and .ca to .hosts and .certificate_authorit #10380
  • Native support for Java plugins (beta) (#10232) #10378
  • 7x index templates #10375

Plugin releases:

logstash-codec-es_bulk - 3.0.7

 - Add documentation about use with http input

logstash-filter-de_dot - 1.0.4

 - fix failure of fieldnames with boolean value "false" #9

logstash-filter-http - 1.0.1

 - Fixed minor documentation issues #9

logstash-filter-jdbc_streaming - 1.0.5

  - Swap out mysql for postgresql for testing #11

logstash-filter-json - 3.0.6

 - Updated documentation with some clarifications and fixes

logstash-filter-memcached - 1.0.0

- Updated to 1.0.0

logstash-filter-metrics - 4.0.6

 - Fixed two minor typos in documentation

logstash-filter-mutate - 3.4.0

- Added ability to directly convert from integer and float to boolean #127

logstash-input-beats - 6.0.0

- Removed obsolete setting congestion_threshold and target_field_for_codec

- Changed default value of `add_hostname` to false

logstash-input-elasticsearch - 4.3.0

 - Added managed slice scrolling with `slices` option

logstash-input-http - 3.3.0

- Added configurable response code option #103

- Added explanation about operation order of codec and additional_codecs #104

logstash-input-http_poller - 5.0.1

- Fixed minor doc and doc formatting issues #107

logstash-input-redis - 3.4.1

 - Changed `redis_type` to `data_type` in .rb file #70 and asciidoc file #71

logstash-output-elasticsearch - 10.0.1

 - Reverted `document_type` obsoletion #844

logstash-output-elasticsearch - 9.4.0

 - Added 'auto' setting for ILM #841

logstash-output-elasticsearch - 9.3.2

 - Fixed sniffing support for 7.x #827

logstash-output-file - 4.2.6

 - Removed JRuby check when using FIFOs #75