21 March 2018

Logstash Lines: Sample plugin for the new Java API

Von Monica Sarbu

Did you know that Logstash 6.2 is already available? Try it and let us know what you think.

Logstash core

In progress: A Sample Input Plugin for the new Java API

We’re still hard at work on our new Java API. As part of this work we have a port of the stdin input plugin.

In progress: Internal Pipelines

The PR for inter-pipeline communications has moved forward. It now uses a copy on read or write approach to correctly handle our use of the Logstash Event API as it occurs in the wild. The previous implementation was correct to spec, but would lead to data errors when used as many plugin authors actually use our API.

Plugin Manager Now Handles Invalid Proxies Better

Previously, use of an invalid proxy with our plugin manager would lead to a confusing error message. We recently patched this.

Logstash integrations

Ongoing Changes

The File Input plugin will soon have the ability to gulp files in a single pass as well as the usual tailing. It will also be able to read gzip files and better detect the uniqueness of a file across renaming, deletions and truncations. And it will drop its dependency on the filewatch library! All of this is now up for review in a tremendous effort by Guy to breathe life into the logstash file input plugin which hadn't had significant changes in more than one year. You can follow the PR #171 for progress.

Other changes/releases:

logstash-codec-cloudtrail - 3.0.4

  • Don't crash when data doesn't contain some particular elements #18

logstash-filter-dns - 3.0.8

  • Fix bug where forward lookups would not cache timeout errors #41

logstash-filter-ruby - 3.1.4

  • Fix return of multiple events when using file based scripts #44

logstash-filter-aggregate - 2.8.0

  • New feature: add 'timeout_timestamp_field' option.  #81
  • When set, this option lets to compute timeout based on event timestamp field (and not system time). It's particularly useful when processing old logs.

logstash-input-beats - 5.0.11

  • Ensure that the keep-alive is sent for ALL pending batches when the pipeline is blocked, not only the batches attempting to write to the queue. #311

logstash-input-beats - 3.1.30

  • Ensure that the keep-alive is sent for ALL pending batches when the pipeline is blocked, not only the batches attempting to write to the queue. #312  

logstash-filter-jdbc_static - 1.0.1

  • Docs: Edit documentation #11