12 December 2017

Logstash Lines: Update for December 12, 2017

By Andrew Cholakian

Hello Logstashers! We're glad to present you all the past couple week's news in convenient digest form!

Added Support for Resetting Logging Back to Default

We've now added support for resetting logging settings changed via the Logstash web API back to their defaults. 

Size caps for log files + gzip on rollover

We've added some much improvements fixes for Logstash's log4j logging. We will cap log files at 100MB per file, and gzip files as they're rolled over automatically. We're targeting 6.2 with this change. 

HTTP Input Now Returns 429s Sometimes/ Runs faster/ No Longer Uses Excessive File Descriptors

The Logstash HTTP input previously exhibited poor behavior when the queue was blocked. If a client connected and timed out, LS would not release the connection, but rather, block indefinitely, causing the client to potentially time out. The HTTP protocol doesn’t deal well with long running requests. This plugin will now either return a 429 (busy) error when Logstash is backlogged, or it will time out the request.

If a 429 error is encountered clients should sleep, backing off exponentially with some random jitter, then retry their request.

This plugin will block if the Logstash queue is blocked and there are available HTTP input threads. This will cause most HTTP clients to time out. Sent events will still be processed in this case. This behavior is not optimal and will be changed in a future release. In the future, this plugin will always return a 429 if the queue is busy, and will not time out in the event of a busy queue.

While doing this work, we discovered that the HTTP input had some synchronization bottlenecks which were unnecessary. Users might see a nice perf boost on multicore systems.

LIR Fixes  / Experimental Execution Fixes

One of the big changes in 6.0 is our work on a new execution backend as well as a new IR for our compiler. We've seen a number of small discrepancies pop up in the 6.x series that we've been ironing out with both the new IR (used by all users) and the new execution engine (will be optional with 6.1). We've done this work in a variety of commits.

Other Fixes