Logstash Lines: Improvements to Plugins
Welcome to Logstash Lines! With these weekly series, we're keeping you up to date with what's new in Logstash, including the latest commits and releases.
Did you know that Logstash 6.2 is already available? Try it and let us know what you think.
Version 4.1.0 of the File input plugin brings new internals and new features!
In pull request #171, Guy Boertje massively improved the file input code structure, reduced external dependencies and brought in new features, of which we'd like to highlight:
- New 'reading' mode: as opposed to 'tailing', this mode assumes files have ends and are not appended to;
- Support for gzip compressed files (only in reading mode);
- Support expiration of sincedb entries by configuring the `sincedb_clean_after` option
For more information, see the documentation for version 4.1.0.
HTTP Input version 3.1.0: goodbye Puma, hello Netty!
A large rewrite of this plugin's internals (#73) has brought more control over TLS/SSL settings, better handling of backpressure by limiting the number of inflight events over all connections and a modest 10% performance increase and lower memory usage. By building the HTTP layer on top of the Netty Java library this rewrite enables further improvements, as demonstrated in this currently open pull request that brings an extra 15 to 20% gain in performance.
Beta stage implementation of SNMP input is complete!
After a lot of demand for SNMP polling, this initial pull request by Colin Surprenant creates a snmp input plugin that can perform get/walk operations on multiple hosts and use mib files to translate OID to field names. Currently it only supports SNMPv2c over UDP but more is planned, see the plugin's roadmap for more information.
New filter: logstash-filter-bytes
Our own Shaunak has contributed a very useful filter that converts human descriptions of storage size to the canonical number of bytes they represent (e.g. "10 KB" is translated to 10240). See the documentation for more examples and uses.
Grok Debugger UI
Added syntax highlighting to the Grok Debugger UI: https://github.com/elastic/
Documentation
Repository: elastic/logstash
Changes in master:
- [DOCS] Fixes links to built-in users #9518
- [DOCS] Removes X-Pack release notes and breaking changes #9509
- Give an example of a single line Hash. #9505
- [DOCS] Enables editing links for X-Pack pages #9500
- [DOCS] Fixes broken link #9479
- [DOCS] Adds new installation package details #9404
Repository: elastic/logstash-docs
Changes in 6.3:
- Docs bump for 6.3.0 #521
Changes in versioned_plugin_docs:
- Fix broken build #537
- Fix broken links #536
- Add branch attribute to resolve doc paths #535
- auto generated update of versioned plugin documentation #533
- auto generated update of versioned plugin documentation #532
- auto generated update of versioned plugin documentation #531
- auto generated update of versioned plugin documentation #530
- Updated versioned plugin docs #522
Changes in master:
Changes in 6.2:
- update docs for 6.2 #529
Changes in 6.x:
- update docs for 6.x #528
More changes
Logstash core
- Fix deep cloning of Timestamp values: #9405
Logstash plugins
Generation of documentation for all versions of all plugins is now done automatically in CI (elastic/infra#4466).
logstash-codec-netflow - 3.13.1
- Fixes exceptions due to concurrent access of IPFIX templates, see issue #134
logstash-codec-netflow - 3.13.0
- Added support for Netflow 9 reduced-size encoding support
- Added support for Barracuda IPFIX Extended Uniflow
logstash-input-google_pubsub - 1.1.0
- Add additional attributes in the `[@metadata][pubsub_message]` field. Fixes #7