07 June 2018

Logstash Lines: Support for TCP and SNMPv1 in SNMP input plugin

By Monica Sarbu

Welcome to Logstash Lines! With these weekly series, we're keeping you up to date with what's new in Logstash, including the latest commits and releases.

Did you know that Logstash 6.2 is already available? Try it and let us know what you think.

This update covers the last two weeks.

Beta3 of SNMP input plugin adds support for TCP and SNMPv1

As of 0.1.0 beta3 we've added support for SNMPv1 and also unlocked the use of TCP, while until now only SNMPv2c over UDP was supported. Work to provide a strong set initial features continues on the road to the first stable release, you can see more about this progress and roadmap here.

Note: To install the plugin beta version you have to explicitly specify the version like this:

$ bin/logstash-plugin install --version 0.1.0.beta3 logstash-input-snmp

Test stability

Our current focus on the core team is test stability. We’ve hit a point where there are a number of flakey tests that have accrued over the years that fail far apart. When you have enough of these combined they can be quite irritating make trusting your builds difficult. To that end, we’ve decided to pause most new feature development to get these rogue tests under control and hope to resume normal ops in the next week or two.

Other changes

Repository: elastic/logstash

  • Remove redundant bootstrap checks #9574
  • Explicitly depend on guava 22 #9622
  • Fix Path.Plugin #8887
  • Add sample config file #9588
  • Port Loggable Module to Java #9587
  • Correctly start x-pack trial for tests #9584
  • Use separate output streams to avoid theoretical keystore corruption #9582
  • Fix the docker build script to see the correct exit code #9567
  • Fix logger handling of ruby nil #9577
  • Load x-pack tests via JUnit #9575
  • Attempted to upgrade JRuby version to 9.17.0 #9572
Documentation

Changes in master:

  • [DOCS] Removes redundant index.asciidoc files #9607
  • [DOCS] Moves X-Pack configuration pages in table of contents #9604
  • [DOCS] Move setup folder to docs #9681
  • [DOCS] Moves management folder to docs #9680
  • [DOCS] Moves settings folder to docs #9679
  • [DOCS] Moves security folder to docs #9678
  • [DOCS] Move monitoring folder to docs #9677

Repositories under: elastic/logstash-plugins

## 1.0.3

  • Fix issue (#25) when index_columns is not specified, the create table statement has a syntax error #26

logstash-filter-kv - 4.1.2

  • Improve trim_key and trim_value to trim any _sequence_ of matching characters from the beginning and ends of the corresponding keys and values; a previous implementation limited trim to a single character from each end, which was surprising. #62
  • Fix issue where we can fail to correctly break up a sequence that includes a partially-quoted value followed by another fully-quoted value by slightly reducing greediness of quoted-value captures. #62

logstash-input-s3 - 3.3.5

  • Avoid plugin crashes when encountering 'bad' files in S3 buckets #136

logstash-output-elasticsearch - 9.1.3

  • Improve plugin behavior when Elasticsearch is down on startup #758

logstash-output-google_cloud_storage - 3.1.0

  • Add support for disabling hostname in the log file names #26
  • Add support for adding a UUID to the log file names  #26

logstash-output-loggly - 4.0.0

  • The plugin now uses the Loggly bulk API.
  •  If you need to modify event batch sizes and max delay between flushes, please adjust the Logstash settings `pipeline.batch.size` and`pipeline.batch.delay` respectively.
  • New settings: `max_event_size` and `max_payload_size`. Both are currently set according to Loggly's published API limits. They only need to be changed if Loggly changes these limits.
  • The plugin now skips events bigger than the API limit for single event size. A proper warning is logged when this happens.
  •  When interpolating `key` field, drop messages where interpolation doesn't resolve (meaning we don't have the API key for the event).
  •  When interpolating `tag` field, revert to default of 'logstash' if interpolation doesn't resolve.
  •  Beef up unit tests significantly.
  •  See pull request #29 for all details.

logstash-codec-cloudtrail - 3.0.5

  • Handle 'sourceIpAddress' fields with non-ip address content by moving them to 'sourceHost' field #22

logstash-codec-netflow - 4.0.1

  • Fixed IPFIX options template parsing for Juniper MX240 JunOS 15.1 

logstash-codec-netflow - 4.0.0

  • Added support for RFC6759 decoding of application_id.
  • This is a breaking change to the way application_id is decoded: The format changes from e.g. 0:40567 to 0..12356..40567**

logstash-input-kafka - 8.1.1

  • Fix race-condition where shutting down a Kafka Input before it has finished starting could cause Logstash to crash #280

logstash-input-snmp - 0.1.0.beta3

  • add tcp transport protocol support, #8
  • add SNMPv1 protocol version support #9

logstash-input-snmp - 0.1.0.beta2

  • add host info in metadata and host field #7

logstash-input-tcp - 5.0.8

  • Reorder shut down of the two event loops to prevent RejectedExecutionException #117

logstash-output-file - 4.2.4

  • Fix a bug where flush interval was being called for each event when enabled #67

logstash-output-s3 - 4.1.4

  • Internal: Revert rake pinning to fix upstream builds #185

logstash-output-s3 - 4.1.3

  • Docs: Fix incorrect characterization of parameters as `required` in example configuration #181
  •  Internal: Pin rake version for jruby-1.7 compatibility #184