Logstash 7.2.0 Release Notes

  • Native support for Java Plugins (GA) #10620. Changes to Java plugins for GA include:

    • BREAKING: The signature for the codec.encode() method was changed to make the codec API easier to use correctly in pipelines with multiple concurrent workers.
    • BREAKING: The return type on the Context::getDlqWriter method was changed to an interface to decouple it from any specific DLQ implementation.
    • BETA: Isolated classloader for Java plugins. When enabled with the pipeline.plugin_classloaders flag, each Java plugin will be loaded with its own parent-last classloader. All dependencies for the Java plugin, with the exception of any classes in org.logstash.* or co.elastic.logstash.* packages, will be loaded first from the plugin’s jar file delegating to the main Logstash classloader only if the dependency is not found within the plugin’s jar. This allows Java plugins to use dependencies that might clash with the dependencies for other Java plugins or Logstash core itself.
    • Gradle tasks to automate the packaging of Java plugins as Ruby gems. The gem task will bootstrap all JRuby dependencies, automatically generate Gemfile and the gemspec file for the plugin as well as the two "glue" Ruby source files for triggering the loading of the Java class files during Logstash startup, and then invoke the necessary commands to create the gem file itself. The gem task also performs a number of validation checks to proactively identify common problems with the packaging of Java plugins. The clean task will remove all of those generated artifacts.
    • Java plugin validation. At pipeline startup, all Java plugins will be checked to ensure that they implement the same version of the Java plugin API present in the current Logstash version.
    • DLQ writer interface provided to Java plugins.
    • Float, URI, and password config types, each of which provide validation for the supplied config value.
    • New built-in Java plugins:

      • A Java-based generator input for testing with the same capabilities as the Ruby generator input as well as an optional eps option to generate events at a given event-per-second rate for situations where as-fast-as-possible event generation is too much.
      • Also includes a jdots codec that mirrors the Ruby dots codec.
      • Java-based sink output that discards any events received. Analogous to the Ruby null plugin though much faster.
  • Documentation: Add details about Elastic Search dependency #10852
  • Fixed parsing of boolean options provided to Java plugins #10848
  • Field reference: handle illegal field references in converted maps #10839
  • Fixes unit test failures on some runs of ConfigCompilerTest::testComplexConfigToPipelineIR #10837
  • Documentation: Value of start_timestamp must to be quoted #10836
  • Build: name rpm/deb oss packages as logstash-oss #10833
  • LIR support for octal literals in pipeline definitions #10828
  • Merge config values in LIR #10832
  • Build: generate tarballs for docker images #10819
  • Documentation: Document copy semantics of QueueWriter::push method #10808
  • Default stack trace size for hot threads to 50 and make it configurable #10793
  • Include G1 in JVM heap metrics #10784
  • Expose Metrics API to Java plugins #10761
  • Documentation: Clarify behavior of ensure_delivery flag #10754
  • Fix JRuby resolv.rb leak #10741
  • Add LogStash::PluginMetadata for simple key/value plugin metadata #10691
  • Fix default codec and buffer handling in Java stdout output #10673
  • Collect and expose codec metrics #10614
  • Enhance GET / API for Metricbeat Merge config values in LIR #10589

Plugins

Es bulk Codec

  • Fixed deeplink to Elasticsearch Reference #18

Dns Filter

  • Fixed JRuby resolver bug for versions after to 9.2.0.0 #51

Geoip Filter

  • Fixed deeplink to Elasticsearch Reference #151

Jdbc streaming Filter

  • Fixes connection leak in pipeline reloads by properly disconnecting on plugin close #26

Azure event hubs Input

  • Updated Azure event hub library dependencies #36

Elasticsearch Input

  • Fixed deeplink to Elasticsearch Reference #103

Elasticsearch Output

  • Added cluster id tracking through the plugin metadata registry #857