Logstash 8.4.0 Release Notes

New features and enhancements

Improvements to the dead letter queue (DLQ)

This release brings significant improvements to help users manage their dead letter queues, including:

  • A new clean_consumed option on the Dead Letter Queue input plugin. It can automatically delete segments from a dead letter queue after all events in the segment have been consumed by a Logstash pipeline.
  • A new age retention policy, enabling the automatic removal of segments from a dead letter queue based on the age of events within those segments.
  • Additional dead letter queue metrics available from the monitoring API #14324

New AWS integration plugin

Several AWS plugins are now bundled in a single AWS integration plugin, enabling easier maintenance and upgrades of AWS-based plugins. They all use version 3 of the AWS Ruby SDK.

JDK17 support

Logstash now comes bundled with JDK17, while still providing compatibility with user-supplied JDK11. The new JDK includes an update pertaining to a potential security vulnerability. Please see our security statement for details.

Logstash M1 download

Logstash is now available for download on M1 equipped MacOS devices, and comes bundled with M1 native JDK17.

Notable issues fixed

  • Remove /etc/systemd/system/logstash.service only when file is installed by Logstash #14200
  • Fix Arcsight module compatibility with Elasticsearch 8.x #13874
  • Ensure that timestamp values are serialized with a minimum of 3 decimal places to guarantee that millisecond precision timestamps match those from Logstash 7.x #14299
  • Fix issue with native Java plugin thread-safety and concurrency #14360
  • Allow the ability to use Ruby codecs inside native Java plugins #13523

Updates to dependencies

  • The bundled JDK has been updated to 17.0.4+8 #14427
  • The version of Sinatra has been updated to 2.2.2 #14454
  • The version of Nokogiri has been updated to 1.13.8 #14454

Plugin releases

Dead Letter Queue Input - 2.0.0

  • Introduce the boolean clean_consumed setting to enable the automatic removal of completely consumed segments. Requires Logstash 8.4.0 or above #43
  • Expose metrics about segments and events cleaned by this plugin #45

Xml Filter - 4.2.0

  • Update Nokogiri dependency version #78

Aws Integration Plugin - 7.0.0

  • This new integration plugin incorporates and replaces the use of the these individual plugins: individual plugins:

    • logstash-input-s3
    • logstash-input-sqs
    • logstash-mixin-aws
    • logstash-output-cloudwatch
    • logstash-output-s3
    • logstash-output-sns
    • logstash-output-sqs
  • This replaces the use of the single combined aws 2.x sdk gem, with the modularized aws 3.x gems.