Logstash 7.13.0 Release Notesedit

New features and enhancementsedit

Progress toward Elastic Common Schema (ECS)edit

In this release, we’ve made more Logstash plugins compatible with the Elastic Common Schema (ECS). This release builds on ECS work in previous releases, and adds ECS-compatibility for these plugins:

ECS compatibility is off-by-default in Logstash 7.x, but will be on-by-default in Logstash 8.0.

Elasticsearch datastreamsedit

The Elasticsearch output plugin now supports Elasticsearch data streams. You can use the plugin to send time series datasets (such as logs, events, and metrics) as well as non-time series data to Elasticsearch.

Elasticsearch data streams store append-only time series data across multiple indices while giving you a single named resource for requests. Data streams are well-suited for logs, events, metrics, and other continuously generated data.

The Elasticsearch output offers data stream options that are designed for indexing time series datasets into Elasticsearch.

Performance improvements and notable issues fixededit

Logstash keystore fixes

Fixed a regression introduced in 7.11 where the bin/logstash-keystore list command would not list secrets from the logstash keystore #12784

Potential Plugin interoperability fixes

Fixed a potential interoperability issue where logstash-filter-date and logstash-filter-geoip were used in the same pipeline. We believe this only manifested in testing rather than actual pipeline #12811

Updates to dependencies

  • Updated jruby to 9.2.16.0 #12699
  • Updated bundled JDK to 11.0.10+9 #12693
  • Updated log4j2 to 1.4.0 and ship log4j 1.2 bridge #12724
  • Updated slf4j to 1.7.30 #12723

Pluginsedit

Cef Codec - 6.2.0

  • Introduce ECS Compatibility mode #83

Clone Filter - 4.1.1

  • [DOC] Add ECS mapping table #25
  • [DOC] Added note that a new type field is added to the clone #23
  • Add ECS compatibility #24

Syslog_pri Filter - 3.1.0

  • Feat: ECS compatibility #9

Beats Input - 6.1.3

  • Fix: safe-guard byte buf allocation #420
  • Updated Jackson dependencies

S3 Input - 3.6.0

  • Fixed unprocessed file with the same last_modified in ingestion. #220
  • [DOC] Added note that only AWS S3 is supported. No other S3 compatible storage solutions are supported. #208
  • [DOC] Added example for exclude_pattern and reordered option descriptions #204

Stdin Input - 3.3.0

  • Feat: ECS support + review dependencies #20

Syslog Input - 3.5.0

  • Feat: ECS compatibility support #63

Jdbc Integration - 5.0.7

  • Feat: try hard to log Java cause (chain) #62
  • Refactored Lookup used in jdbc_streaming and jdbc_static to avoid code duplication. #59

Elasticsearch Output - 11.0.2

  • Validate that required functionality in Elasticsearch is available upon initial connection #1015
  • Fix: DLQ regression shipped in 11.0.0 #1012
  • [DOC] Fixed broken link in list item #1011
  • Feat: Data stream support #988
  • Refactor: reviewed logging format and restored ES (initial) setup error logging
  • Feat: always check ES license #1005