Logstash 8.8.0 Release Notes

Known issues

Logstash 8.8.0 may fail to start when SSL/TLS is enabled in monitoring and/or central management, due to a change introduced in version 11.14.0 of the logstash-output-elasticsearch plugin. When impacted by this issue, Logstash fails to start and logs an error similar to the following:

[logstash.licensechecker.licensereader] Failed to perform request {:message=>"PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target", :exception=>Manticore::ClientProtocolException, :cause=>#<Java::JavaxNetSsl::SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target>}

Resolution

A successful Elasticsearch output plugin update to version 11.15.8 or higher will resolve this issue:

bin/logstash-plugin update logstash-output-elasticsearch

Specify the ca_trusted_fingerprint setting in the logstash.yml. The certificate fingerprint can be extract with:

cat your_ca.cert | openssl x509 -outform der | sha256sum | awk '{print $1}'

Then set the following on logstash.yml using the output from the previous command:

xpack.monitoring.elasticsearch.ssl.ca_trusted_fingerprint: "<value>"
xpack.management.elasticsearch.ssl.ca_trusted_fingerprint: "<value>"

Notable issues fixed

Fix a race condition that prevents Logstash from updating a pipeline’s configuration with in-flight events experiencing connection errors. #14739 This issue primarily manifests following the update of Elasticsearch credentials through Central Management, after credentials expired while events were in-flight. It causes the Elasticsearch Output to get stuck attempting to send events with the expired credentials instead of using the updated ones. To address this problem, Logstash has improved the pipeline shutdown phase functionality to allow an output plugin to request the termination of the in-flight batch of events; hence preventing the need for administrators to manually restart Logstash. Furthermore, when used in combination with a persistent queue to prevent data loss, the batch is eligible for reprocessing on pipeline restart. Plugin developers can now decide whether to make use of such functionality on output plugins. #14940

Updates to dependencies

Updates Bundler to version 2.4 #14995

Plugins

Elasticsearch Filter - 3.15.0

Standardize SSL settings to comply with Logstash’s naming convention #168
Added support for configurable retries with new retry_on_failure and retry_on_status options #160

Memcached Filter - 1.2.0

Upgrade Dalli to 3.x #33

Beats Input - 6.6.0

Standardize SSL settings to comply with Logstash’s naming convention #470

Elasticsearch Input - 4.17.0

Standardize SSL settings to comply with Logstash’s naming convention #185

Http Input - 3.7.0

Standardize SSL settings to comply with Logstash’s naming convention #165

Kafka Integration - 11.2.1

Fix nil exception to empty headers of record during event metadata assignment #140
Added TLS truststore and keystore settings specifically to access the schema registry #137
Added config group_instance_id to use the Kafka’s consumer static membership feature #135
Changed Kafka client to 3.3.1, requires Logstash >= 8.3.0.
Deprecated default value for setting client_dns_lookup forcing to use_all_dns_ips when explicitly used #130
Changed the consumer’s poll from using the one that blocks on metadata retrieval to the one that doesn’t #136

Normalize_config_support Mixin - 1.0.0

Elasticsearch Output - 11.15.1

Fixed race condition during plugin registration phase #1125
Added the ability to negatively acknowledge the batch under processing if the plugin is blocked in a retry-error-loop and a shutdown is requested. #1119
Standardize SSL settings to comply with Logstash’s naming convention #1118

« Logstash 8.8.1 Release Notes Logstash 8.7.1 Release Notes »