Persistent Queue is out of Beta
Back in 5.1.0, we introduced a new feature called persistent queues to improve the durability of Logstash. We received plenty of feedback from the beta program that allowed us to fix bugs and make improvements. Today, we are announcing that persistent queue is a fully supported feature in Logstash! The goals for this release are:
- Provide an at-least-once delivery guarantee against message loss during a normal shutdown as well as when Logstash is terminated abnormally. If Logstash is restarted while events are in-flight, Logstash will attempt to deliver messages stored in the persistent queue until delivery succeeds at least once.
- Provide an out of the box functionality to handle event spikes from your data sources. Before this feature, users had to deploy an external buffering system like Redis just to support Logstash.
Persistent Queue is still disabled by default. You can enable it by changing
queue.type: persisted in the logstash.yml file. We've also updated our deploying and scaling guide to include new architecture recommendations using persistent queues.
5.4.0 has plenty of bug fixes in Logstash core and plugins. The release notes has details of bugs fixed.