Upgrading with the Persistent Queue enabled

If you have the persistent queue (PQ) enabled, please read the section that applies for your upgrade scenario.

Drain the Persistent Queue (version 6.2.x and earlier)

The following applies only if you are upgrading from Logstash version 6.2.x or earlier with the persistent queue (PQ) enabled.

We strive to maintain backward compatibility within a given major release. Serialization issues in Logstash 6.2.x and earlier required us to break that compatibility in version 6.3.0 to ensure correctness of operation. For more technical details, please check our tracking github issue for this matter, #9494.

We strongly recommend that you drain or delete the persistent queue before you upgrade from version 6.2.x and earlier.

To drain the queue:

  1. In the logstash.yml file, set queue.drain: true.
  2. Restart Logstash for this setting to take effect.
  3. Shutdown Logstash (using CTRL+C or SIGTERM), and wait for the queue to empty.

When the queue is empty:

  1. Complete the upgrade.
  2. Restart Logstash.

We have resolved issues with data incompatibilities for version 6.3 and later. These steps won’t be required for future upgrades.

Upgrading from version 6.3 (and later) with Persistent Queues enabled

Upgrading Logstash with persistent queues enabled is supported. The persistent queue directory is self-contained and can be read by a new Logstash instance running the same pipeline. You can safely shut down the original Logstash instance, spin up a new instance, and set path.queue in the logstash.yml settings file to point to the original queue directory. You can also use a mounted drive to make this workflow easier.

Keep in mind that only one Logstash instance can write to path.queue. You cannot have the original instance and the new instance writing to the queue at the same time.