Life of an Event in Logstash

Join me for this presentation to follow the journey of a Logstash event as it travels through the internal Logstash processing pipeline. Understanding Logstash internals will help you make better design decisions for your streaming data processing architecture. You will learn how data is transported in and out of Logstash through its input and output plugins, how Logstash decodes and encodes external data formats using the codec plugins and how the parsing, transformation and enrichment of this data is executed by the filter plugins. We will follow an event as it travels through the pipeline stages and internal queuing, what happens when things start to break downstream, plus how congestion and backpressure is handled. The Logstash pipeline is multithreaded, so you'll also learn how to scale up and take advantage of all your host CPU cores and how to leverage parallelism to acceleratethe processing throughput. Logstash is an amazingly powerful and flexible tool. As a Logstash user, this presentation will help you optimize your logstash configuration and, for developers, you will have much better insights for contributing plugins.