Logstash 1.5.2 released

Logstash 1.5.2 is now available for download! This is a bug fix release which resolves an important issue in file input. Check the changelog for full details!

http input plugin

We are also announcing the availability of a new http input plugin, which is bundled with 1.5.2. This is a general purpose input for sending data to Logstash through http(s). Interesting use cases, you ask? If your existing applications support webhooks, it's now possible to send actionable data through this input to leverage Logstash's rich plugin ecosystem. For example, your Continuous Integration platform such as Jenkins can push build failure notifications to Logstash and index them in Elasticsearch. At the same time you can page developers for critical build failures using conditionals and the PagerDuty output.

Watcher integration

More interestingly, using our recently announced Watcher product, you can set up notifications triggered from data in your Elasticsearch cluster to be escalated through PagerDuty, notify co-workers on HipChat and archived to S3 for long term storage. All this can be accomplished in a single (and simple) Logstash configuration. Watch out for a more detailed blog post about this plugin and its use cases.

Bug Fixes

File Input: Fixed a critical bug where new files added to a directory being watched would cause an error. This issue also happens when wildcards were used to watch files matching a certain pattern (#3473).

Performance improvements: String interpolation is widely used in LS to create keys combining dynamic values from extracted fields. For example, this is used to create an Elasticsearch index name based on the timestamp of an event. We added an optimization which compiles this interpolation template on first use and caches in-memory for subsequent uses. Our performance testing show about 20% increase in throughput for configs that involve a lot of date processing and use field reference syntax (#3425)

Lumberjack input: This input was not handling backpressure properly from downstream plugins and would not timeout client connections, causing clients to constantly reconnect and eventually cause Logstash to run out of memory. We added a circuit breaker to stop accepting new connections when we detect this situation. Please note that `max_clients` setting introduced in v0.1.9 has been deprecated. This setting temporarily solved the problem by configuring an upper limit to the number of LSF connections (#12).

Please download Logstash 1.5.2 and let us know what you think on Twitter (@elastic) or on our forum. You can report any problems on the GitHub issues page.