15 June 2016

Brewing in Beats: Logstashbeat, Nginxupstreambeat, Lambdabeat

By Tudor Golubenco

Welcome to Weekly Beats! With this series, we're keeping you up to date with all that's new in Beats, from the details of work in progress pull requests to releases and learning resources.

New Community Beat: Logstashbeat

Logstash now has a monitoring API, so it can be monitored with Beats :-). Valentin Crettaz has created a new community Beat called Logstashbeat that gathers stats (events, jvm and process) from a running Logstash instance and ship them to Elasticsearch.

New Community Beat: Nginxupstreambeat

Nginxustreambeat is a Beat used for Nginx upstream server status monitoring. The Nginx upstream check module, by the same author,  reads the statistics periodically and indexes them in Elasticsearch. This is very useful when using Nginx as a reverse proxy to load-balance among multiple upstream servers.

New Community Beat: Lambdabeat

Mike Baamonde, one of our Elastic colleagues, created Lambdabeat to collect various AWS Lambda metrics and index them in Elasticsearch. Lambdabeat fetches data from the Cloudwatch API, so you don't need to deploy it anywhere in particular, but it does require valid AWS credentials.

New Generators, now in the main Beats repository

The Beat generator proved to be extremely useful in quickly bootstrapping new Beats and keeping them up-to-date with our changes. As we’re adding different types of generators, like the one that generates a Metricbeat powered Beat, or one for generating a Beat that implements a single Packetbeat protocol, we figured it makes more sense to group these generators in a single place. This place is inside the main Beats repository.

Filebeat: symlinks are no longer followed

Following symlinks to log files in Filebeat was asking for trouble, for example it could happen that the same file was read twice. We will be removing this feature with the next alpha release. Symlinks are now completely ignored from now on.

Fixed backoff logic in outputs

Coming from a community contributor, this PR fixes a bug in the backoff retry code in libbeat.

Metricbeat: automatically document events

Every Metricbeat module so far comes with Dockerized tests to verify it against a real instance of the monitored system. We now take advantage of this and automatically generate samples of the events by running the integration tests.

MongoDB Metricbeat module

A MongoDB module for Metricbeat is currently in progress.