Brewing in Beats: Getting ready for 5.0.0-GA
Welcome to Brewing in Beats! With this series, we're keeping you up to date with all that's new in Beats, from the details of work in progress pull requests to releases and learning resources.
Libbeat: Fix issues when using Kafka as output
The community found the issue that the Kafka output raises a protocol error when Kafka 0.10 is configured. An update of the Sarama library to master solved the issue.
In addition, another issue appeared as the Kafka output connection was closed by broker on SASL/PLAIN. The pull request includes a fix for it, and exports a series of Sarama Kafka metrics that can be used for debugging.
The fix is backported to the 5.0 branch, and it will be part of the 5.0 release.
Metricbeat: Count each unique mountpoint once in fsstats
The system.fsstat metricset contains filesystem metrics aggregated from all mounted filesystems. It includes information like the number of mounted filesystems, total number of the files, total size, total used size etc.
There was an issue reported by the community that the number of mounted filesystems is too high on CentOS, compared with what `df -h` command displays. The problem appeared when two filesystems have the same mounted point, as we were counting them twice. The changes from the pull request count mounted filesystems by the number of unique mounted points.
Filebeat: Ignore duplicate state entries
Fixed an issue in the 5.0 branch where upgrading the registry file from 1.3 could result in duplicate entries and multiple harvesters started on the same file.
Packetbeat: Get the GeoIP information of your clients
Starting with 5.0.0-alpha3, the support in Packetbeat for GeoIP is deprecated, and it will be removed in 6.0. Instead, you can get the GeoIP information of your client's’ IP by using Ingest Node or Logstash. More precisely, by defining a GeoIP processor in the Ingest GeoIP plugin or by defining a filter in Logstash.
For a smooth integration with Ingest Node, Packetbeat adds the client_geoip.location of type geo_point in the mapping template, and then Ingest Node completes it with the GeoIP information, so you can easily visualize it in Kibana without any extra configuration needed.
The change will be available in the 5.1 release. For more details about it, please check our documentation.
Hearbeat: Fix panic when trying to start the scheduler
The pull request fixes the Heartbeat panic when starting the scheduler.