Brewing in Beats: HAProxy module in Metricbeat
Welcome to Brewing in Beats! With this series, we're keeping you up to date with all that's new in Beats, from the details of work in progress pull requests to releases and learning resources.
Metricbeat: new HAProxy module
Filebeat: The return of the Symlinks
Filebeat used to somehow unintentionally follow symlinks when opening files. So we deprecated them in 1.3 and were planning to remove the support for symlinks in 5.0. However, we’ve got some pretty strong pushback from Kubernetes users (K8s uses symlinks to map log files with useful file names), so we’ve listened and added back support for them. However, the symlink support is off by default and marked as experimental at this point.
Filebeat: close_removed and clean_removed are now on by default
This will keep the registry file from growing to quickly. For background: in 1.x the registry file was a map using the filename as a key. In 5.x the registry file is a list. A side effect of this change is that the registry file grows even if the same file names are reused (think of the usual syslog file names). By making these two options on by default, we’re making sure the registry file stays small.
Metricbeat: Improvements to the MySQL module
Store the version in the Elasticsearch templates
We now store the Beats version in the `_meta` section of the Elasticsearch templates we provide. This can help with troubleshooting and perhaps we could use it in the future to provide automatic mappings upgrades.
import_dashboards: automatically select the right Beat
Now that the import_dashboard scripts were rewritten in Go and download the beats-dashboards package when executed, we wanted to make sure that it does the “right thing” when called without any arguments. So we now adjust the binary at build time to have a different `-beat` argument depending on the package.
Packetbeat: Improve handling of HTTP messages larger than 10MB
Old implementation dropped messages larger than 10MB (hard coded), to protect against memory DoS. However, in some cases the transaction was still recorded, but the parsing of the headers was incomplete. This PR makes the support for large messages explicit, by adding a mode to the parser that "sees" the segments without storing them.