Brewing in Beats: Kubernetes module enhancements
Welcome to Brewing in Beats! With this weekly series, we're keeping you up to date with what's new in Beats, including the latest commits and releases.
Interested in a #remote #Job writing #golang code for an open source project used by thousands of companies? https://t.co/tLR5Mrrlec pic.twitter.com/OFDUifSUvS
— Monica Sarbu (@monicasarbu) May 5, 2017
New community Beat: Protologbeat
Created by @hartfordfive, Protologbeat intends to allow any application to easily log messages locally without writing them on disk, and indexes them into Elasticsearch. The application sends the log entries to a local UDP or TCP socket. Protologbeat is a replacement of the older Udplogbeat, by the same author. It can act accept plain-text or JSON logs and also act as a syslog destination replacement.
New community Beats: Varnishlogbeat and Varnishstatbeat
Thanks to @phenomenes, there are two new community Beats created to monitor Varnish. Varnishlogbeat reads log data from a Varnish Shared Memory file and Varnishstatbeat collects stats data from a Varnish Shared Memory file and ships it to Elasticsearch. Both are using vago.
Enhancements to the Kubernetes module in Metricbeat
The state_container metricset is added to the kubernetes module in Metricbeat. It extracts metrics from kube-state-metrics services, and adds info specially useful about:
- Status of deployments, pods and replica sets
- Resource limits, requests and usage by pods
- Node available resources
Also, thanks to @vjsamuel, the event metricset is added to the kubernetes module in Metricbeat. The events are polled from the Kubernetes API server and are sent to Elasticsearch.
The Kubernetes module will be released in version 6.0.
Output the generated Elasticsearch template into a file
At startup, each Beat generates the Elasticsearch template from the fields.yml and loads it to Elasticsearch. With this change, you can configure the name of the generated Elasticsearch template under the setup.template.output_to_file.path
together with its version under setup.template.output_to_file.version
. The version refers to Elasticsearch, and in case it’s not specified, it’s set to the Beat version.
./metricbeat -e -E setup.template.output_to_file.path=template.json
To generate the template for the Elastisearch 2.x version, you can do:
./metricbeat -e -E setup.template.output_to_file.path=template.json -E setup.template.output_to_file.version=2.4.0
This new way of generating the templates will be released in 6.0.
Other changes
Repository: elastic/beats
Affecting all Beats
Changes in master:
Metricbeat
Changes in master:
- Filter relative mounts in system filesystem metricset #4370
- Remove unnecessary print statement in schema apis #4355
- Add debug to Jolokia JMX metricset #4341
- Fix panic on Metricbeat filters #4328
Winlogbeat
Changes in master:
- Improve winlogbeat checkpoint test #4371
Filebeat
Changes in master:
- Bring back prospector.type which went missing #4361
- Remove waitgroup handling from Stopping of harvesters #4336
- Refactor harvester and prospector #4326
- Make state comparison more generic #4190
Infrastructure
Changes in master:
- Improve `make notice` to work on libs not 3 dirs long #4380
- Sync vendor directory #4372
- Small cleanup in Makefile #4369
- Sort NOTICE file by library path #4367
- Remove unused packages from vendor directory #4366
Documentation
Changes in master:
Changes in 5.4:
- Backport: Cherrypick #4140, #4320, and #4197 into 5.4. #4342