Brewing in Beats: Kubernetes module enhancements

Von
Monica Sarbu

Welcome to Brewing in Beats! With this weekly series, we're keeping you up to date with what's new in Beats, including the latest commits and releases.

New community Beat: Protologbeat

Created by @hartfordfive,  Protologbeat intends to allow any application to easily log messages locally without writing them on disk, and indexes them into Elasticsearch. The application sends the log entries to a local UDP or TCP socket. Protologbeat is a replacement of the older Udplogbeat, by the same author. It can act accept plain-text or JSON logs and also act as a syslog destination replacement.

New community Beats: Varnishlogbeat and Varnishstatbeat

Thanks to @phenomenes, there are two new community Beats created to monitor Varnish. Varnishlogbeat reads log data from a Varnish Shared Memory file and Varnishstatbeat collects stats data from a Varnish Shared Memory file and ships it to Elasticsearch. Both are using vago.

Enhancements to the Kubernetes module in Metricbeat

The state_container metricset is added to the kubernetes module in Metricbeat. It extracts metrics from kube-state-metrics services, and adds info specially useful about:

  • Status of deployments, pods and replica sets
  • Resource limits, requests and usage by pods
  • Node available resources

state_container_dashboard.png

Also, thanks to @vjsamuel, the event metricset is added to the kubernetes module in Metricbeat. The events are polled from the Kubernetes API server and are sent to Elasticsearch.

The Kubernetes module will be released in version 6.0.

Output the generated Elasticsearch template into a file

At startup, each Beat generates the Elasticsearch template from the fields.yml and loads it to Elasticsearch. With this change, you can configure the name of the generated Elasticsearch template under the setup.template.output_to_file.path together with its version under setup.template.output_to_file.version. The version refers to Elasticsearch, and in case it’s not specified, it’s set to the Beat version. 

./metricbeat -e -E setup.template.output_to_file.path=template.json

To generate the template for the Elastisearch 2.x version, you can do: 

./metricbeat -e -E setup.template.output_to_file.path=template.json -E setup.template.output_to_file.version=2.4.0

This new way of generating the templates will be released in 6.0.

Other changes

Repository: elastic/beats

Affecting all Beats

Changes in master:

  • Template test cleanup #4357
  • Fix panic due to no type #4331

Metricbeat

Changes in master:

  • Filter relative mounts in system filesystem metricset #4370
  • Remove unnecessary print statement in schema apis #4355
  • Add debug to Jolokia JMX metricset #4341
  • Fix panic on Metricbeat filters #4328

Winlogbeat

Changes in master:

  • Improve winlogbeat checkpoint test #4371

Filebeat

Changes in master:

  • Bring back prospector.type which went missing #4361
  • Remove waitgroup handling from Stopping of harvesters #4336
  • Refactor harvester and prospector #4326
  • Make state comparison more generic #4190

Infrastructure

Changes in master:

  • Improve `make notice` to work on libs not 3 dirs long #4380
  • Sync vendor directory #4372
  • Small cleanup in Makefile #4369
  • Sort NOTICE file by library path #4367
  • Remove unused packages from vendor directory #4366

Documentation

Changes in master:

  • Update processor docs #4330
  • Add docs for Docker images #4312

Changes in 5.4:

  • Backport: Cherrypick #4140, #4320, and #4197 into 5.4. #4342