Brewing in Beats: Enrich events with Docker metadata
Welcome to Brewing in Beats! With this weekly series, we're keeping you up to date with what's new in Beats, including the latest commits and releases.
New processor: add_docker_metadata
Inspired by the add_kubernetes_metadata
processor, this pull request adds the add_docker_metadata
processor to enhance the events with the Docker metadata like name, image, and labels. The processor requires that the container ID to be present in the event fields, and queries the Docker API to get the metadata for that container. For example, the cgroups events generated by the system module in Metricbeat contain the container ID under system.process.cgroup.id
, and you can configure the processor as following to enrich the events generated by each Docker container:
processors: - add_docker_metadata: match_fields: ["system.process.cgroup.id"] host: "unix:///var/run/docker.sock"
In case you want to connect to Docker over TLS, you need to configure a client and a CA certificate.
The processor is released in 6.0.0-alpha2.
Filebeat: Redis module
Build upon the Redis prospector, which was merged the week before, we now have a Filebeat Redis module with two filesets:
- One for collecting and parsing the logs written on disk by Redis
- One that uses the Redis prospector to interrogate the slow logs
A Kibana dashboard is included:
We currently plan to release this module with 6.0.
With this, Redis joins the select group of projects for which we have a Filebeat module to monitor its logs, a Metricbeat module to collect its metrics, and a Packetbeat module/protocol to parse its network traffic.
Metricbeat: New Rabbitmq module
Thanks to @amandahla, Metricbeat gets support for RabbitMQ. The module comes with the node metricset that collects metrics from RabbitMQ nodes by querying the HTTP API /api/nodes.
The module is marked as experimental and it will be released with 6.0.
Other changes
Repository: elastic/beats
Affecting all Beats
Changes in master:
- Only attempt loading the ES template if ES output is enabled #4436
- Add a permissions option to logging.files for all beats #4428
Filebeat
Changes in master:
- Fix typo in the configuration files #4447
- Nginx module: use first not private IP address as remote_ip #4417
Metricbeat
Changes in 5.x:
- Fix type for HAProxy health.last field #4410
Changes in master:
Infrastructure
Changes in master:
- Separate travis jobs in stages #4453
Documentation
Changes in 5.4:
- Add docs for Docker images #4312,
- Provide more detail on log levels #4405,
- Add simple examples that illustrate multiline settings #4408
- Bump docs version to 5.4.1 #4396
Changes in 5.3:
- Bump docs version to 5.3.3 #4438
Changes in master: