Brewing in Beats: Dropwizard module in Metricbeat

Welcome to Brewing in Beats! With this weekly series, we're keeping you up to date with what's new in Beats, including the latest commits and releases.

Add support for dropwizard in Metricbeat

Thanks to the Vijay Samuel, Metricbeat has a new module for the Dropwizard metrics. The collector metricset fetches Dropwizard metrics by querying the HTTP servlets.

If you have a dropwizard project, then you can use the following configuration to automatically report the metrics to Elasticsearch:

- module: dropwizard
     metricsets: ["collector"]
     enabled: true
     period: 10s
     hosts: ["localhost:8080"]
     metrics_path: /metrics/metrics

Enhance Prometheus module in Metricbeat

The current Prometheus collector implementation separates each metric into a separate event, that can generate a lot of data. Thanks to the same Vijay Samuel, there is a community contribution to group metrics under the same MetricFamily. This enables histograms and summaries to be included in the same Elasticsearch document, reducing the amount of documents and making querying easier.

Add Icinga module in Filebeat

Thanks to Blerim Sheqa, Filebeat is capable of processing the main, debug and startup log of the Icinga daemon.

Add support for Tencent Cloud

The add_cloud_metadata processor enhances each event with Cloud metadata. Currently it has support for Amazon Elastic Compute Cloud (EC2), Digital Ocean and Google Compute Engine (GCE). However, a lot of companies in China prefer to use Tencent Cloud and AliCloud. Thanks to Yeer Kunth, add_cloud_metadata processor has also support for them.

Other changes

Repository: elastic/beats

Affecting all Beats

Changes in master:

  • Add (*MapStr).DeepUpdate#4090
  • Add support for templates index_patterns field in ES >= 6.0 #4056
  • Disable _all.norms also for ES 5.X #4055
  • Change dashboards.* config options to setup.dashboards.*#4050
  • Add missing return statement#4045
  • Adding support for include_labels and include_annotations in Kubernetes processor#4043

Filebeat

Changes in 5.4:

  • Properly shut down crawler in case one prospector is misconfigured #4037
  • Fix JSON panic #4042

Changes in master:

  • Add filebeat.config.path as replacement for filebeat.config_dir #4051
  • Properly shut down crawler in case one prospector is misconfigured #4037
  • Add Icinga module to Filebeat #3904
  • Modify clean_removed handling #3827

Metricbeat

Changes in master:

  • Skip empty events without error #4087
  • Allow to overwrite uri in http helper #4084
  • Enhance node_stats metricset with index stats #4077
  • Refactoring prometheus module to aggregate metrics based on metric family #4075
  • Add kubelet module fields #4053
  • Update metricbeat full config to have all modules disabled #4049
  • Add note on how to test kubelet metricbeat module #4044
  • Add gc collection stats to node_stats metricset of ELasticsearch module#4032
  • Unify cluster.name and node.name in elasticsearch metricsets #4031
  • Fix MongoDB dbstats fields mapping #4025
  • Adding dropwizard module #4022
  • Kubernetes annotator #3888

Heartbeat

Changes in master:

  • Change type of ip field in heartbeat #4066

Infrastructure

Changes in master:

  • Simplify environment testing for module #4067
  • Set short_config option to false by default in Metricbeat #4038
  • Update Go version to 1.8.1#4033

Documentation

Changes in 5.4:

  • Backport: Mulitple doc changes into 5.4 #4073

Changes in 5.3:

  • Backport: #4016, #4009, #4017, and #3890 into 5.3 #4074
  • Backport to 5.3: Update curl examples to include content-type #4026

Changes in master:

  • Update CONTRIBUTING.md to Golang 1.8.1 #4094
  • Fix versions.asciidoc #4085
  • Update docs to include more details about the Elasticsearch metricsets #4069
  • Fix doc build #4060
  • Move dev guide topics to separate guide #4030
  • Edit config info about dashboard loading #4016
  • Edit config reload topics #4009
  • Bump LICENSE year in beat templates #4006