19 September 2017

Brewing in Beats: Performance improvements

By Monica Sarbu

Welcome to Brewing in Beats! With this weekly series, we're keeping you up to date with what's new in Beats, including the latest commits and releases.

Last week, we released the second beta release of the Elastic Stack 6.0. Read more details in the blog post about what’s new in the Beats 6.0.0-beta2 release.

Performance improvements

Steffen spent some time last week benchmarking and profiling the publisher pipeline, which was heavily refactored for 6.0. This has resulted in several PRs that are tuning the default configuration and are improving on some slow paths, so that Beats 6.0 will generally be faster and consume less CPU than Beats 5.x on similar loads.

New community Beat: Cloudwatchmetricbeat

Cloudwatchmetricbeat collects the metrics from AWS CloudWatch, and stores them in Elasticsearch for further analysis. The user can configure the metrics that they are interested in. This is a companion for Cloudwatchlogsbeat, also from the community, that can retrieve data from AWS Cloudwatch Log Groups.

Other changes:

Repository: elastic/beats

Affecting all Beats

Changes in master:

  • Allow IndexPrefix to be set. #5161
  • Fix Kubernetes matcher registry lookup #5159

Changes in 6.0:

  • Fix Kubernetes matcher registry lookup #5159
Filebeat

Changes in master:

  • Remove runner creation from every reload check #5141

Changes in 6.0:

  • Add flush timeout setting to Filebeat registrar #5146
  • Remove runner creation from every reload check #5141
Auditbeat

Changes in 6.0:

  • Support receiving events from audit multicast group #5081
Testing

Changes in 5.6:

  • Updated testing env 5.6.1 #5191
Documentation

Changes in master:

  • Update link to certgen command #5197
  • Fix the bullets from the redis docs in Filebeat #5143

Changes in 5.6:

  • Bump docs version 5.6.1 #5192
  • Close changelog for 5.6.1 #5190

Changes in 6.0:

  • Close changelog for 6.0.0-rc1 #5185