Brewing in Beats: Monitoring Beats CPU usage

Did you know that Beats 6.1 is already available? Try it and let us know what you think. If you are curious to see the Beats in action, we just published the Getting Started with Beats webinar.

Monitoring Beats own CPU usage

As part of the Beats central monitoring project, all Beats have gained the ability to read their own CPU times, memory utilization, as well as system statistics like the load. To implement this across our supported operating systems, we needed CGo support. Luckily, we have anticipated this and as of 6.0 we compile all Beats with CGo support regardless of whether they need it or not.

These metrics are reported via the upcoming Beats central monitoring as well as via the log files. The Beats central monitoring is scheduled to be released with 6.2.

Metricbeat: MongoDB per collection statistics

Thanks to @MerlinDMC, the MongoDB module in Metricbeat has a new metricset that fetches per collection statistics. It makes it possible to graph "hot" collections for read, write and other spikes in usage. This new metricset will be present in 6.2.

Metricbeat: New dashboards for the nginx and mongodb modules

The Metricbeat Nginx and MySQL modules get new shiny Time Series Visual Builder (TSVB) based dashboards. Nginx didn’t have a dashboard before, while the MongoDB one was a mix of Timelion and Kibana visualizations.  These new dashboards will be included in the 6.2 release.

Nginx Overview.png

Other changes:

Repository: elastic/beats

Metricbeat

Changes in master:

  • Make sure the active phase status is retrieved for kubernetes pods in state_pod #5980
  • Dedot keys for jolokia metricsets #5957
Packetbeat

Changes in master:

  • Packetbeat, mysql proto, trim SQLs captured from app running #5572
Auditbeat

Changes in master:

  • Check err param in filepath.WalkFunc #6007
  • Do not fail recursive watches due to unaccessible subdir #5958

Changes in 6.1:

  • Check err param in filepath.WalkFunc #6007
Testing

Changes in master:

  • Readd crosscompile builds #5993
  • Start cleaning up condition tests #5960

Changes in 6.1:

  • Follow up changes in ML importer integration tests #5865
Infrastructure

Changes in master:

  • Remove 2x environments #5967
  • Improve Kibana docker image build #5966
Documentation

Changes in master:

  • Clarify naming of multiline settings #6014
  • Fix typo in Filebeat autodiscover examples #6000
  • Changed certgen to certutil #5990
  • Show separate commands for deb/rpm and mac #5828

Changes in 6.1:

  • Fix typo in Filebeat autodiscover examples #6000
  • Add docs for Traefik module #5889
  • Use docker prospector in examples #5963
  • Updated docs for TLS module #5776
  • Reformat module docs #5877)
New community Beats

Changes in 6.1:

  • Add tracebeat to the community beats #5824

Repository: elastic/gosigar

Changes in master:

  • Use the common version of Get(pid) #91

Repository: elastic/go-libaudit

Changes in master:

  • Helper to wait for async message completion #14