Brewing in Beats: Profiling Go Programs

Welcome to Brewing in Beats! With this weekly series, we're keeping you up to date with what's new in Beats, including the latest commits and releases.

Metricbeat: Support for profiling Go programs

We added the golang module in Metricbeat to export profiling metrics useful to do a long-term performance analysis and easily find performance issues such as memory leaks. It comes with two metricsets: expvar and heap. The expvar metricset can collect the variables exposed via expvar except the memory usage metrics that are gathered by the heap metricset. The golang module is experimental and we are planning to release it with 6.0.

golang-metricbeat.png

Fileset for the Linux authorization logs auth

This adds a new fileset under the Filebeat system module: system/auth. It parses the authorization logs (typically /var/log/auth.log or /var/log/secure) and creates events for things like:

  • SSH login attempts
  • Commands executed with sudo
  • New users or groups created

A sample dashboard is included for each of the events above. For example, for the SSH logins:

auth-login.png

This new fileset is scheduled to be released with version 5.4.

Introduce Beat version in the Elasticsearch index and template

By default, each Beat creates an index pattern every day of format beatname-%{+yyyy.MM.dd}, and loads at startup (if doesn’t exist already) the latest template. This means that during upgrades, the new template cannot be applied until the next day, when the index changes.

With this change the Beat version is added to the Elasticsearch index and template. The advantage is that an index always has the correct template applied based on the Beat version, and it helps in case different versions of Beats are running in parallel. Now, the index where the Beat dumps the data is beatname-%{[beat.version]}-%{+yyyy.MM.dd} by default.

This should not have any effects on the Kibana dashboards as beatname-* still applies to all data. As this is a big impact change, it will be available in the 6.0 release.

Changes in 5.2.2:

  • Fix docker hanging when container killed #3612

Changes in 5.3:

Documentation

  • Per module docs in Filebeat #3598
  • Filebeat modules dev guide #3616
  • Added Filebeat Module overview and tutorial #3592

All Beats

  • Fix dashboards config naming #3665

Filebeat, Winlogbeat

  • Fix empty registry file on machine crash #3668
  • Fix permissions on module files #3645

Changes in master:

All Beats

  • Use more global paths for system tests #3591
  • Implement index template generation in Golang #3603
  • Persist beat UUID #3615
  • Make Packetbeat and Filebeat better compatible with community Beats #3635
  • Rename object-type to object_type #3655
  • Add tags to monitoring reports #3660
  • Unify fields generated file across all beats #3677
  • Fix elasticsearch url parsing #3671

Documentation

  • Update doc about how to run single integration test #3624

Packetbeat

  • Packetbeat protocol analyzer configuration enhancements #3518

Infrastructure

  • Use Docker Healthcheck to wait for services to start #3652
  • Set default ES port in Logstash docker-entrypoint #3648
  • Use gvm to manage AppVeyor Go version #3653