Brewing in Beats: Add support for Kafka 2.0

By
Monica Sarbu

Welcome to Brewing in Beats! With these weekly series, we're keeping you up to date with what's new in Beats, including the latest commits and releases.

Did you know that Beats 6.4 is already available? Try it and let us know what you think. 

Add support for Kafka 2.0

Support for Kafka 2.0 is coming, we have updated Sarama recently and we now have a PR about to get merged adding support for the shiny new Kafka in our outputs. #8399

Auditbeat system module

Auditbeat is getting a system module, which will be the base for collecting host specific information like: OS and kernel versions, packages installed (dpkg, rpm, homebrew, etc.), processes started and stopped, etc. We plan to present this information in a curated UI oriented around hosts.

Packetbeat fingerprints

Packetbeat will now report fingerprints of certificates when it captures TLS traffic. This can be useful if you want to check the traffic on your network against an SSL Blacklist. Packetbeat can be configured to compute MD5, SHA1, and SHA256 hashes. See the pull request for more details.

All changes

Repository: elastic/beats

Metricbeat

Changes in master:

  • Fix incorrect type conversion of average response time in Haproxy dashboards #8404
  • Adding elasticsearch/ccr metricset #8335
  • Adding x-pack code for elasticsearch/index metricset #8260

Changes in 6.x:

  • [Kibana metricset] Make integration test conditional on Kibana version #7749
  • Adding x-pack code for elasticsearch/index metricset #8260
  • Added function to return pretty names on "system" module "process summary" metricset #8275
Packetbeat

Changes in master:

  • Calculate fingerprints of TLS certificates #8353
Filebeat

Changes in master:

  • Update geoip data for test_module #8401
  • Reduce errors on filebeat syslog stop #8347

Changes in 6.x:

  • Update geoip data for test_module #8401
Heartbeat

Changes in master:

  • Add required comments to heartbeat monitors #8316

Changes in 6.x:

  • Heartbeat Automatic Reload #8023
Auditbeat

Changes in feature/auditbeat-host:

  • Add skeleton x-pack Auditbeat module #8252

Changes in 6.4:

  • [auditbeat] Improve file monitor flaky test #8367

Changes in master:

  • Change accept to accept4 for 32-bit #8383
  • [auditbeat] Improve file monitor flaky test #8367

Changes in feature-auditbeat-host:

  • [Auditbeat Host] Rename sysinfo module to system #8356

Changes in 6.x:

  • [auditbeat] Improve file monitor flaky test #8367
  • Improve error catching for recursiveWatcher #8087
Processors

Changes in 6.x:

  • Add a reverse dns processor #7927
Packaging

Changes in master:

  • Build version qualifier #8310
Documentation

Changes in 6.4:

  • Add a different doc helper for http and tls configurations #8159
  • Bump docs version for 6.4.1 #8340
  • Revert "Bump docs version for 6.4.1" #8333

Changes in master:

  • Add decode_json_fields processor to Beats config #8344
  • Heartbeat Automatic Reloading Docs #8227
  • Add a different doc helper for http and tls configurations #8159
  • Document known issue in 6.4.0 #8108

Changes in 5.6:

  • Bump docs version for 5.6.12 #8341
  • Docs version bump 5.6.12 #8326
  • Close changelog 5.6.12 #8322

Changes in 6.x:

  • Add a different doc helper for http and tls configurations #8159
  • parameterize beat monitoring docs #8297
  • Document known issue in 6.4.0 #8108