Brewing in Beats: Auditbeat improvements

Welcome to Brewing in Beats! With these weekly series, we're keeping you up to date with what's new in Beats, including the latest commits and releases.

Did you know that Beats 6.2 is already available? Try it and let us know what you think. If you are curious to see the Beats in action, check out the Getting Started with Beats webinar.

Auditbeat Updates

A few minor fixes and enhancements were made to Auditbeat this week. We fixed an issue where user and group names were not being cached by the auditd module. This should result in slightly less CPU load when enriching high volumes of audit events.

Audit message parsing was also enhanced. An issue where AppArmor messages were not fully parsed has been resolved. And we updated the tables used for resolving system call numbers to names for Linux 4.16 (a few new system calls were added since we last updated).

And finally, the error messages generated by Auditbeat when the Linux kernel is compiled without audit support (CONFIG_AUDIT=n) were improved. The error will now tell the user that their kernel does not have audit support.

Heartbeat race condition

When configuring a monitor in Heartbeat to connect to multiple ports, heartbeat might crash due to a race condition on event fields being updated. This is fixed in #6950, by copying the fields that might be updated in parallel.

Documentation

Repository: elastic/beats

Changes in master:

  • Replace references to X-Pack with attributes or remove them #6985
  • [DOCS] Fixes certutil command name #6980
  • [DOCS] Fixes URL for Stack Overview #6946
  • Fixed omission of github.com in git clone path #6851

Changes in 6.3:

  • Backport to 6.3: Fixed omission of github.com in git clone path (#6851) #7002
  • Backport to 6.3: community beats updates #7000
  • Fix double node_stats entry Elasticsearch module docs #6973
  • Set docs version to 6.3.0 in the 6.3 branch #6938

Changes in 6.2:

  • Backport to 6.2: #6557 #7001
  • Backport to 6.2: community beats updates #6999
  • Modify title to indicate that ingest node is used for more than logs #6650,
  • Indicate that Beats monitoring requires ES 6.2 or later #6651,
  • Change title for SEO #6652,
  • Fix descriptions of include_line and exclude_line #6654,
  • Remove old reviewer notes #6655,
  • Docs: use a simple clone in the New Beat guide #6786,
  • Document role required to load dashboards #6849,
  • Fixed omission of github.com in git clone path #6851)
  • Fix double node_stats entry Elasticsearch module docs #6972

Changes in 6.x:

  • Set docs version in the 6.x branch to 6.4.0 #6939

    More changes

    Repository: elastic/beats

    Metricbeat

    Changes in master:

    • Add message rates to rabbitmq queue metricset #6964
    • Metricbeat: Dashboards and visualizations for haproxy #6934
    • Update elasticsearch.node_stats to use ReporterV2 #6917
    • Add `GetProcessedMetrics` method to Prometheus helper #6916
    • Provide same data structure as X-Pack for ES node_stats #6807
    Packetbeat

    Changes in master:

    • Fix out of bounds access in packetbeat's HTTP parser #6997
    Filebeat

    Changes in master:

    • Inherit Kibana credentials from the ES output #6993
    • Fix: protect the registry critical zone when stop/close are called. #6959
    • Commit registry writes to stable storage to avoid corrupt registry files #6877

    Changes in 6.3:

    • Inherit Kibana credentials from the ES output #6993
    Heartbeat

    Changes in master:

    • Fix heartbeat races on event updates #6950
    Auditbeat

    Changes in master:

    • Tentative fix to auditbeat test panic under macOS #6990
    • Better error handling in fsnotify recursive monitor #6949
    Testing

    Changes in 6.3:

    • Change testing to use 6.3 snapshot #6956

    Changes in master:

    • Remove test for comparing short / long config #6975
    • Remove `-` char from docker-compose project name #6966
    • Update URL to -oss artifacts #6965
    • Increase testing timeouts for Elastic Stack #6963
    • Disable fsnotify recursive test under Darwin #6962
    • Add filtering function for data generation #6958
    • Increase health retries for LS image. #6945
    • Change to snapshot testing for Elastic Stack #6944
    • Harden file integrity flaky test #6907
    • Start simplify system tests #6906
    • Fix unstable ceph tests #6773

    Changes in 6.2:

    • Update 6.2 snapshot build to 6.2.5 #6953
    • Cherry-pick #6833 to 6.2: Remove version from docker-compose project name #6919
    Infrastructure

    Changes in master:

    • Update to Golang 1.10.1 #6948

    Repository: elastic/go-sysinfo

    Changes in master:

    • Replace check_license.go for elastic/go-licenser #13

    Repository: elastic/go-libaudit

    Changes in master:

    • Better errors when kernel does not support auditing #33
    • Update Travis testing to Go 1.10.x #31
    • Update syscall and audit message type tables for Linux 4.16 #30
    • Update headers with go-licenser #29
    • Set default UID/GID cache timeout to 1m #28
    • Parse AVC messages from apparmor #27