Beats version 7.0.0-rc1

Check the HEAD diff

Breaking changes

Affecting all Beats

  • On Google Cloud Engine (GCE) the add_cloud_metadata will now trim the project info from the cloud.machine.type and cloud.availability_zone. 10968
  • Add cleanup_timeout option to docker autodiscover, to wait some time before removing configurations after a container is stopped. 10374 10905
  • Empty meta.json file will be treated as a missing meta file. 8558
  • Rename migration.enabled config to migration.6_to_7.enabled. 11284
  • Initialize the Paths before the keystore and save the keystore into data/{beatname}.keystore. 10706
  • Beats Xpack now checks for Basic license on connect. 11296

Auditbeat

  • Process dataset: Only report processes with executable. 11232

Filebeat

  • Set ecs: true in user_agent processors when loading pipelines with Filebeat 7.0.x into Elasticsearch 6.7.x. 10655 10875

Metricbeat

  • Migrate docker module to ECS. 10927

Functionbeat

  • Correctly extract Kinesis Data field from the Kinesis Record. 11141

Bugfixes

Affecting all Beats

  • Reconnections of Kubernetes watchers are now logged at debug level when they are harmless. 10988
  • Add missing host.* fields to fields.yml. 11016
  • Include ip and boolean type when generating index pattern. 10995
  • Using an environment variable for the password when enrolling a beat will now raise an error if the variable doesn’t exist. 10936
  • Cancelling enrollment of a beat will not enroll the beat. 10150
  • Allow to configure Kafka fetching strategy for the topic metadata. 10682

Auditbeat

  • Package: Disable librpm signal handlers. 10694
  • Login: Handle different bad login UTMP types. 10865
  • System module: Fix and unify bucket closing logic. 10897
  • User dataset: Numerous fixes to error handling. 10942

Filebeat

  • Fix errors in filebeat Zeek dashboard and README files. Add notice.log support. 10916
  • Fix a bug when converting NetFlow fields to snake_case. 10950
  • Add on_failure handler for Zeek ingest pipelines. Fix one field name error for notice and add an additional test case. 11004 11105
  • Fix issue preventing docker container events to be stored if the container has a network interface without ip address. 11225 11247
  • Change URLPATH grok pattern to support brackets. 11135 11252
  • Add support for iis log with different address format. 11255 11256

Heartbeat

  • Fix checks for TCP send/receive data 11118

Metricbeat

  • Migrate docker autodiscover to ECS. 10757 10862
  • Fix issue in kubernetes module preventing usage percentages to be properly calculated. 10946
  • Fix for not reusable http client leading to connection leaks in Jolokia module 11014
  • Fix parsing error using GET in Jolokia module. 11075 11071
  • Collect metrics when EC2 instances are not in running state. 11008 11023
  • Change ECS field cloud.provider to aws. 11023
  • Add documentation about jolokia autodiscover fields. 10925 10979
  • Add missing aws.ec2.instance.state.name into fields.yml. 11219 11221
  • Fix ec2 metricset to collect metrics from Cloudwatch with the same timestamp. 11142
  • Fix potential memory leak in stopped docker metricsets 11294

Packetbeat

  • Avoid reporting unknown MongoDB opcodes more than once. 10878

Winlogbeat

  • Prevent Winlogbeat from dropping events with invalid XML. 11006
  • Fix Winlogbeat escaping CR, LF and TAB characters. 11328 11357

Functionbeat

Added

Affecting all Beats

  • Add ip fields to default_field in Elasticsearch template. 11035

Auditbeat

  • Move System module to beta. 10800

Filebeat

  • Add ISO8601 timestamp support in syslog metricset. 8716 10736
  • Add support for loading custom NetFlow and IPFIX field definitions to netflow input. 10945 11223
  • Added categorization fields for SSH login events in the system/auth fileset. 11334

Metricbeat

  • Add filters and pie chart for AWS EC2 dashboard. 10596

Known Issue

Journalbeat

  • Journalbeat requires at least systemd v233 in order to follow entries after journal changes (rotation, vacuum).