Beats version 6.7.0

View commits

Breaking changes

Affecting all Beats

  • Port settings have been deprecated in redis/logstash output and will be removed in 7.0. 9915
  • Update the code of Central Management to align with the new returned format. 10019
  • Allow Central Management to send events back to kibana. 9382
  • Fix panic if fields settting is used to configure hosts.x fields. 10824 10935
  • Introduce query.default_field as part of the template. 11205
  • Beats Xpack now checks for Basic license on connect. 11296

Filebeat

  • Filesets with multiple ingest pipelines added in 8914 only work with Elasticsearch >= 6.5.0 10001
  • Add grok pattern to support redis 5.0.3 log timestamp. 9819 10033
  • Ingesting Elasticsearch audit logs is only supported with Elasticsearch 6.5.0 and above 8852
  • Remove ecs option from user_agent processors when loading pipelines with Filebeat 6.7.x into Elasticsearch < 6.7.0. 10655 11362

Heartbeat

  • Remove monitor generator script that was rarely used. 9648

Bugfixes

Affecting all Beats

  • Fix TLS certificate DoS vulnerability. 10303
  • Fix panic and file unlock in spool on atomic operation (arm, x86-32). File lock was not released when panic occurs, leading to the beat deadlocking on startup. 10289
  • Adding logging traces at debug level when the pipeline client receives the following events: onFilteredOut, onDroppedOnPublish. 9016
  • Do not panic when no tokenizer string is configured for a dissect processor. 8895
  • Fix a issue when remote and local configuration didn’t match when fetching configuration from Central Management. 10587
  • Add ECS-like selectors and dedotting to docker autodiscover. 10757 10862
  • Fix encoding of timestamps when using disk spool. 10099
  • Include ip and boolean type when generating index pattern. 10995
  • Using an environment variable for the password when enrolling a beat will now raise an error if the variable doesn’t exist. 10936
  • Cancelling enrollment of a beat will not enroll the beat. 10150
  • Remove IP fields from default_field in Elasticsearch template. 11399

Auditbeat

  • Package: Disable librpm signal handlers. 10694
  • Login: Handle different bad login UTMP types. 10865
  • Fix hostname references in System module dashbords. 11064
  • User dataset: Numerous fixes to error handling. 10942

Filebeat

  • Support IPv6 addresses with zone id in IIS ingest pipeline. 9836 error log: 9869 access log: 10029
  • Fix bad bytes count in docker input when filtering by stream. 10211
  • Fixed data types for roles and indices fields in elasticsearch/audit fileset 10307
  • Cover empty request data, url and version in Apache2 modulehttps://github.com/elastic/beats/pull/10846[10846]
  • Fix a bug with the convert_timezone option using the incorrect timezone field. 11055 11164
  • Change URLPATH grok pattern to support brackets. 11135 11252
  • Add support for iis log with different address format. 11255 11256
  • Add fix to parse syslog message with priority value 0. 11010

Heartbeat

  • Host header can now be overridden for HTTP requests sent by Heartbeat monitors. 9516
  • Fix checks for TCP send/receive data 10777

Journalbeat

  • Do not stop collecting events when journal entries change. 9994

Metricbeat

  • Fix MongoDB dashboard that had some incorrect field names from status Metricset 9795 9715
  • Fix issue that would prevent collection of processes without command line on Windows. 10196
  • Fixed data type for tags field in docker/container metricset 10307
  • Fixed data type for tags field in docker/image metricset 10307
  • Fixed data type for isr field in kafka/partition metricset 10307
  • Fixed data types for various hosts fields in mongodb/replstatus metricset 10307
  • Added function to close sql database connection. 10355
  • Fix parsing error using GET in Jolokia module. 11075 11071

Winlogbeat

  • Fix Winlogbeat escaping CR, LF and TAB characters. 11328 11357

Functionbeat

  • Correctly extract Kinesis Data field from the Kinesis Record. 11141
  • Add the required permissions to the role when deployment SQS functions. 9152

Added

Affecting all Beats

  • Add ip fields to default_field in Elasticsearch template. 11035
  • Add cleanup_timeout option to docker autodiscover, to wait some time before removing configurations after a container is stopped. 10374 10905

Auditbeat

  • System module process dataset: Add user information to processes. 9963
  • Add system package dataset. 10225
  • Add system module login dataset. 9327
  • Add entity_id fields. 10500
  • Add seven dashboards for the system module. 10511

Filebeat

  • Add field log.source.address and log.file.path to replace source. 9435
  • Support mysql 5.7.22 slowlog starting with time information. 7892 9647
  • Add support for ssl_request_log in apache2 module. 8088 9833
  • Add support for iis 7.5 log format. 9753 9967
  • Add support for MariaDB in the slowlog fileset of mysql module. 9731
  • Add convert_timezone to nginx module. 9839 10148
  • Add support for Percona in the slowlog fileset of mysql module. 6665 10227
  • Added support for ingesting structured Elasticsearch audit logs 8852
  • New iptables module that receives iptables/ip6tables logs over syslog or file. Supports Ubiquiti Firewall extensions. 8781 10176
  • Populate more ECS fields in the Suricata module. 10006

Heartbeat

  • Made monitors.d configuration part of the default config. 9004
  • Autodiscover metadata is now included in events by default. So, if you are using the docker provider for instance, you’ll see the correct fields under the docker key. 10258

Metricbeat

  • Add field event.dataset which is {module}.{metricset}.
  • Add more TCP statuses to socket_summary metricset. 9430
  • Remove experimental tag from ceph metricsets. 9708
  • Add key metricset to the Redis module. 9582 9657
  • Add DeDot for kubernetes labels and annotations. 9860 9939
  • Add docker event metricset. 9856
  • Release Ceph module as GA. 10202
  • Release windows Metricbeat module as GA. 10163
  • Release traefik Metricbeat module as GA. 10166
  • List filesystems on Windows that have an access path but not an assigned letter 8916 10196
  • Release uswgi Metricbeat module GA. 10164
  • Release php_fpm module as GA. 10198
  • Release Memcached module as GA. 10199
  • Release etcd module as GA. 10200
  • Release kubernetes apiserver and event metricsets as GA 10212
  • Release Couchbase module as GA. 10201
  • Release aerospike module as GA. 10203
  • Release envoyproxy module GA. 10223
  • Release mongodb.metrics and mongodb.replstatus as GA. 10242
  • Release mysql.galera_status as Beta. 10242
  • Release postgresql.statement as GA. 10242
  • Release RabbitMQ Metricbeat module GA. 10165
  • Release Dropwizard module as GA. 10240
  • Release Graphite module as GA. 10240
  • Release http.server metricset as GA. 10240
  • Add support for MySQL 8.0 and tests also for Percona and MariaDB. 10261
  • Release use of xpack.enabled: true flag in Elasticsearch and Kibana modules as GA. 10222
  • Release Elastic stack modules (Elasticsearch, Logstash, and Kibana) as GA. 10094
  • Add remaining memory metrics of pods in Kubernetes metricbeat module 10157
  • Added server Metricset to Zookeeper Metricbeat module 8938 10341
  • Add overview dashboard to Zookeeper Metricbeat module 10379

Functionbeat

  • Mark Functionbeat as GA. 10564
  • Functionbeat can now deploy a function for Kinesis. 10116
  • Allow functionbeat to use the keystore. 9009

Deprecated

Filebeat

  • Deprecate field source. Will be replaced by log.source.address and log.file.path in 7.0. 9435

Metricbeat

  • Deprecate field metricset.rtt. Replaced by event.duration which is in nano instead of micro seconds.

Packetbeat

  • Support new TLS version negotiation introduced in TLS 1.3. 8647.

Known Issue

Journalbeat

  • Journalbeat requires at least systemd v233 in order to follow entries after journal changes (rotation, vacuum).