Beats version 6.0.0-beta1

View commits

Breaking changes

Affecting all Beats

  • Rename kubernetes processor to add_kubernetes_metadata. 4473
  • Rename *.full.yml config files to *.reference.yml. 4563
  • The scripts/import_dashboards is removed from packages. Use the setup command instead. 4586
  • Change format of the saved kibana dashboards to have a single JSON file for each dashboard 4413
  • Rename configtest command to test config. 4590
  • Remove setting queue_size and bulk_queue_size. 4650
  • Remove setting dashboard.snapshot and dashboard.snapshot_url. They are no longer needed because the dashboards are included in the packages by default. 4675
  • Beats can no longer be launched from Windows Explorer (GUI), command line is required. 4420

Auditbeat

  • Changed file metricset config to make file.paths a list instead of a dictionary. 4796

Heartbeat

  • Renamed the heartbeat RPM/DEB name to heartbeat-elastic. 4601

Metricbeat

  • Change all system.cpu.*.pct metrics to be scaled by the number of CPU cores. This will make the CPU usage percentages from the system cpu metricset consistent with the system process metricset. The documentation for these metrics already stated that on multi-core systems the percentages could be greater than 100%. 4544
  • Remove filters setting from metricbeat modules. 4699
  • Added type field to filesystem metrics. 4717

Packetbeat

  • Remove the already unsupported pf_ring sniffer option. 4608

Bugfixes

Affecting all Beats

  • Don’t stop with error loading the ES template if the ES output is not enabled. 4436
  • Fix race condition in internal logging rotator. 4519
  • Normalize all times to UTC to ensure proper index naming. 4569
  • Fix issue with loading dashboards to ES 6.0 when .kibana index did not already exist. 4659

Auditbeat

  • Fix file.max_file_size config option for the audit file metricset. 4796

Filebeat

  • Fix issue where the fileset.module could have the wrong value. 4761

Metricbeat

  • Fix issue affecting Windows services timing out at startup. 4491
  • Fix incorrect docker.diskio.total metric calculation. 4507
  • Vsphere module: used memory field corrected. 4461

Packetbeat

  • Enabled /proc/net/tcp6 scanning and fixed ip v6 parsing. 4442

Winlogbeat

  • Removed validation of top-level config keys. This behavior was inconsistent with other Beats and caused maintainability issues. 4657

Added

Affecting all Beats

  • New cli subcommands interface. 4420
  • Allow source path matching in add_docker_metadata processor. 4495
  • Add support for analyzers and multifields in fields.yml. 4574
  • Add support for JSON logging. 4523
  • Add test output command, to test Elasticsearch and Logstash output settings. 4590
  • Introduce configurable event queue settings: queue.mem.events, queue.mem.flush.min_events and queue.mem.flush.timeout. 4650
  • Enable pipelining in Logstash output by default. 4650
  • Added result field to Elasticsearch QueryResult struct for compatibility with 6.x Index and Delete API responses. {issue]4661[4661]
  • The sample dashboards are now included in the Beats packages. 4675
  • Add pattern option to be used in the fields.yml to specify the pattern for a number field. 4731

Auditbeat

  • Added file.hash_types config option for controlling the hash types. 4796
  • Added the ability to specify byte unit suffixes to file.max_file_size. 4796

Filebeat

  • Add experimental Redis module. 4441
  • Nginx module: use the first not-private IP address as the remote_ip. 4417
  • Load Ingest Node pipelines when the Elasticsearch connection is established, instead of only once at startup. 4479
  • Add support for loading Xpack Machine Learning configurations from the modules, and added sample configurations for the Nginx module. 4506 4609
  • Add udp prospector type. 4452
  • Enabled Cgo which means libc is dynamically compiled. 4546
  • Add Beta module config reloading mechanism 4566
  • Remove spooler and publisher components and settings. 4644

Heartbeat

  • Enabled Cgo which means libc is dynamically compiled. 4546

Metricbeat

  • Add random startup delay to each metricset to avoid the thundering herd problem. 4010
  • Add the ability to configure audit rules to the kernel module. 4482
  • Add the ability to configure kernel’s audit failure mode. 4516
  • Add experimental Aerospike module. 4560
  • Vsphere module: collect custom fields from virtual machines. 4464
  • Add test modules command, to test modules expected output. 4656
  • Add processors setting to metricbeat modules. 4699
  • Support npipe protocol (Windows) in Docker module. 4751

Winlogbeat

  • Add the ability to use LevelRaw if Level isn’t populated in the event XML. 4257

Auditbeat

  • Add file integrity metricset to the audit module. 4486