Beats version 5.0.0-beta1

View commits

Breaking changes

Affecting all Beats

  • Change Elasticsearch output index configuration to be based on format strings. If index has been configured, no date will be appended anymore to the index name. 2119
  • Replace output.kafka.use_type by output.kafka.topic accepting a format string. 2188
  • If the path specified by the -c flag is not absolute and -path.config is not specified, it is considered relative to the current working directory. 2245
  • rename tls configurations section to ssl. 2330
  • rename certificate_key configuration to key. 2330
  • replace tls.insecure with ssl.verification_mode setting. 2330
  • replace tls.min/max_version with ssl.supported_protocols setting requiring full protocol name. 2330

Metricbeat

  • Change field type system.process.cpu.start_time from keyword to date. 1565
  • redis/info metricset fields were renamed up according to the naming conventions.

Packetbeat

  • Group HTTP fields under http.request and http.response 2167
  • Export http.request.body and http.response.body when configured under include_body_for 2167
  • Move ignore_outgoing config to packetbeat.ignore_outgoing 2393

Filebeat

  • Set close_inactive default to 5 minutes (was 1 hour before)
  • Set clean_removed and close_removed to true by default

Bugfixes

Affecting all Beats

  • Fix logstash output handles error twice when asynchronous sending fails. 2441
  • Fix Elasticsearch structured error response parsing error. 2229
  • Fixed the run script to allow the overriding of the configuration file. 2171
  • Fix logstash output crash if no hosts are configured. 2325
  • Fix array value support in -E CLI flag. 2521
  • Fix merging array values if -c CLI flag is used multiple times. 2521
  • Fix beats failing to start due to invalid duplicate key error in configuration file. 2521
  • Fix panic on non writable logging directory. 2571

Metricbeat

  • Fix module filters to work properly with drop_event filter. 2249

Packetbeat

  • Fix mapping for some Packetbeat flow metrics that were not marked as being longs. 2177
  • Fix handling of messages larger than the maximum message size (10MB). 2470

Filebeat

  • Fix processor failure in Filebeat when using regex, contain, or equals with the message field. 2178
  • Fix async publisher sending empty events 2455
  • Fix potential issue with multiple harvester per file on large file numbers or slow output 2541

Winlogbeat

  • Fix corrupt registry file that occurs on power loss by disabling file write caching. 2313

Added

Affecting all Beats

  • Add script to generate the Kibana index-pattern from fields.yml. 2122
  • Enhance Redis output key selection based on format string. 2169
  • Configurable Redis keys using filters and format strings. 2169
  • Add format string support to output.kafka.topic. 2188
  • Add output.kafka.topics for more advanced kafka topic selection per event. 2188
  • Add support for Kafka 0.10. 2190
  • Add SASL/PLAIN authentication support to kafka output. 2190
  • Make Kafka metadata update configurable. 2190
  • Add Kafka version setting (optional) enabling kafka broker version support. 2190
  • Add Kafka message timestamp if at least version 0.10 is configured. 2190
  • Add configurable Kafka event key setting. 2284
  • Add settings for configuring the kafka partitioning strategy. 2284
  • Add partitioner settings reachable_only to ignore partitions not reachable by network. 2284
  • Enhance contains condition to work on fields that are arrays of strings. 2237
  • Lookup the configuration file relative to the -path.config CLI flag. 2245
  • Re-write import_dashboards.sh in Golang. 2155
  • Update to Go 1.7. 2306
  • Log total non-zero internal metrics on shutdown. 2349
  • Add support for encrypted private key files by introducing ssl.key_passphrase setting. 2330
  • Add experimental symlink support with symlinks config 2478
  • Improve validation of registry file on startup.

Metricbeat

  • Use the new scaled_float Elasticsearch type for the percentage values. 2156
  • Add experimental cgroup metrics to the system/process MetricSet. 2184
  • Added a PostgreSQL module. 2253
  • Improve mapping by converting half_float to scaled_float and integers to long. 2430
  • Add experimental haproxy module. 2384
  • Add Kibana dashboard for cgroups data 2555

Packetbeat

  • Add Cassandra protocol analyzer to Packetbeat. 1959
  • Match connections with IPv6 addresses to processes 2254
  • Add IP address to -devices command output 2327
  • Add configuration option for the maximum message size. Used to be hard-coded to 10 MB. 2470

Filebeat

  • Introduce close_timeout harvester options 1926
  • Strip BOM from first message in case of BOM files 2351
  • Add harvester_limit option 2417

Deprecated

Affecting all Beats

  • Topology map is deprecated. This applies to the settings: refresh_topology_freq, topology_expire, save_topology, host_topology, password_topology, db_topology.