Beats version 7.16.0edit

View commits

Breaking changesedit

Affecting all Beats

  • Load index templates v2 (composable index templates) by default when talking to ES 7.16 or ES 8.x. Please note that you cannot load templates into Elasticsearch 7.8 or older with this default. To load templates to these ES version, set setup.template.type back to legacy. 28538
  • Previously, RE2 and thus Golang had a bug where (|a)* matched more characters than (|a)+. To stay consistent with PCRE, the bug was fixed. Configurations that rely on the old, buggy behaviour has to be adjusted. See more about Golang bug: https://github.com/golang/go/issues/46123 27543
  • Remove Journalbeat. Use journald input of Filebeat instead. 29131

Heartbeat

  • Change behavior in case of duplicate monitor IDs in configs to be last monitor wins. 29041

Metricbeat

  • Align fields to Beats naming conventions in GCP module. 27231 27974

Functionbeat

  • Support for Google Cloud Functions have been removed, as it has been in Beta for a long time and been broken for a few releases. Please use other tools provided by Elastic to fetch data from GCP (e.g. Filebeat).

Bugfixesedit

Affecting all Beats

  • Fix discovery of Nomad allocations with multiple events during startup. 28700
  • Fix the wrong beat name on monitoring and state endpoint 27755
  • Skip configuration checks in autodiscover for configurations that are already running 29048
  • Fix decode_json_processor to always respect add_error_key 29107
  • Fix add_labels flattening of array values. 29211
  • Skip add_kubernetes_metadata processor when Kubernetes metadata are already present 27689

Auditbeat

  • Fix handling of root and relative paths 24430 28354
  • Fix handling of long file names on Windows. 25334 28517
  • System/socket dataset: Fix uninstallation of return kprobes. 28608 28609
  • Fix auditbeat tracing struct decoding. 28580

Filebeat

  • Update indentation for azure filebeat configuration. 26604
  • Tolerate faults when Windows Event Log session is interrupted 27947 28191
  • Add support for username in Cisco ASA security negotiation logs 26975
  • Relax time parsing and capture group and session type in Cisco ASA module 24710 28325
  • Correctly track bytes read when max_bytes is exceeded. 28317 28352
  • Fix parsing of apache log levels including numbers. 28717
  • Upgrade azure-eventhub SDK reference, contains potential checkpoint fixes. 28919
  • Revert usageDetails api version to 2019-01-01. 28995
  • Fix in aws-s3 input regarding provider discovery through endpoint 28963
  • Fix threatintel.misp filters configuration. 27970
  • Fix opening files on Windows in filestream so open files can be deleted. 29113 29180

Heartbeat

  • Fix broken seccomp filtering and improve security via setcap and setuid when running as root on linux in containers. 27878
  • Log browser zip_url download failures as warn instead of as info. 28440
  • Properly locate base stream in fleet configs. 28455
  • Stop logging params values. 28774
  • Remove accidentally included cups library in Docker images. pull
  • Fix broken monitors with newer versions of image relying on dup3. https://github.com/elastic/beats/pull/28938[pull

Metricbeat

  • beat module respects basepath config option. 28162
  • Fix list_docker.go 28374
  • Fix RDS metadata in Cloudwatch metricset. 29106
  • Errors should be thrown as errors. Metricsets inside metricbeat will now throw errors as the error log level. 27804

Winlogbeat

  • Tolerate faults when Windows Event Log session is interrupted 27947 28191
  • Add ECS 1.9 new users fields 26509
  • Don’t split hyphenated tokens 28483
  • Correctly handle AccessMask if it is an integer or list of masks. 29016

Addededit

Affecting all Beats

  • Allow non-padded base64 data to be decoded by decode_base64_field 27311, 27021
  • The Kafka support library Sarama has been updated to 1.29.1. 27717
  • Kafka is now supported up to version 2.8.0. 27720
  • Add Huawei Cloud provider to add_cloud_metadata. 27607
  • Add default seccomp policy for linux arm64. 27955
  • Add cluster level add_kubernetes_metadata support for centralized enrichment 24621
  • Update cloud.google.com/go library. 28229
  • Add additional metadata to the root HTTP endpoint. 28265
  • Upgrade k8s.io/client-go library. 28228
  • Update ECS to 1.12.0. 27770
  • Fields mapped as match_only_text will automatically fallback to a text mapping when using Elasticsearch versions that do not support match_only_text. 27770
  • Do not load ML jobs to Elasticsearch 8.x from new Beats 7.x releases. 27771
  • Update kubernetes scheduler and controllermanager endpoints in elastic-agent-standalone-kubernetes.yaml with secure ports 28675
  • Add default seccomp policy for Linux arm64. 27955
  • Add http.pprof.enabled option to libbeat to allow http/pprof endpoints on the socket that libbeat creates for metrics. 21965
  • Enable IMDSv2 support for add_cloud_metadata processor on AWS. 22101 28285

Filebeat

  • Add timezone config option to the decode_cef processor. 27232 27727
  • Add timezone config option to the syslog input. 27727
  • Added support for parsing syslog dates containing a leading 0 (e.g. Sep 01) rather than a space. 27775
  • Add base64 Encode functionality to httpjson input. 27681
  • Add join and sprintf functions to httpjson input. 27735
  • Improve memory usage of line reader of log and filestream input. 27782
  • Add ignore_empty_value flag to httpjson split processor. 27880
  • Add support for passing a prefix on S3 bucket list mode for AWS-S3 input 28252 27965
  • Update Cisco ASA/FTD ingest pipeline grok/dissect patterns for multiple message IDs. 26869 26879
  • Add write access to url.value from request.transforms in httpjson input. 27937
  • Add Base64 encoded HMAC and UUID template functions to httpjson input 27873
  • Release checkpoint module as GA. 27814
  • Make aws-cloudwatch input GA. 28161
  • Move processing to ingest node for AWS vpcflow fileset. 28168
  • Release zoom module as GA. 28106
  • Add support for secondary object attribute handling in ThreatIntel MISP module 28124
  • Azure signinlogs - Add support for ManagedIdentitySignInLogs, NonInteractiveUserSignInLogs, and ServicePrincipalSignInLogs. 23653
  • Add base64Decode and base64DecodeNoPad functions to httpsjon templates. 28385
  • Add early_limit config option for Rate-Limiting httpjson. Default rate-limiting for Okta will start when remaining is 1. 28513
  • Add latency config option for aws-cloudwatch input. 28509
  • Added proxy support to threatintel/malwarebazaar. 28533
  • Sophos UTM: Support logs containing hostname in Syslog header. 28638
  • Moving Oracle Filebeat module to GA. 28754
  • Add support in aws-s3 input for S3 notification from SNS to SQS. 28800
  • Add support in aws-s3 input for custom script parsing of S3 notifications. 28946
  • Improve error handling in aws-s3 input for malformed S3 notifications. 28828 28946
  • filestream and log inputs accept null (\u0000) as line terminator. 28998

Heartbeat

  • Support JSON expressions / validation of JSON arrays. 28073
  • Experimental run once mode. 25972
  • Add keyword multi-field mapping for synthetics.step.name. 28452

Metricbeat

  • Enable journald input type in Filebeat. 7955 27351
  • Added a new beta enterprisesearch module for Elastic Enterprise Search 27549
  • Register additional name for storage metricset in the azure module. 28447
  • Update reference to gosigar pacakge for filesystem windows fix. 28909
  • Override Host() on statsd MetricSet 29103
  • Add Linux pressure metricset 27355
  • Add User-Agent header to HTTP requests. 18160 27509

Functionbeat

  • Add support for AWS Kinesis record deaggregation 28241

Winlogbeat

  • Add support for event language selection from config file 19818

Deprecatededit

Affecting all Beats

  • Deprecate setup.template.type. In the future Beats will load data streams instead of regular indices.

Filebeat

  • Deprecate log input in favour of filestream input. 28623