Brewing in Beats: Heartbeat HTTP Body Validation

著者
Andrew Kroh

Welcome to Brewing in Beats! With these weekly series, we're keeping you up to date with what's new in Beats, including the latest commits and releases.

Did you know that Beats 6.2 is already available? Try it and let us know what you think. If you are curious to see the Beats in action, check out the Getting Started with Beats webinar.

Heartbeat HTTP body validation

Heartbeat can now validate that HTTP response bodies match a regular expression. Michael Russell of the Elastic Infrastructure team contributed the enhancement in #6539. Prior to this change, if you wanted to check the response content you had to specify the exact body. But with responses being generated by an HTTP endpoint the output is often not always the same. This new feature will be available in 6.3.

Rename fields with a Beat processor

A new processor has been added in #6292 for renaming fields in an event. This is useful for modifying events that would result in an Elasticsearch error at ingest time. For example you cannot ingest an event like {"c": 1, "c.b": 2} because the value of c cannot simultaneously be a scalar and an object. The processor can be used to rename the key c to c.value. This new feature will be available in 6.3.

Filebeat updates ingest pipelines on setup

The filebeat setup command will now update ingest pipelines used by Filebeat modules. Users that modify a module or its configuration now have an easy way to update the ingest pipeline definition stored in Elasticsearch. This also provides users with a way to load ingest pipelines before starting Filebeat. Checkout the details in #6814. This new feature will be available in 6.3.

Fixed: Filebeat memory leak

If a harvester for tailing a log file failed during its initial setup phase due to an I/O error then a goroutine and some internal state were leaked. A fix for this leak was merged in #6829.

Documentation

Changes in master:

  • Update haproxy.asciidoc #6869
  • Document role required to load dashboards #6849
  • Expose the `keystore.path` in the YAML configuration #6805

All changes

Repository: elastic/beats

Metricbeat

Changes in master:

  • Kubernetes deployment: Add ServiceAccount config to system metricbeat. #6824
  • Remove duplicated filesystems in system module #6819
  • Improvements in docker diskio metricset #6701
Filebeat

Changes in master:

  • Fix memory leak in log harvester (#6797) #6829
  • Add ingest pipeline loading to filebeat setup #6814
  • Fix godoc on registrar gcStates #6806
  • Add new MongoDB module #6283

Changes in 6.2:

  • Defer registry state cleanup #6347
Heartbeat

Changes in master:

  • Use match.Matcher for checking Heartbeat response bodies #6539
Processors

Changes in master:

  • Add rename fields processor #6292
  • Rename host.hostname to host.name in add_host_metadata processor #6752
Testing

Changes in master:

  • Revert docker-compose in Travis CI to version 1.11.1 #6863
  • Print container logs when compose up fails in tests #6847
  • Speed up system tests by caching YAML output #6838
  • Remove version from docker-compose project name #6833
  • Simplify system tests for Elasticsearch #6826
  • Allow system tests to be run from any directory #6825
  • Fix flaky windows diskio tests #6778