24 July 2017

Brewing in Beats: Test command for Metricbeat modules

By Monica Sarbu

Welcome to Brewing in Beats! With this weekly series, we're keeping you up to date with what's new in Beats, including the latest commits and releases.

Test command for Metricbeat modules

This PR adds a new test command for the Metricbeat modules. It goes over all the configured modules and executes a fetch for each, reporting the result back to the user, or any error that happened during the fetch.

$ ./metricbeat test modules
system...
  process_summary...OK
    result: 
    {
     "_namespace": "process.summary",
     "idle": 0,
     "running": 1,
     "sleeping": 236,
     "stopped": 0,
     "total": 238,
     "unknown": 1,
     "zombie": 0
    }
nginx...
  stubstatus...
    error... ERROR error making http request: Get http://127.0.0.1/server-status: dial tcp 127.0.0.1:80: getsockopt: connection refused

This functionality will be released in 6.0.0-beta1.

Load dashboards via Kibana API in version 5.6

In the 5.x versions, the Beat loads the dashboards directly in the Elasticsearch .kibana index, and uses the import_dashboards script for that (available in the Beat package). Starting with 6.0 version, if the Kibana version is greater than 5.6, then the Beat uses the Kibana API to load the dashboards, otherwise it loads them directly to Elasticsearch.

The PR adds the option in the 5.6 version to load the dashboards via the Kibana API from the import_dashboards script, if requested by the user. This would make it possible for Beats 5.6 to import the dashboards in the future Kibana 6.x versions. The default is still to insert the dashboards directly in Elasticsearch, but one can provide a Kibana URL, in which case the Kibana API is used.

This functionality will be released only in 5.6.

Other changes

Repository: elastic/beats

Affecting all Beats

Changes in 5.x:

  • Normalize times to common.Time with UTC time zone #4658

Changes in master:

  • Add pattern option in the fields.yml #4731

Changes in 5.5:

  • Normalize times to common.Time with UTC time zone #4658
Metricbeat

Changes in master:

  • Update metricbeat to use new publisher API #4699
Filebeat

Changes in 5.x:

  • Change grok pattern to fetch correct IP from X-Forwarded-For list #4351
  • Nginx module: use first not private IP address as remote_ip #4417
  • Update ML datafeed settings #4676

Changes in master:

  • Added FBM Redis sample configuration files #4718
Winlogbeat

Changes in master:

  • Fallback on LevelRaw If the Level is not in the RenderingInfo section of the event #4257
  • Move Winlogbeat to new publisher pipeline #4690
Auditbeat

Changes in master:

  • Mark audit metricsets as beta #4727
Dashboards

Changes in master:

  • Few fixes for load dashboards via Kibana API #4721
Machine learning jobs

Changes in 5.x:

  • Check ML job against a minimal version of ES #4681

Changes in master:

  • Check ML job against a minimal version of ES #4681
Testing

Changes in master:

  • Use the beta1-snapshot in the testing env #4711
  • Improve Auditbeat file test on Windows #4698

Changes in 5.5:

  • Update testing env to 5.5.1 #4722
Infrastructure

Changes in master:

  • Generate xunit report for system test #4686
Documentation

Changes in 5.x:

  • Remove invalid links to releases #4707

Changes in master:

  • [minor] Remove link #4728
  • Add Auditbeat docs #4697

Changes in 5.5:

  • Close changelog for 5.5.1 #4720

Repository: elastic/go-libaudit

Changes in master:

  • Apply hex decoding to CWD #11