11 September 2017 Releases

Beats 5.6.0 released

By Monica SarbuTudor Golubenco

Today we are pleased to announce the  Beats 5.6 release, the latest stable release of the 5.x series.

Before going through the highlights, let’s get those handy links out of the way:

Load Machine Learning jobs from Filebeat modules

Filebeat 5.6 release comes with the support to load Machine Learning (ML) job configurations directly from the Filebeat modules. This means that besides Kibana dashboards and Elasticsearch Ingest Node pipelines, Filebeat modules can also contain ML configurations to be used for anomaly detection.

Survey: With this feature, we are further enriching the modules experience for our community. Which Filebeat & Metricbeat modules should we target next to include preconfigured machine learning jobs?  Let us know by filling this short survey. It won't take more than 5 minutes, we promise. 

As part of this release, Filebeat comes with five ML job configurations which were already tuned to work perfectly with the data collected by the Nginx module. If you have Xpack installed and ML enabled, simply run:

filebeat -e -modules=nginx -setup

And you will see the following job configurations created for you:

  • Detect unusual visitor rate
  • Detect unusual response code rates
  • Detect low request rate
  • Detect unusual IP address - high distinct count of urls
  • Detect unusual IP addresses - high request rates

After you let Filebeat ingest some data, simply click the Run button on the ML jobs, and BOOM: anomaly detection for your access logs.

Screen Shot 2017-09-06 at 12.59.29.png

And that’s not all! For any incident detected by ML, you can drill down in dedicated Kibana dashboards which are ready made for you to investigate these incidents.

27842771-3097b33c-610d-11e7-984c-4e1d65cac80b (1).png

Preparing for a smooth transition to 6.0

If you are on a 5.x version of Beats, you will need to upgrade to 5.6 before making the jump to 6.0. This is because we’ve made some changes to 5.6 to smoothen the upgrade experience.

Notably, the _all.norms setting is no longer disabled in the Beats 5.6 mapping templates. This is because Elasticsearch is removing support for the _all field in 6.0, and having settings for it in the template is problematic at upgrade time.

Kibana 6.0 will use a new internal format and a new API for importing and exporting dashboards. To make sure Beats 5.6 will work well with Kibana 6.0, we added support to import the dashboards via the new API in this release already. For this, you can run the `import_dashboards` script with the -kibana flag:

./scripts/import_dashboards -kibana http://localhost:5601

If Kibana 5.x is used, the dashboards are imported directly into Elasticsearch.

./scripts/import_dashboards

Feedback

If you want to make use of the new features added in Beats 5.6.0, please download the latest stable version, install it, and let us know what you think on Twitter (@elastic) or in our forum.