09 August 2016 Releases

Beats 5.0.0-alpha5 released

By Tudor Golubenco

We’re happy to announce the fifth alpha release of the Beats 5.0 series. As we slowly approach the GA release, we're polishing the Beats with lots of small improvements and fixes.  You can find all the details in the release notes below, or read the rest of the blog post for the highlights.

IMPORTANT: This is an alpha release and is intended for testing purposes only. Please do not deploy in production. Yada yada.

Automatically load the right Elasticsearch template

Elasticsearch 5.0 comes with several mapping changes, but it is still able to use the old 2.x templates for backwards compatibility. This is good news because it means that you can use Beats 1.x with either Elasticsearch 2.x or 5.x. However, because we want the Beats 5.x to use the new mapping features, but also be able to work with Elasticsearch 2.x, we added a feature that can query the Elasticsearch version and automatically load the best template for you. This way, you have complete freedom in planning your rolling upgrades.

More filtering flexibility

Beats 5.0.0-alpha5 introduces simple processors (previously called "generic filters") that allow you to flexibly choose which events or fields to drop based on simple conditions. Starting with alpha5, these conditions can also be combined with logical operators (AND/OR/NOT) so they don’t have to be so simple anymore. You can then express conditions like “drop all logs about 200 and 404 responses, except if the response time is larger than 100 milliseconds”.

Filebeat registry file cleanup + more options

Filebeat stores its state about the files it reads in a registry file on disk. This way, it can avoid shipping the same log lines after a restart. With the 5.0.0-alpha5 release, we introduce new configuration options that allow you to configure when to remove entries from this registry file. This means you can make sure that the registry file doesn’t grow forever. On the same note, other Filebeat configuration settings were renamed for consistency and we now have a short guide explaining what each Filebeat component does.

Override settings from the CLI via the -E flag

Do you like how Elasticsearch allows you to set any configuration setting from the command line by using the -E flag (yes, it used to be -D, but was renamed in 5.0 to avoid confusion with the JVM flags)? Well, the same is now possible for all Beats. For example, you can quickly enable the console output by adding -E output.console.pretty=true.

More configuration flexibility

On the same theme with the -E flag, you can now specify multiple configuration files by repeating the -c flag. Settings in subsequent config files override those that precede them. You can use this, for example, for setting defaults in a base configuration file, and overwrite settings via local configs.

We have also standardized on using enabled: false as a way to disable most things in the configuration file. For example, if you want to disable a Packetbeat protocol without commenting out 10 lines of config, add enabled: false. Want to disable an output? The same enabled: false does the trick.

New defaults for the logging verbosity

We did some rethinking of the Beats logging strategy. Part of this, we’ve switched the default log level from ERROR to INFO and reduced the verbosity of most warnings and info messages. To compensate, we added a set of internal metrics which also get logged to the INFO level every 30 seconds. This means you get good visibility into what is happening inside the Beats without getting huge files during periods of high traffic.

Become a Pioneer

A big Thank You to everyone who has tried the previous alpha releases and posted issues or provided feedback. We’d like to also remind you that if you post a valid, non-duplicate bug report during the alpha/beta period against any of the Elastic stack projects, you are entitled to a special gift package.