Product release

Beats 6.2.0 released

We’re pleased to announce the Beats 6.2.0 release, coming with plenty of new features including central monitoring, a passwords keystore, Auditbeat GA, Kubernetes autodiscovery, and a Filebeat module for Osquery.

Beats central monitoring (X-Pack Basic)

The Beats 6.2 release brings Beats central monitoring, as part of X-Pack Basic (Free). The way it works is that Beats are sending their monitoring data like CPU usage, number of published events, number of dropped events to the Elastic cluster, and then you can use the dedicated UI (X-Pack Basic) to monitor the Beats instances.

beats-overview.png

Below is a list of all your Beats instances, together with information like the type of the Beat, the output enabled, version, etc. The list offers filtering, so you can filter for a specific Beat or a specific Beat type.

Beats-list-monitoring.png

If you click on one of the Beats, you are redirected to a different page that shows more information about the evolution of the published events, memory usage, system load, bytes sent to easily identify any issues with the Beat.

Beats-details-monitoring.png

By default, the central monitoring is disabled. To enable it, install the Xpack Elasticsearch and Kibana plugins and enable Xpack monitoring in the configuration file of the Beat.

xpack.monitoring.enabled: true

If Elasticsearch is configured as the output, then the monitoring data is sent to the same Elasticsearch cluster configured in the output. In case a different output is configured, then you need to configure the monitoring Elasticsearch cluster under the xpack.monitoring.elasticsearch section.

Secrets keystore

The Beats configuration files often contain sensitive information. For example, in Metricbeat, you need to define the username and password to connect to the Elasticsearch output or credentials to connect to each individual service that you monitor.

Currently, the sensitive information is found in the plaintext YAML configuration file, but  starting with the 6.2 release, you can define them into an obfuscated keystore on disk and only reference them from the configuration file. Future enhancements will also allow encrypting the keystore with a password.

New CLI commands are provided, to make it easier to create the keystore and add or remove keys to the keystore. Here are some sample commands:

# create new keystore
metricbeat keystore create
# add a new key to the store.
metricbeat keystore add output.elasticsearch.password
# remove a key from the store
metricbeat keystore remove output.elasticsearch.password
# list the configured keys without the sensitive information
metricbeat keystore list

After you create the keystore, you can reference keys from the keystore in the configuration file by using the same syntax that we use for the environment variables:

password: "${output.elasticsearch.password}"

Auditbeat GA

We have first released Auditbeat as Beta in the Beats 6.0 release. Since then, we have worked hard at rounding the rough edges, cleaning up the configuration files, squashing bugs, and collecting your feedback. And the feedback has been constantly great, so we are happy to announce the general availability of Auditbeat as part of the Beats 6.2 release.

Being GA means that Auditbeat is recommended for production, fully supported by Elastic, and that we will only be doing breaking changes in major releases.

If you didn’t hear about Auditbeat before, you can think of it as a friendlier version of auditd that is perfectly integrated with the Elastic Stack. It is based on the Linux audit framework, which means it can hook into every system call and capture them on particular conditions. The Linux Audit Framework can send multiple messages for a single auditable event. For example, a rename syscall causes the kernel to send eight separate messages. Each message describes a different aspect of the activity that is occurring (the syscall itself, file paths, current working directory, process title). Auditbeat is able to combine all of the data from each of the messages into a single event.

You can use Auditbeat to very efficiently detect things like short-lived connections and processes, unauthorised attempts to open files, privilege escalations, and so on. Auditbeat automatically correlates events together, resolves UIDs ito names, and outputs JSON objects directly to Elasticsearch or Logstash.

auditbeat-file-integrity-dashboard.png

Filebeat Osquery module

Osquery is an OS instrumentation framework that uses SQL as an interface for querying the underlying data. The new osquery module in Filebeat collects and indexes the result logs written by osqueryd. Osquery writes the results in JSON, which makes it fairly easy to ingest to the Elastic stack, but the module simplifies the collection and the visualization aspect.

osquery.png

This module comes with dashboards for two of the Osquery packs: it-compliance and ossec-rootkit. It is therefore recommended to enable these packs in the osqueryd configuration.

Feedback

If you want to make use of the new features added in Beats 6.2.0, please download the latest stable version, install it, and let us know what you think on Twitter (@elastic) or in our forum.