13 December 2017 Releases

Beats 6.1.0 released

By Monica SarbuTudor Golubenco

We’re pleased to announce the Beats 6.1.0 release. This is the latest stable version and it comes with lots of new modules and an exciting Autodiscovery feature.

Docker Autodiscovery

Beats 6.1 brings in the first phase of autodiscovery support. Autodiscovery allows the user to configure providers, that watch for system changes and emit events to a common bus. Based on these events, the Autodiscovery system detects situations when there is something new that we can monitor and instantiates new Beats modules for it. In general, Autodiscovery allows the Beats to react and adapt to changes in the ever more dynamic infrastructures.

The first provider watches for Docker events. It supports config mapping from container metadata to config templates, so new modules are created when a container starts.

metricbeat.autodiscover:
  providers:
    - type: docker
      templates:
        - condition:
            equals.docker.container.image: redis
          config:
            - module: redis
              metricsets: ["info", "keyspace"]
              hosts: "${data.host}:${data.port}"

The above is an example configuration that instantiates the Metricbeat Redis module every time a new redis container (defined by having the redis image) is started. Note that the connection information (host/ports) is filled in by the autodiscovery support via a template.

Future releases will add more Autodiscovery providers, for example for Kubernetes events and package managers.

New Metricbeat and Filebeat modules

Each Beats release adds a few new Metricbeat and Filebeat modules, but 6.1 really sets the bar higher. Many of these modules are contributed by our users (Thank You!). Let’s go through the list:

  • Graphite module in Metricbeat, contributed by @vjsamuel. The Graphite protocol is wonderful in its simplicity and very commonly used by monitoring systems. This module makes easy to use Metricbeat as a gateway for Graphite data into the Elastic stack.
  • HTTP server metricset, by the same @vjsamuel. This allows Metricbeat to open an HTTP server and accept arbitrary data in the JSON format. You can use this, for example, for creating custom metrics collection in your programming language of choice, and then push it to the local Metricbeat via this metricset.
  • Etcd module in Metricbeat, contributed by @berfinsari. Etcd is the distributed key-value where Kubernetes stores all of its data, so this fits very well with all the Kubernetes improvements that we added in 6.0.
  • Logstash module in Metricbeat as well as in Filebeat. What about monitoring the Elastic Stack with Beats? Besides Elasticsearch and Kibana, Beats 6.1 is able to monitor Logstash as well, by collecting its metrics and logs.
  • Postgres module in Filebeat. This module collects and parses the error and the slow-logs generated by PostgreSQL.
  • Kafka module in Filebeat. This module collects and parses logs and stacktraces from Kafka.
  • System uptime metricset in Metricbeat. A simple but useful one, reporting the host uptime in milliseconds. It’s available on all our supported operating systems.
  • Windows service metricset in Metricbeat, contributed by @martinscholz83. This metricset collects information about which services are running and data about each of them.
  • OSD tree metricset for the Ceph module in Metricbeat, contributed by @elaron. The Metricbeat support for  Ceph monitoring gets even better with this new metricset.
  • RabbitMQ queue metricset in Metricbeat, contributed by @kvalev. This metricset provides basic RabbitMQ queue information.

TLS support in Packetbeat

Packetbeat 6.1 adds support for the TLS protocol, which is one of the most anticipated Packetbeat features.

It doesn’t mean decrypting traffic, but it parses the initial handshake and extracts data like ciphers supported by the client and the server, the client and server certificate chains, the subject alternative name (SAN), validity dates, raw certificates, and so on. This data is super valuable for debugging TLS issues and also for intrusion detection and auditing.

The implementation also comes with support for the ALPN extension to TLS, which allows Packetbeat to detect, for example, whether HTTP/2 or HTTP/1 are used as an application protocol on top of the TLS connection.

tls-dashboard.png

Docker JSON-file prospector in Filebeat

Filebeat 6.1 comes with an (experimental) Docker prospector that implements the default JSON logging driver. Filebeat could already read Docker logs via the log prospector with JSON decoding enabled, but this new prospector makes things easier for the user. It abstracts the format, so there is no need to manually configure JSON decoding. Here is an example config, which captures the logs from a single container specified by its ID:

prospectors:
   - type: docker
     containers.ids:
       - c3ec7a0bd9640151a768663b7e78c115d5b1a7f87fba572666bacd8065893d41

It also parses the timestamp from the JSON file, something that wasn’t previously possible with Filebeat alone (it required Logstash or Ingest Node). This new prospector works great with the Docker Autodiscovery provider.

New Auditbeat dashboards

Auditbeat 6.1 comes with several use cases in the default configuration file, which makes it easier to get started with. To match the use cases, we also have three new dashboards:

  • Overview - A general dashboard showing a summary of all events.
  • Executions - A dashboard showing process executions (execve and execveat) syscalls.
  • Sockets - A dashboard showing information about related sockets and remote connectivity (e.g. bind, connect, accept, recvfrom).

auditbeat-kernel-sockets-dashboard (1).png

Feedback

If you want to make use of the new features added in Beats 6.1.0, please download the latest stable version, install it, and let us know what you think on Twitter (@elastic) or in our forum.