Brewing in Beats: Packagebeat, Lmsensorsbeat and Soundbeat

Last week we released Beats 1.1.2. The most interesting news and enhancements of the week are:

Packagebeat

Joe Hillenbrand from Elastic has started Packagebeat to gather Linux distribution package information and index them into Elasticsearch. The Beat is used internally by the infra team and it is actively developed and improved.

Lmsensorsbeat

Shane Connely from Elastic created Lmsensorbeat to monitor a variety of I2C/SMBus sensors, such as CPU/motherboard temperatures, fan speeds, voltages, etc. This is another example that creating a new Beat is easy and fun when using the beat-generator to generate all the boilerplate code for you.

Soundbeat

Soundbeat is another Beat developed inhouse by David Pilato from Elastic and it shows that Beats can have applications outside of the operations domain. It reads the MP3 files to extract the sound level for left and right channels using a given period for each sample. He wrote a nice blog post with all the detailed steps that he followed for writing the Beat.

Packaging for community Beats

Coming from a community Beat author, this PR adjusted the Makefile in our Beat packer so that it can be easily used by all community Beats. This means that a single make command can be used by any of the community Beats to get cross-compiled RPMs, DEBs, etc. The Beat generator was also updated to support for this, so it’s really easy to use by the community Beats authors.

Generic filtering

Generic filtering has now support for include_fields and drop_fields actions by merging the pull request in master. The include_fields action specifies a whitelist of fields to export. The drop_fields action defines the fields that are dropped if they exist. By default all fields are exported.

The only fields that cannot be dropped via generic filtering are @timestamp and type, because they are required by the outputs.

Both actions can receive in the fields argument full nested maps. For example to keep only the percentages of the cpu load and remove the cpu ticks, the configuration file looks like:

filter:
 - include_fields:
     fields: [“cpu”]
 - drop_fields:
     fields: [“cpu.user”, “cpu.system”]

Authenticate MySQL connection in Metricbeat/Mysql module

This adds authentication support for connecting to the MySQL server in the mysql module of Metricbeat. You can configure the username and password as different options or you can define them in the DSN connection string: [username[:password]@][protocol[(address)]].

Enhancements in Winlogbeat

Few improvements are done in Winlogbeat to provide the data from the event log messages in a structured format. In addition, more information are exported like activity_id, process_id, thread_id and others.

Add support for double in templates

With this pull request double values are also supported in the Elasticsearch templates for Beats.

Remove count field for Filebeat

All Beats are exporting the count field that was meant to be used for sampling, but never used so far. To cleanup the exported fields, this removes the count field from the exported fields in Filebeat.