Brewing in Beats: Conditional Filtering
Welcome to Brewing in Beats! With this series, we're keeping you up to date with all that's new in Beats, from the details of work in progress pull requests to releases and learning resources.
Last week we have released Beats 1.2.3 containing a few important bug fixes. Besides that, we have worked on a set of improvements that we plan to release with 5.0.0-alpha3:
Conditionals for generic filtering
It is now possible to use conditionals when filtering out fields and events in libbeat. This makes it really easy to remove unneeded data even before sending it over the network. Here is an example filter that drops the HTTP header fields if the code is 200 and the status is OK:
- drop_fields: fields: ["http.request_headers", "http.response_headers"] equals: http.code: 200 status: OK
Metricbeat gets an Nginx module
As anticipated last week, the work has started to normalize the metric names so that users get an uniform experience regardless of the system they monitor. For example, here is the PR for Redis and here is the one for Apache.
Other Metricbeat improvements include:
- Add connection timeout to Redis Module.
- The Dockerfiles use for testing now have a standard place in each module folder.
- The list of imported modules is now automatized.
- The system module is now able to filter processes by regexps, similar with Topbeat.
- The metricset fields are grouped together.
The work on reorganizing the options in our configuration files is almost finished. We now distribute two versions of the configuration files: the default one which contains only the most common options and a “full” one, that contains all non-deprecated options with longer comments. The full versions can be almost used as a reference and it is easy to copy and paste between them.
Another configuration related improvement is that all duration options now accept the same specifiers (e.g 10s, 1ms). This change was done in a backwards compatible way.
We now produce statically linked binaries for Filebeat and Winlogbeat (which don't use CGo) by using simple cross compilation, without Cgo enabled. The advantage of the statically linked binaries is that they tend to be more portable. The Beats that use CGo continue to be dynamically linked against libc and not much else.
We have updated our artifact naming to be consistent with the other Elastic stack projects.