26 April 2018

Brewing in Beats: Syslog input in Filebeat

By Monica Sarbu

Welcome to Brewing in Beats! With these weekly series, we're keeping you up to date with what's new in Beats, including the latest commits and releases.

Did you know that Beats 6.2 is already available? Try it and let us know what you think. If you are curious to see the Beats in action, check out the Getting Started with Beats webinar.

Filebeat: Syslog Input

PR #6842 adds support for accepting and parsing syslog events via UDP and TCP. Currently the parser supports messages formatted as defined in the RFC 3164. At minimum the date and message field must be available in a message. The parser is a state machine build with ragel and allow to parse RFC3164 events with some less than perfect variants, if the received event is a complete RFC3164 we will extract all of them, for us the minimum valid message MUST have the date and the message defined. Because the parser is a Ragel state machine, we expect quite good performance. The syslog input will be available in 6.3. The limitations of the first version are:

  • Doesn't support multiline events, we need to extract the multiline logic from the log input.
  • Only support RFC3164, RFC5424 will require more work on the parser.

Config Reloading going GA

Config reloading allows to add or remove modules in Filebeat and Metricbeat or prospectors in Filebeat without having to restart the Beat. This is especially useful in dynamic environments where the service to monitor or the logs to monitor change frequently. Config reloading has been around for some time but will now be released as GA in 6.3.

Documentation

Repository: elastic/beats

Changes in master:

  • Fix typo #6895
  • Remove config_dir from reference config and update docs #6884
  • Add CHANGELOG-developer file to cover internal changes #6780
Changes in 6.2:
  • Backport doc fixes for libbeat. #6880
  • Update haproxy.asciidoc #6857
  • Bump docs version for 6.2.4 #6839
Changes in 5.6:
  • Docs version bump 5.6.9 #6855

All changes

Repository: elastic/beats

Metricbeat

Changes in master:

  • Metricbeat: Add reference config and default metricset to kvm module #6900
  • Metricbeat: RabbitMQ/Queue disk reads and writes count keys made optional #6887
  • Index metricset for elasticsearch Metricbeat module #6881
  • Don't stop Metricbeat if aerospike server is down #6874
  • Metricbeat: Add mapping for docker metrics per cpu #6843
  • Collect domain memory stat from KVM #6265
Packetbeat

Changes in master:

  • http: Report unmatched requests or responses #6794
Filebeat

Changes in master:

  • Update Filebeat hints to also support modules #6899
  • Make config reloading GA #6891
  • UDP inputs now uses human friendly size to define MaxMessageSize #6886
  • Syslog inputs parses RFC3164 events via TCP or UDP #6842
Auditbeat

Changes in master:

  • Auditbeat recursive file watches for Windows #6893
Testing

Changes in master:

  • Reduce log level of urllib3 in metricbeat tests #6909
  • Update Elastic Stack versions for testing #6898
  • Remove flush interval from Kafka run script #6831