Brewing in Beats: Introducing subcommands

Por
Monica Sarbu

Welcome to Brewing in Beats! With this weekly series, we're keeping you up to date with what's new in Beats, including the latest commits and releases.

Subcommands

This PR introduces subcommands to the command line of the Beats, based on the Cobra library. The benefit of subcommands is that they allow for specific flags for a given command, and also allow for one-time maintenance commands.

Examples: 

$ filebeat help
$ filebeat help setup
$ filebeat version
$ filebeat run -h
$ filebeat run -e  --modules=nginx
$ filebeat setup -e --modules=nginx

The run command is also set as the root command, meaning that filebeat run -e is equivalent with filebeat -e, so backwards compatibility is kept.

The first real use case for subcommands is that we’re replacing the import_dashboards program with the setup command. This allows us to remove the import_dashboards program from the packages, significantly reducing their size.

In the future we plan to have more commands, for example, for listing / enabling / disabling modules.

The subcommands will be introduced in 6.0.

Add TTL option to the Logstash output

Thanks to this PR, by @andrew-d, the Logstash output can be configured with a time-to-live (TTL) option. When the TTL expires, the connection is closed and a new one created. This allows for a better distribution to Logstash instances behind a load balancer.

Since connections from Filebeat to Logstash are sticky, when an instance joins the load balancer, it does not get an equal distribution. For example, if there are 4 instances behind a load balancer and 3 of them are rebooted, then all Filebeat connections will go to the single instance that was not rebooted. By specifying a TTL on the connection, there is an opportunity for the load balancer to distribute connections equally between the instances.

This is currently scheduled to be released with 6.0.

Other changes

Repository: elastic/beats

Affecting all Beats

Changes in master:

  • Updating testing versions for Elastic stack #4485
  • Fix template for output filename and path #4475
  • Move schema package to metricbeat #4466
  • Move libbeat filebeat dependency to libbeat #4465
  • Add a permissions option to logging.files for all beats #4428

Changes in 5.5:

  • Remove empty properties from the template files #4488
Packetbeat

Changes in master:

  • Fixed socket scanning for tcp6 #4442
Filebeat

Changes in 5.x:

  • Fix panic on empty multiline pattern #4377

Changes in master:

  • Load Filebeat modules pipelines on connect #4479
  • Fix redis prospector default type #4476
Documentation

Changes in master:

  • Bump docs version to 6.0.0-alpha2 #4456
  • Close changelog for 6.0.0-alpha2 #4455
  • Misc doc fixes #4451
  • Add add_docker_metadata processor docs #4448
  • Document complex object support in env vars #4200

Changes in 5.4:

  • Backport: Doc changes for #4451 and #4300 into 5.4 #4480
  • Add docs for Docker images #4312,
  • Provide more detail on log levels #4405,
  • Add simple examples that illustrate multiline settings #4408

Changes in 5.5:

  • Backport: Doc changes for #4451 and #4300 into 5.5 #4481
Infrastructure

Changes in master:

  • Add BSD license types in the NOTICE file #4470
  • Fix command line template generation #4463

Repository: elastic/gosigar

Changes in master:

  • Fix hang during init on Windows if WMI is unavailable #74