Brewing in Beats: Kubernetes deployment files

By
Monica Sarbu

Welcome to Brewing in Beats! With this weekly series, we're keeping you up to date with what's new in Beats, including the latest commits and releases.

Kubernetes deployment files

We are making it easier to deploy Filebeat and Metricbeat 6.0 on Kubernetes by providing deployment manifest files. You can find more details about deploying Filebeat and Metricbeat on Kubernetes in the docs, but it can be summarized as: 

 
 curl -L -O https://raw.githubusercontent.com/elastic/beats/6.0/deploy/kubernetes/filebeat-kubernetes.yaml
 # edit the YAML file to set the Elasticsearch connection information
 kubectl create -f filebeat-kubernetes.yaml

The above commands install Filebeat as a DaemonSet, ensuring one agent is running on each Kubernetes node, and configure it to pick up the logs from /var/lib/docker/containers/*, unwrap them from the JSON objects, automatically enhance them with Kubernetes metadata (pod names, labels, etc.), and ship them to Elasticsearch. Metricbeat follows a similar pattern.

In progress: Filebeat modules for Elasticsearch and Logstash

We have work in progress PRs for adding Filebeat modules for Elasticsearch and Logstash. Kibana is coming next. This is part of making the Elastic Stack easier to monitor with the Elastic Stack.

Both modules are planned to be released with 6.1.

Thanks Hacktoberfest

We are pleasantly surprised to see how many of you participated to Beats as part of Hacktoberfest this year, and we would like to thank the following contributors:

We are looking forward for the next Hacktoberfest.

Other changes

Repository: elastic/beats

Affecting all Beats

Changes in master:

  • Fix data race in mockbeat (#5037) #5426
  • Fix tags type to []interface{} in event (#5389) #5395

Changes in 5.6:

  • Fix race condition in logging #4519
  • Add a permissions option to logging.files for all beats #4428
Metricbeat

Changes in master:

  • Fix VSphere config yml #5416
  • Fixing mapping error for mlockall in the Metricbeat Elasticsearch module #5406
  • Changed MySQL active connections viz title to MySQL total connections #5401

Changes in 6.0:

  • Fix VSphere config yml #5416
Packetbeat

Changes in master:

  • Fix missing length check in PgSQL #5457

Changes in 5.6:

  • Fix missing length check in PgSQL #5457
Filebeat

Changes in master:

  • Fix double @timestamp key when using JSON decoding #5436

Changes in 6.0:

  • Fix double @timestamp key when using JSON decoding #5436
Testing

Changes in master:

  • Catch duplicate keys in integration tests #5455
  • Make python tests not executables #5449
  • Fix sha512 check #5420
  • Fix chown timing issues on k8s deploy #5410

Changes in 6.0:

Packaging

Changes in master:

  • Remove kubernetes manifests from packer #5485

Changes in 6.0:

  • Remove kubernetes manifests from packer #5485
Documentation

Changes in master:

  • Fix typo of split_cookie option packetbeat #5474
  • Parameterize Docker Github URL #5460
  • Update screen captures in Filebeat docs #5456
  • Publish kubernetes manifests in the code #5418
  • Add kubernetes manifests #5349
  • Add more context and examples to Filebeat module docs #5341
  • Fix getting started #5326

Changes in 6.0:

  • Backport to 6.0: #5327, #5393, #5270, #5341, #5456 #5445
  • Add kubernetes manifests #5349