18 January 2019

Brewing in Beats: Adding the add_labels and add_tags processors

By Monica Sarbu

Welcome to Brewing in Beats! With these weekly series, we're keeping you up to date with what's new in Beats, including the latest commits and releases.

Did you know that Beats 6.5 is already available? Try it and let us know what you think.

What's new in Beats

Escaping HTML disabled

In the past, we introduced support for disabling HTML escaping in the outputs JSON encoders. With #9914 and the upcoming Beats 7 releases, we will not escape HTML like symbols anymore by default.

New processors: add_labels, add_tags

In Beats, users can already add fields and tags to events using the global or per module local   fields, and tags settings. We recently introduced an add_labels, and add_tags processor (#9973) allowing you to add fields and tags via processors as well. Using these processors we can add fields and tags based on conditions and event contents.

What's new in Stack Monitoring

Repository: elastic/kibana

  • Addresses issues with filtering tables in the new EUI monitoring tables. #27504
  • Rename the Monitoring app in Kibana to Stack Monitoring #28102
  • Ensure all monitoring charts respond to onBrush [bugfix] #28098
  • Fixing issue with EUI table filtering in monitoring UI [bugfix] #27504

Repository: elastic/beats

  • Refactoring: Using common.Version instead of strings #9900
  • Refactoring: Elasticsearch metricbeat module #9896
  • Error handling for multiple pipelines with < ES 6.5 #10001
  • Make watcher threadpool stats optional #9909
  • Allow users to convert timezone in elasticsearch module filesets #9761

What's new in Beats Central Management

Repository: elastic/kibana

Changes in master:

  • propigate errors on index creation fail #28541

What's new with ECS migration

Packetbeat

Changes in master:

  • [Packetbeat] Update DNS protocol to use ECS fields #9941

Filebeat

Changes in master:

  • Convert Filebeat mongodb.log to ECS #10009
  • Convert Filebeat mysql.* to ECS #10008
  • Convert Filebeat nginx.error to ECS #10007
  • Fix apache fields config for ECS migration #9960
  • Convert Filebeat iis.error to ECS and add IPv6 zone support for IIS access logs #9955
  • Convert Filebeat logstash.* to ECS #9935
  • Update event.dataset to contain module and fileset name #9922
  • Convert Filebeat kibana.log to ECS #9301
  • Fixes parsing of GC entries in elasticsearch server log #9603

Changes in 6.x:

  • Parameterizing Painless script literals #9770
  • Allow users to convert timezone in elasticsearch module filesets #9761
  • Fixes parsing of GC entries in elasticsearch server log #9603

Other changes

Repository: elastic/beats

Affecting all Beats

Changes in 6.6:

  • Do not configure aliases in unsupported Elasticsearch versions #9992

Changes in master:

  • Do not configure aliases in unsupported Elasticsearch versions #9992
  • Fix asset registry slice generation #9986
  • Introduce add_labels and add_tags processors #9973
  • Rename process.exe to process.executable for ECS #9949
  • Rename host.name to host.hostname and add config option for name #9943
  • Pretty print exported dashboards to output #9925
  • Do not escape HTML by default anymore #9914
  • Fix config appender registration #9873
  • Allow multiple object types per field. #9772

Changes in 6.x:

  • Do not configure aliases in unsupported Elasticsearch versions #9992
  • Fix config appender registration #9873
Filebeat

Changes in 6.x:

  • Parameterizing Painless script literals #9770
Auditbeat

Changes in 6.6:

  • [Auditbeat] Sort IPv4 addresses before IPv6 #9953
  • [Auditbeat] Add Windows to docs for host dataset #9954

Changes in master:

  • [Auditbeat] Add Windows to docs for host dataset #9954
  • [Auditbeat] Sort IPv4 addresses before IPv6 #9953

Changes in 6.x:

  • [Auditbeat] Sort IPv4 addresses before IPv6 #9953
  • [Auditbeat] Add Windows to docs for host dataset #9954
Testing

Changes in 6.5:

  • Add system tests for Central Management #8762

Changes in master:

  • Fix and reenable ml tests #9942

Changes in 6.x:

  • Minor fixes to error handling #9888
Documentation

Changes in master:

  • [Docs] Remove duplicate metricset pages #9940
  • Update kubernetes.asciidoc #9282
  • [Docs] Add topic about running Journalbeat on docker #9386

Changes in 6.x:

  • Minor grammatical fixes to libbeats configuration file #9511
  • Update kubernetes.asciidoc #9282

Repository: elastic/go-sysinfo

Changes in master:

  • Add Process.PID() #37
  • Implement Processes() for Darwin #35

Repository: elastic/ecs

Changes in master:

  • Update PyYAML to 4.2b1 #298
  • Generate process.args as []string in Go code #296