Migrate log input configurations to filestream

The filestream input has been generally available since 7.14. So it is high time you migrate your existing log input configurations. The filestream input comes with many improvements over the old input, such as configurable order for parsers and more.

While we do not plan to remove the log input from Filebeat, we are not fixing new issues or adding any enhancements to the input. Our focus is on filestream.

In this guide, you learn how to migrate an existing log input configuration. The following example shows three log inputs:

filebeat.inputs:
 - type: log
   enabled: true
   paths:
     - /var/log/java-exceptions*.log
   multiline:
    pattern: '^\['
    negate: true
    match: after
  close_removed: true
  close_renamed: true

- type: log
  enabled: true
  paths:
    - /var/log/my-application*.json
  scan_frequency: 1m
  json.keys_under_root: true

- type: log
  enabled: true
  paths:
    - /var/log/my-old-files*.log
  tail_files: true

For this example, let’s assume that the log input is used to collect logs from the following files. The progress of data collection is shown for each file.

/var/log/java-exceptions1.log (100%)
/var/log/java-exceptions2.log (100%)
/var/log/java-exceptions3.log (75%)
/var/log/java-exceptions4.log (0%)
/var/log/java-exceptions5.log (0%)
/var/log/my-application1.json (100%)
/var/log/my-application2.json (5%)
/var/log/my-application3.json (0%)
/var/log/my-old-files1.json (0%)