When you need to collect lines from multiple files, you can simply configure a single prospector and specify multiple
paths to start a harvester for each file. However, if you want to apply additional prospector-specific
configuration settings (such as
multiline, and so on)
to the lines harvested from specific files, you need to define multiple prospectors in the Filebeat config file.
Within the config file, you can specify multiple prospectors, and each prospector can define multiple paths to crawl, as shown in the following example.
Make sure a file is not defined more than once across all prospectors because this can lead to unexpected behaviour.
filebeat.prospectors: - input_type: log paths: - /var/log/system.log - /var/log/wifi.log - input_type: log paths: - "/var/log/apache2/*" fields: apache: true fields_under_root: true
The config file in the example starts two prospectors (the list of prospectors is a YAML
array, so each prospector begins with a
-). The first prospector has two harvesters,
one harvesting the
system.log file, and the other harvesting
wifi.log. The second prospector
starts a harvester for each file in the
apache2 directory and uses the
option to add a field called
apache to the output.