Now, to indicate that the new
filestream is supposed to take over the files from a previously
log input, we need to add
take_over: true to each new
filestream. This will make sure
that the new
filestream inputs will continue ingesting files from the same offset where the
It’s recommended to enable debug-level logs for Filebeat in order to follow the migration process.
After the first run with
take_over: true the setting can be removed.
take over mode is in beta.
If this parameter is not set, all the files will be re-ingested from the beginning and this will lead to data duplication. Please, double-check that this parameter is set.
logging: level: debug filebeat.inputs: - type: filestream enabled: true id: my-java-collector take_over: true paths: - /var/log/java-exceptions*.log - type: filestream enabled: true id: my-application-input take_over: true paths: - /var/log/my-application*.json - type: filestream enabled: true id: my-old-files take_over: true paths: - /var/log/my-old-files*.log