Use Filebeat to parse and ingest raw, plain-text application logs.
- (Optional) Elastic APM agent for your programming language (for log correlation)
- Raw, plain-text application logs stored on the file system
- Filebeat configured to monitor and capture application logs
- All programming languages/frameworks are supported
- Existing application logs can be ingested
- Does not require modification of the application or its configuration, unless log correlation is required
- Must parse application logs to be useful—meaning writing and maintaining Grok patterns and spending CPU cycles on parsing
- Parsing is tied to the application log format, meaning it can differ per application and needs to be maintained over time
- Log correlation requires modifying the application log format and inject IDs in log messages
Step 1: Use Filebeat to ingest logsedit
- Follow the Filebeat quick start to learn how to install Filebeat and connect to the Elastic Stack.
- Configure filebeat.yaml file to start collecting log data.
Add the following configuration to your
filebeat.yamlfile to start collecting log data.
Step 2: Parse logs at ingest timeedit
A downside of plaintext logs is that you can’t aggregate or search on the fields within the logs. To enable these features, you’ll need to parse the contents of your logs into ECS-compatible fields.
To learn how to use the Grok processor to parse application logs before indexing, see Example: Parse logs in the Common Log Format.