IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
Elastic Docs ›Observability Guide [8.6] ›Application logs
« Application logs ECS logging with Filebeat »

Plaintext logs with Filebeatedit

Use Filebeat to parse and ingest raw, plain-text application logs.

Requirements

  • (Optional) Elastic APM agent for your programming language (for log correlation)
  • Raw, plain-text application logs stored on the file system
  • Filebeat configured to monitor and capture application logs

Pros

  • All programming languages/frameworks are supported
  • Existing application logs can be ingested
  • Does not require modification of the application or its configuration, unless log correlation is required

Cons

  • Must parse application logs to be useful—meaning writing and maintaining Grok patterns and spending CPU cycles on parsing
  • Parsing is tied to the application log format, meaning it can differ per application and needs to be maintained over time
  • Log correlation requires modifying the application log format and inject IDs in log messages

Step 1: Use Filebeat to ingest logsedit

  1. Follow the Filebeat quick start to learn how to install Filebeat and connect to the Elastic Stack.
  2. Configure filebeat.yaml file to start collecting log data.

  3. Add the following configuration to your filebeat.yaml file to start collecting log data.

    filebeat.yaml.

    filebeat.inputs:
- type: filestream 
  paths: /path/to/logs.log

    Reads lines from an active log file.

    A list of glob-based paths that will be crawled and fetched.

Step 2: Parse logs at ingest timeedit

A downside of plaintext logs is that you can’t aggregate or search on the fields within the logs. To enable these features, you’ll need to parse the contents of your logs into ECS-compatible fields.

To learn how to use the Grok processor to parse application logs before indexing, see Example: Parse logs in the Common Log Format.

Step 3: Correlate your logsedit

Correlating your application logs with trace events allows you to:

  • view the context of a log and the parameters a user provided
  • view all logs belonging to a particular trace
  • easily move between logs and traces when debugging application issues in Kibana

Learn more about log correlation in the APM Guide: log correlation, or in any of the agent-specific ingestion guides:

Step 4: View your logs in Kibanaedit

Use the APM or Logs UI to search, filter, and visualize your logs.

Learn moreedit

« Application logs ECS logging with Filebeat »