Elastic Docs› Observability Guide [8.6]› Application logs
« ECS logging with Filebeat APM agent log sending »

APM agent ECS reformatting with Filebeatedit

Elastic APM agents can automatically reformat application logs to Elastic Common Schema (ECS) format without needing to add an ECS logger dependency or modify the application.

Requirements

  • The Elastic APM agent for your programming language
  • Filebeat configured to monitor and capture application logs

Pros

All the benefits of using ECS logging, without having to modify the application or its configuration:

  • Simplicity: no manual parsing with Filebeat, and a configuration can be reused across applications
  • Decently human-readable JSON structure
  • APM log correlation

Cons

  • Requires an Elastic APM agent
  • Not all APM agents support this feature

Supported APM agents/languages

  • Ruby
  • Java

Step 1: Enable APM agent reformattingedit

Enable APM agent ECS reformatting in the supported APM agent:

See the Ruby or Java agent docs.

Step 2: Set up Filebeatedit

  1. Follow the Filebeat quick start to learn how to install Filebeat and connect to the Elastic Stack.

  2. Add the following configuration to your filebeat.yaml file to start collecting log data.

    filebeat.yaml.

    filebeat.inputs:
- type: filestream 
  paths: /path/to/logs.json
  parsers:
    - ndjson:
      keys_under_root: true
      overwrite_keys: true
      add_error_key: true
      expand_keys: true

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

    Use the filestream input to read lines from active log files.
« ECS logging with Filebeat APM agent log sending »