Get startededit

Step 1: Configure application loggingedit

If you are using the Elastic APM Java agent, the easiest way to transform your logs into ECS-compatible JSON format is through the log_ecs_reformatting configuration option. By only setting this option, the Java agent will automatically import the correct ECS-logging library and configure your logging framework to use it instead (OVERRIDE/REPLACE) or in addition to (SHADE) your current configuration. No other changes required! Make sure to check out other Logging configuration options to unlock the full potential of this option.

Otherwise, follow the steps below to manually apply ECS-formatting through your logging framework configuration. The following logging frameworks are supported:

  • Logback (default for Spring Boot)
  • Log4j2
  • Log4j
  • java.util.logging (JUL)
  • JBoss Log Manager

Add the dependencyedit

The minimum required logback version is 1.1.

Download the latest version of Elastic logging: Maven Central

Add a dependency to your application:


If you are not using a dependency management tool, like maven, you have to manually add both logback-ecs-encoder and ecs-logging-core jars to the classpath. For example to the $CATALINA_HOME/lib directory. Other than that, there are no required dependencies.

Use the ECS encoder/formatter/layoutedit

Spring Boot applications

In src/main/resources/logback-spring.xml:

<?xml version="1.0" encoding="UTF-8"?>
    <property name="LOG_FILE" value="${LOG_FILE:-${LOG_PATH:-${LOG_TEMP:-${}}}/spring.log}"/>
    <include resource="org/springframework/boot/logging/logback/defaults.xml"/>
    <include resource="org/springframework/boot/logging/logback/console-appender.xml" />
    <include resource="org/springframework/boot/logging/logback/file-appender.xml" />
    <include resource="co/elastic/logging/logback/boot/ecs-console-appender.xml" />
    <include resource="co/elastic/logging/logback/boot/ecs-file-appender.xml" />
    <root level="INFO">
        <appender-ref ref="ECS_JSON_CONSOLE"/>
        <appender-ref ref="CONSOLE"/>
        <appender-ref ref="ECS_JSON_FILE"/>
        <appender-ref ref="FILE"/>

You also need to configure the following properties to your
# for Spring Boot 2.2.x+
# for older Spring Boot versions

Other applications

All you have to do is to use the co.elastic.logging.logback.EcsEncoder instead of the default pattern encoder in logback.xml

<encoder class="co.elastic.logging.logback.EcsEncoder">

Encoder Parameters

Parameter name Type Default Description



Sets the field so you can filter your logs by a particular service name



Sets the service.version field so you can filter your logs by a particular service version



Sets the service.environment field so you can filter your logs by a particular service environment



Sets the field so you can filter your logs by a particular node of your clustered service




Sets the event.dataset field used by the machine learning job of the Logs app to look for anomalies in the log rate.




Log Markers as tags




Serializes the error.stack_trace as a JSON array where each element is in a new line to improve readability. Note that this requires a slightly more complex Filebeat configuration.




If true, adds the, log.origin.file.line and log.origin.function fields. Note that you also have to set <includeCallerData>true</includeCallerData> on your appenders if you are using the async ones.

To include any custom field in the output, use following syntax:


If you’re using the Elastic APM Java agent, log correlation is enabled by default starting in version 1.30.0. In previous versions, log correlation is off by default, but can be enabled by setting the enable_log_correlation config to true.

Step 2: Configure Filebeatedit

  1. Follow the Filebeat quick start
  2. Add the following configuration to your filebeat.yaml file.

For Filebeat 7.16+


- type: filestream 
  paths: /path/to/logs.json
    - ndjson:
      overwrite_keys: true 
      add_error_key: true 
      expand_keys: true 

  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

Use the filestream input to read lines from active log files.

Values from the decoded JSON object overwrite the fields that Filebeat normally adds (type, source, offset, etc.) in case of conflicts.

Filebeat adds an "error.message" and "error.type: json" key in case of JSON unmarshalling errors.

Filebeat will recursively de-dot keys in the decoded JSON, and expand them into a hierarchical object structure.

Processors enhance your data. See processors to learn more.

For Filebeat < 7.16


- type: log
  paths: /path/to/logs.json
  json.keys_under_root: true
  json.overwrite_keys: true
  json.add_error_key: true
  json.expand_keys: true

- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~

For more information, see the Filebeat reference.

When stackTraceAsArray is enablededit

Filebeat can normally only decode JSON if there is one JSON object per line. When stackTraceAsArray is enabled, there will be a new line for each stack trace element which improves readability. But when combining the multiline settings with a decode_json_fields we can also handle multi-line JSON:

  - type: log
    paths: /path/to/logs.json
    multiline.pattern: '^{'
    multiline.negate: true
    multiline.match: after
  - decode_json_fields:
      fields: message
      target: ""
      overwrite_keys: true
  # flattens the array to a single string
  - script:
        has_fields: ['error.stack_trace']
      lang: javascript
      id: my_filter
      source: >
        function process(event) {
            event.Put("error.stack_trace", event.Get("error.stack_trace").join("\n"));