ECS Logging with Morganedit

This Node.js package provides a formatter for the morgan logging middleware — commonly used with Express — compatible with Elastic Common Schema (ECS) logging. In combination with the Filebeat shipper, you can monitor all your logs in one place in the Elastic Stack.

Setupedit

Step 1: Installedit

$ npm install @elastic/ecs-morgan-format

Step 2: Configureedit

const app = require('express')()
const morgan = require('morgan')
const ecsFormat = require('@elastic/ecs-morgan-format')

app.use(morgan(ecsFormat())) 

// ...
app.get('/', function (req, res) {
  res.send('hello, world!')
})
app.listen(3000)

Pass the ECS formatter to morgan().

Step 3: Configure Filebeatedit

The best way to collect the logs once they are ECS-formatted is with Filebeat:

Follow the Filebeat quick start
Add the following configuration to your filebeat.yaml file.

For Filebeat 7.16+

filebeat.yaml.

filebeat.inputs:
- type: filestream
  paths: /path/to/logs.json
  parsers:
    - ndjson:
        keys_under_root: true
        overwrite_keys: true
        add_error_key: true
        expand_keys: true

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

For Filebeat < 7.16

filebeat.yaml.

filebeat.inputs:
- type: log
  paths: /path/to/logs.json
  json.keys_under_root: true
  json.overwrite_keys: true
  json.add_error_key: true
  json.expand_keys: true

processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~

Make sure your application logs to stdout/stderr.
Follow the Run Filebeat on Kubernetes guide.
Enable hints-based autodiscover (uncomment the corresponding section in filebeat-kubernetes.yaml).
Add these annotations to your pods that log using ECS loggers. This will make sure the logs are parsed appropriately.

annotations:
  co.elastic.logs/json.keys_under_root: true
  co.elastic.logs/json.overwrite_keys: true
  co.elastic.logs/json.add_error_key: true
  co.elastic.logs/json.expand_keys: true

Make sure your application logs to stdout/stderr.
Follow the Run Filebeat on Docker guide.
Enable hints-based autodiscover.
Add these labels to your containers that log using ECS loggers. This will make sure the logs are parsed appropriately.

docker-compose.yml.

labels:
  co.elastic.logs/json.keys_under_root: true
  co.elastic.logs/json.overwrite_keys: true
  co.elastic.logs/json.add_error_key: true
  co.elastic.logs/json.expand_keys: true

For more information, see the Filebeat reference.

Usageedit

const app = require('express')()
const morgan = require('morgan')
const ecsFormat = require('@elastic/ecs-morgan-format')

app.use(morgan(ecsFormat()))

app.get('/', function (req, res) {
  res.send('hello, world!')
})
app.get('/error', function (req, res, next) {
  next(new Error('boom'))
})

app.listen(3000)

Running this script (the full example is here) and making a request (via curl -i localhost:3000/) will produce log output similar to the following:

% node examples/express.js | jq .  # piping to jq for pretty-printing
{
  "@timestamp": "2021-01-16T00:03:23.279Z",
  "log.level": "info",
  "message": "::1 - - [16/Jan/2021:00:03:23 +0000] \"GET / HTTP/1.1\" 200 13 \"-\" \"curl/7.64.1\"",
  "ecs": {
    "version": "1.6.0"
  },
  "http": {
    "version": "1.1",
    "request": {
      "method": "GET",
      "headers": {
        "host": "localhost:3000",
        "accept": "*/*"
      }
    },
    "response": {
      "status_code": 200,
      "headers": {
        "x-powered-by": "Express",
        "content-type": "text/html; charset=utf-8",
        "etag": "W/\"d-HwnTDHB9U/PRbFMN1z1wps51lqk\""
      },
      "body": {
        "bytes": 13
      }
    }
  },
  "url": {
    "path": "/",
    "domain": "localhost",
    "full": "http://localhost:3000/"
  },
  "user_agent": {
    "original": "curl/7.64.1"
  }
}

Format Optionsedit

You can pass any format argument you would normally pass to morgan(), and the log "message" field will use the specified format. The default is combined.

const app = require('express')()
const morgan = require('morgan')
const ecsFormat = require('@elastic/ecs-morgan-format')

app.use(morgan(ecsFormat({ format: 'tiny' }))) 
// ...

If "format" is the only option you are using, you may pass it as ecsFormat('tiny').

log.leveledit

The log.level field will be "error" for response codes >= 500, otherwise "info". For example, running examples/express.js again, a curl -i localhost:3000/error will yield:

% node examples/express.js | jq .
{
  "@timestamp": "2021-01-18T17:52:12.810Z",
  "log.level": "error",
  "message": "::1 - - [18/Jan/2021:17:52:12 +0000] \"GET /error HTTP/1.1\" 500 1416 \"-\" \"curl/7.64.1\"",
  "http": {
    "response": {
      "status_code": 500,
  ...

Integration with APM Tracingedit

This ECS log formatter integrates with Elastic APM tracing. If your Node app is using the Node.js Elastic APM Agent, then fields are added to log records that identify an active trace and the configured service name ("service.name" and "event.dataset"). These fields allow cross linking between traces and logs in Kibana and support log anomaly detection.

For example, running examples/express-with-apm.js and curl -i localhost:3000/ results in a log record with the following:

% node examples/express-with-apm.js | jq .
{
  ...
  "event": {
    "dataset": "express-with-elastic-apm.log"
  },
  "trace": {
    "id": "e097193afa9ac221017b45a1f674601c"
  },
  "transaction": {
    "id": "c6aa5b47e01bad72"
  },
  "service": {
    "name": "express-with-elastic-apm"
  }
}

These IDs match trace data reported by the APM agent.

Integration with Elastic APM can be explicitly disabled via the apmIntegration: false option, for example:

app.use(morgan(ecsFormat({ apmIntegration: false })))

« Introduction ECS Logging with Pino »