Overview

At the high level, a simple Beat has two main components:

  • a component that collects the actual data, and
  • a publisher that sends the data to the specified output, such as Elasticsearch or Logstash.

The publisher is already implemented in libbeat, so you typically only have to worry about the logic specific to your Beat (the code that creates the event and sends it to the publisher). Libbeat also offers common services like configuration management, logging, daemonzing, and Windows service handling, and data processing modules.

Beat overview architecture

The event that you create is a JSON-like object (Golang type map[string]interface{}) that contains the collected data to send to the publisher. At a minimum, the event object must contain a @timestamp field and a type field. Beyond that, events can contain any additional fields, and they can be created as often as necessary.

The following example shows an event object in Lsbeat:

{
  "@timestamp": "2016-07-13T21:33:58.355Z",
  "beat": {
    "hostname": "mar.local",
    "name": "mar.local"
  },
  "directory": false,
  "filename": "winlogbeat.yml",
  "filesize": 2895,
  "modtime": "2016-07-13T20:56:21.000Z",
  "path": "./vendor/github.com/elastic/beats/winlogbeat/winlogbeat.yml",
  "type": "lsbeat"
}

{
  "@timestamp": "2016-07-13T21:33:58.354Z",
  "beat": {
    "hostname": "mar.local",
    "name": "mar.local"
  },
  "directory": true,
  "filename": "system",
  "filesize": 238,
  "modtime": "2016-07-13T20:56:21.000Z",
  "path": "./vendor/github.com/elastic/beats/winlogbeat/tests/system",
  "type": "lsbeat"
}

Now that you have the big picture, let’s dig into the code.