Configuring Flows to Monitor Network Trafficedit

You can configure Packetbeat to collect and report statistics on network flows. A flow is a group of packets sent over the same time period that share common properties, such as the same source and destination address and protocol. You can use this feature to analyze network traffic over specific protocols on your network.

For each flow, Packetbeat reports the number of packets and the total number of bytes sent from the source to the destination. Each flow event also contains information about the source and destination hosts, such as their IP address. For bi-directional flows, Packetbeat reports statistics for the reverse flow.

Packetbeat collects and reports statistics up to and including the transport layer. See Flow Event Fields for more info about the exported data.

Here’s an example of flow events visualized in the Flows dashboard:

./images/flows.png

To configure flows, use the flows option in the packetbeat.yml config file. Flows are enabled by default.

packetbeat.flows:
  timeout: 30s
  period: 10s

See Flows for more info about these configuration options.

Here’s an example of a flow information sent by Packetbeat. See Flow Event Fields for a description of each field.

{
  "@timestamp": "2017-05-03T19:42:40.003Z",
  "beat": {
    "hostname": "host.example.com",
    "name": "host.example.com",
    "version": "5.5.0"
  },
  "connection_id": "AQAAAAAAAAA=",
  "dest": {
    "ip": "192.0.2.0",
    "mac": "fe:ff:20:00:01:00",
    "port": 80,
    "stats": {
      "net_bytes_total": 19236,
      "net_packets_total": 16
    }
  },
  "final": false, 
  "flow_id": "EQwA////DP//////FBgBAAEAAAEAAAD+/yAAAQCR/qDtQdDk3ywNUAABAAAAAAAAAA",
  "last_time": "2017-05-03T19:42:24.151Z",
  "source": {
    "ip": "203.0.113.0",
    "mac": "00:00:01:00:00:00",
    "port": 3372,
    "stats": {
      "net_bytes_total": 1243,
      "net_packets_total": 14
    }
  },
  "start_time": "2017-05-03T19:42:24.151Z",
  "transport": "tcp",
  "type": "flow"
}

Packetbeat sets the final flag to false to indicate that the event contains an intermediate report about a flow that it’s tracking. When the flow completes, Packetbeat sends one last event with final set to true. If you want to aggregate sums of traffic, you need to filter on final:true, or use some other technique, so that you get only the latest update from each flow. You can disable intermediate reports by setting period: -1s.