Setting Traffic Capturing Optionsedit

There are two main ways of deploying Packetbeat:

  • On dedicated servers, getting the traffic from mirror ports or tap devices.
  • On your existing application servers.

The first option has the big advantage that there is no overhead of any kind on your application servers. But it requires dedicated networking gear, which is generally not available on cloud setups.

In both cases, the sniffing performance (reading packets passively from the network) is very important. In the case of a dedicated server, better sniffing performance means that less hardware is required. When Packetbeat is installed on an existing application server, better sniffing performance means less overhead.

Currently Packetbeat has several options for traffic capturing:

  • pcap, which uses the libpcap library and works on most platforms, but it’s not the fastest option.
  • af_packet, which uses memory mapped sniffing. This option is faster than libpcap and doesn’t require a kernel module, but it’s Linux-specific.
  • pf_ring, which makes use of an ntop.org project. This setting provides the best sniffing speed, but it requires recompilation and a kernel module, and it’s Linux-specific.

The pf_ring option provides sniffing speeds in the order of Gigabits per second using only standard hardware, but it requires you to recompile Packetbeat against the ntop libraries and is therefore currently not officially supported by Elastic.

The af_packet option, also known as "memory-mapped sniffing," makes use of a Linux-specific feature. This could be the optimal sniffing mode for both the dedicated server and when Packetbeat is deployed on an existing application server.

The way it works is that both the kernel and the user space program map the same memory zone, and a simple circular buffer is organized in this memory zone. The kernel writes packets into the circular buffer, and the user space program reads from it. The poll system call is used for getting a notification for the first packet available, but the remaining available packets can be simply read via memory access.

This option is not quite as fast as pf_ring (it works to up to 200k packets per second before dropping packets), but it requires no recompilation and no kernel modules, and it’s still a significant improvement over libpcap.

The af_packet sniffer can be further tuned to use more memory in exchange for better performance. The larger the size of the circular buffer, the fewer system calls are needed, which means that fewer CPU cycles are consumed. The default size of the buffer is 30 MB, but you can increase it like this:

packetbeat.interfaces.device: eth0
packetbeat.interfaces.type: af_packet
packetbeat.interfaces.buffer_size_mb: 100

Please see the Network Device (Interfaces) section for more configuration options.