Get up and runningedit

Warning

This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.

To get up and running with infrastructure monitoring, you need:

You might also need to modify UI settings in Kibana to change default behaviors, such as the index pattern used to query the data, and the timestamp field used for sorting. For more information, see Infrastructure UI Settings and Logs UI Settings.

Install Beats shippersedit

To populate the Infrastructure UI with metrics and log data, you need to install and configure the following shippers:

To learn how to install and configure the shippers, do one of the following:

  • Follow the instructions in the Add Data section of the Kibana home page. Click Add log data or Add metrics, and follow the links for the types of data you want to collect.

    add data
  • If your data source isn’t in the list, or you want to install Beats the old fashioned way:

For either approach, you need to enable modules in Filebeat and Metricbeat to populate the Infrastructure UI with data.

Which modules and configuration options do I enable?edit

To populate the Hosts view and add logs, enable:

To populate the Docker view and add logs, enable:

To populate the Kubernetes view and add logs, enable:

Which fields are used for the metrics on the Infrastructure home page?edit

The metrics listed below are provided by the Beats Shippers. Each system type requires their corresponding identity field to be in the same event document:

  • Hosts require host.name
  • Docker containers require container.id
  • Kubernetes pods require kubernetes.pod.uid

For the metrics detail page, event.dataset is a required field. This field is a combination of metricset.module, which is the Metricbeat module name, and metricset.name, which is the sub module name.

Host Metricsedit
CPU Usage
Average of system.cpu.user.pct added to the average of system.cpu.system.pct divided by system.cpu.cores
Memory Usage
Average of system.memory.actual.used.pct
Load
Average of system.load.5
Inbound Traffic
Derivative of the max of system.netowrk.in.bytes scaled to a 1 second rate
Outbound Traffic
Derivative of the max of system.netowrk.out.bytes scaled to a 1 second rate
Log Rate
Derivative of the cumulative sum of the document count scaled to a 1 second rate. This metric relies on the same indices as the logs.
Docker Container Metricsedit
CPU Usage
Average of docker.cpu.total.pct
Memory Usage
Average of docker.memory.usage.pct
Inbound Traffic
Derivative of the max of docker.network.in.bytes scaled to a 1 second rate
Outbound Traffic
Derivative of the max of docker.network.out.bytes scaled to a 1 second rate
Kubernetes Pod Metricsedit
CPU Usage
Average of kubernetes.pod.cpu.usage.node.pct
Memory Usage
Average of kubernetes.pod.memory.usage.node.pct
Inbound Traffic
Derivative of the max of kubernetes.pod.network.rx.bytes scaled to a 1 second rate
Outbound Traffic
Derivative of the max of kubernetes.pod.network.tx.bytes scaled to a 1 second rate

More about container monitoringedit

If you’re monitoring containers, you can use autodiscover to automatically apply configuration changes in response to changes in your containers. To learn how, see:

Known Workaroundsedit

Running Metricbeat ⇐6.5 with Kibana 6.6+
For Kubernetes, you will need to change the "Pod ID" field in the "Configure Source" panel (xpack.infra.sources.default.fields.pod in config/kibana.yml for Kibana 6.6) to kubernetes.pod.name. There is a caveat for this workaround; if you have two pods with the same name, only one will be visible in the UI.