Setting up logs monitoringedit

To set up logs monitoring, you need:

If your data uses nonstandard fields, you may need to modify some of the default configuration settings.

Get Elasticsearch and Kibanaedit

To get started, you can use our hosted Elasticsearch Service on Elastic Cloud (recommended for new users), or you can install Elasticsearch and Kibana locally.

Use our hosted serviceedit

The hosted Elasticsearch Service is available on both AWS and GCP. Try out the Elasticsearch Service for free.

Install Elasticsearch and Kibana locallyedit

Alternatively, you can install Elasticsearch and Kibana locally. Follow the instructions to install Elasticsearch, and to install and start Kibana.

Install Beats shippersedit

To start collecting logs data, you need to install and configure the Filebeat Beats shipper.

You can install and configure Beats shippers for most kinds of data directly from Kibana, or you can install Beats yourself.

Install Beats from Kibanaedit

To install Beats from Kibana, on the machine where you want to collect the data, open a Kibana browser window. In the Observability section displayed on the home page of Kibana, click Add log data. Now follow the instructions for the type of data you want to collect. The instructions include the steps required to download, install, and configure the appropriate Beats modules for your data.

Install Beats yourselfedit

If your data source doesn’t have a Beats module, or if you want to install Beats the old fashioned way, follow the instructions in Filebeat modules quick start and enable modules for the logs you want to collect. If there is no module for the logs you want to collect, see the Filebeat getting started to learn how to configure inputs.

Enable modulesedit

However you install Beats, you need to enable the appropriate modules in Filebeat to start collecting logs data.

To collect logs from your host system, enable:

To collect logs from Docker containers, enable:

To collect logs from Kubernetes pods, enable:

Configure your data sourcesedit

If your logs data has nonstandard fields, you may need to modify some configuration settings in Kibana, such as the index pattern used to query the data, and the timestamp field used for sorting. To modify configuration settings, use the Settings tab in the Logs app. Alternatively, see logs settings for a complete list of logs configuration settings.

More about container monitoringedit

If you’re monitoring Docker containers or Kubernetes pods, you can use autodiscovery to automatically change the configuration settings in response to changes in your containers. Autodiscovery ensures that even when your container configuration changes, data is still collected. To learn how to do this, see Filebeat autodiscover configuration.