Configuring monitoring in Elasticsearchedit

By default, X-Pack monitoring is enabled but data collection is disabled. Advanced monitoring settings enable you to control how frequently data is collected, configure timeouts, and set the retention period for locally-stored monitoring indices. You can also adjust how monitoring data is displayed.

  1. To collect monitoring data about your Elasticsearch cluster:

    1. Verify that the xpack.monitoring.enabled and xpack.monitoring.collection.enabled settings are true on each node in the cluster. By default, data collection is disabled. For more information, see Monitoring Settings.
    2. Optional: Specify which indices you want to monitor.

      By default, the monitoring agent collects data from all Elasticsearch indices. To collect data from particular indices, configure the xpack.monitoring.collection.indices setting. You can specify multiple indices as a comma-separated list or use an index pattern to match multiple indices. For example:

      xpack.monitoring.collection.indices: logstash-*, index1, test2

      You can prepend + or - to explicitly include or exclude index names or patterns. For example, to include all indices that start with test except test3, you could specify +test*,-test3.

    3. Optional: Specify how often to collect monitoring data. The default value for the xpack.monitoring.collection.interval setting 10 seconds. See Monitoring Settings.
  2. Optional: Configure your cluster to route monitoring data from sources such as Kibana, Beats, and Logstash to a monitoring cluster:

    1. Verify that xpack.monitoring.collection.enabled settings are true on each node in the cluster.
    2. Configure X-Pack monitoring across the Elastic Stack.
  3. Identify where to store monitoring data.

    By default, X-Pack monitoring uses a local exporter that indexes monitoring data on the same cluster. See Default exportersedit and Local Exporters.

    Alternatively, you can use an http exporter to send data to a separate monitoring cluster. See HTTP Exporters.

    For more information about typical monitoring architectures, see How Monitoring Works.

  4. If X-Pack security is enabled and you are using an http exporter to send data to a dedicated monitoring cluster:

    1. Create a user on the monitoring cluster that has the remote_monitoring_agent built-in role. For example, the following request creates a remote_monitor user that has the remote_monitoring_agent role:

      POST /_xpack/security/user/remote_monitor
      {
        "password" : "changeme",
        "roles" : [ "remote_monitoring_agent"],
        "full_name" : "Internal Agent For Remote Monitoring"
      }
    2. On each node in the cluster that is being monitored, configure the http exporter to use the appropriate credentials when data is shipped to the monitoring cluster.

      If SSL/TLS is enabled on the monitoring cluster, you must use the HTTPS protocol in the host setting. You must also include the CA certificate in each node’s trusted certificates in order to verify the identities of the nodes in the monitoring cluster.

      The following example specifies the location of the PEM encoded certificate with the certificate_authorities setting:

      xpack.monitoring.exporters:
        id1:
          type: http
          host: ["https://es-mon1:9200", "https://es-mon2:9200"]
          auth:
            username: remote_monitor 
            password: changeme
          ssl:
            certificate_authorities: [ "/path/to/ca.crt" ]
        id2:
          type: local

      The username and password parameters provide the user credentials.

      Alternatively, you can configure trusted certificates using a truststore (a Java Keystore file that contains the certificates):

      xpack.monitoring.exporters:
        id1:
          type: http
          host: ["https://es-mon1:9200", "https://es-mon2:9200"]
          auth:
            username: remote_monitor
            password: changeme
          ssl:
            truststore.path: /path/to/file
            truststore.password: password
        id2:
          type: local
  5. If X-Pack security is enabled and you want to visualize monitoring data in Kibana, you must create users that have access to the Kibana indices and permission to read from the monitoring indices.

    You set up X-Pack monitoring UI users on the cluster where the monitoring data is stored, that is to say the monitoring cluster. To grant all of the necessary permissions, assign users the monitoring_user and kibana_user roles. For more information, see Mapping users and groups to roles.

  6. Optional: Configure the indices that store the monitoring data.