Configure authentication credentialsedit

When sending data to a secured cluster through the elasticsearch output, Filebeat must either provide basic authentication credentials or present a client certificate.

To configure authentication credentials for Filebeat:

  1. Create a writer role that has the following privileges:

    • Cluster: manage_index_templates, monitor, and manage_ingest_pipelines
    • Index: write and create_index on the Filebeat indices

    You can create roles from the Management / Roles UI in Kibana or through the role API. For example, the following request creates a role named filebeat_writer:

    POST _xpack/security/role/filebeat_writer
    {
      "cluster": ["manage_index_templates","monitor","manage_ingest_pipelines"], 
      "indices": [
        {
          "names": [ "filebeat-*" ], 
          "privileges": ["write","create_index"]
        }
      ]
    }

    The manage_ingest_pipelines cluster privilege is required to run Filebeat modules.

    If you use a custom Filebeat index pattern, specify that pattern instead of the default filebeat-* pattern.

  2. If you plan to use index lifecycle management, create a role that has the following privileges. These privileges are required to load index lifecycle policies and create and manage rollover indices:

    • Cluster: manage_ilm
    • Index: write, create_index, manage, and manage_ilm on the Filebeat indices

      POST _xpack/security/role/filebeat_ilm
      {
        "cluster": ["manage_ilm"],
        "indices": [
          {
            "names": [ "filebeat-*","shrink-filebeat-*"],
            "privileges": ["write","create_index","manage","manage_ilm"]
          }
        ]
      }
  3. Assign the writer role to the user that Filebeat will use to connect to Elasticsearch. Make sure you also assign any roles that are required for specific features. For the list of features and required roles, see Filebeat features that require authorization.

    1. To authenticate as a native user, create a user for Filebeat to use internally and assign it the writer role, plus any other roles that are needed.

      You can create users from the Management / Users UI in Kibana or through the user API. For example, following request creates a user named filebeat_internal that has the filebeat_writer and kibana_user roles:

      POST /_xpack/security/user/filebeat_internal
      {
        "password" : "YOUR_PASSWORD",
        "roles" : [ "filebeat_writer","kibana_user"],
        "full_name" : "Internal Filebeat User"
      }
    2. To use PKI authentication, assign the writer role, plus any other roles that are needed, in the role_mapping.yml configuration file. Specify the user by the distinguished name that appears in its certificate:

      filebeat_writer:
        - "cn=Internal Filebeat User,ou=example,o=com"
      kibana_user:
        - "cn=Internal Filebeat User,ou=example,o=com"

      For more information, see Using Role Mapping Files.

  4. In the Filebeat configuration file, specify authentication credentials for the elasticsearch output:

    1. To use basic authentication, configure the username and password settings. For example, the following Filebeat output configuration uses the native filebeat_internal user to connect to Elasticsearch:

      output.elasticsearch:
        hosts: ["localhost:9200"]
        username: "filebeat_internal" 
        password: "YOUR_PASSWORD" 

      You created this user earlier.

      The example shows a hard-coded password, but you should store sensitive values in the secrets keystore.

    2. To use PKI authentication, configure the certificate and key settings:

      output.elasticsearch:
        hosts: ["localhost:9200"]
        ssl.certificate: "/etc/pki/client/cert.pem" 
        ssl.key: "/etc/pki/client/cert.key"

      The distinguished name (DN) in the certificate must be mapped to the filebeat_writer and kibana_user roles in the role_mapping.yml configuration file on each node in the Elasticsearch cluster.