Authentication in X-Pack security is handled by one or more authentication services called realms. A realm is used to resolve and authenticate users based on authentication tokens. X-Pack security provides the following built-in realms:

An internal realm where users are stored in a dedicated Elasticsearch index. This realm supports an authentication token in the form of username and password, and is available by default when no realms are explicitly configured. The users are managed via the user management APIs. See Native user authentication.
A realm that uses an external LDAP server to authenticate the users. This realm supports an authentication token in the form of username and password, and requires explicit configuration in order to be used. See LDAP user authentication.
A realm that uses an external Active Directory Server to authenticate the users. With this realm, users are authenticated by usernames and passwords. See Active Directory user authentication.
A realm that authenticates users using Public Key Infrastructure (PKI). This realm works in conjunction with SSL/TLS and identifies the users through the Distinguished Name (DN) of the client’s X.509 certificates. See PKI user authentication.
An internal realm where users are defined in files stored on each node in the Elasticsearch cluster. This realm supports an authentication token in the form of username and password and is always available. See File-based user authentication.
A realm that facilitates authentication using the SAML 2.0 Web SSO protocol. This realm is designed to support authentication through Kibana and is not intended for use in the REST API. See SAML authentication.
A realm that authenticates a user using Kerberos authentication. Users are authenticated on the basis of Kerberos tickets. See Kerberos authentication.

X-Pack security also supports custom realms. If you need to integrate with another authentication system, you can build a custom realm plugin. For more information, see Integrating with Other Authentication Systems.

Realms live within a realm chain. It is essentially a prioritized list of configured realms (typically of various types). The order of the list determines the order in which the realms will be consulted. You should make sure each configured realm has a distinct order setting. In the event that two or more realms have the same order, they will be processed in name order. During the authentication process, X-Pack security will consult and try to authenticate the request one realm at a time. Once one of the realms successfully authenticates the request, the authentication is considered to be successful and the authenticated user will be associated with the request (which will then proceed to the authorization phase). If a realm cannot authenticate the request, the next in line realm in the chain will be consulted. If all realms in the chain could not authenticate the request, the authentication is then considered to be unsuccessful and an authentication error will be returned (as HTTP status code 401).

Some systems (e.g. Active Directory) have a temporary lock-out period after several successive failed login attempts. If the same username exists in multiple realms, unintentional account lockouts are possible. For more information, please see here.

The default realm chain contains the native and file realms. To explicitly, configure a realm chain, you specify the chain in elasticsearch.yml. When you configure a realm chain, only the realms you specify are used for authentication. To use the native and file realms, you must include them in the chain.

The following snippet configures a realm chain that includes the file and native realms, as well as two LDAP realms and an Active Directory realm.

      type: file
      order: 0

      type: native
      order: 1

      type: ldap
      order: 2
      enabled: false
      url: 'url_to_ldap1'

      type: ldap
      order: 3
      url: 'url_to_ldap2'

      type: active_directory
      order: 4
      url: 'url_to_ad'

As can be seen above, each realm has a unique name that identifies it and each realm type dictates its own set of required and optional settings. That said, there are settings that are common to all realms.

Realm types can roughly be classified in two categories:

Realms that are internal to Elasticsearch and don’t require any communication with external parties. They are fully managed by X-Pack security. There can only be a maximum of one configured realm per internal realm type. X-Pack security provides two internal realm types: native and file.
Realms that require interaction with parties/components external to Elasticsearch, typically, with enterprise grade identity management systems. Unlike internal realms, there can be as many external realms as one would like - each with its own unique name and configuration. X-Pack security provides the following external realm types: ldap, active_directory, saml, kerberos and pki.