Getting Started (Long Version)

Security is installed as an Elasticsearch plugin. The plugin must be installed on every node in the cluster, and every node must be restarted after installation. Plan for a complete cluster restart before beginning the installation process.

Important

Shield 1.0.1 is compatible with Elasticsearch 1.4.2 and newer versions of Elasticsearch 1.4.x.

Configuring your environment

If you install Elasticsearch as a package or you specify a custom configuration directory, the command line tools require you to specify the configuration directory. On Linux systems, add the following line to your .profile file:

export ES_JAVA_OPTS="-Des.path.conf=/etc/elasticsearch"
Note

When using sudo to run commands as a different user, the ES_JAVA_OPTS setting from your profile will not be available in the other user’s environment. You can manually pass the environment variables to the command or you can make the environment variable available by adding the following line to the /etc/sudoers file:

Defaults    env_keep += "ES_JAVA_OPTS"

On Windows systems, the setx command can be used to specify a custom configuration directory:

setx ES_JAVA_OPTS "-Des.path.conf=C:\config"

Shield And Licensing

Shield requires a license to operate and the licensing is managed by a separate plugin. For this reason, the License plugin must be installed (without the license plugin Shield will prevent the node from starting up).

For instructions on how to install the License plugin, please refer to License Management.

Once you have the licensing plugin installed, you may begin working with Shield immediately. When elasticsearch starts for the first time with Shield and the licensing plugin installed, a 30-day trial license for Shield will automatically be generated. If you have a license for Shield that you would like to install, please refer to installing a license.

Important

With a valid license, Shield will be fully operational. Upon license expiry, Shield will operate in a degraded mode, where cluster health, cluster stats, and index stats APIs will be blocked. All other operations will continue operating normally. Additional information can be found at the Shield license expiration section.

Installing the Shield plugin

Follow these steps on every node in the cluster:

  1. From the Elasticsearch home directory, run:

    bin/plugin -i elasticsearch/shield/latest
  2. Restart your Elasticsearch node.

    Before restarting your cluster, consider temporarily disabling shard allocation.

If your server doesn’t have direct Internet access, see manual download for an alternative way to get the Security binaries.

Manual Download

Elasticsearch’s bin/plugin script requires direct Internet access for downloading and installing the security plugin. If your server doesn’t have Internet access, you can download the required binaries from the following link:

https://download.elastic.co/elasticsearch/shield/shield-1.0.2.zip

Transfer the compressed file to your server, then install the plugin with the bin/plugin script:

bin/plugin -i shield -u file://PATH_TO_ZIP_FILE 

Absolute path to Shield plugin zip distribution file (e.g. file:///path/to/file/shield-1.0.2.zip, note the three slashes at the beginning)

Shield Installation Layout

Shield comes with its own set of configuration files and executable tools. These include:

Executables

Shield’s bin directory is located at $ES_HOME/bin/shield. Consider adding this directory to your PATH environment variable.

Configuration

Shield’s config directory is located at <elasticsearch_config>/shield (where <elasticsearch_config> refers to the standard config directory of Elasticsearch - typically at $ES_HOME/config).

Unless otherwise stated, Shield’s settings are placed in the main elasticsearch.yml configuration file.

Message Authentication

Message authentication verifies that a message has not been tampered with or corrupted in transit. To enable message authentication, run the syskeygen tool without any options:

bin/shield/syskeygen

This creates the system key file in Shield’s config directory, e.g. config/shield/system_key. You can customize this file’s location by changing the value of the shield.system_key.file setting in the elasticsearch.yml file.

Important

Because the system key is a symmetric key, the same key must be on every node in the cluster. Copy the key to every node in the cluster after generating it.

Enabling Role-based Access Control

Now that we have Shield installed, we’ll move to configuring the users (and their roles) with which we’ll be able to execute various of APIs on Elasticsearch.

Defining Roles

A role encompasses a set of permissions over the cluster and/or the indices in it. Roles are defined in the $ES_HOME/config/shield/roles.yml file.

Example role definition. 

# All cluster rights
# All operations on all indices
admin: 
  cluster: all
  indices:
    '*': all

# monitoring cluster privileges
# All operations on all indices
power_user: 
  cluster: monitor
  indices:
    '*': all

# Read-only operations on indices
user: 
  indices:
    '*': read

The admin role enables full access to the cluster and all its indices.

The power_user role enables monitoring only access on the cluster and full access on all its indices.

The user role has no cluster wide permissions and only has data read access on all its indices.

For this quick getting started guide, we won’t need to change anything in the roles.yml file that comes out-of-the-box with Shield, as it already defines the roles listed in the snippet above. To learn more on roles and how one can configure them, please see Roles.

Defining Users

Shield supports different authentication realms that authenticate users from different sources. In this example, we’ll use the internal esusers realm that comes with Shield. The esusers realm supports user management using the esusers command line tool from Shield’s bin directory.

Note

The esusers realm is enabled by default when no realms are explicitly configured in elasticsearch.yml. For more information on realms configuration please see Realms.

bin/shield/esusers useradd rdeniro -p taxidriver -r admin
bin/shield/esusers useradd alpacino -p godfather -r user

The example above adds two users:

  • The rdeniro user with password taxidriver, with the admin role in the cluster
  • The alpacino user with password godfather, with the user role in the cluster
Note

To ensure that Elasticsearch can read the user and role information at startup, run esusers useradd as the same user you use to run Elasticsearch. Running the command as root or some other user will update the permissions for the users and users_roles files and prevent Elasticsearch from accessing them.

Now that we’ve defined the roles and the users of the cluster, you can start the Elasticsearch node and we’ll verify that Shield plugin has been loaded.

Verifying Shield Installation

Once your Elasticsearch node is running, you can issue a curl command to verify that Shield has been loaded and is the expected version.

curl --user rdeniro:taxidriver 'localhost:9200/_shield'
{
  "status" : "enabled",
  "name" : "Mimic",
  "cluster_name" : "elasticsearch",
  "version" : {
    "number" : "1.0.2",
    "build_hash" : "ff3c5db809ce0d48e1c9dc8fc5fd04248bc057ff",
    "build_timestamp" : "2015-03-23T15:00:57Z",
    "build_snapshot" : false
  },
  "tagline" : "You know, for security"
}

You can also check the startup logs to verify that the Shield plugin has loaded and the network transports are using Shield. A successful installation will show lines similar to the following:

[2014-10-09 13:47:38,841][INFO ][transport ] [Ezekiel Stane] Using [org.elasticsearch.shield.transport.ShieldServerTransportService] as transport service, overridden by [shield]
[2014-10-09 13:47:38,841][INFO ][transport ] [Ezekiel Stane] Using [org.elasticsearch.shield.transport.netty.ShieldNettyTransport] as transport, overridden by [shield]
[2014-10-09 13:47:38,842][INFO ][http      ] [Ezekiel Stane] Using [org.elasticsearch.shield.transport.netty.ShieldNettyHttpServerTransport] as http transport, overridden by [shield]

In the next section, we’ll use a simple HTTP client to interact with Elasticsearch protected by Shield.

Configuring HTTP REST Clients

Elasticsearch works with standard HTTP basic authentication headers to identify the requester. Since Elasticsearch is stateless, this header must be sent with every request:

Authorization: Basic <TOKEN> 

The <TOKEN> is computed as base64(USERNAME:PASSWORD)

Client examples

Using curl without basic authentication to create an index has the following result:

curl -XPUT 'localhost:9200/idx'
{
  "error":  "AuthenticationException[Missing authentication token]",
  "status": 401
}

Since no user is associated with the request above, the request returns an authentication error. Next, use curl with basic auth to create an index as the rdeniro user:

curl --user rdeniro:taxidriver -XPUT 'localhost:9200/idx'
{
  "acknowledged": true
}

Since the request is executed on behalf of administrative user rdeniro, the create index request authenticates and authorizes successfully, resulting in normal execution of the request. Creating another index as the alpacino user results in the following error:

curl --user alpacino:godfather -XPUT 'localhost:9200/idx2'
{
  "error":  "AuthorizationException[Action [indices:admin/create] is unauthorized for user [alpacino]]",
  "status": 403
}

As user alpacino does not have any index administration rights, the request is rejected with an authorization error.

Next Steps

Now you have a working cluster with authentication and access control enabled. In the Authorization section, we explain how to manage users and their roles. The Authentication section explains how to use Shield’s authentication realms and LDAP integration. The Securing Nodes section discusses enabling SSL/TLS encryption for nodes and clients.