When planning to stand up a new Elasticsearch cluster, it is important to make considerations for implementing authentication and SSL/TLS with Shield as early as possible. Getting this going in the planning and testing phases will help ensure a smoother transition to production with security in place.

In this article, we walk through setting up Shield's native authentication and SSL/TLS with a wildcard certificate. Native authentication is by far the simplest way to manage users and it will be the default in X-Pack for Elasticsearch 5.x.

There are many ways to setup SSL/TLS, and in this post we will go through one of them that I find simple and easy to use for all my development environments. Prior to proceeding, be sure you have a certificate authority (CA) in place. A private (self-signed) or public CA can be used.

Getting Started

1. Install Elasticsearch

2. Install the License and Shield plugins

bin/plugin install elasticsearch/license/latest
bin/plugin install elasticsearch/shield/latest

3. Configure elasticsearch.yml for Shield's file and native authentication. File-based authentication stores users and passwords in file stored locally on each cluster node. With native authentication, users are managed with a REST API and centrally stored in the cluster. In Elasticsearch 2.3.x, file-based authentication must be configured prior to using native authentication to gain access to the cluster. This step will not be required in 5.x.

shield:
  authc:
    realms:
    file1:
      type: file
      order: 0
  native1:
    type: native
    order: 1

4. Create an admin user with the esusers command that will be used for file-based authentication.

bin/shield/esusers useradd admin -p as@m25 -r admin

5. Test authentication to the cluster using a REST client. I like curl, but any REST client will do.

curl -s 'http://node1.kle.moc:9200' -u admin:as@m25
{
  "name" : "roger",
  "cluster_name" : "kermy",
  "version" : {
    "number" : "2.3.4",
    "build_hash" : "e455fd0c13dceca8dbbdbb1665d068ae55dabe3f",
    "build_timestamp" : "2016-06-30T11:24:31Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.0"
  },
  "tagline" : "You Know, for Search"
}

Native Authentication

Use the Shield REST API to configure and manage users for native authentication. To setup the first native admin user, use the file-based credentials.

curl -u admin:as@m25 'http://node1.kle.moc:9200/_shield/user/esadmin' -XPOST -d '
{
  "password": "as@m25",
  "roles": [ "admin" ],
  "full_name": "Search Admin",
  "email": "searchadmin@kle.moc"
}'

Confirm the native user was setup successfully with a REST call.

curl 'http://node1.kle.moc:9200' -u esadmin:as@m25
{
  "name" : "roger",
  "cluster_name" : "kermy",
  "version" : {
    "number" : "2.3.4",
    "build_hash" : "e455fd0c13dceca8dbbdbb1665d068ae55dabe3f",
    "build_timestamp" : "2016-06-30T11:24:31Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.0"
  },
  "tagline" : "You Know, for Search"
}

Configuring SSL/TLS

Create the Java Keystore

In this example, a wildcard certificate will be used as it provides ease of administration across a large Elasticsearch cluster. Wildcards permit a single certificate to be used for every node in the cluster. While this is a convenient way to manage cluster certificates, be sure to take into consideration the risk factors associated with using wildcards.

It is assumed an issuing certificate authority (CA) is already in place. See the article Setting Up a Certificate Authority for help setting up your own CA that can be used with Shield.

1. Create a new Java Keystore by importing the CA certificate that will issue the wildcard certificate. Note: the Java Keystore can be created on any host or workstation with Java installed using the keytool command.

keytool -importcert -keystore shield.jks -file ca.cert.pem -trustcacerts -storepass s3cret -alias ca_cert

2. Create a private key in the Java Keystore.

keytool -storepass s3cret -genkey -alias es-shield -keystore shield.jks -keyalg RSA -keysize 2048 -validity 3650 -dname "cn=*.kle.moc"

3. Create a certificate signing request (CSR) using keytool for requesting a certificate from the issuing CA.

keytool -storepass s3cret -certreq -alias es-shield -keystore shield.jks -keyalg RSA -keysize 2048 -validity 3650 -dname "cn=*.kle.moc" > kermy-shield.csr

4. Once the CA has signed the CSR and returned the certificate in PEM format, import it into the Java Keystore. Be sure the alias lines up with the one used in steps 2 and 3.

keytool -storepass s3cret -importcert -keystore shield.jks -alias es-shield -file kermy-shield.pem
Certificate reply was installed in keystore

Configure Elasticsearch

1. Copy shield.jks to the Elasticsearch configuration directory {path.conf}/shield.

2. Configure elasticsearch.yml for SSL/TLS

shield:
  http.ssl: true
  ssl:
    keystore.path: shield/shield.jks
    keystore.password: s3cret

My full elasticsearch.yml

cluster.name: kermy
node.name: roger
network.host: _ens160:ipv4_
http.port: 9200
transport.tcp.port: 9300
shield:
  http.ssl: true
  ssl:
    keystore.path: shield/shield.jks
    keystore.password: s3cret
  authc:
    realms:
      file1:
        type: file
        order: 0
      native1:
        type: native
        order: 1

And the final curl test with SSL/TLS and native authentication

curl 'https://node1.kle.moc:9200/' -sku esadmin:as@m25 -v
* Trying 192.168.5.50...
* Connected to node1.kle.moc (192.168.5.50) port 9201 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
* Server certificate: *.kle.moc
* Server certificate: Unicorn Ltd CA
* Server auth using Basic with user 'esadmin'
> GET / HTTP/1.1> Host: node1.kle.moc:9200
> Authorization: Basic ZXNhZG1pbjphc0BtMjU=
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/json; charset=UTF-8
< Content-Length: 306
<
{
  "name" : "roger",
  "cluster_name" : "kermy",
  "version" : {
    "number" : "2.3.4",
    "build_hash" : "e455fd0c13dceca8dbbdbb1665d068ae55dabe3f",
    "build_timestamp" : "2016-06-30T11:24:31Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.0"
  },
  "tagline" : "You Know, for Search"
}
* Connection #0 to host node1.kle.moc left intact

Enabling SSL/TLS and authentication should be at the forefront of every service running in your infrastructure, including Elasticsearch. Once this configuration is in place, it can be modeled for further production use when adding new nodes or creating more clusters.

We want to help you ensure that your Elasticsearch cluster is safe and secure. Give Shield, security for Elasticsearch, a try and we welcome your feedback via the forums or file a support ticket.

Context engineering

Vector database

Search powered applications

Logs

Threat protection

Workflows

Elasticsearch

Kibana (Discover, Dashboards)

Elastic Agent Builder

AutoOps

Piped query language

Jina AI search models

Elastic Cloud Serverless

Elastic Cloud Hosted

Self-managed Elasticsearch

Ecommerce search

Customer support search

Search-driven apps

Log analytics

Infrastructure monitoring

Digital experience monitoring

App performance monitoring

AIOps

LLM observability

Next-gen SIEM

Workflows for security

XDR and endpoint security

AI for security

10x your data's value

Cloud providers

Elastic AI Ecosystem

Search AI Partner Program

AV-Comparatives

Forrester Wave™ XDR

Gartner Magic Quadrant Leader

IDC MarketScape

Search

Security

Observability

Get started

Demo gallery

Downloads

Integrations

Docs

Elasticsearch Labs

Elastic Security Labs

Elastic Observability Labs

Blog

Community

Events

Webinars

Discuss

Training

Support

Consulting

Author

Articles by Jason Bryan

Principal Software Engineer, undefined

Getting Started with Elasticsearch and SSL & Native Authentication

Sign up for Elastic Cloud free trial