Elasticsearch is also available as a Docker image. The image is built with X-Pack.
Obtaining Elasticsearch for Docker is as simple as issuing a
docker pull command against the Elastic Docker registry.
The Docker image can be retrieved with the following command:
docker pull docker.elastic.co/elasticsearch/elasticsearch:5.3.3
Elasticsearch can be quickly started for development or testing use with the following command:
docker run -p 9200:9200 -e "http.host=0.0.0.0" -e "transport.host=127.0.0.1" docker.elastic.co/elasticsearch/elasticsearch:5.3.3
vm_max_map_count kernel setting needs to be set to at least
262144 for production use.
Depending on your platform:
vm_map_max_countsetting should be set permanently in /etc/sysctl.conf:
$ grep vm.max_map_count /etc/sysctl.conf vm.max_map_count=262144
To apply the setting on a live system type:
sysctl -w vm.max_map_count=262144
OSX with Docker for Mac
vm_max_map_countsetting must be set within the xhyve virtual machine:
$ screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty
Log in with root and no password. Then configure the
sysctlsetting as you would for Linux:
sysctl -w vm.max_map_count=262144
OSX with Docker Toolbox
vm_max_map_countsetting must be set via docker-machine:
docker-machine ssh sudo sysctl -w vm.max_map_count=262144
The following example brings up a cluster comprising two Elasticsearch nodes.
To bring up the cluster, use the
docker-compose.yml and just type:
docker-compose is not pre-installed with Docker on Linux.
Instructions for installing it can be found on the docker-compose webpage.
elasticsearch1 listens on
elasticsearch2 talks to
elasticsearch1 over a Docker network.
This example also uses Docker named volumes, called
esdata2 which will be created if not already present.
version: '2' services: elasticsearch1: image: docker.elastic.co/elasticsearch/elasticsearch:5.3.3 container_name: elasticsearch1 environment: - cluster.name=docker-cluster - bootstrap.memory_lock=true - "ES_JAVA_OPTS=-Xms512m -Xmx512m" ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 hard: 65536 mem_limit: 1g cap_add: - IPC_LOCK volumes: - esdata1:/usr/share/elasticsearch/data ports: - 9200:9200 networks: - esnet elasticsearch2: image: docker.elastic.co/elasticsearch/elasticsearch:5.3.3 environment: - cluster.name=docker-cluster - bootstrap.memory_lock=true - "ES_JAVA_OPTS=-Xms512m -Xmx512m" - "discovery.zen.ping.unicast.hosts=elasticsearch1" ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 hard: 65536 mem_limit: 1g cap_add: - IPC_LOCK volumes: - esdata2:/usr/share/elasticsearch/data networks: - esnet volumes: esdata1: driver: local esdata2: driver: local networks: esnet: driver: bridge
To stop the cluster, type
docker-compose down. Data volumes will persist, so it’s possible to start the cluster again with the same data using
To destroy the cluster and the data volumes just type
docker-compose down -v.
curl -u elastic http://127.0.0.1:9200/_cat/health Enter host password for user 'elastic': 1472225929 15:38:49 docker-cluster green 2 2 4 2 0 0 0 0 - 100.0%
Log messages go to the console and are handled by the configured Docker logging driver. By default you can access logs with
The image offers several methods for configuring Elasticsearch settings with the conventional approach being to provide customized files, i.e.
elasticsearch.yml, but it’s also possible to use environment variables to set options:
For example, to define the cluster name with
docker run you can pass
-e "cluster.name=mynewclustername". Double quotes are required.
There is a difference between defining default settings and normal settings. The former are prefixed with
default. and cannot override normal settings, if defined.
Create your custom config file and mount this over the image’s corresponding file.
For example, bind-mounting a
docker run can be accomplished with the parameter:
The container runs Elasticsearch as user
elasticsearch using uid:gid
1000:1000. Bind mounted host directories and files, such as
custom_elasticsearch.yml above, need to be accessible by this user. For the data and log dirs, such as
/usr/share/elasticsearch/data, write access is required as well.
In some environments, it may make more sense to prepare a custom image containing your configuration. A
Dockerfile to achieve this may be as simple as:
FROM docker.elastic.co/elasticsearch/elasticsearch:5.3.3 ADD elasticsearch.yml /usr/share/elasticsearch/config/ USER root RUN chown elasticsearch:elasticsearch config/elasticsearch.yml USER elasticsearch
You could then build and try the image with something like:
docker build --tag=elasticsearch-custom . docker run -ti -v /usr/share/elasticsearch/data elasticsearch-custom
We have collected a number of best practices for production use.
Any Docker parameters mentioned below assume the use of
- Elasticsearch inside the container runs as user
1000:1000. If you are bind mounting a local directory or file, ensure it is readable by this user while the data and log dirs additionally require write access.
It is important to correctly set capabilities and ulimits via the Docker CLI. As seen earlier in the example docker-compose.yml, the following options are required:
--cap-add=IPC_LOCK --ulimit memlock=-1:-1 --ulimit nofile=65536:65536
bootstrap.memory_lockis set to
trueas explained in "Disable swapping".
This can be achieved through any of the configuration methods, e.g. by setting the appropriate environments variable with
- The image exposes TCP ports 9200 and 9300. For clusters it is recommended to randomize the published ports with
--publish-all, unless you are pinning one container per host.
- Use the
ES_JAVA_OPTSenvironment variable to set heap size, e.g. to use 16GB use
-e ES_JAVA_OPTS="-Xms16g -Xmx16g"with
docker run. It is also recommended to set a memory limit for the container.
- Pin your deployments to a specific version of the Elasticsearch Docker image, e.g.
Always use a volume bound on
/usr/share/elasticsearch/data, as shown in the production example, for the following reasons:
- The data of your elasticsearch node won’t be lost if the container is killed
- Elasticsearch is I/O sensitive and the Docker storage driver is not ideal for fast I/O
- It allows the use of advanced Docker volume plugins
- If you are using the devicemapper storage driver (default on at least RedHat (rpm) based distributions) make sure you are not using the default
loop-lvmmode. Configure docker-engine to use direct-lvm instead.
- Consider centralizing your logs by using a different logging driver. Also note that the default json-file logging driver is not ideally suited for production use.
You now have a test Elasticsearch environment set up. Before you start serious development or go into production with Elasticsearch, you will need to do some additional setup: