Getting started with the Elastic Stack
editGetting started with the Elastic Stack
editLooking for a guide that shows how to quickly install and configure the Elastic Stack ("ELK")? You’re in the right place! You can install the Elastic Stack on a single VM, or even on your laptop. Install each component in the following order:
Logstash adds powerful data parsing and transformation features, but usually isn’t required. To get started with Logstash, see Getting Started with Logstash.
After completing the installation process, learn how to implement a system monitoring solution that uses Metricbeat to collect server metrics and ship the data to Elasticsearch. Then use Kibana to search and visualize the data.
Before you begin
edit- See the Elastic Support Matrix for information about supported operating systems and product compatibility.
- Verify that your system meets the minimum JVM requirements for Elasticsearch.
Install Elasticsearch
editElasticsearch is a real-time, distributed storage, search, and analytics engine. It can be used for many purposes, but one context where it excels is indexing streams of semi-structured data, such as logs or decoded network packets.
You can run Elasticsearch on your own hardware, or use our hosted Elasticsearch Service on Elastic Cloud. The Elasticsearch Service is available on AWS, Microsoft Azure, and GCP. Try out the Elasticsearch Service for free.
To download and install Elasticsearch, open a terminal window and use the commands that work with your system:
When you start Elasticsearch for the first time, security features such as authentication, authorization and network encryption (TLS) for elasticsearch are enabled by default. The following security configuration occurs automatically:
- Certificates and keys for TLS are generated for the transport and HTTP layers.
-
The TLS configuration settings are written to
elasticsearch.yml
. -
A password is generated for the
elastic
user. - An enrollment token is generated for Kibana.
You can then start Kibana and enter the enrollment token to securely connect Kibana with Elasticsearch. The enrollment token is valid for 30 minutes.
On deb
and rpm
installations, an enrollment token isn’t generated for Kibana
during installation. To generate an enrollment token for Kibana, use the
elasticsearch-create-enrollment-token
tool.
curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.2.3-amd64.deb sudo dpkg -i elasticsearch-8.2.3-amd64.deb sudo /etc/init.d/elasticsearch start
curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.2.3-x86_64.rpm sudo rpm -i elasticsearch-8.2.3-x86_64.rpm sudo service elasticsearch start
curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.2.3-darwin-x86_64.tar.gz tar -xzvf elasticsearch-8.2.3-darwin-x86_64.tar.gz cd elasticsearch-8.2.3 ./bin/elasticsearch
curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.2.3-linux-x86_64.tar.gz tar -xzvf elasticsearch-8.2.3-linux-x86_64.tar.gz cd elasticsearch-8.2.3 ./bin/elasticsearch
- Download the Elasticsearch 8.2.3 Windows zip file from the Elasticsearch download page.
-
Extract the contents of the zip file to a directory on your computer, for
example,
C:\Program Files
. -
Open a command prompt as an Administrator and navigate to the directory that contains the extracted files, for example:
cd C:\Program Files\elasticsearch-8.2.3
-
Start Elasticsearch:
bin\elasticsearch.bat
For other operating systems, go to the Elasticsearch download page.
The default cluster.name and
node.name are elasticsearch
and your hostname,
respectively. If you plan to keep using this cluster or add more nodes, it is a
good idea to change these default values to unique names. For details about
changing these and other settings in the elasticsearch.yml
file, see
Configuring Elasticsearch.
To learn more about installing, configuring, and running Elasticsearch, read the Elasticsearch Reference.
Make sure that Elasticsearch is up and running
editOpen a new terminal and verify that you can connect to your Elasticsearch cluster by
making an authenticated call. Enter the password for the elastic
user when
prompted:
curl --cacert $ES_PATH_CONF/certs/http_ca.crt -u elastic https://localhost:9200
-
--cacert
-
Path to the generated
http_ca.crt
certificate for the HTTP layer. This file is created in the Elasticsearch configuration directory, which is defined by the$ES_PATH_CONF
environment variable.
On Windows, if you don’t have cURL installed, point your browser to the URL.
You should see a response similar to this:
{ "name" : "QtI5dUu", "cluster_name" : "elasticsearch", "cluster_uuid" : "v8OWkR1OQO-rgV8o_lRhEA", "version" : { "number" : "8.2.3", "build_flavor" : "default", "build_type" : "tar", "build_hash" : "f4d76bd413ecfbd5122c3aa5dc85465960f18afe", "build_date" : "2021-10-27T22:47:53.634020433Z", "build_snapshot" : false, "lucene_version" : "9.0.0", "minimum_wire_compatibility_version" : "7.16.0", "minimum_index_compatibility_version" : "7.0.0" }, "tagline" : "You Know, for Search" }
Install Kibana
editKibana is an open source analytics and visualization platform designed to work with Elasticsearch. You use Kibana to search, view, and interact with data stored in Elasticsearch indices. You can easily perform advanced data analysis and visualize your data in a variety of charts, tables, and maps.
Running our hosted Elasticsearch Service on Elastic Cloud? Kibana is enabled automatically in most templates.
We recommend that you install Kibana on the same server as Elasticsearch, but it’s not
required. If you install the products on different servers, you’ll
need to change the URL (IP:PORT) of the Elasticsearch server in the Kibana configuration
file, kibana.yml
, before starting Kibana.
To download and install Kibana, open a terminal window and use the commands that work with your system:
If this is the first time you’re starting Kibana, this command generates a unique link in your terminal to enroll your Kibana instance with Elasticsearch.
- In your terminal, click the generated link to open Kibana in your browser.
- In your browser, paste the enrollment token that was generated in the terminal when you started Elasticsearch, and then click the button to connect your Kibana instance with Elasticsearch.
-
Log in to Kibana as the
elastic
user with the password that was generated when you started Elasticsearch.
If you need to reset the password for the elastic
user or other
built-in users, run the elasticsearch-reset-password
tool.
To generate new enrollment tokens for Kibana or Elasticsearch nodes, run the
elasticsearch-create-enrollment-token
tool.
These tools are available in the Elasticsearch bin
directory.
curl -L -O https://artifacts.elastic.co/downloads/kibana/kibana-8.2.3-linux-x86_64.tar.gz tar xzvf kibana-8.2.3-linux-x86_64.tar.gz cd kibana-8.2.3-linux-x86_64/ ./bin/kibana
curl -L -O https://artifacts.elastic.co/downloads/kibana/kibana-8.2.3-darwin-x86_64.tar.gz tar xzvf kibana-8.2.3-darwin-x86_64.tar.gz cd kibana-8.2.3-darwin-x86_64/ ./bin/kibana
- Download the Kibana 8.2.3 Windows zip file from the Kibana download page.
-
Extract the contents of the zip file to a directory on your computer, for
example,
C:\Program Files
. -
Open a command prompt as an Administrator and navigate to the directory that contains the extracted files, for example:
cd C:\Program Files\kibana-8.2.3-windows
-
Start Kibana:
bin\kibana.bat
For other operating systems, go to the Kibana download page.
To learn more about installing, configuring, and running Kibana, read the Kibana Reference.
Access the Kibana web interface
editTo access the Kibana web interface, point your browser to port 5601
. For example,
http://127.0.0.1:5601.
Install Beats
editThe Beats are open source data shippers that you install as agents on your servers to send operational data to Elasticsearch. Beats can send data directly to Elasticsearch or via Logstash, where you can further process and enhance the data.
Each Beat is a separately installable product. In this guide, you learn how
to install and run Metricbeat with the system
module enabled to collect system
metrics.
To learn more about installing and configuring other Beats, see the Getting Started documentation:
Elastic Beats | To capture |
---|---|
Audit data |
|
Log files |
|
Cloud data |
|
Availability monitoring |
|
Metrics |
|
Network traffic |
|
Windows event logs |
Install Metricbeat
editTo download and install Metricbeat, open a terminal window and use the commands that work with your system:
deb:
curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-8.2.3-amd64.deb sudo dpkg -i metricbeat-8.2.3-amd64.deb
rpm:
curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-8.2.3-x86_64.rpm sudo rpm -vi metricbeat-8.2.3-x86_64.rpm
mac:
curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-8.2.3-darwin-x86_64.tar.gz tar xzvf metricbeat-8.2.3-darwin-x86_64.tar.gz
linux:
curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-8.2.3-linux-x86_64.tar.gz tar xzvf metricbeat-8.2.3-linux-x86_64.tar.gz
win:
- Download the Metricbeat Windows zip file from the Metricbeat download page.
-
Extract the contents of the zip file into
C:\Program Files
. -
Rename the
metricbeat-8.2.3-windows
directory toMetricbeat
. - Open a PowerShell prompt as an Administrator (right-click the PowerShell icon and select Run As Administrator).
-
From the PowerShell prompt, run the following commands to install Metricbeat as a Windows service:
PS > cd 'C:\Program Files\Metricbeat' PS C:\Program Files\Metricbeat> .\install-service-metricbeat.ps1
If script execution is disabled on your system, set the execution policy for the current session to allow the script to run. For example:
PowerShell.exe -ExecutionPolicy UnRestricted -File .\install-service-metricbeat.ps1
.
For other operating systems, go to the Beats download page.
Ship system metrics to Elasticsearch
editMetricbeat provides pre-built modules that you can use to rapidly implement and deploy a system monitoring solution, complete with sample dashboards and data visualizations, in about 5 minutes.
In this section, you learn how to run the system
module to collect metrics
from the operating system and services running on your server. The system module
collects system-level metrics, such as CPU usage, memory, file system, disk IO,
and network IO statistics, as well as statistics for every process
running on your system.
Before you begin: Verify that Elasticsearch and Kibana are running and that Elasticsearch is ready to receive data from Metricbeat.
To set up the system
module and start collecting system metrics:
-
From the Metricbeat install directory, enable the
system
module:deb and rpm:
sudo metricbeat modules enable system
mac and linux:
./metricbeat modules enable system
win:
PS C:\Program Files\Metricbeat> .\metricbeat.exe modules enable system
-
Set up the initial environment:
deb and rpm:
sudo metricbeat setup -e
mac and linux:
./metricbeat setup -e
win:
PS C:\Program Files\Metricbeat> metricbeat.exe setup -e
The
setup
command loads the Kibana dashboards. If the dashboards are already set up, omit this command. The-e
flag is optional and sends output to standard error instead of syslog. -
deb and rpm:
sudo service metricbeat start
mac and linux:
./metricbeat -e
win:
PS C:\Program Files\Metricbeat> Start-Service metricbeat
Metricbeat runs and starts sending system metrics to Elasticsearch.
Visualize system metrics in Kibana
editTo visualize system metrics, open your browser and navigate to the Metricbeat system overview dashboard: http://localhost:5601/app/kibana#/dashboard/Metricbeat-system-overview-ecs
If you don’t see data in Kibana, try changing the date range to a larger range. By default, Kibana shows the last 15 minutes. If you see errors, make sure Metricbeat is running, then refresh the page.
Click Host Overview to see detailed metrics about the selected host.
What’s next?
editCongratulations! You’ve successfully set up the Elastic Stack and securely connected Kibana with Elasticsearch. You learned how to stream system metrics to Elasticsearch and visualize the data in Kibana.
Next, you’ll want to activate your trial license to unlock the full capabilities of the Elastic Stack.
Want to get up and running quickly with metrics monitoring and centralized log analytics? Try out the Metrics app and the Logs app in Kibana. For more details, see Analyze metrics and Monitor logs.
Later, when you’re ready to set up a production environment, also see the Elastic Stack Installation and Upgrade Guide.