Set up Heartbeat monitors
editSet up Heartbeat monitors
editTo configure Heartbeat define a set of monitors
to check your remote hosts.
Specify monitors either directly inside the heartbeat.yml
config file, or in external
dynamically loaded files located in the directory referenced by heartbeat.config.monitors.path
.
One advantage of using external files is that these can be automatically reloaded
without stopping the Heartbeat process.
Each monitor
item is an entry in a yaml list, and so begins with a dash (-).
You can define the type of monitor to use, the hosts to check, and other
optional settings that control Heartbeat behavior.
The following example configures three monitors checking via the icmp
, tcp
, and http
protocols directly inside the heartbeat.yml
file, and demonstrates how to use TCP Echo
and HTTP response verification:
# heartbeat.yml heartbeat.monitors: - type: icmp schedule: '*/5 * * * * * *' hosts: ["myhost"] - type: tcp schedule: '@every 5s' hosts: ["myhost:7"] # default TCP Echo Protocol check.send: "Check" check.receive: "Check" - type: http schedule: '@every 5s' urls: ["http://localhost:80/service/status"] check.response.status: 200 heartbeat.scheduler: limit: 10
Using the heartbeat.yml
configuration file is convenient, but has two drawbacks:
it can become hard to manage with large numbers of monitors, and it will not reload
heartbeat automatically when its contents changes.
Define monitors via the heartbeat.config.monitors
to prevent those issues from
happening to you. To do so you would instead have your heartbeat.yml
file contain the following:
# heartbeat.yml heartbeat.config.monitors: # Directory + glob pattern to search for configuration files path: /path/to/my/monitors.d/*.yml # If enabled, heartbeat will periodically check the config.monitors path for changes reload.enabled: true # How often to check for changes reload.period: 1s
Then, define one or more files in the directory pointed to by heartbeat.config.monitors.path
.
You may specify multiple monitors in a given file if you like. The contents of these files is
monitor definitions only, e.g. what is normally under the heartbeat.monitors
section of
heartbeat.yml
. See below for an example
# /path/to/my/monitors.d/localhost_service_check.yml - type: http schedule: '@every 5s' urls: ["http://localhost:80/service/status"] check.response.status: 200
Monitor options
editYou can specify the following options when defining a Heartbeat monitor in any location. These options are the same for all monitors. Each monitor type has additional configuration options that are specific to that monitor type.
type
editThe type of monitor to run. One of:
-
icmp
: Uses an ICMP (v4 and v6) Echo Request to ping the configured hosts. Requires root access. See ICMP options. -
tcp
: Connects via TCP and optionally verifies the endpoint by sending and/or receiving a custom payload. See TCP options. -
http
: Connects via HTTP and optionally verifies that the host returns the expected response. See HTTP options.
The tcp
and http
monitor types both support SSL/TLS and some proxy
settings.
name
editThe monitor name. This value appears in the exported fields
under the monitor
field as the job name and the type
field as the job type.
enabled
editA Boolean value that specifies whether the module is enabled. If the enabled
option is missing from the configuration block, the module is enabled by
default.
schedule
editA cron-like expression that specifies the task schedule. For example:
-
*/5 * * * * * *
runs the task every 5 seconds (for example, at 10:00:00, 10:00:05, and so on). -
@every 5s
runs the task every 5 seconds from the time when Heartbeat was started.
The schedule
option uses a cron-like syntax based on this cronexpr
implementation,
but adds the @every
keyword.
ipv4
editA Boolean value that specifies whether to ping using the ipv4 protocol if
hostnames are configured. The default is true
.
ipv6
editA Boolean value that specifies whether to ping using the ipv6 protocol
if hostnames are configured. The default is true
.
mode
editIf mode
is any
, the monitor pings only one IP address for a hostname. If
mode
is all
, the monitor pings all resolvable IPs for a hostname. The
mode: all
setting is useful if you are using a DNS-load balancer and want to
ping every IP address for the specified hostname. The default is any
.
timeout
editThe total running time for each ping test. This is the total time allowed for testing the connection and exchanging data. The default is 16 seconds (16s).
If the timeout is exceeded, Heartbeat publishes a service-down
event. If the
value specified for timeout
is greater than schedule
, intermediate checks
will not be executed by the scheduler.
fields
editOptional fields that you can specify to add additional information to the
output. For example, you might add fields that you can use for filtering log
data. Fields can be scalar values, arrays, dictionaries, or any nested
combination of these. By default, the fields that you specify here will be
grouped under a fields
sub-dictionary in the output document. To store the
custom fields as top-level fields, set the fields_under_root
option to true.
If a duplicate field is declared in the general configuration, then its value
will be overwritten by the value declared here.
fields_under_root
editIf this option is set to true, the custom fields
are stored as top-level fields in the output document instead of being grouped
under a fields
sub-dictionary. If the custom field names conflict with other
field names added by Heartbeat, then the custom fields overwrite the other
fields.
tags
editA list of tags that will be sent with the monitor event. This setting is optional.
processors
editA list of processors to apply to the data generated by the monitor.
See Filter and Enhance the exported data for information about specifying processors in your config.
ICMP options
editThese options configure Heartbeat to use ICMP (v4 and v6) Echo Requests to check
the configured hosts. These options are valid when the type
is
icmp
.
hosts
editA list of hosts to ping.
wait
editThe duration to wait before emitting another ICMP Echo Request. The default is 1 second (1s).
TCP options
editThese options configure Heartbeat to connect via TCP and optionally verify the
endpoint by sending and/or receiving a custom payload. These options are valid when
the type
is tcp
.
hosts
editA list of hosts to ping. The entries in the list can be:
-
A plain host name, such as
localhost
, or an IP address. If you specify this option, you must also specify a value forports
. If the monitor is configured to use SSL, Heartbeat establishes an SSL/TLS-based connection. Otherwise, it establishes a plain TCP connection. -
A hostname and port, such as
localhost:12345
. Heartbeat connects to the port on the specified host. If the monitor is configured to use SSL, Heartbeat establishes an SSL/TLS-based connection. Otherwise, it establishes a TCP connection. -
A full URL using the syntax
scheme://<host>:[port]
, where:-
scheme
is one oftcp
,plain
,ssl
ortls
. Iftcp
orplain
is specified, Heartbeat establishes a TCP connection even if the monitor is configured to use SSL. Iftls
orssl
is specified, Heartbeat establishes an SSL connection. However, if the monitor is not configured to use SSL, the system defaults are used (currently not supported on Windows). -
host
is the hostname. -
port
is the port number. Ifport
is missing in the URL, theports
setting is required.
-
ports
editA list of ports to ping if the host specified in hosts
does not contain a port number.
Example configuration:
- type: tcp schedule: '@every 5s' hosts: ["myhost"] ports: [80, 9200, 5044]
check
editAn optional payload string to send to the remote host and the expected answer.
If no payload is specified, the endpoint is assumed to be available if the
connection attempt was successful. If send
is specified without receive
,
any response is accepted as OK. If receive
is specified without send
, no
payload is sent, but the client expects to receive a payload in the form of a
"hello message" or "banner" on connect.
Example configuration:
- type: tcp schedule: '@every 5s' hosts: ["myhost"] ports: [7] check.send: 'Hello World' check.receive: 'Hello World'
proxy_url
editThe URL of the SOCKS5 proxy to use when connecting to the server. The value must be a URL with a scheme of socks5://.
If the SOCKS5 proxy server requires client authentication, then a username and password can be embedded in the URL as shown in the example.
proxy_url: socks5://user:password@socks5-proxy:2233
When using a proxy, hostnames are resolved on the proxy server instead of on
the client. You can change this behavior by setting the
proxy_use_local_resolver
option.
proxy_use_local_resolver
editA Boolean value that determines whether hostnames are resolved locally instead of being resolved on the proxy server. The default value is false, which means that name resolution occurs on the proxy server.
ssl
editThe TLS/SSL connection settings. If the monitor is
configured to use SSL, it will attempt an SSL
handshake. If check
is not configured, the monitor will only check to see if
it can establish an SSL/TLS connection. This check can fail either at TCP level
or during certificate validation.
Example configuration:
- type: tcp schedule: '@every 5s' hosts: ["myhost"] ports: [80, 9200, 5044] ssl: certificate_authorities: ['/etc/ca.crt'] supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]
Also see Specify SSL settings for a full description of the ssl
options.
HTTP options
editThese options configure Heartbeat to connect via HTTP and optionally verify that
the host returns the expected response. These options are valid when the
type
is http
.
urls
editA list of URLs to ping.
Example configuration:
- type: http schedule: '@every 5s' urls: ["http://myhost:80"]
proxy_url
editThe HTTP proxy URL. This setting is optional.
username
editThe username for authenticating with the server. The credentials are passed with the request. This setting is optional.
You need to specify credentials when your check.response
settings require it.
For example, you can check for a 403 response (check.response.status: 403
)
without setting credentials.
password
editThe password for authenticating with the server. This setting is optional.
ssl
editThe TLS/SSL connection settings for use with the HTTPS endpoint. If you don’t specify settings, the system defaults are used.
Example configuration:
- type: http schedule: '@every 5s' urls: ["https://myhost:443"] ssl: certificate_authorities: ['/etc/ca.crt'] supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]
Also see Specify SSL settings for a full description of the ssl
options.
check
editAn optional request
to send to the remote host and the expected response
.
Example configuration:
- type: http schedule: '@every 5s' urls: ["http://myhost:80"] check.request.method: HEAD check.response.status: 200
Under check.request
, specify these options:
-
method
-
The HTTP method to use. Valid values are
"HEAD"
,"GET"
and"POST"
. -
headers
- A dictionary of additional HTTP headers to send.
-
body
- Optional request body content.
Under check.response
, specify these options:
-
status
- The expected status code. If this setting is not configured or it’s set to 0, any status code other than 404 is accepted.
-
headers
- The required response headers.
-
body
- A list of regular expressions to match the the body output. Only a single expression needs to match.
The following configuration shows how to check the response when the body contains JSON:
- type: http schedule: '@every 5s' urls: ["https://myhost:80"] check.request: method: GET headers: 'X-API-Key': '12345-mykey-67890' check.response: status: 200 body: '{"status": "ok"}'
The following configuration shows how to check the response for multiple regex patterns:
- type: http schedule: '@every 5s' urls: ["https://myhost:80"] check.request: method: GET headers: 'X-API-Key': '12345-mykey-67890' check.response: status: 200 body: - hello - world
The following configuration shows how to check the response with a multiline regex:
- type: http schedule: '@every 5s' urls: ["https://myhost:80"] check.request: method: GET headers: 'X-API-Key': '12345-mykey-67890' check.response: status: 200 body: '(?s)first.*second.*third'
Scheduler options
editYou specify options under heartbeat.scheduler
to control the behavior of the task
scheduler.
Example configuration:
heartbeat.scheduler: limit: 10 location: 'UTC-08:00'
In the example, setting limit
to 10 guarantees that only 10 concurrent
I/O tasks will be active. An I/O task can be the actual check or resolving an
address via DNS.
limit
editThe number of concurrent I/O tasks that Heartbeat is allowed to execute. If set to 0, there is no limit. The default is 0.
Most operating systems set a file descriptor limit of 1024. For Heartbeat to
operate correctly and not accidentally block libbeat output, the value that you
specify for limit
should be below the configured ulimit.
location
editThe timezone for the scheduler. By default the scheduler uses localtime.
watch.poll_file
edit
[6.5.0]
Deprecated in 6.5.0. Replaced by using dynamic reloading via the heartbeat.config.monitors
option.
The JSON file to watch for additional monitor configurations. The JSON file can
contain multiple objects, each of which specifies a different monitor config.
Heartbeat checks this file periodically and starts a new monitor instance for
each new JSON object added to the file. For example, imagine that you add
10 new entries to the JSON file, each for a different hostname. When Heartbeat
picks up the changes in the file, it merges the original config
(heartbeat.yml
) plus the JSON objects, and starts a monitor for each new host
that you’ve configured. If you delete an object from the JSON file and it
doesn’t exist in the main config, Heartbeat stops the monitor instance running
for that object.
Each monitor has a unique ID that’s based on parameters like protocol, host, and port. If two monitors have the same ID, Heartbeat uses the settings that are defined in the last JSON object of the merged config. This means that you can specify settings in the JSON file that overwrite the settings in the main config. In this way, the configuration that you specify for the monitor in the main Heartbeat config file acts like a default config that you can live-reconfigure by specifying additional configurations in the external JSON file.
Example configuration:
heartbeat.monitors: - type: tcp schedule: '*/5 * * * * * *' hosts: ["myhost"] watch.poll_file: path: {path.config}/monitors/dynamic.json interval: 5s
-
path
- Specifies the path to the JSON file to check for updates.
-
interval
- Specifies how often Heartbeat checks the file for changes.
To reconfigure the settings specified in the example config, you could define
the following JSON objects in dynamic.json
: