Airflow module

edit

This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.

This module collects metrics from Airflow metrics running a statsd server where airflow will send metrics to. The default metricset is statsd.

Compatibility

edit

The Airflow module is tested with Airflow 2.1.0. It should work with version 2.0.0 and later.

Usage

edit

The Airflow module requires Statsd to receive Statsd metrics. Refer to the link for instructions about how to use Statsd.

Add the following lines to your Airflow configuration file e.g. airflow.cfg ensuring statsd_prefix is left empty and replace %METRICBEAT_HOST% with the address metricbeat is running:

[metrics]
statsd_on = True
statsd_host = %METRICBEAT_HOST%
statsd_port = 8126
statsd_prefix =

Example configuration

The Airflow module supports the standard configuration options that are described in Modules. Here is an example configuration:

metricbeat.modules:
- module: airflow
  host: "localhost"
  port: "8126"
  #ttl: "30s"
  metricsets: [ 'statsd' ]

Metricsets

The following metricsets are available: