This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.
Amazon ECS on Fargate provides a method to retrieve various metadata, network metrics, and Docker stats about tasks and containers. This is referred to as the task metadata endpoint and this endpoint is available per container.
The environment variable is injected by default into the containers of Amazon ECS tasks on Fargate that use platform version 1.4.0 or later and Amazon ECS tasks on Amazon EC2 that are running at least version 1.39.0 of the Amazon ECS container agent.
The awsfargate module is a Metricbeat module which collects AWS fargate metrics from task metadata endpoint.
Introduction to AWS ECS and Fargateedit
Amazon Elastic Container Service (Amazon ECS) is a highly scalable, fast, container management service that makes it easy to run, stop, and manage containers. ECS has two launch types that can define how compute resources will be managed: ECS EC2 and ECS Fargate.
- ECS EC2
ECS EC2 launches containers that run on EC2 instances. Users have to manage EC2 instances. Pricing depends on the number of EC2 instances running.
One can monitor these containers by deploying Metricbeat on the corresponding EC2 instances with the Metricbeat Docker module enabled.
In order to achieve this one will need:
to ensure access to these EC2 instances using ssh keys
coupled with EC2 instances (attach ssh keys on cluster creation using
- to enable shh access for the instances with the proper inbound rules.
- ECS Fargate
ECS Fargate removes the responsibility of provisioning, configuring, and managing the EC2 instances by allowing AWS to manage the EC2 instances. Users only need to specify containers and tasks. Pricing based on the number of tasks.
Task Metadata Endpointedit
Task metadata endpoint returns Docker stats in JSON format for all the containers associated with the task. This endpoint is only available from within the task definition itself, which means Metricbeat needs to be run as a sidecar container within the task definition. Since the metadata endpoint is only accessible from within the Fargate Task, there is no authentication in place.
Currently, we have
task_stats metricset in
This metricset collects runtime CPU metrics, disk I/O metrics, memory metrics,
network metrics and container metadata from both endpoint
The AWS Fargate module supports the standard configuration options that are described in Modules. Here is an example configuration:
metricbeat.modules: - module: awsfargate period: 10s metricsets: - task_stats
The following metricsets are available:
Intro to Kibana
ELK for Logs & Metrics