aws ec2 metricset

The ec2 metricset of aws module allows you to monitor your AWS EC2 instances, including cpu, network, disk and status. ec2 metricset fetches a set of values from Cloudwatch AWS EC2 Metrics.

We fetch the following data:

  • cpu.total.pct: The percentage of allocated EC2 compute units that are currently in use on the instance.
  • cpu.credit_usage: The number of CPU credits spent by the instance for CPU utilization.
  • cpu.credit_balance: The number of earned CPU credits that an instance has accrued since it was launched or started.
  • cpu.surplus_credit_balance: The number of surplus credits that have been spent by an unlimited instance when its CPUCreditBalance value is zero.
  • cpu.surplus_credits_charged: The number of spent surplus credits that are not paid down by earned CPU credits, and which thus incur an additional charge.
  • network.in.packets: The number of packets received on all network interfaces by the instance.
  • network.out.packets: The number of packets sent out on all network interfaces by the instance.
  • network.in.bytes: The number of bytes received on all network interfaces by the instance.
  • network.out.bytes: The number of bytes sent out on all network interfaces by the instance.
  • diskio.read.bytes: Bytes read from all instance store volumes available to the instance.
  • diskio.write.bytes: Bytes written to all instance store volumes available to the instance.
  • diskio.read.ops: Completed read operations from all instance store volumes available to the instance in a specified period of time.
  • diskio.write.ops: Completed write operations to all instance store volumes available to the instance in a specified period of time.
  • status.check_failed: Reports whether the instance has passed both the instance status check and the system status check in the last minute.
  • status.check_failed_system: Reports whether the instance has passed the system status check in the last minute.
  • status.check_failed_instance: Reports whether the instance has passed the instance status check in the last minute.
  • instance.core.count: The number of CPU cores for the instance.
  • instance.image.id: The ID of the image used to launch the instance.
  • instance.monitoring.state: Indicates whether detailed monitoring is enabled.
  • instance.private.dns_name: The private DNS name of the network interface.
  • instance.private.ip: The private IPv4 address associated with the network interface.
  • instance.public.dns_name: The public DNS name of the instance.
  • instance.public.ip: The address of the Elastic IP address (IPv4) bound to the network interface.
  • instance.state.code: The state of the instance, as a 16-bit unsigned integer.
  • instance.threads_per_core: The state of the instance (pending | running | shutting-down | terminated | stopping | stopped).

AWS Permissions

Some specific AWS permissions are required for IAM user to collect AWS EC2 metrics.

ec2:DescribeInstances
ec2:DescribeRegions
cloudwatch:GetMetricData
cloudwatch:ListMetrics

Dashboard

The aws ec2 metricset comes with a predefined dashboard. For example:

metricbeat aws ec2 overview

Fields

For a description of each field in the metricset, see the exported fields section.

Here is an example document generated by this metricset:

{
    "@timestamp": "2017-10-12T08:05:34.853Z",
    "agent": {
        "hostname": "host.example.com",
        "name": "host.example.com"
    },
    "aws": {
        "ec2": {
            "cpu": {
                "credit_balance": 144,
                "credit_usage": 0.001823,
                "surplus_credit_balance": 0,
                "surplus_credits_charged": 0,
                "total": {
                    "pct": 0.033333333333303
                }
            },
            "diskio": {
                "read": {
                    "bytes": 0,
                    "count": 0
                },
                "write": {
                    "bytes": 0,
                    "count": 0
                }
            },
            "instance": {
                "core": {
                    "count": 1
                },
                "image": {
                    "id": "ami-05b3bcf7f311194b3"
                },
                "monitoring": {
                    "state": "disabled"
                },
                "private": {
                    "dns_name": "ip-172-31-10-23.ap-southeast-1.compute.internal",
                    "ip": "172.31.10.23"
                },
                "public": {
                    "dns_name": "ec2-18-136-198-93.ap-southeast-1.compute.amazonaws.com",
                    "ip": "18.136.198.93"
                },
                "state": {
                    "code": 16,
                    "name": "running"
                },
                "threads_per_core": 1
            },
            "network": {
                "in": {
                    "bytes": 56,
                    "packets": 1
                },
                "out": {
                    "bytes": 88,
                    "packets": 1.6
                }
            },
            "status": {
                "check_failed": 0,
                "check_failed_instance": 0,
                "check_failed_system": 0
            }
        }
    },
    "cloud": {
        "availability_zone": "ap-southeast-1b",
        "instance": {
            "id": "i-0c68eeb552231a8d0"
        },
        "machine": {
            "type": "t2.micro"
        },
        "provider": "aws",
        "region": "ap-southeast-1"
    },
    "event": {
        "dataset": "aws.ec2",
        "duration": 115000,
        "module": "aws"
    },
    "metricset": {
        "name": "ec2"
    },
    "service": {
        "name": "ec2",
        "type": "ec2"
    }
}