aws ec2 metricsetedit

The ec2 metricset of aws module allows you to monitor your AWS EC2 instances, including cpu, network, disk and status. ec2 metricset fetches a set of values from Cloudwatch AWS EC2 Metrics.

We fetch the following data:

  • cpu.total.pct: The percentage of allocated EC2 compute units that are currently in use on the instance.
  • cpu.credit_usage: The number of CPU credits spent by the instance for CPU utilization.
  • cpu.credit_balance: The number of earned CPU credits that an instance has accrued since it was launched or started.
  • cpu.surplus_credit_balance: The number of surplus credits that have been spent by an unlimited instance when its CPUCreditBalance value is zero.
  • cpu.surplus_credits_charged: The number of spent surplus credits that are not paid down by earned CPU credits, and which thus incur an additional charge.
  • network.in.packets: The number of packets received on all network interfaces by the instance.
  • network.out.packets: The number of packets sent out on all network interfaces by the instance.
  • network.in.bytes: The number of bytes received on all network interfaces by the instance.
  • network.out.bytes: The number of bytes sent out on all network interfaces by the instance.
  • diskio.read.bytes: Bytes read from all instance store volumes available to the instance.
  • diskio.write.bytes: Bytes written to all instance store volumes available to the instance.
  • diskio.read.ops: Completed read operations from all instance store volumes available to the instance in a specified period of time.
  • diskio.write.ops: Completed write operations to all instance store volumes available to the instance in a specified period of time.
  • status.check_failed: Reports whether the instance has passed both the instance status check and the system status check in the last minute.
  • status.check_failed_system: Reports whether the instance has passed the system status check in the last minute.
  • status.check_failed_instance: Reports whether the instance has passed the instance status check in the last minute.
  • instance.core.count: The number of CPU cores for the instance.
  • instance.image.id: The ID of the image used to launch the instance.
  • instance.monitoring.state: Indicates whether detailed monitoring is enabled.
  • instance.private.dns_name: The private DNS name of the network interface.
  • instance.private.ip: The private IPv4 address associated with the network interface.
  • instance.public.dns_name: The public DNS name of the instance.
  • instance.public.ip: The address of the Elastic IP address (IPv4) bound to the network interface.
  • instance.state.code: The state of the instance, as a 16-bit unsigned integer.
  • instance.threads_per_core: The state of the instance (pending | running | shutting-down | terminated | stopping | stopped).

AWS Permissionsedit

Some specific AWS permissions are required for IAM user to collect AWS EC2 metrics.

ec2:DescribeInstances
ec2:DescribeRegions
cloudwatch:GetMetricData
cloudwatch:ListMetrics

Dashboardedit

The aws ec2 metricset comes with a predefined dashboard. For example:

./images/metricbeat-aws-ec2-overview.png

Fieldsedit

For a description of each field in the metricset, see the exported fields section.

Here is an example document generated by this metricset:

{
    "@timestamp": "2017-10-12T08:05:34.853Z",
    "aws": {
        "ec2": {
            "cpu": {
                "total": {
                    "pct": 0.7000000000116418
                }
            },
            "diskio": {
                "read": {
                    "bytes": 0,
                    "count": 0
                },
                "write": {
                    "bytes": 0,
                    "count": 0
                }
            },
            "instance": {
                "core": {
                    "count": 1
                },
                "image": {
                    "id": "ami-62d35c02"
                },
                "monitoring": {
                    "state": "enabled"
                },
                "private": {
                    "dns_name": "ip-10-240-32-59.us-west-2.compute.internal",
                    "ip": "10.240.32.59"
                },
                "public": {
                    "dns_name": ""
                },
                "state": {
                    "code": 16,
                    "name": "running"
                },
                "threads_per_core": 2
            },
            "network": {
                "in": {
                    "bytes": 16598.6,
                    "packets": 127.8
                },
                "out": {
                    "bytes": 29934.4,
                    "packets": 225.4
                }
            },
            "status": {
                "check_failed": 0,
                "check_failed_instance": 0,
                "check_failed_system": 0
            }
        },
        "tags": {
            "Name": "service-node",
            "aws:autoscaling:groupName": "service-asg"
        }
    },
    "cloud": {
        "availability_zone": "us-west-2b",
        "instance": {
            "id": "i-05cb8b44489d276f4"
        },
        "machine": {
            "type": "c4.large"
        },
        "provider": "aws",
        "region": "us-west-2"
    },
    "event": {
        "dataset": "aws.ec2",
        "duration": 115000,
        "module": "aws"
    },
    "metricset": {
        "name": "ec2"
    },
    "service": {
        "name": "ec2",
        "type": "ec2"
    }
}