aws s3_daily_storage metricset

The s3_daily_storage metricset of aws module allows you to monitor your AWS S3 buckets. s3_daily_storage metricset fetches Cloudwatch daily storage metrics for each S3 bucket from S3 CloudWatch Daily Storage Metrics for Buckets.

AWS Permissions

Some specific AWS permissions are required for IAM user to collect AWS s3_daily_storage metrics.

ec2:DescribeRegions
cloudwatch:GetMetricData
cloudwatch:ListMetrics
sts:GetCallerIdentity
iam:ListAccountAliases

Dashboard

The aws s3_daily_storage metricset and s3_request metricset shares one predefined dashboard. For example:

metricbeat aws s3 overview

Configuration example

- module: aws
  period: 86400s
  metricsets:
    - s3_daily_storage
  access_key_id: '<access_key_id>'
  secret_access_key: '<secret_access_key>'
  session_token: '<session_token>'

Fields

For a description of each field in the metricset, see the exported fields section.

Here is an example document generated by this metricset:

{
    "@timestamp": "2017-10-12T08:05:34.853Z",
    "aws": {
        "s3": {
            "bucket": {
                "name": "test-s3-ks-2"
            }
        },
        "s3_daily_storage": {
            "bucket": {
                "size": {
                    "bytes": 207372
                }
            },
            "number_of_objects": 128
        }
    },
    "cloud": {
        "account": {
            "id": "627959692251",
            "name": "elastic-test"
        },
        "provider": "aws",
        "region": "ap-southeast-1"
    },
    "event": {
        "dataset": "aws.s3_daily_storage",
        "duration": 115000,
        "module": "aws"
    },
    "metricset": {
        "name": "s3_daily_storage",
        "period": 10000
    },
    "service": {
        "type": "aws"
    }
}