Monitor Amazon Simple Storage Service (S3)edit

Amazon Simple Storage Service (S3) is a highly available object storage service that provides durability, security, and scalability. To store data, you create one or more buckets that contain objects. An object consists of a file and optional metadata that describes the file. For each bucket, you can control access to it.

Like most AWS services, Amazon S3 and Amazon CloudWatch are integrated so you can collect, view, and analyze CloudWatch metrics for your S3 buckets to help understand and improve the performance of applications that use Amazon S3. The Elastic Amazon S3 integration collects metrics from Amazon CloudWatch using Elastic Agent.

With the Amazon S3 integration, you can collect these S3 metrics from CloudWatch:

  • Daily storage metrics for buckets. Use these metrics to monitor bucket storage. These metrics are reported once per day by default and are provided to AWS customers at no additional cost.
  • Request metrics. Use these metrics to quickly identify and act on operational issues. These request metrics are available at one-minute intervals after some latency for processing, and they are not enabled by default.
Get startededit

If you plan to collect request metrics, enable them for the S3 buckets you want to monitor. To learn how, refer to the AWS documentation.

To collect S3 metrics, you typically need to install the Elastic Amazon S3 integration and deploy an Elastic Agent locally or on an EC2 instance.

Expand the quick guide to learn how, or skip to the next section if your data is already in Elasticsearch.

Quick guide: Add data

Elastic Agent is currently the preferred way to add S3 metrics. For other ways, refer to Adding data to Elasticsearch.

Dashboardsedit

{kibana-desc} For example, to see an overview of your S3 metrics in Kibana, go to the Dashboard app and navigate to the [Metrics AWS] S3 Overview dashboard.

Screenshot showing the S3 dashboard
Metrics to watchedit

Here are the key metrics that you should watch, organized by category. For a full list of fields exported by the integration, refer to the Amazon S3 integration docs.

  • Daily storage metrics for buckets

    • aws.s3_daily_storage.number_of_objects
    • aws.s3_daily_storage.bucket.size.bytes
  • Request metrics

    • aws.s3_request.requests.total
    • aws.s3_request.requests.get
    • aws.s3_request.requests.put
    • aws.s3_request.requests.delete
    • aws.s3_request.requests.head
    • aws.s3_request.requests.post
    • aws.s3_request.requests.select
    • aws.s3_request.requests.list
    • aws.s3_request.requests.select_scanned.bytes
    • aws.s3_request.requests.select_returned.bytes
    • aws.s3_request.downloaded.bytes
    • aws.s3_request.uploaded.bytes
    • aws.s3_request.downloaded.bytes_per_period
    • aws.s3_request.uploaded.bytes_per_period
    • aws.s3_request.errors.4xx
    • aws.s3_request.errors.5xx
    • aws.s3_request.latency.first_byte.ms
    • aws.s3_request.latency.total_request.ms