Google Cloud Platform carbon metricsetedit
This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.
The carbon metricset is designed to collect Carbon Footprint data from GCP
BigQuery monthly cost detail table. BigQuery is a fully-managed, serverless
data warehouse.
Cloud Carbon export to BigQuery enables you to export detailed Google Cloud carbon footprint data (such as carbon produced by tier and service) automatically throughout the month to a BigQuery dataset that you specify. Then you can access your Cloud Carbon data from BigQuery for detailed analysis using Metricbeat. Please see export carbon footprint data to BigQuery for more details on how to export carbon footprint data.
Metricset-specific configuration notesedit
- dataset_id: (Required) Dataset ID that points to the top-level container which contains the actual carbon footprint tables.
-
table_pattern: (Optional) The name of the table where carbon footprint data
is stored. Default to
carbon_footprint.
Configuration exampleedit
- module: gcp
metricsets:
- carbon
period: 24h
project_id: "your project id"
credentials_file_path: "your JSON credentials file path"
dataset_id: "dataset id"
table_name: "table name"
For a description of each field in the metricset, see the exported fields section.
Here is an example document generated by this metricset:
{
"@timestamp": "2017-10-12T08:05:34.853Z",
"cloud.account.id": "01475F-5B1080-1137E7",
"cloud.project.id": "elastic-bi",
"cloud.project.name": "elastic-containerlib-prod",
"cloud.provider": "gcp",
"event": {
"dataset": "gcp.carbon",
"duration": 115000,
"module": "gcp"
},
"gcp": {
"carbon": {
"project_id": "containerlib-prod-12763",
"project_name": "elastic-containerlib-prod",
"service_id": "24E6-581D-38E5",
"service_description": "BigQuery",
"footprint.scope1": 4.044,
"footprint.scope2.location": 1.797,
"footprint.scope2.market": null,
"footprint.scope3": 2.337,
"footprint.offsets": null
}
},
"metricset": {
"name": "carbon",
"period": 10000
},
"service": {
"type": "gcp"
}
}