Documentation Grafana Agent Flow mode Reference Components reference prometheus.exporter.cloudwatch

prometheus.exporter.cloudwatch

The prometheus.exporter.cloudwatch component embeds yet-another-cloudwatch-exporter, letting you collect CloudWatch metrics, translate them to a prometheus-compatible format and remote write them.

This component lets you scrape CloudWatch metrics in a set of configurations we call jobs. There are two kinds of jobs: discovery and static.

Authentication

Grafana Agent Flow must be running in an environment with access to AWS. The exporter uses the AWS SDK for Go and provides authentication via AWS’s default credential chain. Regardless of the method used to acquire the credentials, some permissions are needed for the exporter to work.

Copy

"tag:GetResources",
"cloudwatch:GetMetricData",
"cloudwatch:GetMetricStatistics",
"cloudwatch:ListMetrics"

The following IAM permissions are required for the Transit Gateway attachment (tgwa) metrics to work.

Copy

"ec2:DescribeTags",
"ec2:DescribeInstances",
"ec2:DescribeRegions",
"ec2:DescribeTransitGateway*"

The following IAM permission is required to discover tagged API Gateway REST APIs:

Copy

"apigateway:GET"

The following IAM permissions are required to discover tagged Database Migration Service (DMS) replication instances and tasks:

Copy

"dms:DescribeReplicationInstances",
"dms:DescribeReplicationTasks"

To use all of the integration features, use the following AWS IAM Policy:

json Copy

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1674249227793",
      "Action": [
        "tag:GetResources",
        "cloudwatch:GetMetricData",
        "cloudwatch:GetMetricStatistics",
        "cloudwatch:ListMetrics",
        "ec2:DescribeTags",
        "ec2:DescribeInstances",
        "ec2:DescribeRegions",
        "ec2:DescribeTransitGateway*",
        "apigateway:GET",
        "dms:DescribeReplicationInstances",
        "dms:DescribeReplicationTasks"
      ],
      "Effect": "Allow",
      "Resource": "*"
    }
  ]
}

Usage

river Copy

prometheus.exporter.cloudwatch "queues" {
	sts_region = "us-east-2"

	discovery {
		type        = "sqs"
		regions     = ["us-east-2"]
		search_tags = {
			"scrape" = "true",
		}

		metric {
			name       = "NumberOfMessagesSent"
			statistics = ["Sum", "Average"]
			period     = "1m"
		}

		metric {
			name       = "NumberOfMessagesReceived"
			statistics = ["Sum", "Average"]
			period     = "1m"
		}
	}
}

Arguments

You can use the following arguments to configure the exporter’s behavior. Omitted fields take their default values.

Blocks

You can use the following blocks inprometheus.exporter.cloudwatch to configure collector-specific options:

discovery block

The discovery block allows the component to scrape CloudWatch metrics with only the AWS service and a list of metrics under that service/namespace. Grafana Agent Flow will find AWS resources in the specified service for which to scrape these metrics, label them appropriately, and export them to Prometheus. For example, if we wanted to scrape CPU utilization and network traffic metrics from all AWS EC2 instances:

river Copy

prometheus.exporter.cloudwatch "discover_instances" {
	sts_region = "us-east-2"

	discovery {
		type    = "AWS/EC2"
		regions = ["us-east-2"]

		metric {
			name       = "CPUUtilization"
			statistics = ["Average"]
			period     = "5m"
		}

		metric {
			name       = "NetworkPacketsIn"
			statistics = ["Average"]
			period     = "5m"
		}
	}
}

You can configure the discovery block one or multiple times to scrape metrics from different services or with different search_tags.

static block

The static block configures the component to scrape a specific set of CloudWatch metrics. The metrics need to be fully qualified with the following specifications:

1. namespace: For example, AWS/EC2, AWS/EBS, CoolApp if it were a custom metric, etc.
2. dimensions: CloudWatch identifies a metric by a set of dimensions, which are essentially label / value pairs. For example, all AWS/EC2 metrics are identified by the InstanceId dimension and the identifier itself.
3. metric: Metric name and statistics.

For example, if you want to scrape the same metrics in the discovery example, but for a specific AWS EC2 instance:

river Copy

prometheus.exporter.cloudwatch "static_instances" {
	sts_region = "us-east-2"

	static "instances" {
		regions    = ["us-east-2"]
		namespace  = "AWS/EC2"
		dimensions = {
			"InstanceId" = "i01u29u12ue1u2c",
		}

		metric {
			name       = "CPUUsage"
			statistics = ["Sum", "Average"]
			period     = "1m"
		}
	}
}

As shown above, static blocks must be specified with a label, which will translate to the name label in the exported metric.

river Copy

static "LABEL" {
    regions    = ["us-east-2"]
    namespace  = "AWS/EC2"
    // ...
}

You can configure the static block one or multiple times to scrape metrics with different sets of dimensions.

All dimensions must be specified when scraping single metrics like the example above. For example, AWS/Logs metrics require Resource, Service, Class, and Type dimensions to be specified. The same applies to CloudWatch custom metrics, all dimensions attached to a metric when saved in CloudWatch are required.

metric block

Represents an AWS Metrics to scrape. To see available metrics, AWS does not keep a documentation page with all available metrics. Follow this guide on how to explore metrics, to easily pick the ones you need.

period and length

period controls primarily the width of the time bucket used for aggregating metrics collected from CloudWatch. length controls how far back in time CloudWatch metrics are considered during each Grafana Agent scrape. If both settings are configured, the time parameters when calling CloudWatch APIs works as follows:

As noted above, if across multiple metrics under the same static or discovery job, there’s different period and/or length the minimum of all periods, and maximum of all lengths is configured.

On the other hand, if length is not configured, both period and length settings will be calculated based on the required period configuration attribute.

If all metrics within a job (discovery or static) have the same period value configured, CloudWatch APIs will be requested for metrics from the scrape time, to periods seconds in the past. The values of these are exported to Prometheus.

On the other hand, if metrics with different periods are configured under an individual job, this works differently. First, two variables are calculated aggregating all periods: length, taking the maximum value of all periods, and the new period value, taking the minimum of all periods. Then, CloudWatch APIs will be requested for metrics from now - length to now, aggregating each in samples for period seconds. For each metric, the most recent sample is exported to CloudWatch.

role block

Represents an AWS IAM Role. If omitted, the AWS role that corresponds to the credentials configured in the environment will be used.

Multiple roles can be useful when scraping metrics from different AWS accounts with a single pair of credentials. In this case, a different role is configured for Grafana Agent to assume before calling AWS APIs. Therefore, the credentials configured in the system need permission to assume the target role. See Granting a user permissions to switch roles in the AWS IAM documentation for more information about how to configure this.

decoupled_scraping block

The decoupled_scraping block configures an optional feature that scrapes CloudWatch metrics in the background on a scheduled interval. When this feature is enabled, CloudWatch metrics are gathered asynchronously at the scheduled interval instead of synchronously when the CloudWatch component is scraped.

The decoupled scraping feature reduces the number of API requests sent to AWS. This feature also prevents component scrape timeouts when you gather high volumes of CloudWatch metrics.

Exported fields

The following fields are exported and can be referenced by other components.

For example, the targets can either be passed to a discovery.relabel component to rewrite the targets’ label sets or to a prometheus.scrape component that collects the exposed metrics.

The exported targets use the configured in-memory traffic address specified by the run command.

Component health

prometheus.exporter.cloudwatch is only reported as unhealthy if given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

prometheus.exporter.cloudwatch does not expose any component-specific debug information.

Debug metrics

prometheus.exporter.cloudwatch does not expose any component-specific debug metrics.

Example

See the examples described under each discovery and static sections.

Supported services in discovery jobs

The following is a list of AWS services that are supported in cloudwatch_exporter discovery jobs. When configuring a discovery job, the type field of each discovery_job must match either the desired job namespace or alias.

• Namespace: CWAgent or Alias: cwagent
• Namespace: AWS/Usage or Alias: usage
• Namespace: AWS/CertificateManager or Alias: acm
• Namespace: AWS/ACMPrivateCA or Alias: acm-pca
• Namespace: AmazonMWAA or Alias: airflow
• Namespace: AWS/MWAA or Alias: mwaa
• Namespace: AWS/ApplicationELB or Alias: alb
• Namespace: AWS/AppStream or Alias: appstream
• Namespace: AWS/Backup or Alias: backup
• Namespace: AWS/ApiGateway or Alias: apigateway
• Namespace: AWS/AmazonMQ or Alias: mq
• Namespace: AWS/AppSync or Alias: appsync
• Namespace: AWS/Athena or Alias: athena
• Namespace: AWS/AutoScaling or Alias: asg
• Namespace: AWS/ElasticBeanstalk or Alias: beanstalk
• Namespace: AWS/Billing or Alias: billing
• Namespace: AWS/Cassandra or Alias: cassandra
• Namespace: AWS/CloudFront or Alias: cloudfront
• Namespace: AWS/Cognito or Alias: cognito-idp
• Namespace: AWS/DMS or Alias: dms
• Namespace: AWS/DDoSProtection or Alias: shield
• Namespace: AWS/DocDB or Alias: docdb
• Namespace: AWS/DX or Alias: dx
• Namespace: AWS/DynamoDB or Alias: dynamodb
• Namespace: AWS/EBS or Alias: ebs
• Namespace: AWS/ElastiCache or Alias: ec
• Namespace: AWS/MemoryDB or Alias: memorydb
• Namespace: AWS/EC2 or Alias: ec2
• Namespace: AWS/EC2Spot or Alias: ec2Spot
• Namespace: AWS/ECS or Alias: ecs-svc
• Namespace: ECS/ContainerInsights or Alias: ecs-containerinsights
• Namespace: AWS/EFS or Alias: efs
• Namespace: AWS/ELB or Alias: elb
• Namespace: AWS/ElasticMapReduce or Alias: emr
• Namespace: AWS/EMRServerless or Alias: emr-serverless
• Namespace: AWS/ES or Alias: es
• Namespace: AWS/Firehose or Alias: firehose
• Namespace: AWS/FSx or Alias: fsx
• Namespace: AWS/GameLift or Alias: gamelift
• Namespace: AWS/GlobalAccelerator or Alias: ga
• Namespace: Glue or Alias: glue
• Namespace: AWS/IoT or Alias: iot
• Namespace: AWS/Kafka or Alias: kafka
• Namespace: AWS/KafkaConnect or Alias: kafkaconnect
• Namespace: AWS/Kinesis or Alias: kinesis
• Namespace: AWS/KinesisAnalytics or Alias: kinesis-analytics
• Namespace: AWS/Lambda or Alias: lambda
• Namespace: AWS/MediaConnect or Alias: mediaconnect
• Namespace: AWS/MediaConvert or Alias: mediaconvert
• Namespace: AWS/MediaLive or Alias: medialive
• Namespace: AWS/MediaTailor or Alias: mediatailor
• Namespace: AWS/Neptune or Alias: neptune
• Namespace: AWS/NetworkFirewall or Alias: nfw
• Namespace: AWS/NATGateway or Alias: ngw
• Namespace: AWS/NetworkELB or Alias: nlb
• Namespace: AWS/PrivateLinkEndpoints or Alias: vpc-endpoint
• Namespace: AWS/PrivateLinkServices or Alias: vpc-endpoint-service
• Namespace: AWS/Prometheus or Alias: amp
• Namespace: AWS/QLDB or Alias: qldb
• Namespace: AWS/RDS or Alias: rds
• Namespace: AWS/Redshift or Alias: redshift
• Namespace: AWS/Route53Resolver or Alias: route53-resolver
• Namespace: AWS/Route53 or Alias: route53
• Namespace: AWS/S3 or Alias: s3
• Namespace: AWS/SES or Alias: ses
• Namespace: AWS/States or Alias: sfn
• Namespace: AWS/SNS or Alias: sns
• Namespace: AWS/SQS or Alias: sqs
• Namespace: AWS/StorageGateway or Alias: storagegateway
• Namespace: AWS/TransitGateway or Alias: tgw
• Namespace: AWS/TrustedAdvisor or Alias: trustedadvisor
• Namespace: AWS/VPN or Alias: vpn
• Namespace: AWS/ClientVPN or Alias: clientvpn
• Namespace: AWS/WAFV2 or Alias: wafv2
• Namespace: AWS/WorkSpaces or Alias: workspaces
• Namespace: AWS/AOSS or Alias: aoss
• Namespace: AWS/SageMaker or Alias: sagemaker
• Namespace: /aws/sagemaker/Endpoints or Alias: sagemaker-endpoints
• Namespace: /aws/sagemaker/TrainingJobs or Alias: sagemaker-training
• Namespace: /aws/sagemaker/ProcessingJobs or Alias: sagemaker-processing
• Namespace: /aws/sagemaker/TransformJobs or Alias: sagemaker-transform
• Namespace: /aws/sagemaker/InferenceRecommendationsJobs or Alias: sagemaker-inf-rec
• Namespace: AWS/Sagemaker/ModelBuildingPipeline or Alias: sagemaker-model-building-pipeline

Compatible components

prometheus.exporter.cloudwatch has exports that can be consumed by the following components:

• Components that consume Targets