Caution
Grafana Alloy is the new name for our distribution of the OTel collector. Grafana Agent has been deprecated and is in Long-Term Support (LTS) through October 31, 2025. Grafana Agent will reach an End-of-Life (EOL) on November 1, 2025. Read more about why we recommend migrating to Grafana Alloy.
Important: This documentation is about an older version. It's relevant only to the release noted, many of the features and functions have been updated or replaced. Please view the current version.
prometheus.operator.podmonitors
BETA: This is a beta component. Beta components are subject to breaking changes, and may be replaced with equivalent functionality that cover the same use case.
prometheus.operator.podmonitors
discovers PodMonitor resources in your kubernetes cluster and scrapes the targets they reference. This component performs three main functions:
- Discover PodMonitor resources from your Kubernetes cluster.
- Discover Pods in your cluster that match those PodMonitors.
- Scrape metrics from those Pods, and forward them to a receiver.
The default configuration assumes the agent is running inside a Kubernetes cluster, and uses the in-cluster config to access the Kubernetes API. It can be run from outside the cluster by supplying connection info in the client
block, but network level access to pods is required to scrape metrics from them.
Usage
prometheus.operator.podmonitors "LABEL" {
forward_to = RECEIVER_LIST
}
Arguments
The following arguments are supported:
Name | Type | Description | Default | Required |
---|---|---|---|---|
forward_to | list(MetricsReceiver) | List of receivers to send scraped metrics to. | yes | |
namespaces | list(string) | List of namespaces to search for PodMonitor resources. If not specified, all namespaces will be searched. | no |
Blocks
The following blocks are supported inside the definition of prometheus.operator.podmonitors
:
Hierarchy | Block | Description | Required |
---|---|---|---|
client | client | Configures Kubernetes client used to find PodMonitors. | no |
client > basic_auth | basic_auth | Configure basic authentication to the Kubernetes API. | no |
client > authorization | authorization | Configure generic authorization to the Kubernetes API. | no |
client > oauth2 | oauth2 | Configure OAuth2 for authenticating to the Kubernetes API. | no |
client > oauth2 > tls_config | tls_config | Configure TLS settings for connecting to the Kubernetes API. | no |
client > tls_config | tls_config | Configure TLS settings for connecting to the Kubernetes API. | no |
selector | selector | Label selector for which PodMonitors to discover. | no |
selector > match_expression | match_expression | Label selector expression for which PodMonitors to discover. | no |
The >
symbol indicates deeper levels of nesting. For example, client > basic_auth
refers to a basic_auth
block defined
inside a client
block.
client block
The client
block configures the Kubernetes client used to discover PodMonitors. If the client
block isn’t provided, the default in-cluster
configuration with the service account of the running Grafana Agent pod is
used.
The following arguments are supported:
Name | Type | Description | Default | Required |
---|---|---|---|---|
api_server | string | URL of the Kubernetes API server. | no | |
kubeconfig_file | string | Path of the kubeconfig file to use for connecting to Kubernetes. | no | |
bearer_token_file | string | File containing a bearer token to authenticate with. | no | |
proxy_url | string | HTTP proxy to proxy requests through. | no | |
follow_redirects | bool | Whether redirects returned by the server should be followed. | true | no |
enable_http2 | bool | Whether HTTP2 is supported for requests. | true | no |
At most one of the following can be provided:
basic_auth block
Name | Type | Description | Default | Required |
---|---|---|---|---|
username | string | Basic auth username. | no | |
password | secret | Basic auth password. | no | |
password_file | string | File containing the basic auth password. | no |
password
and password_file
are mutually exclusive and only one can be
provided inside of a basic_auth
block.
authorization block
Name | Type | Description | Default | Required |
---|---|---|---|---|
type | string | Authorization type, for example, “Bearer”. | no | |
credentials | secret | Secret value. | no | |
credentials_file | string | File containing the secret value. | no |
credential
and credentials_file
are mutually exclusive and only one can be
provided inside of an authorization
block.
oauth2 block
Name | Type | Description | Default | Required |
---|---|---|---|---|
client_id | string | OAuth2 client ID. | no | |
client_secret | secret | OAuth2 client secret. | no | |
client_secret_file | string | File containing the OAuth2 client secret. | no | |
scopes | list(string) | List of scopes to authenticate with. | no | |
token_url | string | URL to fetch the token from. | no | |
endpoint_params | map(string) | Optional parameters to append to the token URL. | no | |
proxy_url | string | Optional proxy URL for OAuth2 requests. | no |
client_secret
and client_secret_file
are mutually exclusive and only one
can be provided inside of an oauth2
block.
The oauth2
block may also contain its own separate tls_config
sub-block.
tls_config block
Name | Type | Description | Default | Required |
---|---|---|---|---|
ca_pem | string | CA PEM-encoded text to validate the server with. | no | |
ca_file | string | CA certificate to validate the server with. | no | |
cert_pem | string | Certificate PEM-encoded text for client authentication. | no | |
cert_file | string | Certificate file for client authentication. | no | |
key_pem | secret | Key PEM-encoded text for client authentication. | no | |
key_file | string | Key file for client authentication. | no | |
server_name | string | ServerName extension to indicate the name of the server. | no | |
insecure_skip_verify | bool | Disables validation of the server certificate. | no | |
min_version | string | Minimum acceptable TLS version. | no |
The following pairs of arguments are mutually exclusive and cannot both be set simultaneously:
ca_pem
andca_file
cert_pem
andcert_file
key_pem
andkey_file
When configuring client authentication, both the client certificate (using
cert_pem
or cert_file
) and the client key (using key_pem
or key_file
)
must be provided.
When min_version
is not provided, the minimum acceptable TLS version is
inherited from Go’s default minimum version, TLS 1.2. If min_version
is
provided, it must be set to one of the following strings:
"TLS10"
(TLS 1.0)"TLS11"
(TLS 1.1)"TLS12"
(TLS 1.2)"TLS13"
(TLS 1.3)
selector block
The selector
block describes a Kubernetes label selector for PodMonitors.
The following arguments are supported:
Name | Type | Description | Default | Required |
---|---|---|---|---|
match_labels | map(string) | Label keys and values used to discover resources. | {} | no |
When the match_labels
argument is empty, all PodMonitor resources will be matched.
match_expression block
The match_expression
block describes a Kubernetes label matcher expression for
PodMonitors discovery.
The following arguments are supported:
Name | Type | Description | Default | Required |
---|---|---|---|---|
key | string | The label name to match against. | yes | |
operator | string | The operator to use when matching. | yes | |
values | list(string) | The values used when matching. | no |
The operator
argument must be one of the following strings:
"In"
"NotIn"
"Exists"
"DoesNotExist"
If there are multiple match_expressions
blocks inside of a selector
block, they are combined together with AND clauses.
Exported fields
prometheus.operator.podmonitors
does not export any fields. It forwards all metrics it scrapes to the receiver configures with the forward_to
argument.
Component health
prometheus.operator.podmonitors
is reported as unhealthy when given an invalid configuration, Prometheus components fail to initialize, or the connection to the Kubernetes API could not be established properly.
Debug information
prometheus.operator.podmonitors
reports the status of the last scrape for each configured
scrape job on the component’s debug endpoint, including discovered labels, and the last scrape time.
It also exposes some debug information for each PodMonitor it has discovered, including any errors found while reconciling the scrape configuration from the PodMonitor.
Debug metrics
Example
This example discovers all PodMonitors in your cluster, and forwards collected metrics to a prometheus.remote_write
component.
prometheus.remote_write "staging" {
// Send metrics to a locally running Mimir.
endpoint {
url = "http://mimir:9009/api/v1/push"
basic_auth {
username = "example-user"
password = "example-password"
}
}
}
prometheus.operator.podmonitors "pods" {
forward_to = [prometheus.remote_write.staging.receiver]
}
This example will limit discovered PodMonitors to ones with the label team=ops
in a specific namespace: my-app
.
prometheus.operator.podmonitors "pods" {
forward_to = [prometheus.remote_write.staging.receiver]
namespaces = ["my-app"]
selector {
match_expression {
key = "team"
operator = "In"
values = ["ops"]
}
}
}