Menu

Caution

Grafana Alloy is the new name for our distribution of the OTel collector. Grafana Agent has been deprecated and is in Long-Term Support (LTS) through October 31, 2025. Grafana Agent will reach an End-of-Life (EOL) on November 1, 2025. Read more about why we recommend migrating to Grafana Alloy.

Important: This documentation is about an older version. It's relevant only to the release noted, many of the features and functions have been updated or replaced. Please view the current version.

Documentationbreadcrumb arrow Grafana Agentbreadcrumb arrow Flow modebreadcrumb arrow Referencebreadcrumb arrow Componentsbreadcrumb arrow prometheus.operator.podmonitors
Beta

prometheus.operator.podmonitors

BETA: This is a beta component. Beta components are subject to breaking changes, and may be replaced with equivalent functionality that cover the same use case.

prometheus.operator.podmonitors discovers PodMonitor resources in your kubernetes cluster and scrapes the targets they reference. This component performs three main functions:

  1. Discover PodMonitor resources from your Kubernetes cluster.
  2. Discover Pods in your cluster that match those PodMonitors.
  3. Scrape metrics from those Pods, and forward them to a receiver.

The default configuration assumes the agent is running inside a Kubernetes cluster, and uses the in-cluster config to access the Kubernetes API. It can be run from outside the cluster by supplying connection info in the client block, but network level access to pods is required to scrape metrics from them.

Usage

river
prometheus.operator.podmonitors "LABEL" {
    forward_to = RECEIVER_LIST
}

Arguments

The following arguments are supported:

NameTypeDescriptionDefaultRequired
forward_tolist(MetricsReceiver)List of receivers to send scraped metrics to.yes
namespaceslist(string)List of namespaces to search for PodMonitor resources. If not specified, all namespaces will be searched.no

Blocks

The following blocks are supported inside the definition of prometheus.operator.podmonitors:

HierarchyBlockDescriptionRequired
clientclientConfigures Kubernetes client used to find PodMonitors.no
client > basic_authbasic_authConfigure basic authentication to the Kubernetes API.no
client > authorizationauthorizationConfigure generic authorization to the Kubernetes API.no
client > oauth2oauth2Configure OAuth2 for authenticating to the Kubernetes API.no
client > oauth2 > tls_configtls_configConfigure TLS settings for connecting to the Kubernetes API.no
client > tls_configtls_configConfigure TLS settings for connecting to the Kubernetes API.no
selectorselectorLabel selector for which PodMonitors to discover.no
selector > match_expressionmatch_expressionLabel selector expression for which PodMonitors to discover.no

The > symbol indicates deeper levels of nesting. For example, client > basic_auth refers to a basic_auth block defined inside a client block.

client block

The client block configures the Kubernetes client used to discover PodMonitors. If the client block isn’t provided, the default in-cluster configuration with the service account of the running Grafana Agent pod is used.

The following arguments are supported:

NameTypeDescriptionDefaultRequired
api_serverstringURL of the Kubernetes API server.no
kubeconfig_filestringPath of the kubeconfig file to use for connecting to Kubernetes.no
bearer_token_filestringFile containing a bearer token to authenticate with.no
proxy_urlstringHTTP proxy to proxy requests through.no
follow_redirectsboolWhether redirects returned by the server should be followed.trueno
enable_http2boolWhether HTTP2 is supported for requests.trueno

At most one of the following can be provided:

basic_auth block

NameTypeDescriptionDefaultRequired
usernamestringBasic auth username.no
passwordsecretBasic auth password.no
password_filestringFile containing the basic auth password.no

password and password_file are mutually exclusive and only one can be provided inside of a basic_auth block.

authorization block

NameTypeDescriptionDefaultRequired
typestringAuthorization type, for example, “Bearer”.no
credentialssecretSecret value.no
credentials_filestringFile containing the secret value.no

credential and credentials_file are mutually exclusive and only one can be provided inside of an authorization block.

oauth2 block

NameTypeDescriptionDefaultRequired
client_idstringOAuth2 client ID.no
client_secretsecretOAuth2 client secret.no
client_secret_filestringFile containing the OAuth2 client secret.no
scopeslist(string)List of scopes to authenticate with.no
token_urlstringURL to fetch the token from.no
endpoint_paramsmap(string)Optional parameters to append to the token URL.no
proxy_urlstringOptional proxy URL for OAuth2 requests.no

client_secret and client_secret_file are mutually exclusive and only one can be provided inside of an oauth2 block.

The oauth2 block may also contain its own separate tls_config sub-block.

tls_config block

NameTypeDescriptionDefaultRequired
ca_pemstringCA PEM-encoded text to validate the server with.no
ca_filestringCA certificate to validate the server with.no
cert_pemstringCertificate PEM-encoded text for client authentication.no
cert_filestringCertificate file for client authentication.no
key_pemsecretKey PEM-encoded text for client authentication.no
key_filestringKey file for client authentication.no
server_namestringServerName extension to indicate the name of the server.no
insecure_skip_verifyboolDisables validation of the server certificate.no
min_versionstringMinimum acceptable TLS version.no

The following pairs of arguments are mutually exclusive and cannot both be set simultaneously:

  • ca_pem and ca_file
  • cert_pem and cert_file
  • key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

When min_version is not provided, the minimum acceptable TLS version is inherited from Go’s default minimum version, TLS 1.2. If min_version is provided, it must be set to one of the following strings:

  • "TLS10" (TLS 1.0)
  • "TLS11" (TLS 1.1)
  • "TLS12" (TLS 1.2)
  • "TLS13" (TLS 1.3)

selector block

The selector block describes a Kubernetes label selector for PodMonitors.

The following arguments are supported:

NameTypeDescriptionDefaultRequired
match_labelsmap(string)Label keys and values used to discover resources.{}no

When the match_labels argument is empty, all PodMonitor resources will be matched.

match_expression block

The match_expression block describes a Kubernetes label matcher expression for PodMonitors discovery.

The following arguments are supported:

NameTypeDescriptionDefaultRequired
keystringThe label name to match against.yes
operatorstringThe operator to use when matching.yes
valueslist(string)The values used when matching.no

The operator argument must be one of the following strings:

  • "In"
  • "NotIn"
  • "Exists"
  • "DoesNotExist"

If there are multiple match_expressions blocks inside of a selector block, they are combined together with AND clauses.

Exported fields

prometheus.operator.podmonitors does not export any fields. It forwards all metrics it scrapes to the receiver configures with the forward_to argument.

Component health

prometheus.operator.podmonitors is reported as unhealthy when given an invalid configuration, Prometheus components fail to initialize, or the connection to the Kubernetes API could not be established properly.

Debug information

prometheus.operator.podmonitors reports the status of the last scrape for each configured scrape job on the component’s debug endpoint, including discovered labels, and the last scrape time.

It also exposes some debug information for each PodMonitor it has discovered, including any errors found while reconciling the scrape configuration from the PodMonitor.

Debug metrics

Example

This example discovers all PodMonitors in your cluster, and forwards collected metrics to a prometheus.remote_write component.

river
prometheus.remote_write "staging" {
  // Send metrics to a locally running Mimir.
  endpoint {
    url = "http://mimir:9009/api/v1/push"

    basic_auth {
      username = "example-user"
      password = "example-password"
    }
  }
}

prometheus.operator.podmonitors "pods" {
    forward_to = [prometheus.remote_write.staging.receiver]
}

This example will limit discovered PodMonitors to ones with the label team=ops in a specific namespace: my-app.

river
prometheus.operator.podmonitors "pods" {
    forward_to = [prometheus.remote_write.staging.receiver]
    namespaces = ["my-app"]
    selector {
        match_expression {
            key = "team"
            operator = "In"
            values = ["ops"]
        }
    }
}