Caution
Grafana Alloy is the new name for our distribution of the OTel collector. Grafana Agent has been deprecated and is in Long-Term Support (LTS) through October 31, 2025. Grafana Agent will reach an End-of-Life (EOL) on November 1, 2025. Read more about why we recommend migrating to Grafana Alloy.
Important: This documentation is about an older version. It's relevant only to the release noted, many of the features and functions have been updated or replaced. Please view the current version.
pyroscope.scrape
BETA: This is a beta component. Beta components are subject to breaking changes, and may be replaced with equivalent functionality that cover the same use case.
pyroscope.scrape
configures a pprof scraping job for a given set of
targets
. The scraped performance profiles are forwarded to the list of receivers passed in
forward_to
.
Multiple pyroscope.scrape
components can be specified by giving them different labels.
Usage
pyroscope.scrape "LABEL" {
targets = TARGET_LIST
forward_to = RECEIVER_LIST
}
Arguments
The component configures and starts a new scrape job to scrape all of the input targets. Multiple scrape jobs can be spawned for a single input target when scraping multiple profile types.
The list of arguments that can be used to configure the block is presented below.
The scrape job name defaults to the component’s unique identifier.
Any omitted fields take on their default values. If conflicting attributes are being passed (e.g., defining both a BearerToken and BearerTokenFile or configuring both Basic Authorization and OAuth2 at the same time), the component reports an error.
The following arguments are supported:
Name | Type | Description | Default | Required |
---|---|---|---|---|
targets | list(map(string)) | List of targets to scrape. | yes | |
forward_to | list(ProfilesReceiver) | List of receivers to send scraped profiles to. | yes | |
job_name | string | The job name to override the job label with. | component name | no |
params | map(list(string)) | A set of query parameters with which the target is scraped. | no | |
scrape_interval | duration | How frequently to scrape the targets of this scrape config. | "15s" | no |
scrape_timeout | duration | The timeout for scraping targets of this config. | "15s" | no |
scheme | string | The URL scheme with which to fetch metrics from targets. | no | |
bearer_token | secret | Bearer token to authenticate with. | no | |
bearer_token_file | string | File containing a bearer token to authenticate with. | no | |
proxy_url | string | HTTP proxy to proxy requests through. | no | |
follow_redirects | bool | Whether redirects returned by the server should be followed. | true | no |
enable_http2 | bool | Whether HTTP2 is supported for requests. | true | no |
At most one of the following can be provided:
Blocks
The following blocks are supported inside the definition of pyroscope.scrape
:
Hierarchy | Block | Description | Required |
---|---|---|---|
basic_auth | basic_auth | Configure basic_auth for authenticating to targets. | no |
authorization | authorization | Configure generic authorization to targets. | no |
oauth2 | oauth2 | Configure OAuth2 for authenticating to targets. | no |
oauth2 > tls_config | tls_config | Configure TLS settings for connecting to targets via OAuth2. | no |
tls_config | tls_config | Configure TLS settings for connecting to targets. | no |
profiling_config | profiling_config | Configure profiling settings for the scrape job. | no |
profiling_config > profile.memory | profile.memory | Collect memory profiles. | no |
profiling_config > profile.block | profile.block | Collect profiles on blocks. | no |
profiling_config > profile.goroutine | profile.goroutine | Collect goroutine profiles. | no |
profiling_config > profile.mutex | profile.mutex | Collect mutex profiles. | no |
profiling_config > profile.process_cpu | profile.process_cpu | Collect CPU profiles. | no |
profiling_config > profile.fgprof | profile.fgprof | Collect fgprof profiles. | no |
profiling_config > profile.godeltaprof_memory | profile.godeltaprof_memory | Collect godeltaprof memory profiles. | no |
profiling_config > profile.godeltaprof_mutex | profile.godeltaprof_mutex | Collect godeltaprof mutex profiles. | no |
profiling_config > profile.godeltaprof_block | profile.godeltaprof_block | Collect godeltaprof block profiles. | no |
profiling_config > profile.custom | profile.custom | Collect custom profiles. | no |
clustering | clustering | Configure the component for when the Agent is running in clustered mode. | no |
The >
symbol indicates deeper levels of nesting. For example,
oauth2 > tls_config
refers to a tls_config
block defined inside
an oauth2
block.
basic_auth block
Name | Type | Description | Default | Required |
---|---|---|---|---|
password_file | string | File containing the basic auth password. | no | |
password | secret | Basic auth password. | no | |
username | string | Basic auth username. | no |
password
and password_file
are mutually exclusive, and only one can be provided inside a basic_auth
block.
authorization block
Name | Type | Description | Default | Required |
---|---|---|---|---|
credentials_file | string | File containing the secret value. | no | |
credentials | secret | Secret value. | no | |
type | string | Authorization type, for example, “Bearer”. | no |
credential
and credentials_file
are mutually exclusive, and only one can be provided inside an authorization
block.
oauth2 block
Name | Type | Description | Default | Required |
---|---|---|---|---|
client_id | string | OAuth2 client ID. | no | |
client_secret_file | string | File containing the OAuth2 client secret. | no | |
client_secret | secret | OAuth2 client secret. | no | |
endpoint_params | map(string) | Optional parameters to append to the token URL. | no | |
proxy_url | string | Optional proxy URL for OAuth2 requests. | no | |
scopes | list(string) | List of scopes to authenticate with. | no | |
token_url | string | URL to fetch the token from. | no |
client_secret
and client_secret_file
are mutually exclusive, and only one can be provided inside an oauth2
block.
The oauth2
block may also contain a separate tls_config
sub-block.
tls_config block
Name | Type | Description | Default | Required |
---|---|---|---|---|
ca_pem | string | CA PEM-encoded text to validate the server with. | no | |
ca_file | string | CA certificate to validate the server with. | no | |
cert_pem | string | Certificate PEM-encoded text for client authentication. | no | |
cert_file | string | Certificate file for client authentication. | no | |
insecure_skip_verify | bool | Disables validation of the server certificate. | no | |
key_file | string | Key file for client authentication. | no | |
key_pem | secret | Key PEM-encoded text for client authentication. | no | |
min_version | string | Minimum acceptable TLS version. | no | |
server_name | string | ServerName extension to indicate the name of the server. | no |
The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:
ca_pem
andca_file
cert_pem
andcert_file
key_pem
andkey_file
When configuring client authentication, both the client certificate (using
cert_pem
or cert_file
) and the client key (using key_pem
or key_file
)
must be provided.
When min_version
is not provided, the minimum acceptable TLS version is
inherited from Go’s default minimum version, TLS 1.2. If min_version
is
provided, it must be set to one of the following strings:
"TLS10"
(TLS 1.0)"TLS11"
(TLS 1.1)"TLS12"
(TLS 1.2)"TLS13"
(TLS 1.3)
profiling_config block
The profiling_config
block configures the profiling settings when scraping
targets.
The block contains the following attributes:
Name | Type | Description | Default | Required |
---|---|---|---|---|
path_prefix | string | The path prefix to use when scraping targets. | no |
profile.memory block
The profile.memory
block collects profiles on memory consumption.
It accepts the following arguments:
Name | Type | Description | Default | Required |
---|---|---|---|---|
enabled | boolean | Enable this profile type to be scraped. | true | no |
path | string | The path to the profile type on the target. | "/debug/pprof/allocs" | no |
delta | boolean | Whether to scrape the profile as a delta. | false | no |
When the delta
argument is true
, a seconds
query parameter is
automatically added to requests.
profile.block block
The profile.block
block collects profiles on process blocking.
It accepts the following arguments:
Name | Type | Description | Default | Required |
---|---|---|---|---|
enabled | boolean | Enable this profile type to be scraped. | true | no |
path | string | The path to the profile type on the target. | "/debug/pprof/block" | no |
delta | boolean | Whether to scrape the profile as a delta. | false | no |
When the delta
argument is true
, a seconds
query parameter is
automatically added to requests.
profile.goroutine block
The profile.goroutine
block collects profiles on the number of goroutines.
It accepts the following arguments:
Name | Type | Description | Default | Required |
---|---|---|---|---|
enabled | boolean | Enable this profile type to be scraped. | true | no |
path | string | The path to the profile type on the target. | "/debug/pprof/goroutine" | no |
delta | boolean | Whether to scrape the profile as a delta. | false | no |
When the delta
argument is true
, a seconds
query parameter is
automatically added to requests.
profile.mutex block
The profile.mutex
block collects profiles on mutexes.
It accepts the following arguments:
Name | Type | Description | Default | Required |
---|---|---|---|---|
enabled | boolean | Enable this profile type to be scraped. | true | no |
path | string | The path to the profile type on the target. | "/debug/pprof/mutex" | no |
delta | boolean | Whether to scrape the profile as a delta. | false | no |
When the delta
argument is true
, a seconds
query parameter is
automatically added to requests.
profile.process_cpu block
The profile.process_cpu
block collects profiles on CPU consumption for the
process.
It accepts the following arguments:
Name | Type | Description | Default | Required |
---|---|---|---|---|
enabled | boolean | Enable this profile type to be scraped. | true | no |
path | string | The path to the profile type on the target. | "/debug/pprof/profile" | no |
delta | boolean | Whether to scrape the profile as a delta. | true | no |
When the delta
argument is true
, a seconds
query parameter is
automatically added to requests.
profile.fgprof block
The profile.fgprof
block collects profiles from an fgprof endpoint.
It accepts the following arguments:
Name | Type | Description | Default | Required |
---|---|---|---|---|
enabled | boolean | Enable this profile type to be scraped. | false | no |
path | string | The path to the profile type on the target. | "/debug/fgprof" | no |
delta | boolean | Whether to scrape the profile as a delta. | true | no |
When the delta
argument is true
, a seconds
query parameter is
automatically added to requests.
profile.godeltaprof_memory block
The profile.godeltaprof_memory
block collects profiles from godeltaprof memory endpoint. The delta is computed on the target.
It accepts the following arguments:
Name | Type | Description | Default | Required |
---|---|---|---|---|
enabled | boolean | Enable this profile type to be scraped. | false | no |
path | string | The path to the profile type on the target. | "/debug/pprof/delta_heap" | no |
profile.godeltaprof_mutex block
The profile.godeltaprof_mutex
block collects profiles from godeltaprof mutex endpoint. The delta is computed on the target.
It accepts the following arguments:
Name | Type | Description | Default | Required |
---|---|---|---|---|
enabled | boolean | Enable this profile type to be scraped. | false | no |
path | string | The path to the profile type on the target. | "/debug/pprof/delta_mutex" | no |
profile.godeltaprof_block block
The profile.godeltaprof_block
block collects profiles from godeltaprof block endpoint. The delta is computed on the target.
It accepts the following arguments:
Name | Type | Description | Default | Required |
---|---|---|---|---|
enabled | boolean | Enable this profile type to be scraped. | false | no |
path | string | The path to the profile type on the target. | "/debug/pprof/delta_block" | no |
profile.custom block
The profile.custom
block allows for collecting profiles from custom
endpoints. Blocks must be specified with a label:
profile.custom "PROFILE_TYPE" {
enabled = true
path = "PROFILE_PATH"
}
Multiple profile.custom
blocks can be specified. Labels assigned to
profile.custom
blocks must be unique across the component.
The profile.custom
block accepts the following arguments:
Name | Type | Description | Default | Required |
---|---|---|---|---|
enabled | boolean | Enable this profile type to be scraped. | yes | |
path | string | The path to the profile type on the target. | yes | |
delta | boolean | Whether to scrape the profile as a delta. | false | no |
When the delta
argument is true
, a seconds
query parameter is
automatically added to requests.
clustering (beta)
Name | Type | Description | Default | Required |
---|---|---|---|---|
enabled | bool | Enables sharing targets with other cluster nodes. | false | yes |
When the agent is using clustering, and enabled
is set to true,
then this pyroscope.scrape
component instance opts-in to participating in the
cluster to distribute scrape load between all cluster nodes.
Clustering causes the set of targets to be locally filtered down to a unique subset per node, where each node is roughly assigned the same number of targets. If the state of the cluster changes, such as a new node joins, then the subset of targets to scrape per node will be recalculated.
When clustering mode is enabled, all agents participating in the cluster must use the same configuration file and have access to the same service discovery APIs.
If the agent is not running in clustered mode, this block is a no-op.
Exported fields
pyroscope.scrape
does not export any fields that can be referenced by other
components.
Component health
pyroscope.scrape
is only reported as unhealthy if given an invalid
configuration.
Debug information
pyroscope.scrape
reports the status of the last scrape for each configured
scrape job on the component’s debug endpoint.
Debug metrics
pyroscope_fanout_latency
(histogram): Write latency for sending to direct and indirect components.
Scraping behavior
The pyroscope.scrape
component borrows the scraping behavior of Prometheus.
Prometheus, and by extension, this component, uses a pull model for scraping
profiles from a given set of targets.
Each scrape target is defined as a set of key-value pairs called labels.
The set of targets can either be static, or dynamically provided periodically
by a service discovery component such as discovery.kubernetes
. The special
label __address__
must always be present and corresponds to the
<host>:<port>
that is used for the scrape request.
The special label service_name
is required and must always be present. If it’s not specified, it is
attempted to be inferred from multiple sources:
__meta_kubernetes_pod_annotation_pyroscope_io_service_name
which is apyroscope.io/service_name
pod annotation.__meta_kubernetes_namespace
and__meta_kubernetes_pod_container_name
__meta_docker_container_name
If service_name
is not specified and could not be inferred it is set to unspecified
.
By default, the scrape job tries to scrape all available targets’ /debug/pprof
endpoints using HTTP, with a scrape interval of 15 seconds and scrape timeout of
15 seconds. The profile paths, protocol scheme, scrape interval and timeout,
query parameters, as well as any other settings can be configured using the
component’s arguments.
The scrape job expects profiles exposed by the endpoint to follow the
pprof protobuf format. All profiles are then propagated
to each receiver listed in the component’s forward_to
argument.
Labels coming from targets, that start with a double underscore __
are
treated as internal, and are removed prior to scraping.
The pyroscope.scrape
component regards a scrape as successful if it
responded with an HTTP 200 OK
status code and returned a body of valid pprof profile.
If the scrape request fails, the component’s debug UI section contains more detailed information about the failure, the last successful scrape, as well as the labels last used for scraping.
The following labels are automatically injected to the scraped profiles and can help pin down a scrape target.
Label | Description |
---|---|
job | The configured job name that the target belongs to. Defaults to the fully formed component name. |
instance | The __address__ or <host>:<port> of the scrape target’s URL. |
service_name | The inferred pyroscope service name |
Example
The following example sets up the scrape job with certain attributes (profiling config, targets) and lets it scrape two local applications (the Agent itself and Pyroscope). The exposed profiles are sent over to the provided list of receivers, as defined by other components.
pyroscope.scrape "local" {
targets = [
{"__address__" = "localhost:4100", "service_name"="pyroscope"},
{"__address__" = "localhost:12345", "service_name"="agent"},
]
forward_to = [pyroscope.write.local.receiver]
profiling_config {
profile.fgprof {
enabled = true
}
profile.block {
enabled = false
}
profile.mutex {
enabled = false
}
}
}
Here are the endpoints that are being scraped every 15 seconds:
http://localhost:4100/debug/pprof/allocs
http://localhost:4100/debug/pprof/goroutine
http://localhost:4100/debug/pprof/profile?seconds=14
http://localhost:4100/debug/fgprof?seconds=14
http://localhost:12345/debug/pprof/allocs
http://localhost:12345/debug/pprof/goroutine
http://localhost:12345/debug/pprof/profile?seconds=14
http://localhost:12345/debug/fgprof?seconds=14