This is documentation for the next version of Alloy. For the latest stable release, go to the latest version.
otelcol.processor.attributes
otelcol.processor.attributes
accepts telemetry data from other otelcol
components and modifies attributes of a span, log, or metric.
It also supports the ability to filter and match input data to determine if it should be included or excluded for attribute modifications.
Note
otelcol.processor.attributes
is a wrapper over the upstream OpenTelemetry Collectorattributes
processor. Bug reports or feature requests will be redirected to the upstream repository, if necessary.
You can specify multiple otelcol.processor.attributes
components by giving them different labels.
Usage
otelcol.processor.attributes "LABEL" {
output {
metrics = [...]
logs = [...]
traces = [...]
}
}
Arguments
otelcol.processor.attributes
doesn’t support any arguments and is configured fully through inner blocks.
Blocks
The following blocks are supported inside the definition of
otelcol.processor.attributes
:
Hierarchy | Block | Description | Required |
---|---|---|---|
output | output | Configures where to send received telemetry data. | yes |
action | action | Actions to take on the attributes of incoming metrics/logs/traces. | no |
include | include | Filter for data included in this processor’s actions. | no |
include > regexp | regexp | Regex cache settings. | no |
include > attribute | attribute | A list of attributes to match against. | no |
include > resource | resource | A list of items to match the resources against. | no |
include > library | library | A list of items to match the implementation library against. | no |
include > log_severity | library | How to match against a log record’s SeverityNumber, if defined. | no |
exclude | exclude | Filter for data excluded from this processor’s actions | no |
exclude > regexp | regexp | Regex cache settings. | no |
exclude > attribute | attribute | A list of attributes to match against. | no |
exclude > resource | resource | A list of items to match the resources against. | no |
exclude > library | library | A list of items to match the implementation library against. | no |
exclude > log_severity | log_severity | How to match against a log record’s SeverityNumber, if defined. | no |
debug_metrics | debug_metrics | Configures the metrics that this component generates to monitor its state. | no |
The >
symbol indicates deeper levels of nesting. For example, include > attribute
refers to an attribute
block defined inside an include
block.
If both an include
block and an exclude
block are specified, the include
properties are checked before the exclude
properties.
action block
The action
block configures how to modify the span, log, or metric.
The following attributes are supported:
Name | Type | Description | Default | Required |
---|---|---|---|---|
key | string | The attribute that the action relates to. | yes | |
action | string | The type of action performed. | yes | |
value | any | The value to populate for the key. | no | |
pattern | string | A regex pattern. | "" | no |
from_attribute | string | The attribute from the input data used to populate the attribute value. | "" | no |
from_context | string | The context value used to populate the attribute value. | "" | no |
converted_type | string | The type to convert the attribute value to. | "" | no |
The value
data type must be either a number, string, or boolean.
The supported values for action
are:
insert
: Inserts a new attribute in input data where the key does not already exist.- The
key
attribute is required. It specifies the attribute to act upon. - One of the
value
,from_attribute
orfrom_context
attributes is required.
- The
update
: Updates an attribute in input data where the key does exist.- The
key
attribute is required. It specifies the attribute to act upon. - One of the
value
,from_attribute
orfrom_context
attributes is required.
- The
upsert
: Either inserts a new attribute in input data where the key does not already exist or updates an attribute in input data where the key does exist.- The
key
attribute is required. It specifies the attribute to act upon. - One of the
value
,from_attribute
orfrom_context
attributes is required:value
specifies the value to populate for the key.from_attribute
specifies the attribute from the input data to use to populate the value. If the attribute doesn’t exist, no action is performed.from_context
specifies the context value used to populate the attribute value. If the key is prefixed withmetadata.
, the values are searched in the receiver’s transport protocol for additional information like gRPC Metadata or HTTP Headers. If the key is prefixed withauth.
, the values are searched in the authentication information set by the server authenticator. Refer to the server authenticator’s documentation part of your pipeline for more information about which attributes are available. If the key isclient.address
, the value will be set to the client address. If the key doesn’t exist, no action is performed. If the key has multiple values the values will be joined with a;
separator.
- The
hash
: Hashes (SHA1) an existing attribute value.- The
key
attribute and/or thepattern
attributes is required.
- The
extract
: Extracts values using a regular expression rule from the input key to target keys specified in the rule. If a target key already exists, it will be overridden. Note: It behaves similarly to the Span Processorto_attributes
setting with the existing attribute as the source.- The
key
attribute is required. It specifies the attribute to extract values from. The value ofkey
is NOT altered. - The
pattern
attribute is required. It is the regex pattern used to extract attributes from the value ofkey
. The submatchers must be named. If attributes already exist, they will be overwritten.
- The
convert
: Converts an existing attribute to a specified type.- The
key
attribute is required. It specifies the attribute to act upon. - The
converted_type
attribute is required and must be one of int, double or string.
- The
delete
: Deletes an attribute from the input data.- The
key
attribute and/or thepattern
attribute is required. It specifies the attribute to act upon.
- The
include block
The include
block provides an option to include data being fed into the action blocks based on the properties of a span, log, or metric records.
The following arguments are supported:
Name | Type | Description | Default | Required |
---|---|---|---|---|
match_type | string | Controls how items to match against are interpreted. | yes | |
log_bodies | list(string) | A list of strings that the LogRecord’s body field must match against. | [] | no |
log_severity_texts | list(string) | A list of strings that the LogRecord’s severity text field must match against. | [] | no |
metric_names | list(string) | A list of strings to match the metric name against. | [] | no |
services | list(string) | A list of items to match the service name against. | [] | no |
span_kinds | list(string) | A list of items to match the span kind against. | [] | no |
span_names | list(string) | A list of items to match the span name against. | [] | no |
match_type
is required and must be set to either "regexp"
or "strict"
.
A match occurs if at least one item in the lists matches.
One of the following is also required:
- For spans, one of
services
,span_names
,span_kinds
, attribute, resource, or library must be specified with a non-empty value for a valid configuration. Thelog_bodies
,log_severity_texts
,log_severity
, andmetric_names
attributes are invalid. - For logs, one of
log_bodies
,log_severity_texts
,log_severity
, attribute, resource, or library must be specified with a non-empty value for a valid configuration. Thespan_names
,span_kinds
,metric_names
, andservices
attributes are invalid. - For metrics,
metric_names
must be specified with a valid non-empty value for a valid configuration. Thespan_names
,span_kinds
,log_bodies
,log_severity_texts
,log_severity
,services
, attribute, resource, and library attributes are invalid.
If the configuration includes filters which are specific to a particular signal type, it is best to include only that signal type in the component’s output.
For example, adding a span_names
filter could cause the component to error if logs are configured in the component’s outputs.
exclude block
The exclude
block provides an option to exclude data from being fed into the action blocks based on the properties of a span, log, or metric records.
Note
Signals excluded by theexclude
block will still be propagated to downstream components as-is. If you would like to not propagate certain signals to downstream components, consider a processor such as otelcol.processor.tail_sampling.
The following arguments are supported:
Name | Type | Description | Default | Required |
---|---|---|---|---|
match_type | string | Controls how items to match against are interpreted. | yes | |
log_bodies | list(string) | A list of strings that the LogRecord’s body field must match against. | [] | no |
log_severity_texts | list(string) | A list of strings that the LogRecord’s severity text field must match against. | [] | no |
metric_names | list(string) | A list of strings to match the metric name against. | [] | no |
services | list(string) | A list of items to match the service name against. | [] | no |
span_kinds | list(string) | A list of items to match the span kind against. | [] | no |
span_names | list(string) | A list of items to match the span name against. | [] | no |
match_type
is required and must be set to either "regexp"
or "strict"
.
A match occurs if at least one item in the lists matches.
One of the following is also required:
- For spans, one of
services
,span_names
,span_kinds
, attribute, resource, or library must be specified with a non-empty value for a valid configuration. Thelog_bodies
,log_severity_texts
,log_severity
, andmetric_names
attributes are invalid. - For logs, one of
log_bodies
,log_severity_texts
,log_severity
, attribute, resource, or library must be specified with a non-empty value for a valid configuration. Thespan_names
,span_kinds
,metric_names
, andservices
attributes are invalid. - For metrics,
metric_names
must be specified with a valid non-empty value for a valid configuration. Thespan_names
,span_kinds
,log_bodies
,log_severity_texts
,log_severity
,services
, attribute, resource, and library attributes are invalid.
If the configuration includes filters which are specific to a particular signal type, it is best to include only that signal type in the component’s output.
For example, adding a span_names
filter could cause the component to error if logs are configured in the component’s outputs.
regexp block
This block is an optional configuration for the match_type
of "regexp"
.
It configures a Least Recently Used (LRU) cache.
The following arguments are supported:
Name | Type | Description | Default | Required |
---|---|---|---|---|
cache_enabled | bool | Determines whether match results are LRU cached. | false | no |
cache_max_num_entries | int | The max number of entries of the LRU cache that stores match results. | 0 | no |
Enabling cache_enabled
could make subsequent matches faster.
Cache size is unlimited unless cache_max_num_entries
is also specified.
cache_max_num_entries
is ignored if cache_enabled
is false.
attribute block
This block specifies an attribute to match against:
- More than one
attribute
block can be defined. - Only
match_type = "strict"
is allowed ifattribute
is specified. - All
attribute
blocks must match exactly for a match to occur.
The following arguments are supported:
Name | Type | Description | Default | Required |
---|---|---|---|---|
key | string | The attribute key. | yes | |
value | any | The attribute value to match against. | no |
If value
isn’t set, any value will match.
The type of value
could be a number, a string, or a boolean.
resource block
This block specifies items to match the resources against:
- More than one
resource
block can be defined. - A match occurs if the input data resources match at least one
resource
block.
The following arguments are supported:
Name | Type | Description | Default | Required |
---|---|---|---|---|
key | string | The resource key. | yes | |
value | any | The resource value to match against. | no |
If value
isn’t set, any value will match.
The type of value
could be a number, a string, or a boolean.
library block
This block specifies properties to match the implementation library against:
- More than one
library
block can be defined. - A match occurs if the span’s implementation library matches at least one
library
block.
The following arguments are supported:
Name | Type | Description | Default | Required |
---|---|---|---|---|
name | string | The attribute key. | yes | |
version | string | The version to match against. | null | no |
If version
is unset, any version matches.
If version
is set to an empty string, it only matches a library version, which is also an empty string.
log_severity block
This block defines how to match based on a log record’s SeverityNumber field.
The following arguments are supported:
Name | Type | Description | Default | Required |
---|---|---|---|---|
match_undefined | bool | Whether logs with “undefined” severity match. | yes | |
min | string | The lowest severity that may be matched. | yes |
If match_undefined
is true, entries with undefined severity will match.
The following table lists the severities supported by OTel.
The value for min
should be one of the values in the “Log Severity” column.
Log Severity | Severity number |
---|---|
TRACE | 1 |
TRACE2 | 2 |
TRACE3 | 3 |
TRACE4 | 4 |
DEBUG | 5 |
DEBUG2 | 6 |
DEBUG3 | 7 |
DEBUG4 | 8 |
INFO | 9 |
INFO2 | 10 |
INFO3 | 11 |
INFO4 | 12 |
WARN | 13 |
WARN2 | 14 |
WARN3 | 15 |
WARN4 | 16 |
ERROR | 17 |
ERROR2 | 18 |
ERROR3 | 19 |
ERROR4 | 20 |
FATAL | 21 |
FATAL2 | 22 |
FATAL3 | 23 |
FATAL4 | 24 |
For example, if the min
attribute in the log_severity
block is “INFO”, then INFO, WARN, ERROR, and FATAL logs will match.
output block
The output
block configures a set of components to forward resulting telemetry data to.
The following arguments are supported:
Name | Type | Description | Default | Required |
---|---|---|---|---|
logs | list(otelcol.Consumer) | List of consumers to send logs to. | [] | no |
metrics | list(otelcol.Consumer) | List of consumers to send metrics to. | [] | no |
traces | list(otelcol.Consumer) | List of consumers to send traces to. | [] | no |
You must specify the output
block, but all its arguments are optional.
By default, telemetry data is dropped.
Configure the metrics
, logs
, and traces
arguments accordingly to send telemetry data to other components.
debug_metrics block
The debug_metrics
block configures the metrics that this component generates to monitor its state.
The following arguments are supported:
Name | Type | Description | Default | Required |
---|---|---|---|---|
disable_high_cardinality_metrics | boolean | Whether to disable certain high cardinality metrics. | true | no |
level | string | Controls the level of detail for metrics emitted by the wrapped collector. | "detailed" | no |
disable_high_cardinality_metrics
is the Grafana Alloy equivalent to the telemetry.disableHighCardinalityMetrics
feature gate in the OpenTelemetry Collector.
It removes attributes that could cause high cardinality metrics.
For example, attributes with IP addresses and port numbers in metrics about HTTP and gRPC connections are removed.
Note
If configured,disable_high_cardinality_metrics
only applies tootelcol.exporter.*
andotelcol.receiver.*
components.
level
is the Alloy equivalent to the telemetry.metrics.level
feature gate in the OpenTelemetry Collector.
Possible values are "none"
, "basic"
, "normal"
and "detailed"
.
Exported fields
The following fields are exported and can be referenced by other components:
Name | Type | Description |
---|---|---|
input | otelcol.Consumer | A value that other components can use to send telemetry data to. |
input
accepts otelcol.Consumer
data for any telemetry signal (metrics, logs, or traces).
Component health
otelcol.processor.attributes
is only reported as unhealthy if given an invalid
configuration.
Debug information
otelcol.processor.attributes
does not expose any component-specific debug
information.
Examples
Various uses of the “action” block
otelcol.receiver.otlp "default" {
http {}
grpc {}
output {
metrics = [otelcol.processor.attributes.default.input]
logs = [otelcol.processor.attributes.default.input]
traces = [otelcol.processor.attributes.default.input]
}
}
otelcol.processor.attributes "default" {
// Inserts a new attribute "attribute1" to spans where
// the key "attribute1" doesn't exist.
// The type of `attribute1` is inferred by the configuration.
// `123` is an integer and is stored as an integer in the attributes.
action {
key = "attribute1"
value = 123
action = "insert"
}
// Inserts a new attribute with a key of "string key" and
// a string value of "anotherkey".
action {
key = "string key"
value = "anotherkey"
action = "insert"
}
// Setting an attribute on all spans.
// Any spans that already had `region` now have value `planet-earth`.
// This can be done to set properties for all traces without
// requiring an instrumentation change.
action {
key = "region"
value = "planet-earth"
action = "upsert"
}
// The following demonstrates copying a value to a new key.
// If a span doesn't contain `user_key`, no new attribute `new_user_key` is created.
action {
key = "new_user_key"
from_attribute = "user_key"
action = "upsert"
}
// Hashing existing attribute values.
action {
key = "user.email"
action = "hash"
}
// Uses the value from key `example_user_key` to upsert attributes
// to the target keys specified in the `pattern`.
// (Insert attributes for target keys that do not exist and update keys that exist.)
// Given example_user_key = /api/v1/document/12345678/update/v1
// then the following attributes will be inserted:
// new_example_user_key: 12345678
// version: v1
//
// Note: Similar to the Span Processor, if a target key already exists,
// it will be updated.
//
// Note: The regex pattern is enclosed in backticks instead of quotation marks.
// This constitutes a raw Alloy syntax string, and lets us avoid the need to escape backslash characters.
action {
key = "example_user_key"
pattern = `\/api\/v1\/document\/(?P<new_user_key>.*)\/update\/(?P<version>.*)$`
action = "extract"
}
// Converting the type of an existing attribute value.
action {
key = "http.status_code"
converted_type = "int"
action = "convert"
}
// Deleting keys from an attribute.
action {
key = "credit_card"
action = "delete"
}
output {
metrics = [otelcol.exporter.otlp.default.input]
logs = [otelcol.exporter.otlp.default.input]
traces = [otelcol.exporter.otlp.default.input]
}
}
otelcol.exporter.otlp "default" {
client {
endpoint = sys.env("OTLP_ENDPOINT")
}
}
Excluding spans based on attributes
For example, the following spans match the properties and won’t be processed by the processor:
- Span1 Name: “svcB”, Attributes: {env: “dev”, test_request: 123, credit_card: 1234}
- Span2 Name: “svcA”, Attributes: {env: “dev”, test_request: false}
The following spans do not match the properties and the processor actions are applied to it:
- Span3 Name: “svcB”, Attributes: {env: 1, test_request: “dev”, credit_card: 1234}
- Span4 Name: “svcC”, Attributes: {env: “dev”, test_request: false}
Note that due to the presence of the services
attribute, this configuration works only for
trace signals. This is why only traces are configured in the output
block.
otelcol.processor.attributes "default" {
exclude {
match_type = "strict"
services = ["svcA", "svcB"]
attribute {
key = "env"
value = "dev"
}
attribute {
key = "test_request"
}
}
action {
key = "credit_card"
action = "delete"
}
action {
key = "duplicate_key"
action = "delete"
}
output {
traces = [otelcol.exporter.otlp.default.input]
}
}
Excluding spans based on resources
A “strict” match_type
means that we must strictly match the resource
key/value pairs.
Note that the resource
attribute is not used for metrics, which is why metrics are not configured
in the component output.
otelcol.processor.attributes "default" {
exclude {
match_type = "strict"
resource {
key = "host.type"
value = "n1-standard-1"
}
}
action {
key = "credit_card"
action = "delete"
}
action {
key = "duplicate_key"
action = "delete"
}
output {
logs = [otelcol.exporter.otlp.default.input]
traces = [otelcol.exporter.otlp.default.input]
}
}
Excluding spans based on a specific library version
A “strict” match_type
means that we must strictly match the library
key/value pairs.
Note that the library
attribute is not used for metrics, which is why metrics are not configured
in the component output.
otelcol.processor.attributes "default" {
exclude {
match_type = "strict"
library {
name = "mongo-java-driver"
version = "3.8.0"
}
}
action {
key = "credit_card"
action = "delete"
}
action {
key = "duplicate_key"
action = "delete"
}
output {
logs = [otelcol.exporter.otlp.default.input]
traces = [otelcol.exporter.otlp.default.input]
}
}
Including and excluding spans based on regex and services
This processor will remove the “token” attribute and will obfuscate the “password” attribute
in spans where the service name matches "auth.*"
and where the span name does not match "login.*"
.
Note that due to the presence of the services
and span_names
attributes, this configuration
works only for trace signals. This is why only traces are configured in the output
block.
otelcol.processor.attributes "default" {
// Specifies the span properties that must exist for the processor to be applied.
include {
// "match_type" defines that "services" is an array of regexp-es.
match_type = "regexp"
// The span service name must match "auth.*" pattern.
services = ["auth.*"]
}
exclude {
// "match_type" defines that "span_names" is an array of regexp-es.
match_type = "regexp"
// The span name must not match "login.*" pattern.
span_names = ["login.*"]
}
action {
key = "password"
action = "update"
value = "obfuscated"
}
action {
key = "token"
action = "delete"
}
output {
traces = [otelcol.exporter.otlp.default.input]
}
}
Including spans based on regex and attributes
The following demonstrates how to process spans with attributes that match a regexp pattern. This processor will obfuscate the “db.statement” attribute in spans where the “db.statement” attribute matches a regex pattern.
otelcol.processor.attributes "default" {
include {
// "match_type" of "regexp" defines that the "value" attributes
// in the "attribute" blocks are regexp-es.
match_type = "regexp"
// This attribute ('db.statement') must exist in the span and match
// the regex ('SELECT \* FROM USERS.*') for a match.
attribute {
key = "db.statement"
value = "SELECT \* FROM USERS.*"
}
}
action {
key = "db.statement"
action = "update"
value = "SELECT * FROM USERS [obfuscated]"
}
output {
metrics = [otelcol.exporter.otlp.default.input]
logs = [otelcol.exporter.otlp.default.input]
traces = [otelcol.exporter.otlp.default.input]
}
}
Including spans based on regex of log body
This processor will remove the “token” attribute and will obfuscate the “password” attribute in spans where the log body matches “AUTH.*”.
Note that due to the presence of the log_bodies
attribute, this configuration works only for
log signals. This is why only logs are configured in the output
block.
otelcol.processor.attributes "default" {
include {
match_type = "regexp"
log_bodies = ["AUTH.*"]
}
action {
key = "password"
action = "update"
value = "obfuscated"
}
action {
key = "token"
action = "delete"
}
output {
logs = [otelcol.exporter.otlp.default.input]
}
}
Including spans based on regex of log severity
The following demonstrates how to process logs that have a severity level which is equal to or higher than
the level specified in the log_severity
block. This processor will remove the “token” attribute and will
obfuscate the “password” attribute in logs where the severity is at least “INFO”.
Note that due to the presence of the log_severity
attribute, this configuration works only for
log signals. This is why only logs are configured in the output
block.
otelcol.processor.attributes "default" {
include {
match_type = "regexp"
log_severity {
min = "INFO"
match_undefined = true
}
}
action {
key = "password"
action = "update"
value = "obfuscated"
}
action {
key = "token"
action = "delete"
}
output {
logs = [otelcol.exporter.otlp.default.input]
}
}
Including spans based on regex of log severity text
The following demonstrates how to process logs that have a severity text that match regexp patterns. This processor will remove the “token” attribute and will obfuscate the “password” attribute in logs where severity matches “info”.
Note that due to the presence of the log_severity_texts
attribute, this configuration works only for
log signals. This is why only logs are configured in the output
block.
otelcol.processor.attributes "default" {
include {
match_type = "regexp"
log_severity_texts = ["info.*"]
}
action {
key = "password"
action = "update"
value = "obfuscated"
}
action {
key = "token"
action = "delete"
}
output {
logs = [otelcol.exporter.otlp.default.input]
}
}
Including metrics based on metric names
The following demonstrates how to process metrics that have a name starting with “counter”. This processor will add a label called “important_label” with a value of “label_val” to the metric. If the label already exists, its value will be updated.
Note that due to the presence of the metric_names
attribute, this configuration works only for
metric signals. This is why only metrics are configured in the output
block.
otelcol.processor.attributes "default" {
include {
match_type = "regexp"
metric_names = ["counter.*"]
}
action {
key = "important_label"
action = "upsert"
value = "label_val"
}
output {
metrics = [otelcol.exporter.otlp.default.input]
}
}
Compatible components
otelcol.processor.attributes
can accept arguments from the following components:
- Components that export OpenTelemetry
otelcol.Consumer
otelcol.processor.attributes
has exports that can be consumed by the following components:
- Components that consume OpenTelemetry
otelcol.Consumer
Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.