This is documentation for the next version of Alloy. For the latest stable release, go to the latest version.

Documentation Grafana Alloy Reference Components otelcol otelcol.receiver.filelog

otelcol.receiver.filelog

Public preview: This is a public preview component. Public preview components are subject to breaking changes, and may be replaced with equivalent functionality that cover the same use case. The stability.level flag must be set to public-preview or below to use the component.

otelcol.receiver.filelog reads log entries from files and forwards them to other otelcol.* components.

You can specify multiple otelcol.receiver.filelog components by giving them different labels.

Usage

alloy Copy

otelcol.receiver.filelog "<LABEL>" {
  include = [...]
  output {
    logs    = [...]
  }
}

Arguments

You can use the following arguments with otelcol.receiver.filelog:

encoding must be one of utf-8, utf-16le, utf-16be, ascii, big5, or nop. Refer to the upstream receiver documentation for more details.

start_at must be one of beginning or end. The header block may only be used if start_at is beginning.

compression must be either `` or gzip.

operators

The operators list is a list of stanza operators that transform the log entries after they have been read.

For example, if container logs are being collected you may want to utilize the stanza container parser operator to add relevant attributes to the log entries.

alloy Copy

otelcol.receiver.filelog "default" {
    ...
    operators = [
      {
        type = "container"
      }
    ]
}

Blocks

The following blocks are supported inside the definition of otelcol.receiver.filelog:

The > symbol indicates deeper levels of nesting. For example, ordering_criteria > sort_by refers to a sort_by block defined inside a ordering_criteria block.

output

Required

The output block configures a set of components to forward resulting telemetry data to.

The following arguments are supported:

You must specify the output block, but all its arguments are optional. By default, telemetry data is dropped. Configure the metrics, logs, and traces arguments accordingly to send telemetry data to other components.

debug_metrics

The debug_metrics block configures the metrics that this component generates to monitor its state.

The following arguments are supported:

disable_high_cardinality_metrics is the Grafana Alloy equivalent to the telemetry.disableHighCardinalityMetrics feature gate in the OpenTelemetry Collector. It removes attributes that could cause high cardinality metrics. For example, attributes with IP addresses and port numbers in metrics about HTTP and gRPC connections are removed.

level is the Alloy equivalent to the telemetry.metrics.level feature gate in the OpenTelemetry Collector. Possible values are "none", "basic", "normal" and "detailed".

header

The header block configures logic for parsing a log header line into additional attributes added to each log entry. It may only be used when start_at is set to beginning. The following arguments are supported:

If a header block is not set, no log lines will be treated as header metadata.

The metadata_operators list is a list of stanza operators that parses metadata from the header. Any attributes created from the embedded operators pipeline will be applied to all log entries in the file.

For example, you might use a regex_parser to process a header line that has been identified by the pattern expression. The following example shows a fictitious header line, and then the header block that would parse an environment attribute from it.

text Copy

HEADER_IDENTIFIER env="production"
...

alloy Copy

otelcol.receiver.filelog "default" {
    ...
    header {
      pattern = '^HEADER_IDENTIFIER .*$'
      metadata_operators = [
        {
          type = "regex_parser"
          regex = 'env="(?P<environment>.+)"'
        }
      ]
    }
}

multiline

The multiline block configures logic for splitting incoming log entries. The following arguments are supported:

A multiline block must contain either line_start_pattern or line_end_pattern.

If a multiline block isn’t set, log entries will not be split.

ordering_criteria

The ordering_criteria block configures the order in which log files discovered will be processed. The following arguments are supported:

sort_by

The sort_by repeatable block configures the way the fields parsed in the ordering_criteria block will be applied to sort the discovered log files. The following arguments are supported:

sort_type must be one of numeric, lexicographic, timestamp, or mtime. When using numeric, lexicographic, or timestamp sort_type, a named capture group defined in the regex attribute in ordering_criteria must be provided in regex_key. When using mtime sort_type, the file’s modified time will be used to sort.

The location and layout arguments are only applicable when sort_type is timestamp.

The location argument specifies a Time Zone identifier. The available locations depend on the local IANA Time Zone database. Refer to the list of tz database time zones in Wikipedia for a non-comprehensive list.

retry_on_failure

The retry_on_failure block configures the retry behavior when the receiver encounters an error downstream in the pipeline. A backoff algorithm is used to delay the retry upon subsequent failures. The following arguments are supported:

If max_elapsed_time is set to 0 data is never discarded.

Exported fields

otelcol.receiver.filelog doesn’t export any fields.

Component health

otelcol.receiver.filelog is only reported as unhealthy if given an invalid configuration.

Debug metrics

otelcol.receiver.filelog doesn’t expose any component-specific debug metrics.

Example

This example reads log entries using the otelcol.receiver.filelog receiver and they are logged by a otelcol.exporter.debug component. It expects the logs to start with an ISO8601 compatible timestamp and parses it from the log using the regex_parser operator.

alloy Copy

otelcol.receiver.filelog "default" {
  include = ["/var/log/*.log"]
  operators = [{
    type = "regex_parser",
    regex = "^(?P<timestamp>\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}\\.\\d{3,6}Z)",
    timestamp = {
      parse_from = "attributes.timestamp",
      layout = "%Y-%m-%dT%H:%M:%S.%fZ",
      location = "UTC",
    },
  }]
  output {
      logs = [otelcol.exporter.debug.default.input]
  }
}

otelcol.exporter.debug "default" {}

Compatible components

otelcol.receiver.filelog can accept arguments from the following components:

• Components that export OpenTelemetry otelcol.Consumer