Grafana Agent 0.29.0 release: New OpenTelemetry components

Today the Grafana Agent team is excited to announce the release of Grafana Agent v0.29.0. This September, we introduced a new way to easily run and configure Grafana Agent called Grafana Agent Flow, our new dynamic configuration runtime built on components.

Within Flow, we are also embracing Grafana Labs’ big tent philosophy by introducing OpenTelemetry (OTel) Collector components and converters for traces, metrics, and logs in Agent v0.29.0.

Big tent strategy at scale with Flow and OpenTelemetry

With our big tent philosophy in mind, our underlying goal is to make Grafana Agent the best tool for handling telemetry signals from various sources and remote endpoints. In the past, adding new exporters and receivers to Grafana Agent required a deep understanding of its codebase to develop and test the feature. With this new release, we can showcase how easy it is to add new components and convert from one format to another. Flow and OpenTelemetry components allow us to iterate, refine, and adopt this big tent mentality at a speed and scale we haven’t had before. This release includes 11 new Flow components, all built around OTel.

Flow components allow developers to create new functionality for converting, exporting, and receiving signals with low friction. A developer who wants to create a new component can use one of the existing components as a template and need only define what data the component receives and what data the component exports. The component subsystem handles moving data to and from the component, when to instantiate, and when to update. This operation was covered previously in Introducing programmable pipelines with Grafana Agent Flow.

New OpenTelemetry components in Grafana Agent Flow

The new OTel components allow receiving traces, logs, and metrics. These signals can be received via otelcol.receiver.otlp and then sent via otelcol.exporter.otlp or otelcol.exporter.otlphttp to compatible endpoints.

Copy

otelcol.receiver.otlp "otel" {
    output {
           metrics = [otelcol.exporter.otlp.default.input]
    }
}

otelcol.exporter.otlp "default" {
  client {
    endpoint = env("OTLP_ENDPOINT")
  }
}

OTel metric conversion to Prometheus metrics

A new important feature enabled by Flow is the ability to convert signals from one format to another so that you can use the best tools for your needs.

With otelcol.receiver.otlp, the Grafana Agent can receive metrics and convert them with otelcol.exporter.prometheus. The following workflow illustrates the process.

The Flow configuration below illustrates this new feature.

Copy

otelcol.receiver.prometheus "otel" {
    output {
           metrics = [otelcol.exporter.prometheus.default.input]
    }
}
otelcol.exporter.prometheus "default" {
    forward_to = [prometheus.remote_write.default]
}

prometheus.remote_write "default" {
    endpoint {
            url = "http://localhost:9009/api/prom/push"
    }
}

In this example, the Grafana Agent creates an OTel receiver for metrics, transforms the OTel metric series to a Prometheus format, and then sends those metrics to the remote_write. The power of this approach is that it allows you to use Prometheus relabel rules and any other Prometheus feature with metrics coming from OTel. This union combines the battle-tested code of Prometheus and OTel, no matter where they originate.

The future of the Grafana Agent 

We believe that the future of Grafana Agent is Flow and are happy to announce that in v0.30.0, Flow will be moving from experimental to beta. 

The big tent paradigm paired with Grafana Agent Flow components allows Grafana Agent to be a broker and transformer of your observability data. Instead of changing your existing applications to write to a new endpoint, by adding Grafana Agent as an intermediary, you can transform the data transparently. 

The current components are just the beginning. We need the help of the community. Let us know what you want to see and which features are essential. We encourage anyone interested in adding more functionality to submit new components or create issues on our Grafana Agent GitHub board. We would also love for you to drop by the Grafana Labs Community Slack in our #agent channel or attend our community call on November 23.