Grafana Agent v0.30: Flow adds support for logging pipelines and graduates to beta

Grafana Agent v0.30 is here!

The past couple of Grafana Agent releases have been pretty exciting for us. We introduced Agent Flow as a new way to configure, run, and debug telemetry pipelines. We also announced OpenTelemetry Collector components to expand on our Big Tent philosophy and allow users to switch seamlessly between the Prometheus and OTel ecosystems.

This latest release continues that momentum by introducing Loki components for building logging pipelines and marking Flow mode as beta!

The release also contains enhancements to Grafana Agent Operator, bug fixes for integrations, as well as support for a new and exciting Prometheus feature — native histograms. Check our release notes for a more detailed view of the v0.30 release features and updates.

Loki components

After support for metrics and traces in Agent Flow, it was time for logs to take the spotlight. Release v0.30 introduces a new set of components to enable logging pipelines. This brings Flow a step closer to feature parity with the static Agent and can handle all telemetry signals that the static Agent mode does.

Agent Flow’s discovery.file component lets you expose log files on disk as targets and feed them into a loki.source.file component to be tailed.

You can transform log entries and their labels using either loki.relabel or loki.process, while loki.write takes care of shipping entries to a Loki instance.

Here’s a Flow pipeline that uses glob patterns to discover files on disk, reads from them, and ships them over to a local Loki instance:

river Copy

discovery.file "varlog" {
	path_targets = [
		{__path__ = "/var/log/nginx/**", srv = "nginx"},
		{__path__ = "/var/log/httpd/*.log", srv = "https"},
		{__path__ = "/var/log/mysql/mysql_error.log", srv = "mysql"},
	]
}

loki.source.file "httpd" {
	targets    = discovery.file.varlog.targets
	forward_to = [loki.write.onprem.receiver]
}

loki.write "onprem" {
	endpoint {
		url = "http://loki:3100"
	}
}

We’ve already got plans to add components for converting back and forth between Loki and OpenTelemetry logs, so stay tuned for more news on that.

Flow is now beta

Release v0.30 also brings the graduation of Flow mode from Experimental to Beta!

To use Flow, set the AGENT_MODE environment variable to flow. The Agent’s mode still defaults to static. The EXPERIMENTAL_ENABLE_FLOW environment variable is marked as deprecated and will be dropped in v0.32.

We will continue to iterate on the current set of components, as well as introduce new ones to reach complete feature parity with the static mode and the OTel Collector, as well as improve performance and stability.

We always appreciate the community’s opinions and help. Please let us know how Flow works for you and what features you’d like to see next. And feel free to post any questions you may have on our dedicated GitHub discussion, on the Grafana Labs Community Slack, or directly in the Agent repo. We look forward to hearing your feedback!