Menu
Documentationbreadcrumb arrow Grafana Lokibreadcrumb arrow Release notesbreadcrumb arrow v3.4
Open source

v3.4

Grafana Labs and the Loki team are excited to announce the release of Loki 3.4. Here’s a summary of new enhancements and important fixes.

For a full list of all changes and fixes, refer to the CHANGELOG.

Features and enhancements

Key features in Loki 3.4.0 include the following:

  • Time sharding ingestion: Add per-tenant time sharding for long out-of-order ingestion. The setting shard_streams.time_sharding_enabled is a per-tenant config that splits incoming log streams by injecting a new __time_shard__ synthetic label in such a way that no stream will ever be longer than 1h (or, rather, max_chunk_age/2). This allows the ingestion of very old logs, because the logs in these new streams will never be “too far behind” by design. (#14711) (0d6d68d).

  • Thanos object client: Previously Loki and Grafana Enterprise Logs used different Go packages to talk to the underlying object store. Migrating to Thanos means that there is now a single object client across Loki and GEL, and across the other Grafana backend databases: Mimir and Pyroscope.(#14891) (fb6789d), (#14638) (8958eb9), (#14867) (947a66f), (#14793) (cfc3819), (#14937) (a629212). The documentation now includes configuration examples using Thanos-based storage clients.

  • API: Add support for Parquet file format as a query response format. (#15408) (d0c11a6).

  • Structured metadata: Introduces a new feature,based on per-tenant configuration, that allows for extraction of “fields” into structured metadata at ingest time. Fields can either be regular labels, structured metadata keys, or keys from logfmt or json formatted log lines. (#15188) (7033091).

  • lambda-promtail: Adds support for relabeling log entries in lambda-promtail using Prometheus-style relabel configs. This allows for more flexible label manipulation and filtering of log entries before they are sent to Loki. (#15600) (c41a8b4).

  • Kafka: Loki 3.4 adds an Apache Kafka development environment to support ongoing development work(#15603) (226e9f1).

  • Helm charts: Loki 3.4 includes multiple improvements to the Helm charts, including: Added support for overrides exporter (#15630) (db72164), Expose topologySpreadConstraints for admin-api pods (#14995) (1b93086), Expose topologySpreadConstraints for distributed deployment (#14853) (d075107), Compute ConfigMap/Secret checksum only over .data content (#15177) (990f71c), Do not explicitly template managed-by, as it should be set at install time by the installation manager. (#14763) (10b7478).

  • Query acceleration with Bloom filters (experimental): The Bloom filters feature introduced in Loki 3.0 remains experimental. In Loki 3.3, there was a major shift in how Loki leverages bloom filters — pivoting from free-text search to harnessing the power of structured metadata. In Loki 3.4, work continues on the new Bloom Planner (#14874) (b646861).

  • Docs: This release includes a heavily revised labels topic and a new Cardinality topic. This release also marks the return of the sizing guidance page, newly updated based on Grafana Labs experience running Loki. Continuing updates to the Query section include a revised logcli topic and new logcli tutorial. There is also two additional new tutorials: Deploying Loki on Azure tutorial and Kubernetes Monitoring Helm tutorial.

Note

Note that when appropriate, documentation updates have been backported to previous releases.

Other improvements include the following:

  • analytics: Add TLS config to the analytics client (#15227) (2c5eabd).
  • approx_topk: Reduce memory usage of HyperLogLog in approx_topk. (#15559) (bef2043).
  • approx_topk: Improve approx_topk performance by reducing allocations. (#15450) (04994ca).
  • blooms: Add task timing and sizing metrics (#15032) (79cccfd).
  • blooms: Apply task timeout in bloom builder (#14988) (f92dde0).
  • blooms: Prefetch bloom blocks as soon as they are built (#15050) (b406015).
  • blooms: Compute chunkrefs for series right before sending task to builder (#14808) (66e6b1c).
  • blooms: Replace JSON lib for encoding/decoding metas (#14767) (57f16a0).
  • distributor: Add a metric ingestion time SM sanitization (#15222) (e9d0c3e).
  • docker compose: enable feature toggle (#14858) (8f243ee).
  • docker driver: Add ARM64 build and release of the Docker driver in Drone pipeline(#9247) (e950251).
  • lambda-promtail: Add terraform variable for s3 bucket notification filter prefix (#15381) (3dd8d7d).
  • logcli: Add opt-in support for ProxyFromEnvironment in logcli (#11742) (#14950) (c56b95d).
  • logcli: Include common labels (#15611) (639ac74).
  • logql: Allow line_format bypass template execution if possible (#15411) (2150fbc).
  • metrics: Add lines skipped metric to pattern ingesters (#14997) (dea5d78).
  • operator: Add support for managed GCP WorkloadIdentity (#14752) (7635a5c).
  • otel/labels: Add otlp attribute deployment.environment.name to list of default labels (#14747) (2d407bd).
  • querier: Present DF bytes values in queryable format (#15272) (f5d62bd).
  • querier: Use context propagation to call the same ingester in GetChunksID as Query (#15186) (70d9587).
  • ruler: Add Rule Name and Type to Query Tags (#15055) (b272573).
  • storage: Add objstore support for Swift using thanos.io/objstore (#11672) (44523e0).
  • storage: Skip writeback for chunks fetched by queriers older than a duration (#15605) (3b8d993).
  • storage: Support config timeout for AlibabaCloud OSS requests (#14856) (ac2e21f).
  • storage: Allow TSDB index creation in memory only (#14732) (831c0d5).
  • structured metadata: Sanitize structured metadata during ingestion in the distributor (#15141) (be4f17e).

Deprecations

One of the focuses of Loki 3.0 was cleaning up unused code and old features that had been previously deprecated but not removed. Loki 3.0 removed a number of previous deprecations and introduces some new deprecations. Some of the main areas with changes include:

To learn more about breaking changes in this release, refer to the Upgrade guide.

Caution

Grafana Alloy is the new name for our distribution of the OTel collector. Grafana Agent has been deprecated and is in Long-Term Support (LTS) through October 31, 2025. Grafana Agent will reach an End-of-Life (EOL) on November 1, 2025. Read more about why we recommend migrating to Grafana Alloy.

Upgrade Considerations

For important upgrade guidance, refer to the Upgrade Guide.

  • BREAKING CHANGE - Docker image: Remove wget from Promtail docker image (#15101).

Bug fixes

3.4.2 (2025-02-14)

  • blooms: Initialize bloom gateway client only once (backport release-3.4.x) (#16268) (1b9829b)
  • ci: do not disable CGO in Makefile (backport release-3.4.x) (#16272) (4fa045d)

3.4.1 (2025-02-12)

  • docker: Update the build info in docker images (#16225) (4484080).

3.4.0 (2025-02-12)

  • aggregated_metrics: Fix the IsError method causing retries (#15296) (7a7b3b1).

  • azureblob: Return error if Azure returns no service principal token (#13195) (e98a86b).

  • blooms: Copy chunks from ForSeries (#14863) (bfc2890).

  • blooms: Do not restart builders when planner disconnects (#14783) (7669385).

  • blooms: Enable metas cache on backend when running in ssd mode (#15275) (4cce207).

  • blooms: Fix check for skipping most recent data when filtering blooms (#15300) (78d3c44).

  • blooms: Fix strategy logger and add task test (#14857) (ab2721d).

  • blooms: Fix tenants slice on loadTenantTables (#14900) (a8f7851).

  • blooms: Match series to newest block only (#15481) (5846ea2).

  • blooms: Skip multi-tenant TSDBs during bloom planning (#14770) (76d35cc).

  • blooms: Fix data race in bloombuild tests (#15360) (aa7b849).

  • ci: Fix test compile error (#15404) (db2b6db).

  • ci: Remove submodule once and for all (#15495) (80e8e60).

  • ci: Revert “fixed Publish Rendered Helm Chart Diff workflow” (#15506) (6c49cc0).

  • ci: Transform TestChunkSize into a benchmark (#15361) (fed43db).

  • ci: Update Python version for Helm CI (#15420) (bdbb009).

  • deps: Downgrade hyperloglog to 0.2.0 (#15424) (#15426) (3b0d815).

  • deps: Update aws-sdk-go-v2 monorepo (#14820) (a355840).

  • deps: Update module cloud.google.com/go/bigtable to v1.34.0 (#15581) (9f28f69).

  • deps: Update module cloud.google.com/go/pubsub to v1.45.3 (#15294) (672fb08).

  • deps: Update module cloud.google.com/go/storage to v1.50.0 (#15690) (efd3ec3).

  • deps: Update module github.com/alecthomas/chroma/v2 to v2.15.0 (#15596) (a2d3057).

  • deps: Update module github.com/alicebob/miniredis/v2 to v2.34.0 (#15501) (61b4ddf).

  • deps: Update module github.com/aliyun/aliyun-oss-go-sdk to v3 (#14976) (64081c0).

  • deps: Update module github.com/aws/aws-sdk-go-v2 to v1.32.8 (#15680) (42c43ec).

  • deps: Update module github.com/aws/aws-sdk-go-v2/config to v1.28.9 (#15663) (c5c9a2e).

  • deps: Update module github.com/aws/aws-sdk-go-v2/service/s3 to v1.72.2 (#15685) (0e329e1).

  • deps: Update module github.com/axiomhq/hyperloglog to v0.2.3 (#15641) (b6df82e).

  • deps: Update module github.com/azure/azure-storage-blob-go to v0.15.0 (#14746) (ad9bdcd).

  • deps: Update module github.com/baidubce/bce-sdk-go to v0.9.213 (#15686) (95dac7c).

  • deps: Update module github.com/bmatcuk/doublestar to v4 (#14977) (a2b66d3).

  • deps: Update module github.com/docker/docker to v27.3.1+incompatible (#14753) (b4a926c).

  • deps: Update module github.com/fatih/color to v1.18.0 (#14754) (8fc9b28).

  • deps: Update module github.com/fsnotify/fsnotify to v1.8.0 (#14755) (0704f5d).

  • deps: Update module github.com/go-redis/redis/v8 to v9 (#14978) (33a2148).

  • deps: Update module github.com/gocql/gocql to v1 (#15016) (2a51081).

  • deps: Update module github.com/golang/snappy to v0.0.4 (#15415) (035f871).

  • deps: Update module github.com/grafana/loki/v3 to v3.2.1 (#14826) (b8509fc).

  • deps: Update module github.com/grpc-ecosystem/go-grpc-middleware/v2 to v2.2.0 (#15392) (6b7c3fc).

  • deps: Update module github.com/hashicorp/consul/api to v1.31.0 (#15540) (8207a9b).

  • deps: Update module github.com/hashicorp/golang-lru to v2 (#14979) (3abb3b1).

  • deps: Update module github.com/heroku/x to v0.4.1 (#15370) (20f5414).

  • deps: Update module github.com/ibm/go-sdk-core/v5 to v5.18.5 (#15673) (b7ec8e3).

  • deps: Update module github.com/ibm/ibm-cos-sdk-go to v1.12.0 (#15228) (5689e2e).

  • deps: Update module github.com/ibm/sarama to v1.45.0 (#15636) (9e13abd).

  • deps: Update module github.com/imdario/mergo to v1 (#15035) (be24862).

  • deps: Update module github.com/influxdata/telegraf to v1.33.1 (#15688) (d57492c).

  • deps: Update module github.com/klauspost/compress to v1.17.11 (#15451) (6e8bd9d).

  • deps: Update module github.com/masterminds/sprig/v3 to v3.3.0 (#14899) (8874878).

  • deps: Update module github.com/minio/minio-go/v7 to v7.0.82 (#15417) (7b8d82e).

  • deps: Update module github.com/opentracing-contrib/go-stdlib to v1.1.0 (#15221) (9f7892a).

  • deps: Update module github.com/pierrec/lz4 to v2.6.1+incompatible (#15502) (60f031a).

  • deps: Update module github.com/pierrec/lz4/v4 to v4.1.22 (#15400) (36bda72).

  • deps: Update module github.com/prometheus/common to v0.61.0 (#15352) (860c25d).

  • deps: Update module github.com/schollz/progressbar/v3 to v3.18.0 (#15691) (5f902b4).

  • deps: Update module github.com/shirou/gopsutil/v4 to v4.24.12 (#15569) (bc43878).

  • deps: Update module github.com/sony/gobreaker/v2 to v2.1.0 (#15556) (d188d06).

  • deps: Update module github.com/spf13/afero to v1.12.0 (#15696) (94cfb90).

  • deps: Update module github.com/stretchr/testify to v1.10.0 (#15091) (f65ab13).

  • deps: Update module github.com/twmb/franz-go/pkg/kadm to v1.14.0 (#14911) (1b59525).

  • deps: Update module go.opentelemetry.io/collector/pdata to v1.22.0 (#15452) (661ae79).

  • deps: Update module go.opentelemetry.io/otel/trace to v1.33.0 (#15454) (82c2c56).

  • deps: Update module golang.org/x/crypto to v0.32.0 (#15610) (db1188f).

  • deps: Update module golang.org/x/net to v0.34.0 (#15612) (0b25012).

  • deps: Update module golang.org/x/oauth2 to v0.25.0 (#15590) (ccee7f9).

  • deps: Update module golang.org/x/sync to v0.10.0 (#15253) (3150f07).

  • deps: Update module golang.org/x/sys to v0.29.0 (#15591) (efb9a81).

  • deps: Update module golang.org/x/text to v0.21.0 (#15266) (818446c).

  • deps: Update module golang.org/x/time to v0.9.0 (#15592) (5401e78).

  • deps: Update module google.golang.org/api to v0.214.0 (#15519) (e34944d).

  • deps: Update module google.golang.org/grpc to v1.68.1 (#15269) (b607fe0).

  • deps: Update module google.golang.org/protobuf to v1.36.2 (#15635) (74885a2).

  • deps: Update module gopkg.in/alecthomas/kingpin.v2 to v2.4.0 (#14970) (37d0bb8).

  • deps: Update module k8s.io/apimachinery to v0.32.0 (#15398) (fada02c).

  • deps: Update module k8s.io/klog to v2 (#15046) (734caf5).

  • distributor: Fix data race in distributor tests (#15423) (b8168a8).

  • distributor: Return error 422 (Unprocessable Content) when push request does not contain any streams (#13706) (17bf32b).

  • docs: Correct typo in architecture name (amr64 -> arm64) (#15707) (068f81d).

  • docs: Improve docs for min and max table offsets (#14890) (fd9d332).

  • docs: Change “did received” to “did receive” in NOTES.txt (#15584) (9191eaf).

  • helm/loki: Add namespace to Loki templates for ruler and index-gateway (#14838) (733d99f).

  • helm/loki: Allow prefixing of rollout-group value and ingester name (#15063) (f4accc1).

  • helm: Add appProtocol: tcp to headless backend grpc ports (#14507) (400f5af).

  • helm: Add conditional to include ruler config only when enabled (#15385) (3badbb3).

  • helm: Add default wal dir to ruler config (#14920) (1a6751f).

  • helm: Fix statefulset templates to not show diffs in ArgoCD (#15192) (3c71c83).

  • helm: Split ingester HPA when zoneAwareReplication is enabled (#14565) (80e46f7).

  • helm: FIx Yaml templates incorrectly annotated (#15207) (acd4d11).

  • helm: Disable Helm test pod when test.enabled=false (#15346) (1fb4e2b).

  • ingester: data race in ingester test (#15465) (04f621e).

  • ingester: Add stream creation error logging (#15640) (f63c16f).

  • kafka: Ensure partition-reader starts up correctly (#14845) (b2f3d2e).

  • kafka: Fix incorrect error check in autoCommitLoop (#15149) (4b5925a).

  • kafka: Read from beginning if no committed offset (#14865) (1b6da11).

  • kafka: Use KafkaEndOffset instead of -1 (#15099) (2113526).

  • kafka: Use separate variable to track the consume offset (#15095) (161a192).

  • Keep the service_name label after discovery in push.ParseLokiRequest before calculating retention and passing to the usage tracker (#14960) (71d4d6a)

  • lambda-promtail: Correct S3 bucket owner label key to prevent retrieval errors (#14913) (85aaf63).

  • lambda-promtail: Remove s3 bucket ownership verification (#15383) (bf73014).

  • logcli: Check for errors before checking for exists when fetching data (#14780) (9e7d2f2).

  • logql: Allow comparison against zero bytes (#15217) (17f1972).

  • memberlist: Respect interface names config for advertise address (#13940) (751324c).

  • metric_aggregation: Fix duplicate metrics registration (#15142) (215f994).

  • metrics: Correctly propagate index stats to metrics.go log line (#14941) (ff111dc).

  • metrics: Enable metric aggregation in loki-local-config.yaml (#15302) (c96b750).

  • operator: Add log attribute for level to structured metadata (#14776) (036c131).

  • operator: Fix maximum OpenShift version (#14764) (cc496c6).

  • operator: Fix operator release pipeline warnings (#14817) (e707a3d).

  • operator: Fix RBAC permission for poddisruptionbudgets (#15107) (f00d07a).

  • operator: Publish images on docker hub upon release (#14786) (126ebed).

  • operator: Update kube-rbac-proxy to upstream registry (#14809) (568d22f).

  • otlp: Write protobuf status on error (#15097) (63a2442).

  • pattern ingester: Remove enabled field in loki-local-config.yaml (#15059) (7410f10).

  • promtail: Fix API level node filtering of Kubernetes SD to only fetch pod metadata for pods of the current node (#14267) (d2b6fd8).

  • promtail: Fix Windows forward event crash. #16155 (1c0d558).

  • querier: Fix goroutine leak in queryrange downstreamer (#15665) (5f476a3).

  • querier: Legacy label endpoint route (#14933) (8ce321e).

  • querier: Mitigate ingester race between Query & GetChunkIDs (#15178) (bd46e4c).

  • querier: Show correct setting in error message (#14842) (f5ae015).

  • queries: Bytes unit detection in detected fields (#15525) (8e260fe).

  • Remove unsafe pkg usage from util.mempool (#15428) (e6d82b9).

  • ruler: Fix loki ruler generator url left parameter url encoding (#15601) (e347eb7).

  • Shut down delete client on local rule-evaluator (#15345) (42469cc).

  • storage: Data race in chunk client hedging tests (#15466) (fad06ee).

  • storage: Do not retain copy of chunk while indexing a new chunk in tsdb while processing delete requests (#15541) (ff19955).

  • structured metadata: Structured metadata not always passed down for processing deletes (#15526) (a3808b6).

  • wal: Ensure successive WAL replays don’t overwrite each other (#14848) (ec95ed1).