V2.4

Loki 2.4 focuses on two items:

Making it easier to get your logs into Loki.
Making Loki easier to run and operate. Anyone not using Kubernetes or avoiding microservices should pay close attention: this release has a lot for you.

Features and enhancements

Loki no longer requires logs to be sent in perfect chronological order. Support for out of order logs is one of the most highly requested features for Loki. The strict ordering constraint has been removed.
Scaling Loki is now easier with a hybrid deployment mode that falls between our single binary and our microservices. The Simple scalable deployment scales Loki with new read and write targets. Where previously you would have needed Kubernetes and the microservices approach to start tapping into Loki’s potential, it’s now possible to do this in a simpler way.
The new common section results in a 70% smaller Loki configuration. Pair that with updated defaults and Loki comes out of the box with more appropriate defaults and limits. Check out the example local configuration as the new reference for running Loki.
Recording rules are no longer an experimental feature. We’ve given them a more resilient implementation which leverages the existing write ahead log code in Prometheus.
The new Promtail Kafka Consumer can easily get your logs out of Kafka and into Loki.
There are nice LogQL enhancements, thanks to the amazing Loki community. LogQL now has group_left and group_right. And, the label_format and line_format functions now support working with dates and times.
Another great community contribution allows Promtail to accept ndjson and plaintext log files over HTTP.

All in all, about 260 PR’s went into Loki 2.4, and we thank everyone for helping us make the best Loki yet.

For a full list of all changes, look at the CHANGELOG.

Upgrade Considerations

Please read the upgrade guide before updating Loki.

We made a lot of changes to Loki’s configuration as part of this release. We have tried our best to make sure changes are compatible with existing configurations, however some changes to default limits may impact users who didn’t have values explicitly set for these limits in their configuration files.

v2.4.0 changes to defaults

Here is a list of limit defaults that have changed in v2.4.0:

config	new default	old default
ingestion_rate_strategy	“global”	“local”
max_global_streams_per_user	5000	0 (no limit)
max_query_length	“721h”	“0h” (no limit)
max_query_parallelism	32	14
max_streams_per_user	0 (no limit)	10000
reject_old_samples	true	false
reject_old_samples_max_age	“168h”	“336h”
per_stream_rate_limit	3MB	-
per_stream_rate_limit_burst	15MB	-

We have enabled the in-memory FIFO cache by default when no existing external cache is configured. This helps improve Loki’s performance, however it will also allow Loki to consume more memory.

v2.4.2 changes to defaults

Here is a list of limit defaults that have changed in v2.4.2, add in PR 5077:

config	new default	old default
parallelise_shardable_queries	true	false
split_queries_by_interval	30m	0s
query_ingesters_within	3h	0s
max_chunk_age	2h	1h
max_concurrent	10	20

Bug fixes

2.4.2 bug fixes

2.4.2 fixes these bugs:

PR 4968 trevorwhitney: Fixes a bug in which querying ingesters wrongly returns a ruler, causing the internal server error code = Unimplemented.
PR 4875 trevorwhitney: Honor the replication factor specified in the common configuration block when memberlist is the consistent hash ring store.
PR 4792 AndreZiviani: Corrects the default values of configuration options in the documentation for:
- scheduler_dns_lookup_period
- min_ready_duration
- final_sleep
- max_transfer_retries
- chunk_retain_period
- chunk_target_size
- batch_size
- timeout (for Redis requests)

2.4.1 bug fixes

2.4.1 fixes two bugs in the 2.4.0 release:

PR 4687 owen-d: Eliminate a compactor panic when not using a tenant overrides file.
PR 4681 slim-bean: Correct the initialization of the read target. Incorrect initialization resulted in query gaps after a chunk was flushed and before the querier downloaded new index tables.

2.4.0 bug fixes

Here are some of the more important fixes in version 2.4.0:

PR 4598 kavirajk: Fix the IP matcher lexer to differentiate a filter from an identifier.
PR 4563 cyriltovena: Fixes the Series function to properly handle sharding.
PR 4518 slim-bean: Fixes a bug in which items are returned to a sync.Pool incorrectly.
PR 4411 slim-bean: Fixes a frontend bug which caused it to wait for results which would never arrive.
PR 4238 liguozhong: Fixes a distributor goroutine leak.

Security fixes

2.4.0 security fixes

Version 2.4.0 contains one security-related fix:

PR 4627 Explicitly define allowed HTTP methods on HTTP endpoints.
A community user noticed that all Loki endpoints would respond to HTTP OPTIONS requests, and the proxy they placed in front of Loki responsible for HTTP authentication would pass OPTIONS requests unauthenticated to Loki, allowing users to make unauthenticated/unauthorized queries to Loki. The fix restricts the type of HTTP methods allowed on each endpoint and disallows OPTIONS requests.