This is documentation for the next version of Loki. For the latest stable release, go to the latest version.
V2.4
Loki 2.4 focuses on two items:
- Making it easier to get your logs into Loki.
- Making Loki easier to run and operate. Anyone not using Kubernetes or avoiding microservices should pay close attention: this release has a lot for you.
Features and enhancements
- Loki no longer requires logs to be sent in perfect chronological order. Support for out of order logs is one of the most highly requested features for Loki. The strict ordering constraint has been removed.
- Scaling Loki is now easier with a hybrid deployment mode that falls between our single binary and our microservices. The Simple scalable deployment scales Loki with new
read
andwrite
targets. Where previously you would have needed Kubernetes and the microservices approach to start tapping into Loki’s potential, it’s now possible to do this in a simpler way. - The new
common
section results in a 70% smaller Loki configuration. Pair that with updated defaults and Loki comes out of the box with more appropriate defaults and limits. Check out the example local configuration as the new reference for running Loki. - Recording rules are no longer an experimental feature. We’ve given them a more resilient implementation which leverages the existing write ahead log code in Prometheus.
- The new Promtail Kafka Consumer can easily get your logs out of Kafka and into Loki.
- There are nice LogQL enhancements, thanks to the amazing Loki community. LogQL now has group_left and group_right. And, the
label_format
andline_format
functions now support working with dates and times. - Another great community contribution allows Promtail to accept ndjson and plaintext log files over HTTP.
All in all, about 260 PR’s went into Loki 2.4, and we thank everyone for helping us make the best Loki yet.
For a full list of all changes, look at the CHANGELOG.
Upgrade Considerations
Please read the upgrade guide before updating Loki.
We made a lot of changes to Loki’s configuration as part of this release. We have tried our best to make sure changes are compatible with existing configurations, however some changes to default limits may impact users who didn’t have values explicitly set for these limits in their configuration files.
v2.4.0 changes to defaults
Here is a list of limit defaults that have changed in v2.4.0:
config | new default | old default |
---|---|---|
ingestion_rate_strategy | “global” | “local” |
max_global_streams_per_user | 5000 | 0 (no limit) |
max_query_length | “721h” | “0h” (no limit) |
max_query_parallelism | 32 | 14 |
max_streams_per_user | 0 (no limit) | 10000 |
reject_old_samples | true | false |
reject_old_samples_max_age | “168h” | “336h” |
per_stream_rate_limit | 3MB | - |
per_stream_rate_limit_burst | 15MB | - |
We have enabled the in-memory FIFO cache by default when no existing external cache is configured. This helps improve Loki’s performance, however it will also allow Loki to consume more memory.
v2.4.2 changes to defaults
Here is a list of limit defaults that have changed in v2.4.2, add in PR 5077:
config | new default | old default |
---|---|---|
parallelise_shardable_queries | true | false |
split_queries_by_interval | 30m | 0s |
query_ingesters_within | 3h | 0s |
max_chunk_age | 2h | 1h |
max_concurrent | 10 | 20 |
Bug fixes
2.4.2 bug fixes
2.4.2 fixes these bugs:
- PR 4968 trevorwhitney: Fixes a bug in which querying ingesters wrongly returns a ruler,
causing the internal server error
code = Unimplemented
. - PR 4875 trevorwhitney: Honor the replication factor specified in the common configuration block when
memberlist
is the consistent hash ring store. - PR 4792 AndreZiviani: Corrects the default values of configuration options in the documentation for:
scheduler_dns_lookup_period
min_ready_duration
final_sleep
max_transfer_retries
chunk_retain_period
chunk_target_size
batch_size
timeout
(for Redis requests)
2.4.1 bug fixes
2.4.1 fixes two bugs in the 2.4.0 release:
- PR 4687 owen-d: Eliminate a compactor panic when not using a tenant overrides file.
- PR 4681 slim-bean: Correct the initialization of the
read
target. Incorrect initialization resulted in query gaps after a chunk was flushed and before the querier downloaded new index tables.
2.4.0 bug fixes
Here are some of the more important fixes in version 2.4.0:
- PR 4598 kavirajk: Fix the IP matcher lexer to differentiate a filter from an identifier.
- PR 4563 cyriltovena: Fixes the Series function to properly handle sharding.
- PR 4518 slim-bean: Fixes a bug in which items are returned to a sync.Pool incorrectly.
- PR 4411 slim-bean: Fixes a frontend bug which caused it to wait for results which would never arrive.
- PR 4238 liguozhong: Fixes a distributor goroutine leak.
Security fixes
2.4.0 security fixes
Version 2.4.0 contains one security-related fix:
PR 4627 Explicitly define allowed HTTP methods on HTTP endpoints.
A community user noticed that all Loki endpoints would respond to HTTP OPTIONS requests, and the proxy they placed in front of Loki responsible for HTTP authentication would pass OPTIONS requests unauthenticated to Loki, allowing users to make unauthenticated/unauthorized queries to Loki. The fix restricts the type of HTTP methods allowed on each endpoint and disallows OPTIONS requests.