Loki’s Path to GA: Version 0.2.0

What Took So Long!?

Friday, August 2, marked the second beta release for Loki, a long overdue version 0.2.0. Why did it take so long? In large part this was my fault. Having done some work to create a release process for version 0.1.0, I found myself focusing on other things, so improving that process ended up on the backburner. This entire time, in the back of my mind, I was delaying a new release until I could improve that process.

Thankfully for all of us, sh0rez showed up to the project and has joined Grafana Labs for the summer, making significant contributions to our release processes. Things are back on track to start making more frequent Loki releases.

So what has changed since 0.1.0? A lot! When looking at the merged PRs, I quickly realized there were way too many to individually call out. I did my best trying to summarize the more significant changes into the changelog, and I’ll talk about some of them here.

What’s New?

Performance

I think I’m most excited to see the performance improvement work done by cyriltovena, who started by adding support to fetch labels from the chunk store. This allows us to configure a lower chunk retention in memory. Then he took a deep dive into how chunks were loaded and memory was allocated to make sure queries were no longer causing Out Of Memory issues. With all these improvements, we’ve seen a significant improvement in query performance, most notably in Loki’s memory usage:

Features

The log processing pipeline continues to get more robust. There’s a new stage for manipulating string values using Go’s templating language. Improvements have been made to the timestamp parsing to support Unix monotonically increasing timestamps as well as support dates without a year or time zone (common in a lot of syslog formats).

The fluent plugin saw some challenges, with a naming conflict that was resolved, as well as some updates to dependencies and bug fixes.

A Docker plugin was created, and support for reading the systemd journal as well. (We are still working out some packaging challenges with this change, so it’s currently only available if you build promtail yourself.)

The loki-canary application was introduced to help us monitor Loki’s accuracy in storing logs.

Improvements continue for live tailing of logs and to the logcli.

And as mentioned at the beginning, big improvements were made to the build and release tooling to both simplify our Makefile and automatically generate multiple architecture container images and pre-compiled binaries.

What’s Next?

More frequent releases! We continue to march Loki towards the first GA release, and to help that process we are going to target bi-weekly to weekly releases, depending on changes. This should help ensure all features and contributions are visible in the changelog and make it easier to keep current with the rapid improvements the project is undergoing.

Community improvements are also underway, too, with governance documentation recently added to the project as well as plans to start a regular community call.

Thanks!

Thanks so much to everyone who has contributed to Loki. We couldn’t be more excited by the community involvement on GitHub and in Slack and are excited by what’s to come.

Check out the GitHub release page, where you can get started with Loki 0.2.0, and subscribe for notifications of all new Loki releases!