Grafana Mimir Q&A with Grafana Labs CEO Raj Dutt

When Grafana Labs CEO and co-founder Raj Dutt announced to the team that the company would be launching Grafana Mimir, we knew that along with the public announcement, we would want to publish a Q&A. We collected questions from Grafanistas and Raj has answered them here.

What is Grafana Mimir?

Grafana Mimir is our next-generation, open source scalable time series database for metrics; it is the combination of Cortex and code previously only shipped as part of our commercial products. Mimir is hosted at https://github.com/grafana/mimir and licensed under AGPLv3. 

With Mimir, we’ve extended Cortex to support unlimited cardinality using a horizontally scalable, “split” compactor architecture. This has been tested and hardened in production to support more than 1 billion active series. Mimir also supports blazing fast high cardinality queries through a horizontally scalable, sharded query engine. We ship production-ready Grafana dashboards, alerting and playbooks to make operating Grafana Mimir easy. There are tools and automation for self-care: TSDB introspection, bucket analysis, traffic analysis, and more. And Mimir is easier to deploy than Cortex, with better and more extensive documentation. 

Mimir isn’t just the best scalable Prometheus-compatible TSDB; we will soon be shipping native support for OpenTelemetry, Graphite, Influx, and Datadog-compatible metrics ingestion. In addition, we have taken this opportunity to remove a bunch of legacy code, such as chunk-based storage engine and SQL-based config store. We’ve also drastically simplified the configuration, removing unnecessary options, updating the defaults to match those we run in production.

Why was this change made now?

The time felt right. There are a lot of improvements in Grafana Cloud Metrics and Grafana Enterprise Metrics that we wanted to open source. At the same time, it did not fit into our strategy to license those features under Apache 2.0.

The only way to fulfill both constraints was to license the code under AGPLv3.

So is Mimir a fork of Cortex? Why didn’t you add these features to Cortex?

Yes, Mimir is a fork of Cortex.  

At Grafana Labs we firmly believe in the power of open source; it’s in our DNA. We also believe you get the best OSS by having a sustainable business behind it, one that can employ the world’s best developers to drive the projects forward. That’s our mission at Grafana Labs.

Cortex is a CNCF project licensed under the Apache 2.0 license, and the vast majority of contributions were coming from Grafanistas. Cortex is used by some of the world’s largest cloud providers and ISVs, who are able to offer Cortex at a lower cost because they do not invest the same amount in developing the project. 

This led us down the path of keeping many of our improvements to Cortex closed source.  Rather than keep the code closed, we think we’ve found a way of making it open and building a sustainable business. We feel like the combination of the AGPLv3 license and a CLA achieves this.

All CNCF projects must be Apache 2.0-licensed. This restriction also prevents us from contributing our improvements back to Cortex.

Why is it called Grafana Mimir? How do you pronounce Mimir?

Same as with Grafana for graphs, Loki for logs, and Tempo for traces, we knew our metrics project’s name needed to start with M. As to the specific name of Mimir, for the same reason why we spell Grafana with an “f”: our Scandinavian roots.

We pronounce it /mɪ’mir/.

Why is Grafana Mimir launching with 2.0?

We see Grafana Mimir 2.0 as the natural progression from the Cortex 1.x line.

Grafana Mimir has the resilience and scalability of a 2.x release. Grafana Mimir powers Grafana Cloud Metrics and Grafana Enterprise Metrics, and has done so for months. From single-person organizations to the largest banks and insurance companies, we are using this codebase to deliver our services both on premises and as-a-service in the cloud.

In the process of making Grafana Mimir simpler to operate, we’ve removed a lot of previously required configuration options, changed defaults, increased automated self-care, and added lots of new features. Some of those changes are breaking (in the good sense), so following Semantic Versioning conventions, a new major version is mandated.

Can I run Grafana Mimir myself? Can others use Grafana Mimir to offer Prometheus as a Service?

Grafana Mimir is an AGPLv3-licensed project, so as long as you honor the terms of the license, you are welcome to use the project for any purpose.

How will the choice of the AGPLv3 license affect Mimir adoption?

The work we’re doing on Grafana Mimir to make it easier to get started with and operate, combined with the effort we’re putting into configuration and documentation, will massively improve adoption. Grafana Mimir is significantly easier to install and maintain than Cortex ever was.

We don’t think the choice of license will particularly impact adoption, either negatively or positively. If anything, we saw adoption increase after relicensing Grafana, Grafana Loki, and Grafana Tempo.

While there are a lot of uncertainties and misunderstandings about AGPLv3 in the market, most companies who have performed their own assessments do not have a problem running APGLv3-licensed code. For the few who do, we offer Grafana Enterprise Metrics under a commercial license that allows those users to use the product without any open source copyleft concerns. 

What will happen to the Cortex project?

Historically, Grafana Labs has sponsored the majority of work on Cortex. While Grafanistas will no longer be asked to work on Cortex as part of their day job, they are welcome to contribute in their free time — as is anyone else.

The future of the Cortex project is up to the maintainers of the Cortex project. We strongly believe that individuals, not companies, are members of any given open source project. As such, Grafanistas are welcome to continue as maintainers of Cortex or any other project. If maintainers choose to step down, that is their own personal decision.

We’ve done our best to set up Cortex for future success. Just a few months ago, all Cortex maintainers worked at Grafana Labs. We have worked hard to onboard new Cortex maintainers. As of today, Cortex has three non-Grafana Labs maintainers.

What does this mean for the Prometheus project?

Cortex has always been a separate project from Prometheus with separate (but somewhat overlapping) maintainers and governance. Prometheus is maintained by a large group of people, many independent and others working for a large number of different organizations. Many Grafanistas contribute to Prometheus, and forking and relicensing Prometheus would not make sense for us, nor for the wider community. There is no intention of doing anything like this.

If anything, we are increasing our total investment into Prometheus. We have worked to upstream changes from Cortex into Prometheus in the past, have already upstreamed changes from GEM/GCM to Prometheus, have several PRs open to upstream more, and will continue to do so. (See these PRs, these, these, and these.)

Can other projects (such as Prometheus or Thanos) reuse code from Grafana Mimir? 

While moving code from Apache 2.0 to AGPLv3 is always possible, the other way is not possible without approval from the copyright holder. For other projects to use the code in Mimir, they must be AGPLv3-licensed.

What is dskit, and how does it relate to Grafana Mimir?

dskit is a Golang library for building distributed systems, licensed under Apache 2.0. It contains various useful data structures and algorithms originally from Cortex. We started the project to decouple Grafana Loki and Grafana Tempo from Cortex, so the maintainers of those projects could get more involved in the maintenance of dskit. Grafana Mimir uses the dskit project.

We had wanted Cortex to get all the future improvements we were making to dskit and had migrated code from Cortex there. After the community voiced concerns about this, we reverted all the changes.

What does this mean for users of Grafana Cloud and Grafana Enterprise Metrics?

Grafana Cloud Metrics and Grafana Enterprise Metrics have been based on Grafana Mimir for months. With the release of Grafana Mimir, customers of our products can now see more of the inner workings — for example, how we achieve such impressive performance.

Grafana Cloud is continuously released based on the very tip of Grafana Mimir. The upcoming Grafana Enterprise Metrics 2.0 release is based on Grafana Mimir 2.0, adding our Enterprise features on top. Going forward, Grafana Mimir and Grafana Enterprise Metrics will be released at the same time.

How do we choose when to add a feature to Grafana Mimir versus adding it to Grafana Enterprise Metrics?

The features of Grafana Cloud Metrics and Grafana Enterprise Metrics that aren’t part of Grafana Mimir are targeted at very large organizations with specific needs around things like authentication, reporting, and fine-grained access control. On principle, we try to never wholesale replace components or features in our open source projects with “better” versions in our products; the features in our products tend to be purely additive.

Will we accept outside contributors or even outside maintainers for Grafana Mimir?

Yes. As a matter of fact, we encourage everyone to contribute if they want to. Grafana Mimir is a truly open source project with an OSI-approved license that promotes user freedom and its own governance and maintainers. Most other Grafana Labs projects have outside maintainers and team members. The Grafana Mimir community calls take place on the last Thursday of every month at 14:30 UTC; join us for the first one on March 31! 

If I choose Grafana Mimir, will I be locked in? What options do I have to migrate away from Grafana Mimir in the future? 

We operate on the premise that our community and customers choose to use our projects and products because they’re awesome, not because they have to, or because they cannot afford to migrate away. As such, we embrace open source technology and open standards to prevent lock-in.

We have worked to ensure that Mimir has 100% PromQL compatibility, which prevents you from getting locked in by having Mimir-specific extensions to PromQL. We use the same on-disk format for Prometheus TSDB, allowing you to easily move to any other Prometheus TSDB-compatible project. Features such as the scalable compactor rely on sharded TSDB blocks, which no other project supports, but compacting these blocks together is possible.

We also ensure 100% compatibility with Prometheus remote write, allowing you to send your metrics to any Prometheus remote write compatible solution.

Is all this really about hyperscale cloud vendors?

No. We have great partnerships with Amazon, Azure, Alibaba, Tencent, and others. The release of Mimir is the culmination of a journey that we started last year to have an AGPL-licensed “LGTM” stack — Loki, Grafana, Tempo, and now Mimir.

To learn more, read the Grafana Mimir announcement blog post, and register for the April 26 webinar, Intro to Grafana Mimir, the open source time series database that scales to 1 billion metrics & beyond.