Grafana Labs founders on the future of observability and how to scale an open source company
“Overwhelming.”
It was the only word Grafana Labs CEO and Co-founder Raj Dutt could use to describe how it felt to look out at the sea of more than 600 Grafanistas gathered together in Whistler, British Columbia, for the first company-wide employee event in two years.
“Intellectually, everyone knows that we’ve grown a lot as a company, and it just feels both long overdue, overwhelming, and just generally awesome to be here and meet everyone who’s helped us get to this point,” Raj told the crowd.
If we were plotting points on a graph to visualize the growth of Grafana Labs since it was founded in 2014 by Raj, Chief Technology Office Anthony Woods, and Chief Grafana Officer Torkel Ödegaard, who created Grafana, we would see a steep line of growth that has led to not only establishing Grafana as the de facto visualization platform with more than 50k stars on GitHub and 950,000 installations to date. Grafana Labs has also expanded its suite of open source projects to include Grafana Loki for logs, Grafana Mimir for metrics, Grafana Tempo for traces, and Grafana OnCall for on-call management; not to mention our self-managed Grafana Enterprise and hosted Grafana Cloud services as well.
But it’s the people and culture of Grafana Labs — which fosters transparency, autonomy, and accountability — that has proven to be the cornerstone of the company. No matter how big the organization gets or how fast it scales, success will happen with “our souls intact. I have no interest in pushing another 10% of growth if it means losing our souls,” Raj explained. “I really want to make sure that we’re in this together. We are building relationships that are going to last beyond this company, whatever our personal journeys are.”
In May 2022, our shared journey led us to GrafanaFest, a celebration of Grafanistas from more than 40 countries all over the world. During a week filled with team bonding, outdoor adventures, and a lot of Grafana swag, the Grafana Labs co-founders sat down for a special live taping of “Grafana’s Big Tent” podcast with hosts Mat Ryer, Matt Toback, and Tom Wilkie, where they answered questions from the audience about where they see the company going, how they work through disagreements, and the difference between high performance and burnout.
This transcript has been edited for length and clarity.
Grafana Labs today: “It makes me immensely proud”
Tom Wilkie: This question is for all of you — how does it feel, guys?
Raj Dutt: I was in a hotel room in Singapore for two years. [In early 2020, Raj went to visit his mother in Singapore and ended up locked down when the pandemic hit.] During those two years, the company grew so much. We raised hundreds of millions of dollars. We hired hundreds of people. We launched lots of products. We acquired companies, we got new investors, we got new board members — without meeting a single person. And that was really frustrating. Ordering a bottle of champagne to celebrate our series D to my room and to drink it by myself at 9 a.m. was not cool. So to answer your question, this feels, pardon my French, f—ing awesome!
Tom Wilkie: Torkel, in your wildest expectations did you ever think you’d have this many users, this many employees, this many GitHub stars?
Torkel Ödegaard: No! I thought it was just an open source project; maybe we’ll use it in my team that I was in at the time. It’s really incredible. The company is so much bigger than Grafana today. We have so many other products, but [the fact] that the company’s named after Grafana means a lot to me. It makes me immensely proud that we have built all this together, but it has the name that I came up with!
Tom: About the name. Why have we got the “f”? Why is it not a “ph”?
Torkel: That’s how you spell graph in Swedish.
Tom: Probably should have known that.
Matt Toback: We had 10 minutes slated for that answer.
Anthony Woods: This is just wild. When I talked to Raj about doing a startup, there’s no way we thought we’d ever get to this stage. And it’s been such a great journey. But it’s been tough over the last few years. I just remember Zoom calls where we would be signing term sheets and be like, “Yay …. now, back to work. Thanks.”
Tom: And you’ve been trapped in Western Australia.
Anthony: I have. It’s Australia, It’s a penal colony. I’m just in prison there. [laughs] But it’s been frustrating not being able to travel. This is my first trip in two and a half years, and I think now I have trips planned for every month for the rest of the year, which is good. We’ll see how long that lasts. But it’s so great to be here seeing everybody, and personally it really is just mind blowing. I was just overwhelmed last night, seeing people having drinks and just the fact that we filled the room. It was just confusing. I didn’t understand how our company could fill such a space.
Tom: Serious question. Are the pubs still shut on Sundays? Because when I came and visited you, it was impossible to get a beer on a Sunday.
Anthony: No, they’re open, but they close at 10 p.m. every day of the week, which is really irritating — especially when you’ve been stuck there for two years.
Mat: It is “ask me anything,” although I didn’t expect Tom to ask him about pub opening times.
On navigating conflicts: “We have a basis of trust”
Question: Have you ever had a massive disagreement and have not been able to talk to each other?
Raj: We’ve definitely had disagreements. We’ve definitely gotten annoyed at each other. Anthony’s definitely gotten annoyed at me. Anthony’s wife is getting very annoyed at me. [laughs] But we just try to work things out. We have an unwritten rule that we’ll try to talk stuff through.
Anthony: I agree. I don’t think we’ve ever had any major issues. Often just talking things through, we always get to a conclusion. I certainly feel like if you’re having a disagreement with someone and you can’t convince them of your idea, then it probably means it’s not a good idea.
“We’ve been willing to do things that have failed and just been like, ‘Oh well, that told us something.’ So we haven’t been afraid of ambiguity. We’ve kind of embraced it.”
— Raj Dutt
Torkel: We have a basis of trust, and we know we all have good intentions. If we have misunderstandings, we just need to talk it through.
Anthony: I think being remote really makes a big difference. Because we don’t get into this situation where you’re in somebody’s face and you can’t avoid it. We can always take time to be thoughtful about what it is we want to say or what we want to present and then go and talk to the person. So I think being a remote company has really been an advantage for us.
Mat: That’s really interesting because you can see that in the company. Today is actually my one year Grafana anniversary, and that is what I noticed spread throughout the company when I joined is that attitude of being authentic, trusting each other, all these things we talked about.
Tom: Your remote-first point also really resonates with me. I’ve rage-quit most companies I’ve worked for. [laughs] I really do feel that if you have a disagreement, you go and sit back at your desk and you’re kind of fuming. But the remote-first way, you go out for a walk, you go and watch some YouTube, you go and do some cooking or whatever it is to calm down and realize the other person means well. They want the same thing you do. We just have to get there.
Matt: What’s the day after a rage-quit feel like?
Tom: It’s good! I’m usually in a pub until 10 p.m.
On Americans v. Europeans
Question: All three of you are from different countries and different cultures. [Raj is based in the U.S. Torkel lives in Sweden. Anthony, as we learned, has been trapped in Australia.] I’m curious if you’ve learned anything fun or interesting about each other’s cultures in your time working together.
Matt: Phone starts with an F…. [laughter]
Raj: I think Americans are generally more optimistic than Europeans.
Anthony: I think Europeans are more honest.
Torkel: We take more time off.
Raj: The Europeans know how to have better work-life balance, which was definitely a point of discussion several years ago when we decided to align on 30 days global vacation across the whole company, which made Sweden go [yawns] “Okay, whatever” and made the U.S. go “Wow!”
Matt: Do you remember at the beginning, in the first summer, Torkel was like, “Oh, and I’m going to be on vacation in July.” And I was like, “When?” He was like, “July … and a little bit of June and the first bit of August.”
On building and flying the company at the same time
Question: When I joined the company, sometimes I’ve been brought in on projects and I’m like, “How the heck do I even do this?” and David Kaltschmidt [Senior Director of Engineering] would say, “We’re building and flying at the same time.” So as all of you were building and flying Grafana Labs, how did you guys navigate ambiguity?
Raj: With great difficulty and unsuccessfully? Personally speaking, I actually enjoy being in a state of ambiguity without a lot of process and sort of building things as we go. That’s always been something that I’ve enjoyed in previous startups, but the challenge is combining that with scaling things and removing the ambiguity when you need to. I think we’ve just iterated. We’ve zigged and zagged as we go, we’ve been willing to do things that have failed and just been like, “Oh well, that told us something.” So we haven’t been afraid of the ambiguity. We’ve kind of embraced it.
Tom: That’s part of the company culture, right? Perfect is the enemy of good. We really do prioritize shipping something before shipping the perfect thing, so that really helps in these situations.
Anthony: I’m a big believer that when there’s ambiguity, if you’ve got two choices and you’re not sure which one is better then it doesn’t matter, just pick one.
Raj: That’s a really good point. Speed can be a weapon in terms of decision making. As we scale, obviously decisions are slower and slower for good reason. But what Anthony just said in terms of pick one or maybe try quick experiments on both — do something. Perfection is the enemy of progress.
On what keeps them inspired
Question: What keeps you motivated in your day-to-day tasks? I’m looking for some inspiration.
Torkel: For me, it’s simple. We constantly try to push that barrier of ease of use for existing users and new users and areas where Grafana can innovate. I think there’s still a lot to explore and improve and rethink.
Raj: At Anthony’s and my last company [Voxel], we were selling and hosting data center services and cloud services, and we watched how open source completely commoditized the whole infrastructure space. We saw that, we were a part of that, but we kind of got swept away in that. One of the things that motivates me about Grafana Labs is we think that same commoditization with observability is happening. Open source is going to win in observability. But instead of getting swept up in that, we have the opportunity to be on the crest of the wave literally making that happen. We have the opportunity to be the dominant company that changes the entire industry as open source and open standards take over. That’s really exciting for one.
The other thing is just seeing people scale. Regardless of what happens in the future — whether it’s a long-term dominant company in the future, an IPO, an acquisition, who knows? — I really hope that the relationships that we all build with the people here and the people that are going to join us are going to continue onto our next thing. That’s something that’s personally a big part of who I am. That’s why this guy’s here [points to Anthony]. That’s why so many of the ex-Voxel crew are investors in this company. And that’s what really motivates me. Regardless of what we do together, I hope that everyone here has [those connections], because that’s so powerful.
And of course money, but that’s third. Honestly, the first two are really what get me out of bed every day.
Anthony: The thing that motivates me and gets me real excited is that we’ve done a lot of great things, but I also see so many areas of opportunity for us to do better. And it’s always great being reflective: This is good; now if we do that, look how much better we could be. There’s a lot of areas that we can continue to work on and improve. And I like knowing that because if we got into a situation where we tried everything and it wasn’t working, then that’s a problem. But we’ve got so much opportunity to keep growing, to keep doing new things. We don’t know where this will end. There’s so much we can just keep on doing.
Mat: The answers were really inspirational. But by the way, you can’t put acceptance criteria on questions! Make it inspirational, please? Can you answer the question in a British accent, please?
On what’s next in observability
Question: What’s the next big thing you foresee in the observability space? [someone chimes in from the audience] In a British accent!
Raj: [in his best British accent] Well, I mean, you’ve got platforms that are expanding …
Tom: That’s passable actually!
Matt: Was that British?
Mat: I didn’t notice it, so yes!
Raj: I’m going to stop the British accent …
Mat: I will too.
Raj: There’s so many ways to answer this question, but one interesting thing to me is how the definition of observability is widening. You see this happening across the industry whatever you want to call it — shift left, shift right, shift up, shift down, earlier in the software development life cycle, later in the software development life cycle — observability isn’t necessarily its own thing. It’s part of a continuum of how we build and run software. It’s just an exciting time to be in observability.
Torkel: One trend that I could see coming is that increasing standardization of development platforms and application platforms where the observability experiences are going to be so much more pre-made, out-of-the box because you are using some standardized application development frameworks where all the metrics are already in place, and the experience in Grafana is going to be already made for you. I can see that kind of evolution to more manually built dashboards if you use those more standardized development platforms.
On avoiding burnout at a high-growth company
Question: Grafana Labs is growing rapidly. When you grow this quickly, you also stretch the teams. Because of the pure acceleration of new people coming in, what are your views on doing this quick growth while also maintaining the qualities and the culture at the same time within the teams and within the company?
We’re in a very competitive industry and sometimes teams are going to have to stretch. … Sometimes you’re going to be up late trying to fix something and you’re going to be stressed out and you know what? That is fine. What is not fine is if that becomes the norm."
— Raj Dutt
Anthony: That’s a really important topic and I think that’s been our biggest concern and risk that we see as a company: Scaling and being able to maintain our culture as we do it. It’s something that we really put a lot of effort into. We really want to have great hiring practices. The People Ops team has been great around recruiting. Onboarding is so important to make sure people understand how we work because as we get bigger, it’s getting more and more challenging to do that. And it’s great to see that we built these processes because we really want to make sure we maintain this really great culture that we feel.
Raj: The stretching point is really interesting. I guess a two-part answer to that. Look, we’re a startup, we’re a high-growth company. We’re in a very competitive industry and sometimes teams are going to have to stretch. People are going to have to stretch. Sometimes you’re going to have a really intense week. Sometimes you’re going to be up late trying to fix something and you’re going to be stressed out and you know what? That is fine. What is not fine is if that becomes the norm. I don’t want to get away from the fact that this is a challenge. This is a startup. This is a high-performance company, and we expect everyone to bring their A-game and be able to really step up, if necessary. But the commitment that we will make and the People Ops team and hopefully all our leaders will make is if that becomes the normal and you’re stretched for weeks on end or months on end, that’s really bad. It’s just going to burn people out. So stretching is okay, but consistent stretching over time or a culture where you’re just overworked, that’s what we really want to avoid.
Mat: That deserves a round applause.
Don’t want to miss the latest episodes? You can now subscribe to “Grafana’s Big Tent” podcast on Apple Podcasts and Spotify.
