Grafana

The Business of Democratizing Metrics 

Some startup companies toil away invisibly for years. Though their products often have mission-critical value for users, they may be so far behind the scenes that they never get the glory.

That’s not the case for Grafana, an open source software project for visualizing and analyzing metrics. And given what the platform does and how it looks, it’s easy to see why it has taken the spotlight in this data-driven world. 

“It’s such a visual and appealing way to present data, and you can get at-a-glance insight from it,” says Torkel Ödegaard, the creator and lead developer of Grafana, and co-founder of Grafana Labs. “Once a couple of people within a company start using it, they put the dashboards up on a TV or big screen, and people who walk by become intrigued by it.”

The Grafana dashboards have become so recognizable, in fact, that during the first Falcon 9 rocket launch last year, they were easy to spot, displayed on a screen at Space X’s control center. “I couldn’t believe it at first,” says Ödegaard. “But zoom in and you can see it.” And then a few weeks later, he adds, “There was a promotional video from Microsoft showing off their underwater data centers. And there were a couple of people standing around pointing at a Grafana dashboard.”

Solving a Basic Need

That’s the beauty of open source projects like Grafana, which anyone—from the hobbyist and hacker to the Microsofts of the world—can pick up and experiment with. Grafana now boasts 100,000 active installations, but the project started in 2013 as Ödegaard’s attempt to scratch his own itch. “As a consultant in Sweden, I felt a lack of insight into the applications that we were writing, the behavior in production, where the bottlenecks were, and what sort of usage we were doing,” he says. 

He started using a popular open source database, Graphite, but found its user interface lacking. Then, he discovered Kibana, which visualizes Elasticsearch logs, and was inspired by its user interface. “I started Grafana to do something similar as Kibana, but focused on time series metrics.  My goal was to make time series data accessible for a wider audience, to make it easier to build dashboards, to make graphs and dashboards more interactive,” he says. 

And, importantly, more beautiful. “In the open-source world in particular, it’s unfortunate that few projects get to spend enough effort on design or user experience,” says Grafana Labs co-founder and CEO Raj Dutt. “From the first version, Torkel has put a lot of effort and care into the user experience, and how it looks and flows” 

That combination led to Grafana going viral within the monitoring and metrics community. And with its system of plugins, which allows users to hook just about any kind of data source into Grafana, the project has become the ‘de facto’ standard for visualizing metrics. 

Building an Open Source Business

Meanwhile, Dutt and Anthony Woods were looking to build a business together in the monitoring space. They both loved the Grafana Project, and in 2014 reached out to Ödegaard and asked him to come to New York City for a weeklong consulting gig. After that quick visit, the three decided to co-found a new business, Grafana Labs (originally raintank), dedicated to accelerating the Grafana Project. 

“We just got along so well,” says Woods, who is a co-founder and CTO of Grafana Labs. Plus, Ödegaard had just left a consulting gig in order to work on the Grafana project full-time, and was thinking about how to make the project sustainable; Woods remarked: “It was definitely good timing.”

For Dutt and Woods, the fact that Grafana was open source was critical. “A lot of times, open source projects are acquired or strategically merged and basically the acquiring company closes it off or makes it a more proprietary thing,” says Dutt. “One of the fundamental tenets of Grafana Labs is we believe open source is a better way.” Adds Woods: “We want to provide quality open source software for the community to leverage and use; that’s part of our core strategy.” 

Grafana v.2.0 was released by the newly-formed Grafana Labs in the summer of 2015. “There’s just been ever-increasing popularity,” says Ödegaard. “New startups and established enterprise companies alike depend on Grafana, and we’re always impressed by the huge variety of use cases people find for it.” 

This variety is made possible because the platform is essentially agnostic about what data is displayed. Aside from the typical application metrics and infrastructure monitoring, Grafana is used for scientific purposes, for industrial and agricultural sensors, and energy management. One user even created a dashboard to show the wait times at hospital emergency rooms in Japan.

And increasingly, and importantly for Grafana Labs, it’s being used by big enterprises. “We often first hear from large users after they’ve deployed and started using Grafana. They already have an internal advocate. There’s very little sales process at that stage, says Dutt. “They’ve checked it out, they like the fact that it’s open, that it’s powerful yet easy to use and looks great, and that it works with all their different data sources. And then it becomes a question of, ‘Now we’re depending on this software we downloaded, and we want to get the most out of it. We want it to fit better, we want to de-risk it, and we want support for it.’ And so open source model gives us the ultimate distribution channel”

‘Open SaaS’

To that end, Grafana now sells two products: subscriptions for users who want to run Grafana themselves, and GrafanaCloud, an enterprise-level, SaaS version. 
“When you look at the monitoring landscape, you’ve got all the open source players, and on the commercial side you’ve got all these new SaaS players,” says Dutt. “We think there’s a third choice, which offers the convenience of SaaS—you sign up, and if you pay your bill, it just works for however much data you send it—with the flexibility of open source. All our software is open source so there’s no lock-in. You own your own data. You can combine other data sources with the platform. It’s the best of both open source and SaaS. We call it Open SaaS.”

Specifically, GrafanaCloud solves a challenge that many users have with storing all their metrics: volume. 

“Five years ago, people would measure every 5 minutes; now people want it every 10 seconds,” says Woods. “People used to measure maybe a handful of metrics per server; now people are collecting hundreds of different metrics per server.  Also, with cloud computing there’s a lot of dynamic infrastructure—virtual machines come and go all the time. We provide a hosted metrics platform, and people can send us the metrics and we handle the rest. It integrates directly with Grafana, which people are familiar with and has a great user experience. So we solve the complexity of scaling the infrastructure.”

Operating this back end for customers made their infrastructure choices crucial. “Since launching Grafana Cloud, we’ve grown to thousands of hosted Grafana instances running on thousands of containers,” says Dutt. “We run a mission critical service, and it’s really important for us to have a really reliable and consistent platform with great performance, that can scale with our customers growth”

The Equinix Metal Impact 

GrafanaCloud operates across different clouds, between Google, Amazon and Equinix Metal, depending on users’ needs. It can deploy its service across multiple vendors for maximum reliability. “Our preference is to expand across Equinix Metal because of the quality and performance of their platform” says Dutt. 

“It’s about having the performance and the consistency of bare metal, and knowing exactly what we’re getting down to the exact CPU in the machine. We’re building really large Cassandra and Kubernetes clusters so the Type 2s just give tremendous storage and memory capabilities.” Adds Woods: “The heavier the workload, the more Equinix Metal shines.”

Equinix Metal's geographic expansion has been beneficial, too. “Our customers are all around the world and the location of their cloud setup is important,” says Dutt. “We started with Equinix Metal in New York and recently launched GrafanaCloud with Equinix Metal in San Jose. As Equinix Metal expands its footprint in Europe, we anticipate expanding as well.”

Democratizing Metrics

Grafana 5.0 will be coming out later this year, with several major new features. The project is seeing most new features and fixes come from external power users in the community. The Grafana Labs team recently collaborated with WalMart Labs to develop a dashboard versioning history feature, released this month. “These companies are so aligned with the Grafana Project that they assign resources to help us make the open source project better, far faster than we could do it alone.”

There are also more plugins coming for Grafana. “We have an ever-expanding catalog on Grafana.com of integrations and dashboards that, whatever you’re doing—whether you’re monitoring MySQL or MongoDB or a Linux server or your home thermometer—there are easy ways to get started and get data into our platform,” says Dutt. “There’s a tremendous opportunity just going wide with that in terms of plugging into the ecosystem.”

It’s all part of one of the company’s main tenets: Democratize metrics. “It means making software easy to use, but it also means owning your data, and bringing data from wherever you need to,” says Dutt. “We really think that this stuff shouldn’t be left to the data scientists or the developers. If you’re in the marketing department and you understand what your metrics are, and how they’re related, you should be able to create an awesome visualization for your team. When Grafana becomes easier to use, it empowers people to leverage their data to innovate more quickly.”