How Exabeam gained insight and control over 20 million active time series with Grafana Cloud

Nobody enjoys a migration, but sometimes they come right when you need one.

Cybersecurity leader Exabeam is all too familiar with this experience. Last year, the team was running another observability platform and a self-hosted monitoring infrastructure that was hitting its limits. Their on-prem stack was taxed dealing with 500,000 time series metrics, and with the addition of Exabeam’s next-gen apps, that number quickly doubled.

Luckily, they were already amid plans to migrate to Grafana Cloud as part of a company-wide effort to move from an on-premises environment to the cloud and streamline all their observability efforts into one platform. Over the course of four months, the engineers performed the delicate dance of migrating their legacy stacks to hosted Grafana while simultaneously maintaining local environments for customers. They also had to consistently reconfigure limits and quotas within Grafana Cloud as releases continued to roll out during the process, increasing their observability data consumption.

They didn’t do it all on their own. “Grafana worked with us to figure out our usage, to put in limits, and to help us with the cardinality, so our experience has really improved from the beginning,” says Yonit Lustig, Lead SRE at Exabeam. Grafana Cloud is now home to more than 20 million active time series and hundreds of dashboards used to track just about everything in Exabeam systems.

A square observability peg in a round hole

Exabeam has multiple offerings, including a security analytics tool and a cloud native security information and event management (SIEM) platform designed to harness the scale of the cloud. They ingest petabytes of logs and events from hundreds of different vendors and products supporting DNS, firewalls, and cloud infrastructure and applications. That data is then normalized, parsed, and analyzed to model user behavior and provide insights to Exabeam customers. Those two primary applications are underpinned by a fleet of more than 100 microservices, each with its own observability requirements.

“We want to ensure there are real-time alerts, debugging, and investigations if the services are down or not performing to the expected SLOs,” says Dinesh Maheshwari, Technical Architect at Exabeam. “We need to get alerts on those and we need to measure all of it.”

That’s hard to do with a struggling observability stack. To erase that toil, Exabeam decided to pivot to a cloud-first approach and began migrating to cloud native solutions. But there was one caveat: Exabeam engineers still had to account for older systems designed to work with off-the-shelf software like Hadoop and MongoDB, at least temporarily, to ensure a consistent experience for customers.

At the same time they were managing legacy tools, they were also acclimating to cloud native services like Google Cloud Storage and Cloud Spanner. Those services require a different approach to observability, which meant they also had to reassess their existing on-premises observability tools — ultimately they decided they needed to consolidate all of their observability efforts into one tool.

Two (expensive) birds, one stone

Exabeam conducted proof-of-concept experiments to migrate to a single platform that had enough power and flexibility to address their ever-growing set of customers, metrics, and logs.

They also needed a solution built for distributed, cloud native applications.

Though they had experience using another observability tool in their older stack, the team ultimately felt it lacked the right levers to control the increasing flow of data into the new cloud native platform. As a result, they were paying to ship a portion of their data that was never really used, yet still added to their costs. Compare that to their experience with Grafana Labs. When the Exabeam team faced a similar situation with an unexpected spike in metrics in Grafana Cloud, “the Grafana Labs team was able to troubleshoot issues, and ultimately helped us optimize our spend,” Maheshwari says. As a result, Exabeam’s observability costs were cut dramatically. “There was much more transparency and visibility into how the costs were calculated,” Maheshwari says.

Beyond cost, Exabeam’s engineers embraced using Grafana Cloud almost immediately, in part, because the UI was clean, well-organized, and intuitive. More importantly, the open and composable cloud platform proved it could meet the wide-ranging needs of the Exabeam team — without any compromises.

Now with a fully managed Grafana Cloud stack that leverages Grafana Cloud Metrics and Grafana Cloud Logs as well as at least eight different data sources and four Grafana Cloud integrations, including Kubernetes Monitoring and AWS CloudWatch, “Grafana was able to support what we needed to build and will continue to build,” says Maheshwari.

Eyes wide open

While some observability platforms are designed for simplicity, that won’t cut it for data-driven organizations like Exabeam that need to dig into the nuances that exist in their multi-faceted systems.

For Exabeam’s engineers, their eyes have been opened to a solution that has increased their productivity and made their lives easier. Lustig says she now only spends 10% of her day managing and supporting engineers using the infrastructure, and the entire team has quickly become comfortable building alerts in Grafana and connecting them to the rest of their engineering tools. Instead of watching dashboards all day, it’s now easier to troubleshoot issues when they arise because they can quickly assess those issues in Grafana.

This becomes even more important in the multi-tenant world of the cloud, where an incident might be impacting multiple customers at once.

“We’re a data-driven company; our decisions are more accurate and representative of real world scenarios versus just relying on anecdotes,” Maheshwari says. “When you’re not data driven, decisions are very subjective and often lead to undesired results.”

As Exabeam’s data and systems grow and evolve, they need a partner that gives them the necessary visibility to continue scaling with confidence. They’re confident that the flexibility and power of Grafana Cloud will help them achieve their goals no matter where they go next.