Reduce costs, optimize log volumes, and maintain full observability: Introducing Adaptive Logs

When excessive log volumes flood your environment, finding the crucial information becomes a stressful and daunting task. Rising costs add to the pressure, straining your resources without delivering real value.

You need a way to optimize your costs while maintaining full visibility into your system. That’s why we built Adaptive Logs, which is now generally available in Grafana Cloud for all tiers (including our forever-free tier). As we demonstrated at ObservabilityCON 2024, this new Grafana Cloud feature helps you lower your observability costs by reducing the volume of unnecessary logs. And with just a few clicks, you can also improve your log management experience so you can focus on what’s truly valuable.

Adaptive Logs leverages AI/ML techniques to analyze observability data at a scale that wouldn’t be feasible with manual processes. It identifies commonly ingested log patterns and creates a set of customized optimization recommendations based on how frequently those patterns are queried. Like Adaptive Metrics — which has delivered, on average, a 35% reduction in metrics costs for more than 1,500 organizations — Adaptive Logs recommends dropping telemetry that you aren’t actually using. This alleviates the stress of rising costs while ensuring you maintain comprehensive insights.

“Adaptive Logs helps reduce noise, making it easier to spot valuable logs and ultimately saves us costs,” says Andrew Qu, Software Engineer II at TeleTracking, an early adopter of Adaptive Logs that is already seeing a 50% reduction in log volumes, a leading indicator of potential cost savings.

Reduce costs, stay within budget, and cut down on noise

Adaptive Logs analyzes your ingestion and query behaviors in real time, providing daily recommendations that reflect your most recent usage patterns. You can then apply those recommendations with the push of a button, helping you stay within your budget and enabling you to put those savings to better use elsewhere.

And by cutting through that clutter, you can also observe the state of your systems more easily.

“Without Adaptive Logs, even at lower log levels, our dev teams are still generating many logs that never get reviewed,” says TeleTracking’s Andrew Qu. “This makes it significantly harder and longer to sift through logs and find what they’re looking for, especially when searching back over days or weeks.”

TeleTracking’s engineering team was also experiencing alert fatigue from unactionable alerts, making it hard to focus on important ones. With Adaptive Logs, those concerns have largely been addressed. “The engineers would prefer if all or most of their logs were important or useful. Adaptive Logs help ensure that error logs are significant, making it easier to identify and respond to critical issues,” Qu says.

Identify a percentage of logs that can be dropped via recommendations

Adaptive Logs optimizes your log spend by analyzing your Grafana Cloud Logs environment to identify low-value logs that can be dropped. It begins by grouping log lines that match similar patterns using its recommendation engine. Then, it analyzes the query behavior in your Grafana Cloud Logs instance, tracking how often these grouped log patterns are returned in queries over the past 15 days.

Based on this analysis, Adaptive Logs generates recommendations for each pattern group, suggesting a drop rate — i.e, the percentage of those logs that can be safely dropped without affecting your observability.

You can review, edit, and apply these recommendations in the Adaptive Logs plugin UI. Once applied, logs are dropped upon ingestion into Grafana Cloud, reducing unnecessary log storage costs while ensuring that your observability remains intact.

Start using Adaptive Logs today

Getting started with Adaptive Logs is easy: Go to the Grafana Cost Management hub, where you can access the Adaptive Logs plugin. In the Adaptive Logs UI, you’ll find recommendations on which logs to drop based on usage patterns. From there, you have the flexibility to review and edit these recommendations to fit your needs. Once you’re satisfied, simply press Apply drop rates to implement the changes.

And if you want to learn more about how to get the most out of this new feature, check out the Adaptive Logs docs.

Adaptive Logs is available for no additional cost on all Grafana Cloud tiers, so if you haven’t already, sign up for a Grafana Cloud account to take advantage of this powerful tool.