Observability trends and predictions for 2024: CI/CD observability is in. Spiking costs are out.
Note: If you’d like to learn more about where the observability industry is today and where it’s going, check out the final results of the Grafana Labs Observability Survey 2024.
From AI to OTel, 2023 was a transformative year for open source observability. While the advancements we made in open source observability will be a catalyst for our continued work in 2024, there is even more innovation on the horizon.
We asked seven Grafanistas to share their predictions for which observability trends are on their “In” list for 2024. Here’s what they had to say:
AI in observability: More thoughtful about where, when, and how it’s used
AI was ubiquitous in 2023. But with the hype comes promises, and with promises comes, inevitably, disappointment. It’s crucial to approach AI in observability with a realistic perspective, understanding that while AI holds the potential to revolutionize the field, it is not a panacea to everything. That’s why 2024 will be the year observability practitioners learn to strike the right balance between human expertise and AI-driven automation.
This year, AI in observability will be driven by time and cost savings. Users will seek more thoughtful solutions that abstract away time-consuming tasks by automating routine processes. We won’t see AI take jobs from SREs, DevOps specialists, or engineers. Rather, AI will become more of a trusted tool to understand systems quickly through signal correlation, anomaly detection, root cause analysis, and performance optimization. AI-driven tools will free up valuable human resources, allowing teams to focus on more strategic and creative aspects of system management.
– Marc Chipouras, Senior Director of Engineering/Office of the CTO at Grafana Labs
Reducing observability costs: Better budget management, fewer surprises
With companies still facing economic headwinds — and uncertainty — CIOs and IT leaders will continue to reel in their spending. The journey is full of challenges, especially related to measuring costs, attributing them to teams and services, and then prioritizing cost-optimization efforts. It’ll be increasingly important for companies that provide SaaS services to develop adequate metrics (usually service-dependent) that can be tracked over time —for example, a per-service “unit cost” (that is, how much does a “unit of service” cost, with a unit being, for example, “qps”?).
– JuanJo Ciarlante, Principal Engineer at Grafana Labs
FinOps: More financial observability capabilities built in
As companies continue to optimize their IT budgets for impact and efficiency, we’ll see a more data-driven approach in the world of FinOps. As with other aspects of operations, data collection, organization, and analysis are key. Leveraging the power of modern observability tooling will give early adopters a key edge in managing and optimizing their spend.
As an added benefit, this will also enable companies to correlate cost and profit centers more closely. By using the real-time data from the actual production systems, companies will ultimately build a holistic understanding of what makes their company and business work. Successful businesses will be able to understand why and where they spend money in the cloud and on-prem, allowing for deliberate, well-informed choices and trade-offs and avoiding the trap of saving too much in areas that generate revenue.
– Richard “RichiH” Hartmann, Director of Community and Office of the CTO at Grafana Labs
Application and infrastructure observability converge in the OSS space
With increasingly complicated distributed architectures and layers of infrastructures (EC2, Kubernetes, Lambda, etc.), it is critical to combine the insights from both applications and infrastructure to identify and resolve performance issues. For example, if you are seeing 500 errors in your applications due to database queries failing, you need to quickly identify the database in which the errors are happening, what is failing inside the database, where it is running, and how to fix it. This cannot be achieved with siloed infrastructure and application observability.
OpenTelemetry is starting to lead the way in application observability, and Prometheus is an indisputable leader in infrastructure observability. We will see both projects collaborate closely, bringing together the two worlds and providing a single overview of everything that is connected to run reliable applications. The Prometheus project is already on its way to becoming the best metrics backend for OpenTelemetry, and in the future, we will see OpenTelemetry adopt semantic conventions that combine the best of OTel and Prometheus. Right now, according to early results from Grafana Labs’ 2024 Observability Survey, 34% of developers use both OpenTelemetry and Prometheus in some capacity — and we’ll see that increase as these two projects continue to intersect.
– Goutham Veeramachaneni, Product Manager at Grafana Labs
OpenTelemetry: Continuous profiling officially supported
OpenTelemetry reached GA last November, meaning we have all three signals marked as stable: traces, metrics, and now logs. In 2024, as we continue to stabilize more tools like semantic conventions and consistent configuration across components, we’ll hopefully be able to start our graduation process at the Cloud Native Computing Foundation (CNCF).
But one of the most exciting areas of innovation we’ll see is in continuous profiling, which I believe we’ll see evolve into a pivotal aspect of OpenTelemetry’s capabilities. Profiling is poised to undergo transformative advancements, enhancing our ability to gain deeper insights into application performance. In 2024, we anticipate profiling to officially become a supported signal in OpenTelemetry, aligned with traces, metrics, and logs. This will offer a granular view of code execution and resource utilization.
– Juraci Paixão Kröhling, Principal Software Engineer at Grafana Labs and OpenTelemetry maintainer
CI/CD observability: Monitor pipelines with OpenTelemetry
A reliable CI/CD pipeline is the backbone of every modern project, yet there’s limited visibility into its processes, often requiring manual review and analysis of build outputs. That’s why in 2024, more organizations will leverage OpenTelemetry standards to bring observability to CI/CD pipelines, making the software delivery process fully observable. The shift to redefine CI/CD observability will ensure CI/CD pipelines are more reliable and stay performant over time, help to identify flakiness, and build a future where — no matter your system or your observability solution — you’ll be able to effortlessly gain full visibility over the software delivery process.
– Dimitris Sotirakis, Senior Software Engineer at Grafana Labs and Giordano Ricci, Senior Software Engineer at Grafana Labs
Learn more about the latest innovations from Grafana Labs in open source and in Grafana Cloud, our fully managed observability platform.
