Grafana Cloud Metrics: A guide to what metrics to monitor and best practices
Metrics are the cornerstone of an observable system – they tell you a system’s measured outputs, granting visibility into what your customers are experiencing and when there’s a problem. However, not all methods for recording and saving metrics from a system’s output are alike. The best method for shipping your system’s metrics to Grafana Cloud depends on many factors, varying from the source of your metrics data to your familiarity with observability tools.
Grafana Cloud customers typically consider one or more of the following types of questions before determining their preferred method for shipping metrics:
- What is my preferred time series database?
- How familiar am I with using programs for collecting and aggregating metrics?
- What is the quantity and scale of metrics I’d like to monitor?
- What is my application’s runtime environment?
This guide will help you better understand the possibilities and the trade-offs. By the end, you should be able to identify what’s the best way to ship metrics from your system to Grafana Cloud.
Why ship metrics to Grafana Cloud?
Creating, deploying, and managing your applications and core infrastructure are complicated enough without having to worry about hosting a redundant metrics or time series database to allow resilient monitoring and data observability.
Grafana Cloud is a managed service, letting you plug in and start playing around almost instantly with open source tools such as Grafana, Grafana Mimir, and Grafana Loki – skipping the time-consuming installation and maintenance required when deploying services on your own.
Once you’re up and running with Grafana Cloud, you’ll be able to enjoy the peace of mind knowing that we’ll automatically scale with your usage. That’s not to mention the security fixes and release upgrades that we handle on behalf of your organization on a regular basis.
What metrics should you ship to Grafana Cloud?
The metrics you choose to monitor depend on your goals and the outputs of your system. With an endless list of possibilities, it may be helpful to focus on metrics that tell you insights into the “The Four Golden Signals” of monitoring:
- Latency: The time it takes to service a request.
- Traffic: How much demand is being placed on your system.
- Errors: The rate of failures in your system.
- Saturation: How “full” is your system — and critically, how much capacity remains.
Once you’ve narrowed down the metrics you want to ship, we recommend reviewing the typical measurement structure of a metric. That way you’ll get a sense of just how granular (or not) you can go with your metrics and a feel for how you’ll query your metrics using Grafana.
For our examples in this guide, we’ll go with Graphite, which is one of the easiest and most flexible methods for pushing metrics to Grafana Cloud.
Each metric is composed of the following properties:
name // Graphite style name (required)
interval // the resolution of the metric in seconds (required)
value // float64 value (required)
time // unix timestamp in seconds (required)
tags // list of key=value pairs of tags (optional)
Metric name syntax
<namespace>.<instrumented section>.<target (noun)>.<action (past tense verb)>
E.g. accounts.authentication.password.succeeded
Credit: Practical Guide to Graphite Monitoring
Metric tags (optional)
["key1=value1", "key2=value2"]
E.g. ["location=usa", "environment=production"]
Once you select relevant metric names and tags, you’ll want to add:
- The time - a Unix timestamp in seconds
- The interval - the time between metric data points
- The value - a float that tells us how much or how little was measured
Bringing it together, we get the following example Graphite metric:
{
"name": "accounts.authentication.password.succeeded ",
"interval": 10,
"value": 1.0,
"tags": \["location=usa", "environment=production"],
"time": 1660850859
}Now it’s time for you to start shipping metrics that are relevant to you and your systems. Read on for additional pointers on how to ship metrics to Grafana Cloud.
How can you ship metrics to Grafana Cloud?
The most commonly used methods for shipping metrics to Grafana Cloud are outlined below:
Most flexible: HTTP Write
Recommended for people who want to ship metrics regarding their systems over HTTP. This is a great option for application developers, Raspberry Pi hackers, Linux administrators, and more given the universal nature of the HTTP interface.
Graphite examples
Prometheus examples (via InfluxDB Line Protocol)
Easiest: Agent
Dozens of integrations for popular server tools like Linux, Kubernetes, Nginx, and Docker. Simply copy-paste a command on your server to get started and access prebuilt dashboards to quickly visualize your metrics within minutes.
Recommended for people wanting to know performance metrics of their infrastructure.
Best for Prometheus power users: Remote write
Most difficult. Suitable for use cases with a high volume of metrics (> 100k active series) sent over a dedicated socket.
What’s next? Once your metric data is flowing into Grafana Cloud, the next step is to begin building visualizations so that you can monitor those metrics over time. We recommend visiting the Explore tool in your Grafana instance.
Grafana Explore
The Explore tool lets you quickly and easily build queries that return your metrics in the form of graphs and tables, which you can then add to dashboards that show the trends in your data. It’s a perfect place to start digging into your data and figure out which parts are best suited for dashboards, alerting, and more.
Using Explore
From within Explore, below are the components you’ll need to adjust based on your needs.
- Select a data source
You’ll see a dropdown in the top-left corner to select your data source. If you’re following the HTTP Write examples above, you’ll want to select either the grafanacloud--graphite data source or the grafancloud- -prom data source. - Write a query
Explore offers a prebuilt query builder below the data source selector. Click the dropdowns beside Series to quickly select a metric series for your query. Or, if you’re already experienced with the query language for your data source, Explore lets you write queries from scratch, too. - Visualize
Click Run query in the top-right corner of Explore to execute a query. Explore will show your metrics in a graph and table so that you can quickly view your metric data. And finally, click to Add to dashboard once you’re ready to monitor a query over time.
Learn more about Grafana Explore.
Learn more about Grafana Cloud Metrics
If you have any questions about shipping metrics to Grafana Cloud, please don’t hesitate to reach out to our team!
Below are some additional resources on best practices, configurations, and use cases.
Grafana Cloud resources
Graphite resources
Grafana Cloud documentation
Blog posts
- How to monitor an Umbrel server running a Bitcoin node with Grafana Cloud
- Monitoring a hard water softener at home with Grafana Cloud
Prometheus resources
Grafana Cloud documentation
Blog posts
- The future of Prometheus remote write
- New in Grafana Mimir: Ingest Graphite, Datadog, Influx, and Prometheus metrics into a single storage backend
- How to monitor a Tesla with Grafana Cloud
If you’re not already using Grafana Cloud — the easiest way to get started with observability — sign up now for a free 14-day trial of Grafana Cloud Pro, with unlimited metrics, logs, traces, and users, long-term retention, and access to one Enterprise plugin.
