Grafana Alerting video: How to create alerts in Grafana 9
With the Grafana 9.0 release, we rolled out the new and improved Grafana Alerting experience, which is now the default alerting system across all of our products.
Along with introducing significant improvements to Grafana Alerting based on community feedback and more robust alerting documentation to guide our users, we also created easy-to-follow video tutorials to help you get started with creating alerts.
The 3 alert rules in Grafana Alerting
First, let’s go over the different types of alert rules you can create in Grafana 9.0:
- Grafana-managed alert
A Grafana-managed alert is a rule that is stored and executed by Grafana itself. It can span multiple different data sources like SQL, Prometheus, InfluxDB, etc. and can handle complex conditions.
- Mimir- or Loki-managed alert
Mimir or Loki alerts consist only of one query that is written in PromQL or LogQL. Those rules are stored and executed on the Mimir or Loki ruler and are completely decoupled from the Grafana runtime itself.
- Mimir or Loki recording rule
Recording rules can be used to precompute the result of expensive queries. The result will then be much faster than executing the query each time.
In the following video tutorials, we will be focusing on setting up Grafana-managed alerts and Mimir-managed alerts.
How to create Grafana-managed alerts
In this video tutorial, we are creating an alert on a simple query: We want to know if our HTTP server is returning 500 internal server errors. We have a Prometheus server that is collecting metrics from our HTTP server, and we want to create an alert when the status code is 500.
After adding a rate function to see the rate at which our server is returning 500 internal server errors, we need to reduce the data to a single value to make this query alertable. In the Operation dropdown menu, we can choose either a classic condition or we can use a reduce and math expression.
- Classic conditions provide simple aggregation functions, such as average min, max, and sum. They let us create alerts when a chosen value exceeds the threshold.
- Reduce and math expressions let us create more complex alerting conditions. These are particularly useful when we want to create an alert rule that uses multiple queries to determine whether the alert should fire. (Note: When using a reduce and math expression, if the condition is greater than or equal to 1, we can simply create a reduce expression without the math expression.)
When creating a Grafana-managed alert, it’s possible to set both the evaluation interval and the duration. We set up this alert example to evaluate every minute, and for the alert to fire if we are seeing 500 internal server errors for at least 5 minutes.
The demo then walks through how to name and organize alerts into folders followed by assigning the rule to an alert group — a feature in Grafana Alerting that is the equivalent to those in Prometheus. You can also add annotations and more context to your alerting rule, such as a summary or runbooks.
For more information, check out our Grafana-managed alerts documentation.
Create a Mimir-managed alert rule in less than 90 seconds
A Grafana Mimir-managed alert rule may be useful if latency is high or expensive queries are running. In this video demo, Grafana acts as a proxy and becomes your single pane of glass for all your alerts.
To create a new Mimir alert rule, click on New alert rule and select Mimir or Loki alert. Then users can simply enter the rule name and select the Mimir data source. The demo will also showcase how easy it is to use namespaces and groups to logically group your alert rules as you create them.
After learning how to create an alert query and define alert conditions, the demo will take you through how to add annotations and more context to your alerting rule, such as a summary or runbooks. Users can also add custom labels that will filter or route the alert to specific teams.
Finally, after you click Save and Exit, the rule will be processed by the Mimir alert ruler and saved.
For more information, check out our Grafana Mimir-managed alerts documentation.
Learn more about Grafana Alerting
The new and improved features in the Grafana Alerting system are now enabled by default for all users in Grafana 9. While Grafana users currently have the option to roll back to the former alerting experience based on dashboard panels (now referred to as “legacy alerting”), we will officially remove that functionality in Grafana 10.
To learn more about Grafana Alerting, please read our Grafana Alerting documentation or visit the Grafana Alerting page.
The Grafana Alerting team also provided a deep dive into the latest features in our GrafanaCONline 2022 session “Alerting in Grafana 9: What’s new and improved,” which is available on demand.
Grafana Cloud is the easiest way to get started with metrics, logs, traces, and dashboards. We have a generous free forever tier and plans for every use case. Sign up for free now!
