This is documentation for the next version of Grafana. For the latest stable release, go to the latest version.
Labels and annotations template examples
Templating allows you to add dynamic data from queries to alert labels and annotations. Dynamic data enhances alert context, making it easier for responders to quickly assess and address the issue.
This page provides common examples for templating labels and annotations. For more information on templating, refer to:
Annotation example
Annotations add extra details to alert instances and are often used to provide helpful information for identifying the issue and guiding the response.
A common use case for annotations is to display the specific query value or threshold that triggered the alert.
For example, you can display the query value from the $values
variable to inform about the CPU value that triggered the alert.
CPU usage has exceeded 80% ({{ $values.A.value }}) for the last 5 minutes.
Alternatively, you can use the index()
function to retrieve the query value as follows.
CPU usage has exceeded 80% ({{ index $values "A" }}) for the last 5 minutes.
CPU usage has exceeded 80% (81.2345) for the last 5 minutes.
Include labels for extra details
To provide additional context, you can include labels from the query. For instance, access the $labels
variable to display a label that informs about the affected instance:
CPU usage for {{ $labels.instance }} has exceeded 80% ({{ $values.A.Value }}) for the last 5 minutes.
CPU usage for Instance 1 has exceeded 80% (81.2345) for the last 5 minutes.
Annotations can also be used to provide a summary of key alert labels, such as the environment and alert severity. For instance, you can display a summary of the alert with important labels like:
Alert triggered in {{ $labels.environment }} with severity {{ $labels.severity }}
Alert triggered in production with severity critical.
Print a range query
To print the value of an instant query you can print its Ref ID using the index
function or the $values
variable:
{{ $values.A.Value }}
For range queries, reduce them from a time series to an instant vector using a reduce expression. You can then print the result by referencing its Ref ID. For example, if the reduce expression averages A
with the Ref ID B
, you would then print $values.B
:
{{ $values.B.Value }}
Humanize the value of a query
To print the humanized value of an instant query, use the humanize
function:
{{ humanize $values.A.Value }}
Alternatively:
{{ humanize (index $values "A").Value }}
554.9
To print the value of an instant query as a percentage, use the humanizePercentage
function:
{{ humanizePercentage $values.A.Value }}
10%
For additional functions to display or format data, refer to:
Label example
Labels determine how alerts are routed and managed, ensuring that notifications reach the right teams at the right time. If the labels returned by your queries don’t fully capture the necessary context, you can create a new label and sets its value based on query data.
Based on query value
Here’s an example of creating a severity
label based on a query value:
{{ if (gt $values.A.Value 90.0) -}}
critical
{{ else if (gt $values.A.Value 80.0) -}}
high
{{ else if (gt $values.A.Value 60.0) -}}
medium
{{ else -}}
low
{{- end }}
In this example, the severity
label is determined by the query value:
critical
for values above 90,high
for values above 80,medium
for values above 60,- and
low
for anything below.
You can then use the severity
label to control how alerts are handled. For instance, you could send critical
alerts immediately, while routing low
severity alerts to a team for further investigation.
Note
You should avoid displaying query values in labels, as this may create many alert instances—one for each distinct label value. Instead, use annotations to convey query values.
Based on query label
You can use labels to differentiate alerts coming from various environments (e.g., production, staging, dev). For example, you may want to add a label that sets the environment based on the instance’s label. Here’s how you can template it:
{{ if eq $labels.instance "prod-server-1" }}production
{{ else if eq $labels.instance "staging-server-1" }}staging
{{ else }}development
{{ end }}
This would print:
- For instance
prod-server-1
, the label would beproduction
. - For
staging-server-1
, the label would bestaging
. - All other instances would be labeled
development
.
To make this template more flexible, you can use a regular expression that matches the instance name with the instance name prefix using the match()
function:
{{ if match "^prod-server-.*" $labels.instance }}production
{{ else if match "^staging-server-.*" $labels.instance}}staging
{{ else }}development
{{ end }}
Legacy Alerting templates
For users working with Grafana’s legacy alerting system, templates can still be utilized to extract useful information from alert conditions. However, it’s important to note that you cannot use $labels
to print labels from the query if you are using classic conditions, and must use $values
instead. The reason for this is classic conditions discard these labels to enforce uni-dimensional behavior (at most one alert per alert rule). If classic conditions didn’t discard these labels, then queries that returned many time series would cause alerts to flap between firing and resolved constantly as the labels would change every time the alert rule was evaluated.
Instead, the $values
variable contains the reduced values of all time series for all conditions that are firing. For example, if you have an alert rule with a query A that returns two time series, and a classic condition B with two conditions, then $values
would contain B0
, B1
, B2
and B3
. If the classic condition B had just one condition, then $values
would contain just B0
and B1
.
Print all labels from a classic condition
To print all labels of all firing time series use the following template (make sure to replace B
in the regular expression with the Ref ID of the classic condition if it’s different):
{{ range $k, $v := $values -}}
{{ if (match "B[0-9]+" $k) -}}
{{ $k }}: {{ $v.Labels }}{{ end }}
{{ end }}
For example, a classic condition for two time series exceeding a single condition would print:
B0: instance=server1
B1: instance=server2
If the classic condition has two or more conditions, and a time series exceeds multiple conditions at the same time, then its labels will be duplicated for each condition that is exceeded:
B0: instance=server1
B1: instance=server2
B2: instance=server1
B3: instance=server2
If you need to print unique labels you should consider changing your alert rules from uni-dimensional to multi-dimensional instead. You can do this by replacing your classic condition with reduce and math expressions.
Print all values from a classic condition
To print all values from a classic condition take the previous example and replace $v.Labels
with $v.Value
:
{{ range $k, $v := $values -}}
{{ if (match "B[0-9]+" $k) -}}
{{ $k }}: {{ $v.Value }}{{ end }}
{{ end }}
For example, a classic condition for two time series exceeding a single condition would print:
B0: 81.2345
B1: 84.5678
If the classic condition has two or more conditions, and a time series exceeds multiple conditions at the same time, then $values
will contain the values of all conditions:
B0: 81.2345
B1: 92.3456
B2: 84.5678
B3: 95.6789