Menu
Documentationbreadcrumb arrow Grafana Cloudbreadcrumb arrow Alerts and IRMbreadcrumb arrow Alertingbreadcrumb arrow Configure alert rulesbreadcrumb arrow Template annotations and labelsbreadcrumb arrow Examples
Grafana Cloud

Labels and annotations template examples

Templating allows you to add dynamic data from queries to alert labels and annotations. Dynamic data enhances alert context, making it easier for responders to quickly assess and address the issue.

This page provides common examples for templating labels and annotations. For more information on templating, refer to:

Annotation example

Annotations add extra details to alert instances and are often used to provide helpful information for identifying the issue and guiding the response.

A common use case for annotations is to display the specific query value or threshold that triggered the alert.

For example, you can display the query value from the $values variable to inform about the CPU value that triggered the alert.

CPU usage has exceeded 80% ({{ $values.A.value }}) for the last 5 minutes.

Alternatively, you can use the index() function to retrieve the query value as follows.

CPU usage has exceeded 80% ({{ index $values "A" }}) for the last 5 minutes.
template_output 
CPU usage has exceeded 80% (81.2345) for the last 5 minutes.

Include labels for extra details

To provide additional context, you can include labels from the query. For instance, access the $labels variable to display a label that informs about the affected instance:

CPU usage for {{ $labels.instance }} has exceeded 80% ({{ $values.A.Value }}) for the last 5 minutes.
template_output 
CPU usage for Instance 1 has exceeded 80% (81.2345) for the last 5 minutes.

Annotations can also be used to provide a summary of key alert labels, such as the environment and alert severity. For instance, you can display a summary of the alert with important labels like:

Alert triggered in {{ $labels.environment }} with severity {{ $labels.severity }}
template_output 
Alert triggered in production with severity critical.

To print the value of an instant query you can print its Ref ID using the index function or the $values variable:

{{ $values.A.Value }}

For range queries, reduce them from a time series to an instant vector using a reduce expression. You can then print the result by referencing its Ref ID. For example, if the reduce expression averages A with the Ref ID B, you would then print $values.B:

{{ $values.B.Value }}

Humanize the value of a query

To print the humanized value of an instant query, use the humanize function:

{{ humanize $values.A.Value }}

Alternatively:

{{ humanize (index $values "A").Value }}
template_output 
554.9

To print the value of an instant query as a percentage, use the humanizePercentage function:

{{ humanizePercentage $values.A.Value }}
template_output 
10%

For additional functions to display or format data, refer to:

Label example

Labels determine how alerts are routed and managed, ensuring that notifications reach the right teams at the right time. If the labels returned by your queries don’t fully capture the necessary context, you can create a new label and sets its value based on query data.

Based on query value

Here’s an example of creating a severity label based on a query value:

Go 
{{ if (gt $values.A.Value 90.0) -}}
critical
{{ else if (gt $values.A.Value 80.0) -}}
high
{{ else if (gt $values.A.Value 60.0) -}}
medium
{{ else -}}
low
{{- end }}

In this example, the severity label is determined by the query value:

  • critical for values above 90,
  • high for values above 80,
  • medium for values above 60,
  • and low for anything below.

You can then use the severity label to control how alerts are handled. For instance, you could send critical alerts immediately, while routing low severity alerts to a team for further investigation.

Note

You should avoid displaying query values in labels, as this may create many alert instances—one for each distinct label value. Instead, use annotations to convey query values.

Based on query label

You can use labels to differentiate alerts coming from various environments (e.g., production, staging, dev). For example, you may want to add a label that sets the environment based on the instance’s label. Here’s how you can template it:

Go 
{{ if eq $labels.instance "prod-server-1" }}production
{{ else if eq $labels.instance "staging-server-1" }}staging
{{ else }}development
{{ end }}

This would print:

  • For instance prod-server-1, the label would be production.
  • For staging-server-1, the label would be staging.
  • All other instances would be labeled development.

To make this template more flexible, you can use a regular expression that matches the instance name with the instance name prefix using the match() function:

Go 
{{ if match "^prod-server-.*" $labels.instance }}production
{{ else if match "^staging-server-.*" $labels.instance}}staging
{{ else }}development
{{ end }}