Annotation and label template reference
Annotations and labels in alert rules can be defined using plain text. However, you can also define templates to customize their values with dynamic data from alert rule queries.
For example, you can template the
summary annotation to include information from query values, providing relevant alert context for responders. Refer to Template annotations and labels for various use cases.
In templates, variables represent dynamic values from queries, while functions perform actions to transform or format this data.
Variables
Variables represent dynamic values from alert rule queries that can be displayed or accessed in your templates.
The
$ and
. symbols are used to reference variables and their properties. You can reference variables directly in your alert rule definitions using the
$ symbol followed by the variable name. Similarly, you can access properties of variables using the dot (
.) notation in alert rule templates.
{{ $values.A.Value }}
Templates are based on the Go templating system. Refer to Template language for additional information.
The following variables are available when templating annotations and labels:
Variables
Description
$labels
Contains all labels from the query, only query labels.
$values
Contains the labels and floating point values of all instant queries and expressions, indexed by their Ref IDs.
$value
A string containing the labels and values of all instant queries; threshold, reduce and math expressions, and classic conditions in the alert rule. When a single data source is used, it returns the value of the query. It is generally recommended to use $values.
$labels
The
$labels variable contains all labels from the query. It excludes user-configured and reserved labels, containing only query labels.
{{< figure src="/media/docs/alerting/query-labels-and-values.png" max-width="1200px" caption="An alert rule displaying labels and value from a query." >}}
For example, suppose you have a query that returns CPU usage for all of your servers, and you have an alert rule that fires when any of your servers have exceeded 80% CPU usage for the last 5 minutes. You want to add a summary annotation to the alert that tells you which server is experiencing high CPU usage. With the
$labels variable you can write a template that prints a human-readable sentence such as:
CPU usage for {{ $labels.instance }} has exceeded 80% for the last 5 minutes
The outcome of this template would be:
CPU usage for server1 has exceeded 80% for the last 5 minutes
If you are using a classic condition then
$labels will not contain any labels from the query. Classic conditions discard these labels to enforce uni-dimensional behavior (at most one alert per alert rule). If you want to use labels from the query in your template then use the example here.
$values
The
$values variable is a table containing the labels and floating point values of all instant queries and expressions, indexed by their Ref IDs (e.g,
A,
B,
C, etc.). It does not contain the results of range queries, as they can return hundreds or thousands of rows.
Each Ref IDs, such as
$values.A, has the following properties
Property
Type
Description
Value
Float
The value returned by the instant query or expression.
Labels
Key/value pairs
The labels associated with the instant query or expression.
To print labels from a specific query or expression, use its Ref ID and the
Labels property:
{{ $values.A.Labels.instance }}
$values is indexed by Ref ID (
A,
B,
C), not by the alert name.
For most alert messages, use
$labels instead.
$labels contains the query labels for the alert instance, so you don't need the Ref ID:
{{ $labels.instance }}
Here's the previous example printing now the value of the instant query with Ref ID
A:
{{ $values.A.Value }} CPU usage for {{ $labels.instance }} over the last 5 minutes.
If the alert has the label
instance=server1 and the query returns
81.2345, the template would print:
81.2345 CPU usage for instance1 over the last 5 minutes.
If the query in Ref ID
A is a range query rather than an instant query then add a reduce expression with Ref ID
B and replace
$values.A.Value with
$values.B.Value:
{{ $values.B.Value }} CPU usage for {{ $labels.instance }} over the last 5 minutes.
Alternatively, you can use the
index() function to retrieve the query value:
{{ index $values "B" }} CPU usage for {{ index $labels "instance" }} over the last 5 minutes.
{{< admonition type="note" >}}
Variable names that start with a number (for example,
1B) are not valid identifiers in Go templates.
To access a value or label whose key starts with a number, use the
index function:
{{ index $values "1B" }} CPU usage for {{ index $labels "1instance" }} over the last 5 minutes.
Using
{{ $values.1B.Value }} is invalid and causes the template code to render as plain text.
{{< /admonition >}}
$value
The
$value variable is a string containing the labels and values of all instant queries; threshold, reduce and math expressions, and classic conditions in the alert rule.
When a single data source is used in the alert rule,
$value will return the query value directly.
This example prints the
$value variable:
{{ $value }}: CPU usage has exceeded 80% for the last 5 minutes.
When using multiple data sources, it would display something like this:
[ var='A' labels={instance=instance1} value=81.234, , [ var='B' labels={instance=instance2} value=1 ] ]: CPU usage has exceeded 80% for the last 5 minutes.
But with a single data source, it would display just the value of the query:
81.234: CPU usage has exceeded 80% for the last 5 minutes.
Instead, we recommend using $values, which contains the same information as
$value but is structured in an easier-to-use table format.
Functions
Functions can perform actions in templates such as transforming or formatting data.
Note that the functions provided by Go's template language, such as
index,
and,
printf, and
len, are available, along with many others.
In addition, the following functions are also available for templating annotations and labels:
Numbers
Name
Arguments
Returns
Description
humanize
number or string
string
Humanizes decimal numbers.
humanize1024
number or string
string
Like
humanize, but uses 1024 as the base rather than 1000.
humanizeDuration
number or string
string
Humanizes a duration in seconds.
humanizePercentage
number or string
string
Humanizes a ratio value to a percentage.
humanizeTimestamp
number or string
string
Humanizes a Unix timestamp.
toTime
number or string
time
Converts a Unix timestamp in seconds to time.
Strings
Name
Arguments
Returns
Description
title
string
string
Capitalizes the first character of each word.
toUpper
string
string
Returns all text in uppercase.
toLower
string
string
Returns all text in lowercase.
stripPort
string
string
Returns only host.
match
pattern, text
boolean
Matches the text against a regular expression pattern.
reReplaceAll
pattern, replacement, text
string
Replaces text matching the regular expression.
graphLink
expr
string
Returns the path to the graphical view in
Explore for the given expression and data source.
tableLink
expr
string
Returns the path to the tabular view in
Explore for the given expression and data source.
parseDuration
string
float
Parses a duration string such as "1h" into the number of seconds it represents.
stripDomain
string
string
Returns the result of removing the domain part of a FQDN.
Others
Name
Arguments
Returns
Description
args
[]interface{}
map[string]interface{}
Translates a list of objects to a map with keys arg0, arg1 etc.
safeHtml
string
string
Marks string as HTML not requiring auto-escaping.
externalURL
none
string
Returns the external URL of the Grafana server as configured in the ini file(s).
pathPrefix
none
string
Returns the path of the Grafana server as configured in the ini file(s).
For further context on these functions, note that templating in Grafana is based on the Prometheus template implementation, enabling the use of these functions and Prometheus-like templates for formatting alert messages within Grafana.
humanize
The
humanize function humanizes decimal numbers:
{{ humanize 1000.0 }}
1k
humanize1024
The
humanize1024 function works similarly to
humanize but uses 1024 as the base rather than 1000:
{{ humanize1024 1024.0 }}
1ki
humanizeDuration
The
humanizeDuration function humanizes a duration in seconds:
{{ humanizeDuration 60.0 }}
1m 0s
humanizePercentage
The
humanizePercentage function humanizes a ratio value between 0 and 1 to a percentage:
{{ humanizePercentage 0.2 }}
20%
humanizeTimestamp
The
humanizeTimestamp function humanizes a Unix timestamp:
{{ humanizeTimestamp 1577836800.0 }}
2020-01-01 00:00:00 +0000 UTC
toTime
The
toTime function converts a Unix timestamp in seconds to time.:
{{ toTime 1727802106 }}
2024-10-01 17:01:46 +0000 UTC
title
The
title function capitalizes the first character of each word:
{{ title "hello, world!" }}
Hello, World!
toUpper
The
toUpper function returns all text in uppercase:
{{ toUpper "Hello, world!" }}
HELLO, WORLD!
toLower
The
toLower function returns all text in lowercase:
{{ toLower "Hello, world!" }}
hello, world!
stripPort
The
stripPort splits string into host and port, then returns only host:
{{ stripPort "example.com:8080" }}
example.com
match
The
match function matches the text against a regular expression pattern:
{{ match "a.*" "abc" }}
true
reReplaceAll
The
reReplaceAll function replaces text matching the regular expression:
{{ reReplaceAll "localhost:(.*)" "example.com:$1" "localhost:8080" }}
example.com:8080
graphLink
The
graphLink function returns the path to the graphical view in Explore for the given expression and data source:
{{ graphLink "{\"expr\": \"up\", \"datasource\": \"gdev-prometheus\"}" }}
/explore?left=["now-1h","now","gdev-prometheus",{"datasource":"gdev-prometheus","expr":"up","instant":false,"range":true}]
parseDuration
The
parseDuration function parses a duration string such as "1h" into the number of seconds it represents.
{{ parseDuration "1h" }}
3600
stripDomain
The
stripDomain removes the domain part of a FQDN, leaving port untouched:
{{ stripDomain "example.com:8080" }}
example:8080
tableLink
The
tableLink function returns the path to the tabular view in Explore for the given expression and data source:
{{ tableLink "{\"expr\": \"up\", \"datasource\": \"gdev-prometheus\"}" }}
/explore?left=["now-1h","now","gdev-prometheus",{"datasource":"gdev-prometheus","expr":"up","instant":true,"range":false}]
args
The
args function translates a list of objects to a map with keys arg0, arg1 etc. This is intended to allow multiple arguments to be passed to templates:
{{define "x"}}{{.arg0}} {{.arg1}}{{end}}{{template "x" (args 1 "2")}}
1 2
safeHtml
The
safeHtml function marks string as HTML not requiring auto-escaping:
{{ safeHtml "Text"}}
Text
externalURL
The
externalURL function returns the external URL of the Grafana server as configured in the ini file(s):
{{ externalURL }}
https://example.com/grafana
pathPrefix
The
pathPrefix function returns the path of the Grafana server as configured in the ini file(s):
{{ pathPrefix }}
/grafana
Differences with notification templates
Both notification templates and alert rule templates use the Go templating system. However, the functions and variables available in notification templates differ from those used in annotations and labels templates, which are described in this documentation.
Annotation and label templates operate in the context of an individual alert instance, while notification templates apply to a notification that includes a group of alert(s).
For example, notification templates provide the
.Alerts variable, which includes the list of all firing and resolved alerts in the notification. This variable is not available in alert rule templates, which operate within the context of a single alert instance.
Additionally, you cannot reuse templates for labels and annotations as you can with notification templates. Instead, you need to write each template inline within the label or annotation fields and manually copy them wherever you want to reuse the templates.