Skip to main content
Use an alert rule Description to define the text shown on firing and recovery events. You can reference alert labels, query values, related query results, and safe Sprig functions with Go text/template, then render the result as Text or Markdown.
The template variables and functions on this page require alert engine v0.42.0 or later. Reading query-provided additional information from $annotations with $<query name>.<field name> keys requires v0.53.0 or later.

How it works

When a rule fires or recovers, the alert engine renders the template after label enrichment completes and sends the result as the event description. You can use the common variables below, or access the full event with .FieldName.
For day-to-day templates, prefer $labels, $values, $value, $relates, $status, and $checkMode. Use the other short variables only for advanced templates or compatibility cases.

Built-in variables

Example:

Additional information from query results

When a primary query explicitly selects label fields, its other non-value fields become additional information in $annotations. Query-provided information uses $<query name>.<field name> as its key; see Query result field mapping:
The syntax is the same for one or multiple queries and for Threshold evaluation, Data exists, and No data modes. The $ prefix is reserved for query fields, so a custom field configured on the rule cannot begin with $. See Query result field mapping for field classification.

Custom fields

Custom fields configured on the rule are carried into the alert event’s $annotations. The field value itself also supports variables and template rendering:
  • Simple variables: ${label_name} forms (for example, ${service}) are replaced with the corresponding label value of the alert event at render time. For example, a value of https://a.com/runbook/${service} has ${service} replaced with the alert event’s service label value.
  • Go template: When the value contains {{, Monitors renders the whole value with the same Go template rules as the Description. You can use the built-in variables and functions on this page.
  • title field: A field named title uses its rendered value as the event title.
A field name must begin with an English letter and may contain only English letters, digits, and underscores (for example, runbook_url). The console validates this rule, so a name that does not match it cannot be saved. Names beginning with $ are reserved for fields produced by queries and cannot be used as custom field names. Examples:

Root object fields

Besides the common variables above, you can access these event fields with .FieldName: Related query results are available through $relates. The map key is the related query name, such as R1. Example:

Show values at recovery time in recovery notifications

When an alert recovers, the description and custom fields (including title) are rendered again, and $status is recovered. Which check the numbers in $values come from depends on the check mode and the Recovery mode.

Values for each recovery mode

In the two modes that use a recovery query, main query values are the values at recovery time if the main query still returns the alerting object. Otherwise, they’re the values from the most recent firing or repeat notification. The recovery query only decides whether the alert recovers, and its results aren’t available to the template. To show other values in recovery notifications, use a related query.
When there are multiple values, $value is 0. When a rule has multiple queries or a query returns multiple value fields, read values by key with getvalue $values "<key>".

Threshold evaluation: reference query values directly

With the first two recovery modes, branch on $status in the template and reference the same key. In this example, main query A returns CPU usage:
With Recover if no data, the recovering check finds no data, so $values only has the values from the most recent notification. To show values at recovery time, use a related query: related queries run every time a notification is sent, including recovery notifications, regardless of the check mode and recovery mode. Query the values you want to show, then reference them in the template through $relates. Related query results from SQL data sources are returned as text, so read them with $row.Field "<column name>". Related query example: the main query detects error logs by service, and related query R1 counts errors in the last 5 minutes.

Go template built-in functions

These functions are provided by Go text/template.

Alert engine custom functions

These functions are registered by the alert engine without a prefix.

Sprig functions

The alert engine registers safe Sprig functions with standard Sprig names and also with a sprig_ prefix; if a standard name conflicts with a custom function, the custom function keeps the standard name and the Sprig function stays available through the sprig_ prefix. Sprig functions come from Masterminds/sprig. Check that repository for upstream function behavior. Common examples:
Important regex note: regexFind and sprig_regexFind return the full matched substring. They do not return a capture group. To extract a capture group, use regexReplaceAll or sprig_regexReplaceAll:
The result is:

Common Sprig functions

Disabled Sprig functions

For predictable and safe rendering, Sprig functions that read environment variables, perform DNS lookup, generate random values, create certificates or keys, encrypt or decrypt, or deliberately fail the template are unavailable (for example env, expandenv, getHostByName, bcrypt, genPrivateKey, encryptAES, uuidv4, randAlphaNum, fail), in both the standard and the sprig_ prefixed form. A template that uses one fails with an error like function "env" not defined. Remove the function or replace it with deterministic template logic.

Complete example: Extract MySQL Unknown Column

Troubleshooting

function "contains" not defined means the alert engine is too old to have enabled standard Sprig names. Upgrade the alert engine, or use sprig_contains as a workaround. function "env" not defined means the template used a disabled Sprig function; see Disabled Sprig functions. regexFind or sprig_regexFind returning Unknown column 'x' instead of x is expected. Use regexReplaceAll or sprig_regexReplaceAll to extract capture groups.