Skip to main content
Monitors supports using standard SQL syntax to query MySQL and trigger alerts based on query results.

Core concepts

Related queries and recovery queries support the ${label_name} variable and the Go template {{ $labels.<label_name> }}. The main query doesn’t support variables; a main query containing ${...} is rejected at save time with variables are only supported in relate and recovery queries.
Variables require alert engine v0.76.0 or later; if any instance in the cluster serving the rule’s datasources is older, saving or enabling the rule is rejected with an error that starts with MySQL relate or recovery queries with variables require and names the cluster and the required version. Rules that use no variables are not subject to this version requirement.
The SQL after variable substitution is only known at runtime, so it can’t be pre-checked at save time. The alert engine renders the variables per alert instance and then checks that the rendered text is a read-only query; if it isn’t a read-only query, that evaluation run fails.

Query restrictions

A MySQL rule’s SQL must be a single read-only statement. The main query, relate queries, and recovery queries are checked by the same rules, and a rule that breaks them is rejected at save time with an error that starts with the field name, for example rule config invalid: query(A) expr: ….
  • The statement must start with SELECT, WITH, SHOW, DESC, DESCRIBE, or EXPLAIN; anything else is refused with MySQL statement must start with SELECT, WITH, SHOW, DESC, DESCRIBE, or EXPLAIN.
  • INTO, INSERT, UPDATE, and DELETE may not appear anywhere in the statement, which is what stops SELECT … INTO OUTFILE, WITH … DELETE, and INSERT … SELECT. The message is MySQL keyword INTO is not allowed, with the keyword upper-cased, and the INSERT(...) string function is not affected.
  • Only one statement is allowed; more than one returns multiple MySQL statements are not allowed.
  • Executable comments are refused with executable SQL comments are not allowed. This covers MySQL /*! … */, MariaDB /*M! … */, and TiDB /*T! … */; ordinary /* … */ comments are fine.
  • EXPLAIN ANALYZE is refused with MySQL EXPLAIN ANALYZE is not allowed; plain EXPLAIN and DESC work.
This is a lexical read-only policy that stops obvious writes; a read-only datasource account remains the real boundary.

Threshold evaluation mode

This mode is suitable for scenarios requiring threshold comparison on aggregated values.

Configuration

  1. Query Statement: Write SQL aggregate query, returning value columns and (optional) label columns.
    • Example: Count error log quantity by service in the last 5 minutes (assuming there’s a log table).
  2. Field Mapping:
    • Value fields: Select error_cnt for threshold evaluation.
    • Label fields: Select service_name to identify the alert object. After you select label fields, other non-value fields are carried with the alert as additional information.
    • See Query result field mapping for the complete behavior.
  3. Threshold Conditions:
    • Use $A.field_name to reference values.
    • Example: Critical: $A.error_cnt > 50, Warning: $A.error_cnt > 10.

How it works

Monitors distinguishes alert objects by their label fields and evaluates thresholds with their value fields. If Label fields is empty, every returned field except the value fields becomes a label.

Recovery logic

See Alert recovery for the recovery options. A recovery query is a separate SQL statement that supports the ${label_name} variable and the Go template {{ $labels.<label_name> }}. These rules are subject to the same version requirement as above: if any instance in the cluster is older than v0.76.0, saving or enabling the rule is rejected.

Data exists mode

This mode is suitable for scenarios where filter logic is written directly in SQL.

Configuration

  1. Query Statement: Use HAVING clause in SQL to directly filter out anomalous data.
    • Example: Directly query services with error count exceeding 50.
  2. Evaluation Rules: As long as SQL query returns data (Result Set is not empty), triggers alert.
Data exists mode cannot distinguish alert levels; use threshold evaluation when you need levels.

Recovery logic

See Alert recovery for the recovery options.

No data mode

This mode is used to monitor scenarios where “data is expected but actually missing”.

Configuration

  1. Query Statement: Write a SQL query that is expected to continuously return data.
    • Example: Query heartbeat reports from all probes.
  2. Evaluation Rules: If a probe_id appeared in previous cycles but cannot be found in current and N consecutive cycles, triggers “No Data” alert.

Recovery logic

See Alert recovery for the recovery options.

Best practices

Always include time range filtering in WHERE clause and ensure the time field has an index, otherwise it may cause full table scan.Recommended syntax: log_time > now() - INTERVAL 5 MINUTE
Monitors converts column names returned by MySQL to lowercase. Use lowercase names for both label fields and value fields.