Investigation entries

​

How it works

• An entry is a weak navigation reference. Saving the configuration and rendering the snapshot never run a query and never read the dashboard. Entries do not participate in alert evaluation, alert identity, instance identity or lifecycle, and they trigger no automatic investigation task.
• An entry follows the event’s data source. A rule can match several data sources by type plus name pattern, but each alert event comes from exactly one of them. A query entry runs on the event’s own data source by default, so the jump lands on the data source that fired the alert rather than on a fixed or default one. You can also name a different data source explicitly (for example to drill into a log data source).
• Rendering happens when the alert is ingested. When monit-webapi accepts an event, it renders the entries configured on the rule at that moment — using the final event time, the event’s raw data labels and the event data source — into a snapshot (the Investigation Context), which is stored and delivered with the event. The investigation block in the active alert detail drawer, the Investigation block in an incident’s comprehensive info panel (see Incident details) and AI SRE all read that same snapshot.
• The block has a different name on each side: it is called Investigation entries in Monitors and Investigation in incident details, but both render the same event snapshot.
• Order affects display only. Entries can be dragged into any order, but the first one carries no default, primary or recommended meaning, and the order never reaches alert evaluation or lifecycle.

​

Entry types

​

Adding and editing entries

1. Click Add dashboard entry or Add query entry to open the entry dialog. The dialog title distinguishes new from existing entries (Add dashboard link / Edit dashboard link / Add query link / Edit query link).
2. Fill in Purpose and Time range (shared by both types), then the fields for that type.
3. Click OK to add the entry to the form. OK only adds it to the form; the entry takes effect when you save the rule.
4. Each row offers three actions: Preview investigation entry (eye icon), Edit (pencil icon) and Delete (trash icon). When you view a saved rule’s details, the edit action becomes Details and the dialog footer offers only Close.

​

Shared fields

• The anchor is the alert event time: start = event time − before, end = event time + after.
• The window is written into the snapshot exactly as configured, and it never drifts with the current time when you open it later. You can adjust the time range after opening the dashboard or the workbench, but adjustments made in the page are not written back to the rule or to snapshots already produced.
• An end time in the future is allowed (that is what a non-zero “after” produces): the entry is available as soon as the alert fires, without waiting for later data.
• The system does not infer, warn about or validate whether the entry’s window covers the alert’s evaluation window.

​

Dashboard entry fields

• A dashboard with no variables shows This dashboard has no variables.
• Variable values are rendered as Go templates; see Template variables. Datasource variables are marked (datasource) and their placeholder is {{ $datasource.id }}; other variables are prompted with Leave empty to use the dashboard default, e.g. {{ $labels.<variable name> }}.
• Empty means not configured, and the dashboard applies its own rule for the default. Configured but failing to render, or rendering to an empty string, leaves the variable unresolved (it does not fall back to the default) and shows an error on the entry.
• A variable that is configured but no longer exists in the dashboard (or was renamed) is still listed and marked No longer in dashboard, so you can clean it up — otherwise it would affect the dashboard’s variable resolution invisibly.
• Deleting, renaming or changing the type of a variable, or invalidating its candidates, never updates the rule.

​

Query entry fields

​

Template variables

• Read a simple label directly: {{ $labels.service }}. For label names containing dots or other special characters, use {{ index $labels "service.name" }}.
• Accessing a missing label or a key that $datasource does not have fails explicitly (including through index) instead of silently producing an empty string, which would turn into an unfiltered query.
• The dot (.) only means a local data context such as range / with; it is not the full alert event. Variables available in the Description template such as $values, $value, $annotations and $appendLabels are not available here, and the root object cannot be used to bypass that restriction.
• Conditionals, loops, local variables and built-in functions of Go templates are supported. The template is rendered once: a {{ }} produced by the rendering is not executed again.

{{ $labels.service_name }}
{{ $datasource.id }}

event_name:="webhook.attempt.completed"
service:="{{ $labels.service_name }}"
outcome:="failure"

​

Previewing an entry

• Prerequisites: the rule must be saved (otherwise it says Save the rule first, then preview once it has an alert.) and the rule must currently have an active alert (otherwise This rule has no active alerts. Preview will be available once an alert fires.). The dialog picks a sample from the rule’s active alerts, with paging above 20.
• The preview uses the selected alert’s event snapshot and does not include unsaved changes in the form, so save the rule and wait for it to fire before previewing a new configuration.
• Clicking an entry in the dialog opens the target in a new tab. If that alert has no snapshot matching this entry, it says This alert has no matching snapshot for this entry. Select an alert generated after the configuration took effect.; if the alert has no snapshot at all, it says This alert has no investigation links. Select an alert generated after the configuration took effect.

​

Validation and limits

• Dashboard entries: the same dashboard plus the same target object is a duplicate, regardless of variable values and time range. Duplicates cannot be used to express a variable fallback or multiple instances.
• Query entries: de-duplicated by the normalized data source, query (mode, expression, arguments, minimum step) and time range; the event data source (no data source named) counts as its own distinct value. Several different queries may be configured on the same data source.

​

Edge version requirement at save time

• When it applies: the saved rule carries non-empty entries and the request either submitted non-empty entries explicitly, or kept existing entries while changing the data source scope (data source type, name patterns or exact bindings). A rule with no entries, an explicit empty array, and requests that only change other fields are not checked.
• What it checks: every data source the rule currently matches, and the active instances of the Edge clusters those data sources belong to. Data sources without any active instance are skipped; a rule that matches no data source at all is allowed to save.
• To pass: every checked instance must be at version v0.75.3 or later. An unrecognizable version counts as failing.
• On rejection: no field is saved, and the request returns an upgrade error listing the minimum version and the failing data sources, Edge clusters, instance IPs and versions (bounded in number). The message looks like: investigation_targets require monit-edge v0.75.3 or later on every active instance serving the rule's datasources; upgrade cluster "<cluster>" <instance IP>:<port> (v0.75.2).
• The gate covers every write path: create, update, batch field updates that can change entries or the data source scope, and native-bundle and legacy JSON imports. Rules imported from external formats (Prometheus, Nightingale and so on) always have an empty entry list and are unaffected.
• After a successful save, a newly added or downgraded low-version instance does not invalidate the saved rule; the check runs again only when you next submit entries or change the data source scope.

​

Snapshot semantics and failures

• If you change the entries on the rule, alerts already accepted (active alert details, the Investigation block on an incident) keep showing the old snapshot;
• Repeat notifications and terminal events reuse the same snapshot, shifting each entry’s window by the difference in event time; the rule is not read again;
• Deleting the rule does not remove historical snapshots — the incident side then shows The alert rule was deleted, so no investigation entries were generated.

​

Related pages