In Flashduty On-call
You can get the integration push URL in either of the following ways.Use a dedicated integration
- In the Flashduty console, select Channel and open a channel
- Select Configuration → Integrations → Private integration, then click Add integration
- Select Mackerel, then click Save
- Open the generated integration card and copy the Push URL
Use a shared integration
- In the Flashduty console, go to Integration Center → Alert Events
- Select Mackerel and enter an integration name
- Configure the default route and select a channel. You can add more rules under Routing after creation
- Click Save and copy the generated Push URL
Configure Mackerel
1
Add a Webhook notification channel in Mackerel
Where: Mackerel › Channels › Webhook
Alert group, host status, host registration, host retirement, and monitor change events do not create alerts; Flashduty acknowledges them and returns success.Recovery needs no separate setting: when an alert closes, Mackerel sends another notification (
alert.isOpen is false) and Flashduty recovers the alert.If alert.isOpen is true and alert.status is none of critical, unknown, warning or ok, Flashduty returns a parameter error. Notifications whose event is not alert (such as host status or monitor changes) are acknowledged and ignored. Save the channel.2
Add the channel to a notification group
Where: Mackerel › Channels
3
Trigger an alert in Mackerel and verify
Where: Mackerel › Monitors
- Pick a monitor (for example, a host metric monitor) and temporarily lower its threshold so that it opens an alert
- Confirm that Flashduty receives an active alert
- Restore the threshold. After Mackerel closes the alert, confirm that the Flashduty alert recovers. You can also click Close alert on the Mackerel alert page to close it manually; Flashduty recovers the alert in the same way The checklist item “Receive the first event” is then complete as well.
Alert key
Flashduty uses the Mackerel alert’salert.id as the Alert Key. A Mackerel alert keeps the same ID from the moment it opens, through status changes, until it closes, so these notifications merge into one Flashduty alert.
When the same monitor fires again after its alert closed, Mackerel creates a new alert ID and Flashduty creates a new alert. Changes to the monitor name, message, metric value, or host status do not change the Alert Key. Alert notifications without alert.id are rejected.
Status and severity
Mackerel ranks Unknown at the same level as Critical, so Flashduty maps it to Critical. Closing an alert manually, deleting its monitor, retiring its host, or deleting its service all close the alert (
alert.isOpen is false), and Flashduty recovers the alert.
If alert.isOpen is true and alert.status is none of critical, unknown, warning or ok, Flashduty returns a parameter error. Notifications whose event is not alert (such as host status or monitor changes) are acknowledged and ignored.
Labels
The alert
message and the monitor memo are written to the alert description.
Troubleshooting
- Flashduty returns a parameter error: Make sure the URL is complete and includes
integration_key - No alerts arrive: Make sure the channel subscribes to the alert (
alert) event, the channel’s notification group includes the monitor, the host status isworking(Mackerel does not send alert notifications for hosts instandby,maintenance, orpoweroff), and the monitor’s notifications are not muted - Only Critical alerts arrive: The notification group has Critical only turned on, so Warning alerts are not sent
- The channel shows as suspended: Mackerel suspends a channel after its notifications keep failing for a period of time. Fix the push URL, then click the warning icon next to the channel to unsuspend it
- Restricting source IPs: See the Mackerel FAQ entry on the source IP addresses of notifications