In Flashduty On-call
You can get the integration push URL in either of the following ways.
Use a dedicated integration
- In the Flashduty console, go to Channels and open a channel
- Select Configuration → Integrations → Private integration, then click Add an integration
- Select Split and click Save
- Open the new integration card and copy the push URL
Use a shared integration
- In the Flashduty console, go to Integration Center → Alert Events
- Select Split and enter an integration name
- Configure the default route and select a channel. You can add more rules under Routes after creation
- Click Save and copy the generated push URL
Configure Harness FME
1
Check permissions
You need FME Administrator access and the account-level permission to edit connectors (
core_connector_edit): Harness stores a metric alert webhook as a connector.2
Add the metric alert webhook
- In the FME navigation menu, click FME Settings → Integrations and select Webhooks in the categories menu
- On Outgoing Webhook (Metric Alerts), click Add and select the project to connect
- Under Environments, select the environments that should send alerts, for example production
- Select the alert types that trigger a delivery (one or more):
- All alert policies
- Metric alert policies
- Key metric significance
- Guardrail metric significance
- Enter the full Flashduty push URL as the webhook URL
- Click Save
3
Test
Before saving, you can click Send Test Message to check connectivity. If the test message is not a degradation alert, Flashduty returns success and creates no alert. If it is a complete metric alert sample, Flashduty creates an alert that you close by hand.
4
Turn on the auto-resolve timeout
FME sends a delivery only when an alert fires. Nothing is sent when the alert is dismissed or auto-resolved, so Flashduty never receives a recovery event. In the channel that receives these alerts, turn on the auto-resolve timeout. We suggest 24 hours, matching FME’s default 24-hour monitoring window (if you changed the window, use its length), counted from Incident trigger. Closing the incident also closes its alerts.
Payload
Each FME delivery is one JSON object whose
type is METRIC_ALERT. Flashduty uses these fields:
The alert title is
<feature flag or experiment name> degraded <metric name> in <environment name>. Every alert also carries the label source=split-io.
Alert Key
Flashduty builds the Alert Key from the feature flag or experiment ID, the environment ID and the metric ID:
- When the same feature flag degrades the same metric in the same environment again (including through another treatment or targeting rule), the delivery merges into the alert that is still firing
- When one metric fires both an alert policy alert and a significance alert, they merge into one alert
- Different feature flags, environments or metrics each create their own alert
- FME retries a failed delivery once after 300 milliseconds; the retry merges into the original alert
Severity
These deliveries return success and create no alert:
data.directionisDESIRED(the metric moved in the desired direction). Key metric significance alerts and alert policies with a threshold of 0 send both improvements and degradationstypeis notMETRIC_ALERT
METRIC_ALERT delivery without source.id, data.environment.id or data.metric.id is rejected.
FAQ
Why did the Flashduty alert not close after the FME alert auto-resolved?
Why did the Flashduty alert not close after the FME alert auto-resolved?
FME’s metric alert webhook sends a delivery only when an alert fires; there is no recovery event. Turn on the channel’s auto-resolve timeout, or close the alert in Flashduty by hand.
Why don't significance alerts for improvements show up in Flashduty?
Why don't significance alerts for improvements show up in Flashduty?
A delivery with
direction set to DESIRED means the metric moved in the desired direction and needs no on-call action, so Flashduty creates no alert.What is the difference between FME alert emails and the webhook?
What is the difference between FME alert emails and the webhook?
FME can also email alerts to any address. The webhook sends structured fields, which lets Flashduty merge alerts by feature flag, environment and metric. We recommend the webhook.
Troubleshooting
- Flashduty returns a parameter error: make sure the push URL is complete (including
integration_key) - FME shows an alert but Flashduty does not: make sure the webhook includes that environment and that alert type; improvements (
DESIRED) create no alert