Skip to main content
Use an Ofelia webhook to send the results of scheduled jobs to Flashduty On-call. Each job on each Ofelia host maps to one Flashduty alert: a failed run opens a Critical alert, and a later successful run recovers it. This page covers the webhook feature of netresearch/ofelia (verified with v1.0.1). The upstream mcuadros/ofelia has no webhook feature, so this integration cannot be used with it.

In Flashduty On-call

You can get the integration push URL in either of the following ways.

Use a dedicated integration

  1. In the Flashduty console, select Channel and open a channel
  2. Select Configuration → Integrations → Private integration, then click Add integration
  3. Select Ofelia, then click Save
  4. Open the generated integration card and copy the Push URL

Use a shared integration

  1. In the Flashduty console, go to Integration Center → Alert Events
  2. Select Ofelia and enter an integration name
  3. Configure the default route and select a channel. You can add more rules under Routing after creation
  4. Click Save and copy the generated Push URL

Configure Ofelia

Ofelia talks to the Docker API even when it only runs job-local jobs, so the Ofelia container must mount /var/run/docker.sock; without it Ofelia exits at startup. With docker run, add -v /var/run/docker.sock:/var/run/docker.sock:ro.
1

Define a webhook in Ofelia

Where: /etc/ofelia/config.ini › [webhook "flashduty"]/etc/ofelia/config.ini is Ofelia’s default config file (the official image reads it too). If you start Ofelia with --config pointing elsewhere, edit that file instead.
trigger = always is required. Ofelia’s default is error, which sends failed runs only. Flashduty would never see a successful run, so the alert would not recover.
If you set webhook-allowed-hosts in [global], add api.flashcat.cloud to the list.
2

Attach the webhook to jobs in Ofelia

Where: /etc/ofelia/config.ini › [job-exec "backup-database"] › webhooksIn each job section you want to monitor, set webhooks to the webhook name from the previous step (separate several names with commas):
  • With Docker labels, set ofelia.job-exec.backup-database.webhooks: "flashduty" on the job’s container
  • To send for every job, set webhook-webhooks = flashduty in [global] (label: ofelia.webhook-webhooks)
3

Pin the hostname in Ofelia

Where: docker-compose.yml › services.ofelia › hostname (--hostname for docker run)Flashduty uses the hostname to tell apart jobs with the same name on different machines. Inside a container, the hostname defaults to the container ID, which changes when the container is recreated, and a successful run could then no longer recover an alert opened under the old ID. Set a fixed value, for example hostname: ofelia-prod-01.
4

Verify the lifecycle

Where: the command of a monitored job on the Ofelia hostOfelia has no test-send feature, so verify with real runs:
  1. Make a job fail once (for example, temporarily change its command to false) and confirm that Flashduty receives a Critical alert and the checklist’s “first event received” item turns green
  2. Restore the command, wait for the next successful run, and confirm that the alert recovers

Alert key

Flashduty builds the Alert Key from host.hostname and job.name, joined by a separator that cannot appear in either, then hashed with MD5. In our verification, failed and successful runs of the same job carried the same hostname and job name. execution.id differs on every run, so it is not part of the Alert Key.
  • Two different jobs on one host are two alerts, and the same job name on two hosts is also two alerts
  • Changes to the command, schedule, duration, or error text do not change the Alert Key
  • Jobs of different types in one Ofelia (job-exec, job-run, and so on) that share a name are treated as one alert, so keep job names unique
  • If job.name is missing, Flashduty returns 400 rather than writing a failure and its recovery to the wrong alert

Status and severity

A job skipped by no-overlap means the previous run is still going, so the job’s health has not changed. A skipped run therefore leaves an open alert alone. Repeated failures update the same alert. If a job stops running altogether (deleted or disabled), its alert does not recover. Close it manually, or turn on auto-close after timeout for the channel.

Alert content

  • Title: Ofelia job "<job name>" failed, or Ofelia job "<job name>" succeeded on recovery
  • Description: execution.error, followed by execution.stderr on a new line
  • Labels: check, job_name, job_type, job_schedule, execution_status, duration, resource / host (the hostname), ofelia_version, source=ofelia
The job command and stdout never reach labels or the description, because a command can carry credentials.

Troubleshooting

  • Flashduty receives nothing: confirm the job has webhooks = flashduty and look for Webhook error in the Ofelia log. Ofelia retries a failed webhook 3 times by default, 5 seconds apart
  • Failures arrive but never recover: confirm the webhook has trigger = always and that the hostname did not change between runs
  • Flashduty returns a parameter error: confirm the webhook has no other preset. The body must be the JSON that the json-post preset sends
  • Docker labels have no effect: webhook labels are only processed on the container with ofelia.service: "true". When an INI webhook and a label webhook share a name, the INI one wins
  • “host not in allowed hosts list”: add api.flashcat.cloud to [global] webhook-allowed-hosts
For more options, see Ofelia’s Webhook Notifications.