Skip to main content
OpenNMS Horizon uses notices to tell operators about events in the network. This integration provides a webhook notification command that pushes every notice OpenNMS sends to Flashduty. Each OpenNMS notice maps to one Flashduty alert: it triggers when a node, interface, or service goes down, and closes when the matching recovery event auto-acknowledges the notice.

​
In Flashduty On-call

You can get the integration push URL in either of the following ways.

​
Use a dedicated integration

  1. In the Flashduty console, select Channel and open a channel
  2. Select Configuration → Integrations → Private integration, then click Add an integration
  3. Select OpenNMS and click Save
  4. Open the new integration card and copy the Push URL

​
Use a shared integration

  1. In the Flashduty console, select Integration Center → Alert Events
  2. Select OpenNMS and enter an integration name
  3. Configure the default route and select a channel. You can add more rules under Routes after creation
  4. Click Save and copy the generated Push URL

​
Configure OpenNMS

This integration uses the OpenNMS webhook notification strategy WebhookNotificationStrategy, which requires OpenNMS Horizon 36.0.3 or later. Earlier versions do not have this strategy, so upgrade first. Below, $OPENNMS_HOME is the OpenNMS installation directory, /opt/opennms in the official container image. The OpenNMS server must be able to reach the Flashduty push URL.
1

Add the Flashduty notification command

Edit $OPENNMS_HOME/etc/notificationCommands.xml, add the following command before </notification-commands>, and replace the address in -url with the push URL you copied:
Notes:
  • Every ${name} in the body needs an <argument> with the matching name; a missing one is replaced with an empty string. notice_id is the field Flashduty uses to identify the alert, so do not remove it
  • If the push URL contains & (for example, because you appended other query parameters), write it as &amp; in the XML
After saving, reload Notifd; until then the flashduty command does not appear in the destination path wizard in the next step. On the OpenNMS server, run $OPENNMS_HOME/bin/send-event.pl -p 'daemonName Notifd' uei.opennms.org/internal/reloadDaemonConfig. The official container image has no Perl, so send the same event through the REST API instead (replace <OpenNMS> with the OpenNMS address and enter the admin password when prompted):
2

Create the receiving user and destination path

OpenNMS runs the notification command once for each target user in a destination path. To push each notice only once, create a dedicated user for Flashduty:
  1. Select Administration → Configure OpenNMS, open Configure Users under Configure Users, Groups and On-Call Roles, and add the user flashduty. Do not give this user a duty schedule
  2. Go back to Configure OpenNMS, select Configure Notifications → Configure Destination Paths under Event Management, and click New Path
  3. Name it Flashduty, click Edit, select only flashduty under Send to Selected Users, and click Next Step
  4. Select the flashduty command, set Auto Notify to On, click Next Step, and then click Finish
Auto Notify must be On, which the wizard preselects. With Off or Auto, OpenNMS does not send the resolution notice for a notice that someone has already acknowledged by hand in OpenNMS, and the Flashduty alert does not close.
3

Choose the event notifications to push

Under Configure Notifications → Configure Event Notifications, edit each event notification you want to push to Flashduty and change its destination path to Flashduty. To keep email as well, add a second notification for the same event that uses the Flashduty path. We recommend pushing these notifications, which have recovery events:Do not use the Flashduty path for informational notifications such as nodeAdded and interfaceDeleted, or for threshold Rearmed notifications. They have no recovery event, so each one would create an alert that never closes on its own.
4

(Optional) Push the event severity

OpenNMS does not pass the event severity to notification commands on its own. To tell alert severities apart in Flashduty, edit $OPENNMS_HOME/etc/notifications.xml and add the following line to every <notification> pushed to Flashduty, before </notification>:
OpenNMS replaces %severity% with the severity of the triggering event (Critical, Major, Minor, and so on). Without this line, every alert in Flashduty has Critical severity.
5

Turn on notifications

Notifications are off in a new OpenNMS installation. Select Administration → Configure OpenNMS, set Notification Status under Event Management to On, and click Update. The bell icon in the top menu bar turns green when notifications are on.
6

Verify the lifecycle

Make a service on a monitored node stop responding (for example, stop the HTTP service on the node), wait for OpenNMS to detect it on its next poll, and confirm that Flashduty receives an active alert. Then restore the service and confirm that the alert closes.You can also send events by hand on the OpenNMS server. Replace the node ID and IP address with the values of a monitored node:
The official container image has no Perl to run send-event.pl; send the same events through the REST API instead:
OpenNMS has no button to test a notification command on its own.

​
Alert Key

Flashduty uses the OpenNMS notice ID notice_id (${noticeid}) as the Alert Key. The notice ID stays the same when an escalation step resends the notice and when OpenNMS sends the resolution notice after a recovery event auto-acknowledges it, so all of these land on the same alert. When the same node or service goes down again, OpenNMS creates a new notice and Flashduty creates a new alert. When one event matches several event notifications pushed to Flashduty, each notice creates its own alert. Notice IDs are unique only within one OpenNMS instance. Use a separate integration for each OpenNMS instance instead of pushing several instances to the same push URL. Flashduty rejects a request that lacks notice_id, and OpenNMS logs the failed push in notifd.log.

​
Alert lifecycle

OpenNMS configures auto-acknowledgement (auto-acknowledge) for outage events in notifd-configuration.xml: when the recovery event arrives, OpenNMS acknowledges the matching outage notice and sends the original notice again to its original recipients, with RESOLVED: in front of the subject and text. Flashduty checks whether the subject starts with RESOLVED:: The default auto-acknowledgements are nodeUp→nodeDown, interfaceUp→interfaceDown, nodeRegainedService→nodeLostService, serviceResponsive→serviceUnresponsive, and wideSpreadOutageResolved→wideSpreadOutage.
Keep resolution-prefix="RESOLVED: " on each auto-acknowledge. If you change or remove the prefix, resolution notices are pushed as new triggers and the alerts do not close.
To push other notices that have no auto-acknowledgement (for example, threshold alerts), add the following line inside <notifd-configuration> in notifd-configuration.xml, before all <auto-acknowledge> elements:
When the event’s alarm is cleared by its clearing event, OpenNMS acknowledges the notice and sends the resolution notice. After the change, reload Notifd as in the first step. For notices that still get no resolution notice, turn on the auto-resolve timeout in the channel that receives this integration’s alerts, set Window timing start to Incident trigger, and set the timeout to 24 hours.

​
Severity

Severity comes from severity in the request, which is the OpenNMS event severity that %severity% resolves to in the optional step: In a resolution notice, severity is the numeric severity id OpenNMS stores (for example 5 for Minor and 6 for Major). Flashduty converts ids 1–7 back to severity names, so the recovery event keeps the severity the alert was triggered with.

​
Alert content

  • Title: the notice subject subject without the RESOLVED: prefix; OpenNMS notice #<notice ID> when the subject is empty
  • Description: the notice text message without the RESOLVED: prefix
  • Labels: notice_id, event_id, event_uei, node_id, interface, service, severity (the severity name; the numeric id in a resolution notice is converted to its name). Labels with empty values are not added

​
Troubleshooting

The log messages below are in $OPENNMS_HOME/logs/notifd.log.
  • Nothing is pushed: confirm that the notification bell in the top menu bar is green, and that the event notification is On and uses the Flashduty destination path
  • The log says org.opennms.netmgt.notifd.WebhookNotificationStrategy cannot be loaded: OpenNMS is older than Horizon 36.0.3
  • The log shows the rendered -body is not valid JSON: the -body template was changed. Copy the command above again
  • The log shows Webhook returned status 4xx: confirm that -url is the full push URL, including integration_key
  • The log shows I/O problem posting to webhook: the OpenNMS server cannot connect to the push URL within 3 seconds. Check DNS, firewalls, and proxies. To use the system proxy, add an <argument> whose <switch> is -useSystemProxy and whose <substitution> is true
  • The same notice is pushed several times: the destination path targets a group or several users. Select only the flashduty user
  • The alert does not recover: confirm that the recovery event is in the auto-acknowledge list, that resolution-prefix is still RESOLVED: , and that Auto Notify on the destination path is On
For details on the webhook notification strategy, see Webhook Notifications in the OpenNMS documentation.