Skip to main content

​
Use Cases

Alert responders are maintained in the source monitoring system and frequently adjusted; you want to sync them to Flashduty On-call in real-time.
Customer A has a self-developed big data task system where internal personnel can create various data batch processing tasks. Each task can specify a primary responder and secondary responder. When a batch processing task fails, the system will first notify the primary responder. If the alert is not resolved after 30 minutes, it escalates to the secondary responder.
Customer B uses Zabbix for host monitoring and has set a responsible person tag for each host. They want host alerts to notify the corresponding responder based on this tag.
Customer C has a self-developed monitoring system with many alert policies. Each policy is configured to notify a specific WeCom group. The customer has decided to migrate incident response to Flashduty but wants to maintain the policy-to-WeCom-group relationships from the source monitoring system and dynamically route alerts to WeCom groups based on these relationships.

​
Implementation

Add specific labels or query parameters to adjust assignment targets in Flashduty On-call. Parameter values can contain multiple emails, team names, or bot tokens separated by ,. Dynamic assignment supports two modes:
ModeDescriptionUse case
Replace (reset)Replace the original targets in the specified level with the targets from dynamic labelsYour monitoring system or external CMDB is the single source of truth for responders
Append (append)Add the targets from dynamic labels to the original targets in the specified level, with automatic deduplicationKeep the default on-call team while adding service owners, business groups, or temporary responders
Level numbers start from 0. For example, layer_person_reset_0_emails points to escalation rule level 1, and layer_person_append_1_emails points to escalation rule level 2.

​
Responder and team parameters

TargetReplace parameterAppend parameterValue
Responder emailslayer_person_reset_(\d)_emailslayer_person_append_(\d)_emailsMember emails, separated by ,
Team nameslayer_person_reset_(\d)_team_nameslayer_person_append_(\d)_team_namesTeam names, separated by ,

​
Group bot parameters

TargetReplace parameterAppend parameterValue
WeCom group botlayer_webhook_reset_(\d)_wecomslayer_webhook_append_(\d)_wecomsBot tokens, separated by ,
Dingtalk group botlayer_webhook_reset_(\d)_dingtalkslayer_webhook_append_(\d)_dingtalksBot tokens, separated by ,
Feishu/Lark group botlayer_webhook_reset_(\d)_feishuslayer_webhook_append_(\d)_feishusBot tokens, separated by ,
When an incident is triggered, Flashduty matches existing escalation rules. After matching an escalation rule, it assigns or escalates according to the levels in that rule. If these parameters are set, the system automatically replaces or appends assignment targets and group chat channels.In the matched escalation rule, everything remains unchanged except for the assignment targets and group chat targets, so the rule acts as a template escalation rule.
reset has higher priority than append. Responders and teams share the same responder group dimension: if any layer_person_reset_* parameter is set for a level, layer_person_append_* parameters for that level are not merged. Group bots are evaluated by bot type. For example, if both layer_webhook_reset_0_wecoms and layer_webhook_append_0_wecoms are set, level 1 uses the WeCom bot from reset; appending Feishu/Lark or Dingtalk bots at the same level still takes effect.

​
Push Example

​
Step 1: Set Up Template Escalation Rule

Configure an escalation rule for the channel. As shown below, this channel has only one assignment level, with the responder set to “Toutie Tech”, and also pushes to a WeCom group chat with a token ending in 5b96.
Template Escalation Rule Configuration

​
Step 2: Set Alert Labels

Using custom alert event integration as an example, push a sample alert to the target channel:
  • Set layer_person_reset_0_emails label to replace level 1 responders with guoyuhang and yushuangyu
  • Set layer_webhook_reset_0_wecoms label to replace level 1 WeCom group chat token with a token ending in d9c0
  • Set layer_person_append_0_emails label to append wangwu to level 1
  • Set layer_webhook_append_0_feishus label to append one Feishu/Lark group bot to level 1
curl --location --request POST 'https://api.flashcat.cloud/event/push/alert/standard?integration_key=your-integration-key' \
--header 'Content-Type: application/json' \
--data-raw '{
    "event_status": "Warning",
    "alert_key": "lasdfl2xzasd0262",
    "description": "cpu idle lower than 20%",
    "title_rule": "$cluster::$resource::$check",
    "labels": {
        "service": "engine",
        "cluster":"nj",
        "resource":"es.nj.01",
        "check":"cpu.idle<20%",
        "metric":"node_cpu_seconds_total",
        "layer_person_reset_0_emails": "guoyuhang@flashcat.cloud,yushuangyu@flashcat.cloud",
        "layer_webhook_reset_0_wecoms":"90dbb66b-af39-4235-956c-636a9c1ed9c0",
        "layer_person_append_0_emails": "wangwu@flashcat.cloud",
        "layer_webhook_append_0_feishus":"feishu-bot-token"
    }
}'

​
Step 3: View Incident Assignment Timeline

As shown below, the target incident is triggered and assigned normally. The incident responders and target group chats are replaced or appended according to the dynamic labels.
Dynamic Assignment Result Display

​
FAQ

1

Option 1: Manually Add Labels

If your system supports manually adding labels, such as Prometheus or Nightingale, we recommend adding specific labels directly in the alert policy.
2

Option 2: Use Label Enhancement

If your system already has related labels but in a different format or naming convention. For example, your hosts have team labels and you need to find the corresponding responder based on the team - in this case, you can use the label enhancement feature to generate responder-related labels based on team labels.For details, see Configure Label Enhancement.

Label Enhancement

Automatically generate new labels based on existing labels

Escalation Rules

Learn how to configure escalation rules