Skip to content

feat(alerts): escalation chains that page step by step until someone acknowledges - #2304

Merged
Makisuo merged 39 commits into
mainfrom
feat/alert-escalation-chains
Nov 6, 2026
Merged

feat(alerts): escalation chains that page step by step until someone acknowledges#2304
Makisuo merged 39 commits into
mainfrom
feat/alert-escalation-chains

Merge remote-tracking branch 'origin/main' into feat/alert-escalation…

62aeecf
Select commit
Loading
Failed to load commit list.
Maple Review Bot / Maple / review succeeded Nov 6, 2026 in 2m 42s

Confidence 10/10 · No issues found

🟢 Confidence 10/10 · safe to merge. Chain advance is a compare-and-set pinned by twelve PGlite cases and eleven planner table tests; nothing new since c51738f needed filing.
no findings · tests covered · risk high · 3/3 new units observable

Adds escalation chains: snapshotted runs advanced by the alert lane, sent through the existing delivery outbox, with v2 CRUD, rule-tier wiring and MCP tools. No findings; merge after #2287. It does not touch the open McpQueryError or VcsWebhookEnqueueFailure issues.

  • planEscalationAdvance steps a chain by position; applyEscalationPlan commits each step with a compare-and-set
  • Runs snapshot chain and trigger payload at incident open or tier escalation
  • An ack pauses a run, an unack resumes it, a resolve ends it
  • v2 /v2/alerts/escalation_chains CRUD plus escalation_chain_id on rule tiers

Before merge

Production impact

Open errors in the changed files (3)
Issue Service Occurrences File
@maple/mcp/errors/McpQueryError (4 issues) maple-chat 30 apps/ai/src/mcp/tools/registry.ts
@maple/api/routes/VcsWebhookEnqueueFailure (2 issues) maple-api 9 apps/api/src/runtime/http-graph.ts
@maple/mcp/errors/McpQueryError (4 issues) maple-ai 12 apps/ai/src/mcp/tools/registry.ts

After this merges, Maple checks whether they stop.

Production traffic of the changed files (last 7 days)
File Calls/day Busiest operations
packages/backend/src/services/alerts/AlertsService.ts 364 AlertsService.prepareIncidentDeliveries 201/day, 0.0% err
AlertsService.advanceEscalations 163/day, 0.0% err
packages/backend/src/services/alerts/EscalationChainsService.ts 15 EscalationChainsService.list 15/day, 0.0% err
packages/domain/src/http/v2/escalation-chains.ts 15 GET /v2/alerts/escalation_chains 15/day, 0.0% err
Telemetry this change adds and removes (15)
  • ➖ span name AlertsService.prepareIncidentDeliveries (~201/day today) · packages/backend/src/services/alerts/AlertsService.ts:1420
  • ➕ span name McpTool.createEscalationChain · apps/ai/src/mcp/tools/create-escalation-chain.ts:41
  • ➕ span name McpTool.deleteEscalationChain · apps/ai/src/mcp/tools/delete-escalation-chain.ts:23
  • ➕ span name McpTool.listEscalationChains · apps/ai/src/mcp/tools/list-escalation-chains.ts:21
  • ➕ span name McpTool.updateEscalationChain · apps/ai/src/mcp/tools/update-escalation-chain.ts:39
  • ➕ span name AlertsService.prepareIncidentNotification · packages/backend/src/services/alerts/AlertsService.ts:1424
  • ➕ span name AlertsService.prepareEscalationStart · packages/backend/src/services/alerts/AlertsService.ts:1609
  • ➕ span name AlertsService.staleEscalationSteps · packages/backend/src/services/alerts/AlertsService.ts:2400
  • ➕ span name AlertsService.activeSilenceForEscalation · packages/backend/src/services/alerts/AlertsService.ts:6018
  • ➕ span name AlertsService.advanceEscalationRun · packages/backend/src/services/alerts/AlertsService.ts:6056
  • ➕ attribute maple.escalation.silenced · packages/backend/src/services/alerts/AlertsService.ts:6072
  • ➕ span name EscalationChainsService.get · packages/backend/src/services/alerts/EscalationChainsService.ts:216
  • ➕ span name EscalationChainsService.create · packages/backend/src/services/alerts/EscalationChainsService.ts:226
  • ➕ span name EscalationChainsService.update · packages/backend/src/services/alerts/EscalationChainsService.ts:277
  • ➕ span name EscalationChainsService.delete · packages/backend/src/services/alerts/EscalationChainsService.ts:324
What was checked
  • Step 1 re-uses the tier's own delivery key, so the drain dedupes it (makeAlertDeliveryKey, alerting-core/src/index.ts:327)
  • A silenced step defers 60s and is realigned to the catch-up timestamp, so the held page is not sent twice
  • alert_escalation_runs_live_idx on (status, next_step_at) serves the due-run scan (migration.sql:40)
Observability coverage: 3 of 3 changes observable
Change Kind Observable Evidence
v2 escalation-chain routes (/v2/alerts/escalation-chains) inbound HTTP yes maple-api reports GET /v2/alerts/escalation_chains in route traffic; EscalationChainsService.list/create/update/delete are Effect.fn spans
packages/backend/src/services/alerts/AlertsService.ts escalation tick background cron consumer yes AlertScheduler.escalationTick wrapped by AlertScheduler.escalationTick span + Effect.annotateCurrentSpan
packages/backend/src/services/alerts/EscalationChainsService.ts service method yes EscalationChainsService.* are Effect.fn spans

62aeecf · Reply "won't fix" to dismiss a finding, or mention @maple-review-bot to ask about one.

View more details on Maple Review Bot