The Runbook-Attached Alert Pattern
Every alert links to a runbook. Without it, the alert is a ping with no action. The pattern, the enforcement, and the runbook gaps it surfaces.
The rule
One rule: every alert links to a runbook. Enforce it in tooling, not in code review, or it slips within a quarter.
- Required field. Every alert configuration has a runbook_url field; missing field, the alert cannot be saved.
- Link must resolve. CI checks the URL; broken links fail the build, not the on-call.
- One-click access. The runbook link is rendered in every page payload; on-call clicks once, never searches.
- Template per alert type. Standard headings (symptom, impact, first action, escalation); reduces blank-page friction.
Gaps the rule surfaces
The rule's value is what it forces you to confront. Three classes of latent debt show up the moment you require the link.
- Alerts without playbooks. The rule forces the team to write one or admit the alert is not actionable and remove it.
- Stale runbooks. Vague or outdated runbooks get fixed when linked from a real, firing alert.
- Duplicate alerts. Two alerts pointing at the same runbook reveal duplication; collapse them.
- Ownership gaps. Alerts whose runbooks have no clear owner expose holes in service ownership.
The trade
The rule introduces friction. The friction is the feature; alerts that cannot justify a runbook were never worth waking someone up for.
- Authoring cost. 30 to 60 minutes per runbook; pays back the first time the on-call follows it.
- Resistance to new alerts. Engineers think twice before adding alerts; the alert pool stays small.
- Smaller pool. The team converges on alerts that are real and actionable; signal-to-noise improves.
- Onboarding win. New on-call engineers get up to speed in days because the linked runbooks are the curriculum.