Feature: Incident Replay

Replay past incidents.

Overview

The Nova incident replay feature replays past incidents to test new detection logic. Synthetic tests cover what the team thought to write; replayed real incidents catch the cases the team did not anticipate, including the ones they already responded to once.

• Replay past incidents. Per-incident full data replay; the new detection sees the same data the on-call saw.
• Detection regression testing. Per-rule replay against past data; supports detection quality by catching false negatives before they ship.
• Onboarding tool. New on-callers replay past incidents; supports velocity by building muscle on real cases.
• What-if analysis plus PM evidence. Per-incident alternative response analysis; per-PM replay-supported timeline.

The approach

The practical approach: replay on every rule change, onboarding replay for new on-callers, what-if analysis for learning, PM evidence via replay link, documented per-team practice. The team’s discipline produces real detection quality instead of detection rules that pass synthetic tests but miss real incidents.

• Replay on rule change. Per-rule change replay against past incidents; the regression catches before deploy.
• Onboarding replay. Per-new-on-caller a replay session; the new engineer builds muscle without waiting for the next real incident.
• What-if analysis. Per-incident alternative path explored; supports learning from the incident the team already responded to.
• PM evidence plus documented practice. Per-PM replay link supports investigation; per-team replay practice committed for operational reviews.

Why this compounds

Replay discipline compounds across incidents. Each replay grows the team’s incident expertise; the next on-call walks in with more reps than the previous cohort.

• Better detection quality. Replay catches detection regressions; the new rule is verified against historical data.
• Better onboarding. New on-callers learn from real incidents; ramp accelerates without waiting for incident frequency.
• Better PM quality. Replay supports timeline construction; the PM has data, not memory.
• Institutional knowledge. Each replay teaches incident patterns; the team’s incident learning muscle grows.

Incident replay is a product feature that pays off across years. Nova AI Ops invests in incident learning as a first-class surface.