Canary Metric Divergence Detection
Detect when canary metrics diverge from baseline before the SLO breach. The detection logic and the gate it enables.
Compare
Same metric on canary vs baseline. Statistical test (e.g., KS test) for distribution drift.
Tunable significance threshold. Higher = fewer false alarms; lower = catches subtle drift.
The gate
Canary deployment paused if divergence is detected. Engineer reviews; promotes or rolls back.
Automated. Saves the manual 'is the canary OK?' check.
Limits
Statistical tests need data. Low-traffic services have less reliable divergence detection.
False positives are real. Tune; do not disable.