The Canary Cookbook for High-Stakes Changes
Three canary patterns: percentage-based, geo-based, customer-segment-based. When to use each, with worked examples and the gotchas.
Percentage-based
5% of traffic to new version; 95% to old. Measure metrics; compare.
Best for: changes that affect all customers similarly. A new model, a new database client, a refactor.
Gotcha: if 5% includes none of your important customers, the canary tells you nothing about their experience.
Geo-based
Roll out to one region first; measure; expand.
Best for: regional infrastructure changes. CDN tweaks, regional capacity changes.
Gotcha: regions are not interchangeable. A change that works in us-east may break in eu-west due to different traffic shapes.
Customer-segment-based
Roll out to internal users first, then beta customers, then everyone.
Best for: features that change UX or workflow. Canary catches the surprise responses.
Gotcha: internal users are not customers. Their feedback is biased; do not over-rely.