Checklist
Give the canary a fair comparison and a real stopping rule
Five easy successes can be a very convincing test of the wrong workload.
When it fits
- A limited rollout is being used to decide whether a change should reach everyone.
When to avoid it
- A canary still exposes real work to risk. Stateful replay, shared caches and duplicated side effects require isolation, not just a smaller sample.
Checklist
- Include the important workload types and enough time for delayed effects to appear.
- Compare attributable outcome metrics, not just a green deployment status.
- Assign a stop or rollback action and the person authorized to trigger it.
Why it matters
Choose exposure that can reveal the relevant failure, then compare it with an appropriate unchanged control. Define the harmful signals and response before starting. Keep shared infrastructure in view: a bad canary can affect the control and hide the difference you expected to measure.
An example
A new batch transformation is tried on representative record types, including exceptions, while the unchanged path supplies a comparison.
Check your result
The rollout decision states which workloads and observation period were actually covered.
Keep this limit in mind
- A canary still exposes real work to risk. Stateful replay, shared caches and duplicated side effects require isolation, not just a smaller sample.
Evidence and sources
Canary evaluation needs representative exposure, attributable metrics and attention to shared failure domains between canary and control.
The source does not justify a universal safe sample size, duration or rollout percentage.
Canarying Releases · Canary population; Metrics Should Be Representative and Attributable