Protocol
Keep every rerun result when the failure is intermittent
Rerunning until green edits the story, not the software.
Before you use it
- Use when
- An unchanged test alternates between passing and failing.
Keep this limit in mind
- No fixed number of clean reruns proves the absence of intermittent defects. A test-environment failure may still reveal a real production risk.
An idea to explore.
Try this idea.
Inspect the example, try the idea in your situation, then check your next move.
Why it matters
Preserve the sequence of outcomes and the conditions of each attempt. First establish that the result varies without the intended code change. Then investigate candidate dependencies such as concurrency, test order or external services instead of treating the final pass as a repair.
Steps
- Record version, input, environment, attempt and outcome together.
- Compare failing and passing attempts for a specific condition that changed.
- Test that condition deliberately in a safe environment and retain failures in the report.
An example
A batch test passes alone but fails after another test leaves shared state behind. An isolated green run did not cover that sequence.
Check your result
The evidence contains unsuccessful attempts and distinguishes a suspected condition from a confirmed cause.
Save and practise this idea
My learning
Read. Recall. Repeat.
Save this idea and return to it on any device.
Keep exploring
Try it in a task guide
These guides use this edition of the method alongside other ideas for the same task.
Keep every rerun result and compare failures with successes.Debug an Intermittent Failure or Flaky TestKeep successful and failed checks as evidence instead of reporting only the green run.
Verify an AI Agent Actually Finished the Work
Connected ideas
You are readingKeep every rerun result when the failure is intermittent
Evidence and sources
A flaky test can pass and fail on the same code, so one successful rerun does not establish that the underlying problem was fixed.
Repeated attempts reveal variability but do not by themselves identify its cause or certify reliability.
Where do our flaky tests come from? · Opening account of nondeterministic tests
All sources (1)
Edition and review
Version 0.1.0
No review details are supplied for this edition.