Protocol

Keep every rerun result when the failure is intermittent

Rerunning until green edits the story, not the software.

About 1 min read

Before you use it

Use when
  • An unchanged test alternates between passing and failing.

Keep this limit in mind

  • No fixed number of clean reruns proves the absence of intermittent defects. A test-environment failure may still reveal a real production risk.

An idea to explore.

Try this idea.

Inspect the example, try the idea in your situation, then check your next move.

Keep reading

Why it matters

Preserve the sequence of outcomes and the conditions of each attempt. First establish that the result varies without the intended code change. Then investigate candidate dependencies such as concurrency, test order or external services instead of treating the final pass as a repair.

Steps

  1. Record version, input, environment, attempt and outcome together.
  2. Compare failing and passing attempts for a specific condition that changed.
  3. Test that condition deliberately in a safe environment and retain failures in the report.

An example

A batch test passes alone but fails after another test leaves shared state behind. An isolated green run did not cover that sequence.

Check your result

The evidence contains unsuccessful attempts and distinguishes a suspected condition from a confirmed cause.

Save and practise this idea

My learning

Read. Recall. Repeat.

Save this idea and return to it on any device.

Use email
Keep exploring

Try it in a task guide

These guides use this edition of the method alongside other ideas for the same task.

Keep every rerun result and compare failures with successes.
Debug an Intermittent Failure or Flaky Test
Keep successful and failed checks as evidence instead of reporting only the green run.
Verify an AI Agent Actually Finished the Work

Connected ideas

You are readingKeep every rerun result when the failure is intermittent

Evidence and sources
Supports

A flaky test can pass and fail on the same code, so one successful rerun does not establish that the underlying problem was fixed.

Repeated attempts reveal variability but do not by themselves identify its cause or certify reliability.

Where do our flaky tests come from? · Opening account of nondeterministic tests

All sources (1)
Edition and review

Version 0.1.0

No review details are supplied for this edition.

Read this idea as JSON · Release details