Protocol

Review test changes independently from the code they excuse

Green can mean the test moved, not the bug.

About 1 min read

Before you use it

Use when
  • The same agent changes implementation and tests until CI turns green.

Keep this limit in mind

  • Human-written tests can also be poor; independence reduces correlated failure but does not guarantee correctness.

An idea to explore.

Try this idea.

Inspect the example, try the idea in your situation, then check your next move.

Keep reading

Why it matters

Inspect deleted tests, weaker assertions, new mocks and changed fixtures separately from implementation logic. Ask whether each test still checks the intended behavior. For important invariants, preserve at least one independently authored or reviewed test that the implementation agent did not redefine.

Steps

  1. Deleted tests justified.
  2. Assertions not weakened silently.
  3. Mocks do not remove the behavior under test.
  4. Old regression intent preserved.
  5. Critical invariant has independent review.

An example

An agent replaces an exact authorization result with 'not null'; the test diff is rejected even though CI is green.

Check your result

A passing suite still exercises the intended contract rather than the agent's preferred implementation.

Save and practise this idea

My learning

Read. Recall. Repeat.

Save this idea and return to it on any device.

Use email
Keep exploring

Connected ideas

You are readingReview test changes independently from the code they excuse

Evidence and sources
Supports

OWASP recommends independent scrutiny of AI-generated test changes because an agent can make a suite pass by weakening, deleting or misdirecting tests.

This is a threat model and practice recommendation, not evidence that every agent routinely corrupts tests.

Secure Coding with AI Cheat Sheet · Test Fabrication and Test Deletion

All sources (1)
Edition and review

Version 0.1.0

No review details are supplied for this edition.

Read this idea as JSON · Release details