Protocol

Give the second check evidence the first answer did not create

Another confident paragraph is not automatically another line of evidence.

About 1 min read

Before you use it

Use when
  • You are tempted to verify an AI result by asking the same assistant whether it is sure.

Keep this limit in mind

  • Older self-correction studies do not establish the limits of every current model. The external test can also be wrong, so inspect its assumptions.

An idea to explore.

Try this idea.

Inspect the example, try the idea in your situation, then check your next move.

Keep reading

Why it matters

Choose a check with a different failure route: execute a calculation, inspect the cited passage, compare with a controlled fixture or ask a qualified reviewer to assess the original evidence. Self-correction can help, and training can improve it, but a repeated assurance is not proof that an external check occurred.

Steps

  1. Identify the exact claim or output that matters.
  2. Choose a check capable of finding its likely failure independently of the generated explanation.
  3. Record the observed result and any disagreement instead of asking for reassurance again.

An example

For a generated reconciliation formula, use a small hand-checked dataset with missing, extra and duplicate IDs.

Check your result

The verification report names an actual external observation or executed test.

Save and practise this idea

My learning

Read. Recall. Repeat.

Save this idea and return to it on any device.

Use email
Keep exploring

Try it in a task guide

These guides use this edition of the method alongside other ideas for the same task.

Use evidence the first answer did not create.
Verify an AI Output Before You Act
Require evidence with a failure path independent from the agent's own completion claim.
Verify an AI Agent Actually Finished the Work

Connected ideas

You are readingGive the second check evidence the first answer did not create

Evidence and sources
Supports

The cited ICLR study found that intrinsic reasoning self-correction without external feedback could fail or degrade performance in its tested settings.

This finding is model-, training- and task-dependent; it is not evidence that self-correction is universally impossible.

Large Language Models Cannot Self-Correct Reasoning Yet · Abstract

Limits

SCoRe reports improved self-correction through dedicated reinforcement-learning training, limiting a universal claim that models cannot self-correct.

Improved self-correction does not establish that a particular result passed an external check.

Training Language Models to Self-Correct via Reinforcement Learning · Abstract

All sources (2)
Edition and review

Version 0.1.0

No review details are supplied for this edition.

Read this idea as JSON · Release details