Protocol

Check stalled work before stopping the agent

A progress alarm should start a diagnosis. It should not declare failure just because a counter stayed flat: an investigation can rule out a cause before it fixes anything.

About 2 min read

Before you use it

Use when
  • An agent is still acting but the task state is not improving.

Keep this limit in mind

  • Some valid work has delayed visible progress; choose signals that fit the task rather than using one generic iteration count.

Current editorial reading

Compare evidence of progress

Choose a signal linked to the task and a review interval before the run. At each check, save the last verified result, new evidence and remaining work. If these stay unchanged, inspect the recent actions and choose a different approach, a bounded wait or a request for help. For a failing build, an unchanged test count may still hide a useful finding that eliminates one suspected cause.

Keep the alarm separate from the verdict

This is an editorial monitoring heuristic, not a validated three-iteration rule. The cited paper is a narrative synthesis of agent failures; it does not establish an effective stagnation threshold. Its authors did not reproduce the underlying experiments. Test the alarm on both genuine loops and useful slow work. Record false alarms as well as missed stalls, and adjust the signal or interval when either is unacceptable.

Take it with you

Produce a short diagnosis and a justified next action. An unchanged metric is a reason to inspect progress, not sufficient evidence that progress is impossible.

Sources and evidence (1)

What we checked

Provides context

The synthesis describes failures in composed, long-horizon agent tasks and proposes reporting partial progress alongside final success.

Selected full-text passages inspected in the 16-page PDF after HTML retrieval failed. These observations motivate review; they do not measure the benefit of this proposed alarm.

Beyond the Leaderboard: A Synthesis of Tool-Use, Planning, and Reasoning Failures in Large Language Model Agents · arXiv v1 PDF, sections 7.1–7.4 and 8.1, pp. 12–13

Limits

The paper is a narrative review, does not reproduce its underlying experiments, and mainly covers English text and web tasks.

These are the authors’ stated review limitations. No specific stopping threshold or detector accuracy is claimed. Underlying benchmark experiments were not independently reviewed in this pass.

Beyond the Leaderboard: A Synthesis of Tool-Use, Planning, and Reasoning Failures in Large Language Model Agents · arXiv v1 PDF, section 9, p. 14

Editorial sources (1)

Editorial version 1.0.0 · Evidence checked 2026-10-04

Save and practise this idea

My learning

Read. Recall. Repeat.

Save this idea and return to it on any device.

Use email
Keep exploring

More from this collection

Make the agent loop finish, recover and stay inspectable

Treat repeated state-action pairs as a loop alarmRetry after a state change; replan after the same failure
Original edition evidence and sources

These records belong to the original released edition. The current editorial reading has its own evidence above; neither set silently replaces the other.

Supports

Long-horizon agent failure syntheses report compounding planning/execution failures and stalled behavior, supporting explicit progress diagnostics rather than outcome-only monitoring.

Some valid work has delayed visible progress; choose signals that fit the task rather than using one generic iteration count.

Beyond the Leaderboard: A Synthesis of Tool-Use, Planning, and Reasoning Failures in Large Language Model Agents · See source record

All sources (1)
Original edition, downloads and provenance

The reading above is editorial version 1.0.0, bound to this exact source edition. The original data and saved downloads retain that edition and the separate editorial layer. They may contain earlier wording; use the current reading above when interpreting it.

Version 0.1.0

No review details are supplied for this edition.

Read the original edition and overlay as JSON · Release details