Protocol

Keep a human baseline for important AI tasks

Without a baseline, 'better with AI' can mean 'faster than I remember.'

About 1 min read

Before you use it

Use when
  • AI output looks impressive but you do not know whether it actually improves the work.

Keep this limit in mind

  • Small samples are directional and can be biased by case selection; do not convert them into universal productivity claims.

An idea to explore.

Try this idea.

Inspect the example, try the idea in your situation, then check your next move.

Keep reading

Why it matters

For a small representative sample, compare AI-assisted performance with the current human or non-AI method using the same acceptance criteria. Measure quality, time and review effort separately. Repeat only often enough to detect meaningful capability changes; this is a calibration tool, not permanent double work.

Steps

  1. The same task and acceptance criteria are used in both conditions.
  2. Quality is judged independently from speed.
  3. Review or correction time is included in the AI-assisted cost.
  4. Cases include at least one known edge condition.
  5. The comparison is saved with the model and workflow version.

An example

Compare five real specification summaries produced with and without AI, including correction time and factual misses.

Check your result

You can say what AI changed relative to the current method instead of comparing it with an imagined baseline.

Save and practise this idea

My learning

Read. Recall. Repeat.

Save this idea and return to it on any device.

Use email
Keep exploring

Connected ideas

You are readingKeep a human baseline for important AI tasks

Evidence and sources
Supports

The 2026 Organization Science field experiment found strong AI-assisted gains on a set of tasks within the tested capability frontier but lower correctness on a selected complex task outside that frontier.

The experiment used a particular model generation and consulting task set; current frontier location must be re-estimated for today's model and workflow.

Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality · Abstract and results

Provides context

Across the recent field and lab studies, AI's effect on speed and quality varies by task and workflow, so a productivity claim should measure these outcomes separately rather than assume faster means better.

This synthesis draws a practical measurement implication from heterogeneous studies; it is not a pooled meta-analysis.

Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality · Abstract and task-contingent results

All sources (1)
Edition and review

Version 0.1.0

No review details are supplied for this edition.

Read this idea as JSON · Release details