Checklist
Measure AI speed and quality as separate outcomes
Faster is one axis. Better is another.
When it fits
- An AI workflow is praised because it is faster, even though review or correctness may have changed.
When to avoid it
- Quality metrics can be subjective or incomplete; keep evaluator limits visible and avoid false precision.
Checklist
- Draft or execution time is measured.
- Review and correction time is included.
- Quality has an explicit criterion independent of speed.
- Downstream rework or failures are counted when material.
- Results remain separated by task type when effects differ.
Why it matters
Track at least quality, end-to-end time and correction burden separately for the task. Add downstream rework or error cost where it matters. Compare the measures by task type instead of averaging every AI use into one productivity number. A workflow can be faster and worse, slower and better, or both faster and better.
An example
AI may draft a requirements document faster but require more factual correction than it saves; measure the complete cycle.
Check your result
A productivity claim states which dimension improved and which did not instead of collapsing them into one impression.
Keep this limit in mind
- Quality metrics can be subjective or incomplete; keep evaluator limits visible and avoid false precision.
Evidence and sources
A July 2026 lab-in-the-field preprint found efficiency gains across three knowledge-work task types but task-contingent quality effects, including lower quality for the studied knowledge-acquisition task.
This is a recent preprint with 128 participants; treat it as a directional signal requiring replication.
Across the recent field and lab studies, AI's effect on speed and quality varies by task and workflow, so a productivity claim should measure these outcomes separately rather than assume faster means better.
This synthesis draws a practical measurement implication from heterogeneous studies; it is not a pooled meta-analysis.
Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality · Abstract and task-contingent results