Protocol
Build a task bank for variation, not repetition count
Fifty questions can still be one question wearing different numbers.
When it fits
- When a practice bank contains many items but most are surface copies of the same decision.
When to avoid it
- A large bank is not automatically representative of the real domain. Sample from actual tasks and update the map when new failure modes appear.
Why it matters
Classify practice items by the decision they require, the cues that distinguish them, important edge cases and transfer distance. Add items where the bank is thin, not where generation is easiest. Sample across these dimensions so coverage improves as the bank grows.
Steps
- The bank can show coverage of distinct decisions and confusions, not only item count.
An example
A grammar bank tracks which tense contrast each sentence requires instead of celebrating another hundred sentences with the same obvious cue.
Check your result
The bank can show coverage of distinct decisions and confusions, not only item count.
Keep this limit in mind
- A large bank is not automatically representative of the real domain. Sample from actual tasks and update the map when new failure modes appear.
Evidence and sources
A meta-analysis of interleaving found a moderate overall benefit (g = 0.42) but strong moderation by material type and category similarity, including conditions where blocking performed better.
Interleaving should be selected for a discrimination problem rather than used as a universal mixing rule.
Similarity matters: A meta-analysis of interleaved learning and its moderators · Abstract
A meta-analysis of adaptive training found that effects varied by instructional intervention, with adaptive difficulty showing the strongest results among the examined categories.
The literature was heterogeneous and much of the context was military training; this does not validate every adaptive-learning product.
Adaptive training instructional interventions: A meta-analysis · Abstract