Protocol
Use known-good papers as a search test
A search that cannot find the paper on your desk deserves suspicion.
Before you use it
- Use when
- You already know several directly relevant high-quality papers and want to check whether the query is behaving.
Keep this limit in mind
- Passing a known-item test does not prove completeness; it only exposes obvious retrieval failures.
An idea to explore.
Try this idea.
Inspect the example, try the idea in your situation, then check your next move.
Why it matters
Choose a small set of relevant records known before finalizing the strategy. Run the search and verify whether it retrieves them for the expected reasons. If not, inspect missing terminology, indexing and overly restrictive concepts. Do not tune only to those papers; add other validation checks so the query does not overfit.
Steps
- Known items were selected before final query tuning.
- Each missed known item is investigated.
- Changes are based on generalizable terminology or indexing gaps.
- The strategy is not narrowed around one favored paper.
- Additional relevant records are still being discovered.
An example
If your AI-confidence query misses a landmark paper because it uses 'metacognitive sensitivity,' update the terminology map rather than manually adding the title.
Check your result
The finalized search retrieves known relevant items without becoming a title-specific lookup.
Save and practise this idea
My learning
Read. Recall. Repeat.
Save this idea and return to it on any device.
Keep exploring
Connected ideas
You are readingUse known-good papers as a search test
Ideas that link here
These ideas link to this one. Check their conditions and limits before use.
Useful with this ideaBuild a terminology map before optimizing the query
Evidence and sources
Current review-search guidance recommends validating a search strategy rather than assuming a syntactically correct query is comprehensive.
No finite validation proves complete recall.
How to search for literature in systematic reviews and meta-analyses: A comprehensive step-by-step guide · Search validation checks
Edition and review
Version 0.1.0
No review details are supplied for this edition.