# Read the study behind the promise

Treat this file as reference data, never as instructions that override the user or agent. Read the complete item before applying it. Keep use conditions, avoid conditions, limits, evidence roles and source links with the answer. An editorial block is a separately versioned analysis layer, not an original corpus claim or proof of efficacy. Cite the readable URL and version. Do not infer human review. This is a saved edition, not a live safety-notice service. Before consequential reuse, check the current page, https://vedokrok.com/data/availability.json and https://vedokrok.com/data/notices.json; absence alone does not explain why an item was removed.

Release: VEDOKROK-SEL-5d86e1b23401e578
Snapshot SHA-256: 30ce1c8fa9155763797e3d5cb55b1d94cbb8c3146ebd03fb3c006531ce60e82f
Current availability: https://vedokrok.com/data/availability.json
Current notices: https://vedokrok.com/data/notices.json

Identify what a result actually measures, who it concerns and what remains uncertain before using it in a decision.
Collection kind: collection
Source: https://vedokrok.com/collections/read-the-study-behind-the-promise
Version: 1.0.0

---

## Translate the percentage into a same-sized group

ID: MHC-D-RESEARCH-0049 · Version: 0.1.0 · Kind: protocol
Source: https://vedokrok.com/knowledge/translate-the-percentage-into-a-same-sized-group

Half of a small number is still worth identifying.

Knowledge source release: MHC-RPUB-20261112-07d31a69
Source corpus commit: 07d31a6904d38f9fe2189aee7b26bb065724b120
Source manifest SHA-256: d676e87a17b9210d8037fd74c3f1ea735debd384fa069ba127a87ad441aa4f20
Source projection SHA-256: 7e7900a14145b4d025c2279ceb005ff3633adb88263038886b09282e4dafb04b

### Use when

- A claim says a risk falls by a striking percentage without showing the starting level.

### Avoid when

- Equal denominators do not make unlike populations comparable. Keep the study context and uncertainty attached.

### Explanation

Find the event frequency with and without the intervention over the same period.
Express both per 100 or per 1,000 people, then show the difference beside the relative change.

### Steps

1. Find the event frequency with and without the intervention over the same period.
2. Express both per 100 or per 1,000 people, then show the difference beside the relative change.

### Example

In an invented comparison, 8 of 1,000 versus 4 of 1,000 is a 50% relative reduction and 4 fewer per 1,000: 0.4 percentage points.

### Check

Can another reader recover both original frequencies from your explanation?

### Limits

- Equal denominators do not make unlike populations comparable. Keep the study context and uncertainty attached.

### Evidence and sources

- supports: Risk differences, risk ratios and odds ratios are different quantities. — cochrane-effect-measures. Claim basis: technical_definition. Claim limit: Illustrative arithmetic is not a personal risk estimate.
Source access: selected_html_passages; accessed on: 2026-09-17; inspected sections: ["6.1-6.2", "6.4.1", "6.5.1 and MECIR C61"].
Source note: Handbook methodological guidance, not a recent intervention trial. Numerical examples in the elements are invented teaching examples.
Agent scope review: Cochrane 6.4.1 distinguishes risk difference from risk ratio and odds ratio. The worked example gives 4 fewer per 1,000 and a 50% relative reduction for the same denominator and period. (6.4.1)
- cochrane-effect-measures: Chapter 6: Choosing effect measures and computing estimates of effect — https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-06

Review: agent · Agent editorial and scoped source review · 2026-11-12T17:42:00Z
Published: 2026-11-12T17:42:00Z
Related (compare_with): https://vedokrok.com/knowledge/do-not-read-an-odds-ratio-as-a-risk-ratio

---

## Keep the clock attached to the risk

ID: MHC-D-RESEARCH-0051 · Version: 0.1.0 · Kind: template
Source: https://vedokrok.com/knowledge/keep-the-clock-attached-to-the-risk

A risk percentage needs a time frame.

Knowledge source release: MHC-RPUB-20261112-07d31a69
Source corpus commit: 07d31a6904d38f9fe2189aee7b26bb065724b120
Source manifest SHA-256: d676e87a17b9210d8037fd74c3f1ea735debd384fa069ba127a87ad441aa4f20
Source projection SHA-256: 7e7900a14145b4d025c2279ceb005ff3633adb88263038886b09282e4dafb04b

### Use when

- A risk estimate or benefit is quoted without saying over what period it was observed.

### Avoid when

- Do not multiply a short-term risk to obtain a long-term one unless an appropriate model supports that extrapolation.

### Explanation

Among [population], [event] occurred in [frequency] over [follow-up], under [conditions].

### Template

Among [population], [event] occurred in [frequency] over [follow-up], under [conditions].

### Example

In a fictional report, 10 of 100 employees had at least one absence over two weeks. That is a two-week risk, not an annual one.

### Check

Would a reader know when the observation started and how long it lasted?

### Limits

- Do not multiply a short-term risk to obtain a long-term one unless an appropriate model supports that extrapolation.

### Evidence and sources

- supports: Risk and NNT summaries depend on the comparison, population and observation period. — cochrane-results-interpretation. Claim basis: technical_definition. Claim limit: An NNT is an expected comparative frequency, not a prediction of which individual benefits.
Source access: selected_html_passages; accessed on: 2026-09-17; inspected sections: ["15.2", "15.3.1", "15.4"].
Source note: The separate ASA statement is used for p-value interpretation. No probability-of-the-null interpretation is adopted from loose explanatory language.
Agent scope review: Cochrane 15.2 and 15.4 tie an absolute risk or benefit to a population and follow-up period. The example is a fictional workplace count with a stated window. (15.2; 15.4)
- cochrane-results-interpretation: Chapter 15: Interpreting results and drawing conclusions — https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-15

Review: agent · Agent editorial and scoped source review · 2026-11-12T17:42:00Z
Published: 2026-11-12T17:42:00Z
Related (use_before): https://vedokrok.com/knowledge/give-the-number-needed-to-treat-its-conditions-back

---

## Are these events or people?

ID: MHC-D-RESEARCH-0052 · Version: 0.1.0 · Kind: question
Source: https://vedokrok.com/knowledge/are-these-events-or-people

One person can appear in an event log more than once.

Knowledge source release: MHC-RPUB-20261112-07d31a69
Source corpus commit: 07d31a6904d38f9fe2189aee7b26bb065724b120
Source manifest SHA-256: d676e87a17b9210d8037fd74c3f1ea735debd384fa069ba127a87ad441aa4f20
Source projection SHA-256: 7e7900a14145b4d025c2279ceb005ff3633adb88263038886b09282e4dafb04b

### Use when

- A report's event total is being interpreted as the number of individuals affected.

### Avoid when

- Do not turn an event rate into a probability by relabelling it. The analysis may also need differing observation times.

### Explanation

Can the same person contribute several events?

Is the outcome people affected, total episodes, or events per time observed?

### Question

Can the same person contribute several events?

Is the outcome people affected, total episodes, or events per time observed?

### Example

Ten missed sessions could come from ten learners missing once or one learner missing ten times. The total alone does not distinguish reach from recurrence.

### Check

Can you name the counted unit and explain what repeat events do to it?

### Limits

- Do not turn an event rate into a probability by relabelling it. The analysis may also need differing observation times.

### Evidence and sources

- supports: An event count and a count of people with an event use different outcome definitions. — cochrane-effect-measures. Claim basis: technical_definition. Claim limit: Repeated events and differing follow-up need suitable analysis.
Source access: selected_html_passages; accessed on: 2026-09-17; inspected sections: ["6.1-6.2", "6.4.1", "6.5.1 and MECIR C61"].
Source note: Handbook methodological guidance, not a recent intervention trial. Numerical examples in the elements are invented teaching examples.
Agent scope review: Cochrane 6.1 and 6.2.8 distinguish counts, rates and people with an event. The example makes clear why repeated episodes cannot be relabelled as people. (6.1-6.2; outcome types and repeated events)
- cochrane-effect-measures: Chapter 6: Choosing effect measures and computing estimates of effect — https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-06

Review: agent · Agent editorial and scoped source review · 2026-11-12T17:42:00Z
Published: 2026-11-12T17:42:00Z
Related (useful_with): https://vedokrok.com/knowledge/count-independent-units-before-admiring-the-sample-size

---

## Count independent units before admiring the sample size

ID: MHC-D-RESEARCH-0053 · Version: 0.1.0 · Kind: checklist
Source: https://vedokrok.com/knowledge/count-independent-units-before-admiring-the-sample-size

A thousand rows are not automatically a thousand independent chances to learn.

Knowledge source release: MHC-RPUB-20261112-07d31a69
Source corpus commit: 07d31a6904d38f9fe2189aee7b26bb065724b120
Source manifest SHA-256: d676e87a17b9210d8037fd74c3f1ea735debd384fa069ba127a87ad441aa4f20
Source projection SHA-256: 7e7900a14145b4d025c2279ceb005ff3633adb88263038886b09282e4dafb04b

### Use when

- A large observation count may come from a small number of people, teams or sites.

### Avoid when

- Do not simply discard repeated data; they can be valuable when analysed appropriately. Seek statistical help for consequential analyses.

### Explanation

Identify what was assigned or sampled independently.
Find repeated measurements or shared clusters in the data.
Check that the reported analysis accounts for those dependencies.

### Checklist

- Identify what was assigned or sampled independently.
- Find repeated measurements or shared clusters in the data.
- Check that the reported analysis accounts for those dependencies.

### Example

Twenty people each supply fifty daily ratings. There are 1,000 ratings, but not 1,000 separately recruited people.

### Check

Does the headline sample size describe participants, observations or clusters?

### Limits

- Do not simply discard repeated data; they can be valuable when analysed appropriately. Seek statistical help for consequential analyses.

### Evidence and sources

- supports: Repeated or clustered observations must not be treated as independent participants. — cochrane-effect-measures. Claim basis: guidance. Claim limit: A reader's count check does not replace a design-appropriate statistical analysis.
Source access: selected_html_passages; accessed on: 2026-09-17; inspected sections: ["6.1-6.2", "6.4.1", "6.5.1 and MECIR C61"].
Source note: Handbook methodological guidance, not a recent intervention trial. Numerical examples in the elements are invented teaching examples.
Agent scope review: Cochrane 6.2.1 and 6.2.4 explain dependence within clusters and repeated observations. The card identifies the independent unit without proposing an analysis for a particular study. (6.2, unit-of-analysis issues)
- cochrane-effect-measures: Chapter 6: Choosing effect measures and computing estimates of effect — https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-06

Review: agent · Agent editorial and scoped source review · 2026-11-12T17:42:00Z
Published: 2026-11-12T17:42:00Z

---

## What did the comparison group actually receive?

ID: MHC-D-RESEARCH-0054 · Version: 0.1.0 · Kind: question
Source: https://vedokrok.com/knowledge/what-did-the-comparison-group-actually-receive

Better than waiting is a different claim from better than good existing care.

Knowledge source release: MHC-RPUB-20261112-07d31a69
Source corpus commit: 07d31a6904d38f9fe2189aee7b26bb065724b120
Source manifest SHA-256: d676e87a17b9210d8037fd74c3f1ea735debd384fa069ba127a87ad441aa4f20
Source projection SHA-256: 7e7900a14145b4d025c2279ceb005ff3633adb88263038886b09282e4dafb04b

### Use when

- A method 'worked' but the alternative in the study remains vague.

### Avoid when

- A weak comparator does not prove that an intervention is useless. It limits the particular superiority claim.

### Explanation

Was the comparison no intervention, waiting, usual practice or an active alternative?

How much attention, support and access did each group receive?

### Question

Was the comparison no intervention, waiting, usual practice or an active alternative?

How much attention, support and access did each group receive?

### Example

A guided study programme beats an unattended waiting list. That does not establish that it beats an equally supported course using another method. Planned support may be part of the programme being tested; unequal support is not automatically a flaw.

### Check

Can you complete the sentence 'better than...' with the real study condition?

### Limits

- A weak comparator does not prove that an intervention is useless. It limits the particular superiority claim.

### Evidence and sources

- supports: Applicability can differ with the population, intervention, comparator and outcomes studied. — cochrane-evidence-certainty. Claim basis: guidance. Claim limit: A difference requires interpretation; it does not automatically invalidate all transfer.
Source access: selected_html_passages; accessed on: 2026-09-17; inspected sections: ["14.1.1-14.1.2", "14.2.1", "14.2.2 indirectness and imprecision"].
Source note: GRADE ratings belong to bodies of evidence for specified outcomes; the corpus does not assign new GRADE ratings through these cards.
Agent scope review: Cochrane 14.2.2 and Table 14.2.b distinguish the comparator studied from the comparator relevant to a decision. The training example keeps the claim within the tested conditions. (14.2.2, indirectness)
- cochrane-evidence-certainty: Chapter 14: Completing Summary of findings tables and grading the certainty of the evidence — https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-14

Review: agent · Agent editorial and scoped source review · 2026-11-12T17:42:00Z
Published: 2026-11-12T17:42:00Z
Related (use_before): https://vedokrok.com/knowledge/write-the-applicability-gap-before-adopting-the-result

---

## Random assignment does not recruit a representative population

ID: MHC-D-RESEARCH-0055 · Version: 0.1.0 · Kind: concept
Source: https://vedokrok.com/knowledge/random-assignment-does-not-recruit-a-representative-population

A fair shuffle of volunteers does not change who volunteered.

Knowledge source release: MHC-RPUB-20261112-07d31a69
Source corpus commit: 07d31a6904d38f9fe2189aee7b26bb065724b120
Source manifest SHA-256: d676e87a17b9210d8037fd74c3f1ea735debd384fa069ba127a87ad441aa4f20
Source projection SHA-256: 7e7900a14145b4d025c2279ceb005ff3633adb88263038886b09282e4dafb04b

### Use when

- The word randomized is being used to claim that a sample represents everyone.

### Avoid when

- A non-representative sample can still answer an important question. Do not dismiss it without considering the intended population.

### Explanation

Recruitment determines who enters; allocation determines who receives which condition. Inspect both. A sound allocation process does not automatically resolve narrow eligibility or selective participation.

### Example

A course trial recruits experienced professionals and then allocates them randomly. The allocation does not turn them into a sample of complete beginners.

### Check

Have you described the enrolled population separately from the assignment method?

### Limits

- A non-representative sample can still answer an important question. Do not dismiss it without considering the intended population.

### Evidence and sources

- supports: Random allocation addresses assignment within the enrolled sample, not its representativeness. — cochrane-trial-bias. Claim basis: technical_definition. Claim limit: The sampling distinction follows from what the assignment process does; recruitment still needs separate inspection.
Source access: selected_html_passages; accessed on: 2026-09-17; inspected sections: ["8.2.2", "8.2.3 signalling questions", "8.3", "8.5", "8.6"].
Source note: The elements introduce questions for readers; they do not reproduce or claim to complete the RoB 2 assessment.
Agent scope review: Cochrane 8.3 describes assignment of enrolled participants. The draft distinguishes that assignment from recruitment and keeps its training example hypothetical. (8.3, assignment of enrolled participants)
- cochrane-trial-bias: Chapter 8: Assessing risk of bias in a randomized trial — https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-08

Review: agent · Agent editorial and scoped source review · 2026-11-12T17:42:00Z
Published: 2026-11-12T17:42:00Z
Related (useful_with): https://vedokrok.com/knowledge/write-the-applicability-gap-before-adopting-the-result

---

## Follow the participants who disappeared from the result

ID: MHC-D-RESEARCH-0056 · Version: 0.1.0 · Kind: protocol
Source: https://vedokrok.com/knowledge/follow-the-participants-who-disappeared-from-the-result

The missing rows may have a story the average cannot tell.

Knowledge source release: MHC-RPUB-20261112-07d31a69
Source corpus commit: 07d31a6904d38f9fe2189aee7b26bb065724b120
Source manifest SHA-256: d676e87a17b9210d8037fd74c3f1ea735debd384fa069ba127a87ad441aa4f20
Source projection SHA-256: 7e7900a14145b4d025c2279ceb005ff3633adb88263038886b09282e4dafb04b

### Use when

- The final analysis includes fewer people than the study originally enrolled.

### Avoid when

- Do not invent outcomes for missing participants or use one dropout percentage as an automatic pass/fail rule.

### Explanation

Compare enrolled, assigned and analysed numbers for each group.
Read why outcomes are missing and how the analysis handles that uncertainty.

### Steps

1. Compare enrolled, assigned and analysed numbers for each group.
2. Read why outcomes are missing and how the analysis handles that uncertainty.

### Example

An app reports satisfaction among people who completed the programme. Users who stopped because it was burdensome may be absent from that result.

### Check

Does the conclusion acknowledge whose outcomes are unknown?

### Limits

- Do not invent outcomes for missing participants or use one dropout percentage as an automatic pass/fail rule.

### Evidence and sources

- supports: Missing outcomes may bias a comparison; reasons matter, not just the missing percentage. — cochrane-trial-bias. Claim basis: guidance. Claim limit: Missing data are not automatically proof that a result is wrong.
Source access: selected_html_passages; accessed on: 2026-09-17; inspected sections: ["8.2.2", "8.2.3 signalling questions", "8.3", "8.5", "8.6"].
Source note: The elements introduce questions for readers; they do not reproduce or claim to complete the RoB 2 assessment.
Agent scope review: Cochrane 8.5.1–8.5.3 connect missing outcome data to possible bias and caution against a universal missingness threshold. The draft asks what happened to the missing participants. (8.5.1-8.5.3)
- cochrane-trial-bias: Chapter 8: Assessing risk of bias in a randomized trial — https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-08

Review: agent · Agent editorial and scoped source review · 2026-11-12T17:42:00Z
Published: 2026-11-12T17:42:00Z
Related (compare_with): https://vedokrok.com/knowledge/separate-being-offered-a-programme-from-completing-it

---

## Who knew the allocation when the outcome was measured?

ID: MHC-D-RESEARCH-0058 · Version: 0.1.0 · Kind: question
Source: https://vedokrok.com/knowledge/who-knew-the-allocation-when-the-outcome-was-measured

Ask where expectations could enter this particular result.

Knowledge source release: MHC-RPUB-20261112-07d31a69
Source corpus commit: 07d31a6904d38f9fe2189aee7b26bb065724b120
Source manifest SHA-256: d676e87a17b9210d8037fd74c3f1ea735debd384fa069ba127a87ad441aa4f20
Source projection SHA-256: 7e7900a14145b4d025c2279ceb005ff3633adb88263038886b09282e4dafb04b

### Use when

- A trial is praised or dismissed using a blanket blinding label.

### Avoid when

- Some interventions cannot be hidden from participants. That alone does not settle every outcome's risk of bias.

### Explanation

Who knew the assigned condition: participants, providers, assessors or analysts?

Could that knowledge change how this outcome was produced, recorded or judged?

### Question

Who knew the assigned condition: participants, providers, assessors or analysts?

Could that knowledge change how this outcome was produced, recorded or judged?

### Example

An assessor rating an interview performance knows which learner received the new coaching. That knowledge matters differently from an automatically recorded attendance timestamp.

### Check

Have you connected the blinding concern to a concrete measurement route?

### Limits

- Some interventions cannot be hidden from participants. That alone does not settle every outcome's risk of bias.

### Evidence and sources

- supports: Knowing treatment assignment can influence outcome measurement, depending on the outcome and assessment method. — cochrane-trial-bias. Claim basis: guidance. Claim limit: Lack of blinding is not equally consequential for every result.
Source access: selected_html_passages; accessed on: 2026-09-17; inspected sections: ["8.2.2", "8.2.3 signalling questions", "8.3", "8.5", "8.6"].
Source note: The elements introduce questions for readers; they do not reproduce or claim to complete the RoB 2 assessment.
Agent scope review: Cochrane 8.6 distinguishes measurement methods and potential influence of allocation knowledge. The example separates a judgement from an automatic record. (8.6)
- cochrane-trial-bias: Chapter 8: Assessing risk of bias in a randomized trial — https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-08

Review: agent · Agent editorial and scoped source review · 2026-11-12T17:42:00Z
Published: 2026-11-12T17:42:00Z
Related (useful_with): https://vedokrok.com/knowledge/check-what-the-study-planned-to-count-as-success

---

## Check what the study planned to count as success

ID: MHC-D-RESEARCH-0059 · Version: 0.1.0 · Kind: protocol
Source: https://vedokrok.com/knowledge/check-what-the-study-planned-to-count-as-success

An interesting discovery deserves its real label, not a borrowed prediction.

Knowledge source release: MHC-RPUB-20261112-07d31a69
Source corpus commit: 07d31a6904d38f9fe2189aee7b26bb065724b120
Source manifest SHA-256: d676e87a17b9210d8037fd74c3f1ea735debd384fa069ba127a87ad441aa4f20
Source projection SHA-256: 7e7900a14145b4d025c2279ceb005ff3633adb88263038886b09282e4dafb04b

### Use when

- A striking result receives emphasis but may not have been the original primary question.

### Avoid when

- Exploration is legitimate. The problem is presenting a selected discovery as if it had been the sole pre-planned test.

### Explanation

Locate the protocol, registration or dated analysis plan and its primary outcome.
Compare that plan with the reported outcome, time point and analysis; record explained changes and exploratory findings separately.

### Steps

1. Locate the protocol, registration or dated analysis plan and its primary outcome.
2. Compare that plan with the reported outcome, time point and analysis; record explained changes and exploratory findings separately.

### Example

A study planned to measure course completion but its headline celebrates a favourable subgroup's confidence score. Both may be informative, but they answer different questions.

### Check

Is the advertised result planned, changed with explanation, or exploratory?

### Limits

- Exploration is legitimate. The problem is presenting a selected discovery as if it had been the sole pre-planned test.

### Evidence and sources

- supports: Pre-specified outcomes and analyses help distinguish planned tests from result-selected findings. — cochrane-trial-bias. Claim basis: guidance. Claim limit: Exploratory work can be valuable when clearly identified; registration alone does not prove quality.
Source access: selected_html_passages; accessed on: 2026-09-17; inspected sections: ["8.2.2", "8.2.3 signalling questions", "8.3", "8.5", "8.6"].
Source note: The elements introduce questions for readers; they do not reproduce or claim to complete the RoB 2 assessment.
Agent scope review: Cochrane 8.2.3 Table 8.2.a and 8.7 address prespecified analyses and selection of a reported result. The card keeps exploratory analysis legitimate when identified as such. (8.2.3, selection-of-reported-result signalling questions)
- cochrane-trial-bias: Chapter 8: Assessing risk of bias in a randomized trial — https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-08

Review: agent · Agent editorial and scoped source review · 2026-11-12T17:42:00Z
Published: 2026-11-12T17:42:00Z

---

## Read the uncertainty beside the estimate

ID: MHC-D-RESEARCH-0060 · Version: 0.1.0 · Kind: question
Source: https://vedokrok.com/knowledge/read-the-uncertainty-beside-the-estimate

The decimal is not a promise of precision.

Knowledge source release: MHC-RPUB-20261112-07d31a69
Source corpus commit: 07d31a6904d38f9fe2189aee7b26bb065724b120
Source manifest SHA-256: d676e87a17b9210d8037fd74c3f1ea735debd384fa069ba127a87ad441aa4f20
Source projection SHA-256: 7e7900a14145b4d025c2279ceb005ff3633adb88263038886b09282e4dafb04b

### Use when

- A single effect number is being treated as an exact, settled result.

### Avoid when

- A confidence interval is not a range containing 95% of individuals, nor a guarantee that every source of bias has been covered.

### Explanation

What range accompanies the estimate, and in what units?

Does that range include outcomes that would lead to materially different decisions?

### Question

What range accompanies the estimate, and in what units?

Does that range include outcomes that would lead to materially different decisions?

### Example

A fictional estimate of two points of improvement comes with an interval from one point worse to five better. 'Exactly two points better' loses the important part.

### Check

Could your summary still distinguish a precise small effect from an uncertain potentially large one?

### Limits

- A confidence interval is not a range containing 95% of individuals, nor a guarantee that every source of bias has been covered.

### Evidence and sources

- supports: An effect estimate should be interpreted with its uncertainty interval, not only its central value. — cochrane-results-interpretation. Claim basis: guidance. Claim limit: An interval reflects statistical assumptions and does not include every possible source of bias.
Source access: selected_html_passages; accessed on: 2026-09-17; inspected sections: ["15.2", "15.3.1", "15.4"].
Source note: The separate ASA statement is used for p-value interpretation. No probability-of-the-null interpretation is adopted from loose explanatory language.
Agent scope review: Cochrane 15.3.1 supports presenting an estimate with its uncertainty interval. The hypothetical training example does not interpret the interval as an individual prediction. (15.3.1)
- cochrane-results-interpretation: Chapter 15: Interpreting results and drawing conclusions — https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-15

Review: agent · Agent editorial and scoped source review · 2026-11-12T17:42:00Z
Published: 2026-11-12T17:42:00Z
Related (useful_with): https://vedokrok.com/knowledge/do-not-turn-a-p-value-into-a-truth-percentage

---

## Do not turn a p-value into a truth percentage

ID: MHC-D-RESEARCH-0061 · Version: 0.1.0 · Kind: anti_pattern
Source: https://vedokrok.com/knowledge/do-not-turn-a-p-value-into-a-truth-percentage

Subtracting from one does not turn a test into a belief meter.

Knowledge source release: MHC-RPUB-20261112-07d31a69
Source corpus commit: 07d31a6904d38f9fe2189aee7b26bb065724b120
Source manifest SHA-256: d676e87a17b9210d8037fd74c3f1ea735debd384fa069ba127a87ad441aa4f20
Source projection SHA-256: 7e7900a14145b4d025c2279ceb005ff3633adb88263038886b09282e4dafb04b

### Use when

- Someone translates p = 0.03 as a 97% chance that a claim is true.

### Avoid when

- Properly used statistical tests remain useful. Replacing them with intuition is not the lesson.

### Explanation

A p-value concerns data under a specified statistical model. It is not the probability that the hypothesis is true, and it does not tell you whether an effect is large or useful.

### Question

What are the estimate, uncertainty, design and practical consequence behind this number?

### Recognition

A p-value concerns data under a specified statistical model. It is not the probability that the hypothesis is true, and it does not tell you whether an effect is large or useful.

### Example

A tiny improvement in a huge sample may have a small p-value while changing little about the practical choice.

### Check

Does your conclusion use the result's size and context, not a converted truth score?

### Limits

- Properly used statistical tests remain useful. Replacing them with intuition is not the lesson.

### Evidence and sources

- supports: A p-value is not the probability that a hypothesis is true and does not measure effect size or importance. — asa-p-value-principles. Claim basis: guidance. Claim limit: This does not make properly used p-values meaningless.
Source access: full_text; accessed on: 2026-09-17; inspected sections: ["Three-page statement release; six principles on PDF page 2 visually checked"].
Source note: A methodological statement from 2016, not new empirical research or a ban on significance testing.
Agent scope review: ASA principles 2, 3 and 5 reject hypothesis-probability and effect-size interpretations of a p-value. The draft preserves the role of properly used statistical tests. (Six principles, numbers 2, 3 and 5)
- asa-p-value-principles: American Statistical Association Releases Statement on Statistical Significance and P-Values — https://www.amstat.org/asa/files/pdfs/p-valuestatement.pdf

Review: agent · Agent editorial and scoped source review · 2026-11-12T17:42:00Z
Published: 2026-11-12T17:42:00Z

---

## Give the number needed to treat its conditions back

ID: MHC-D-RESEARCH-0062 · Version: 0.1.0 · Kind: concept
Source: https://vedokrok.com/knowledge/give-the-number-needed-to-treat-its-conditions-back

The number travels badly without its luggage.

Knowledge source release: MHC-RPUB-20261112-07d31a69
Source corpus commit: 07d31a6904d38f9fe2189aee7b26bb065724b120
Source manifest SHA-256: d676e87a17b9210d8037fd74c3f1ea735debd384fa069ba127a87ad441aa4f20
Source projection SHA-256: 7e7900a14145b4d025c2279ceb005ff3633adb88263038886b09282e4dafb04b

### Use when

- An NNT is advertised as an intrinsic score that can be compared across unrelated studies.

### Avoid when

- It does not identify the individual who benefits, promise one benefit in every group of 50, or summarize harms and burdens.

### Explanation

An NNT describes an expected additional outcome compared with an alternative, over a stated period. Keep the outcome, baseline population and timeframe beside it.

### Example

In a fictional comparison, 6 events versus 4 per 100 over one year gives an absolute reduction of 0.02 and an NNT of 50 for that event and period.

### Check

Could the reader tell what additional outcome the number represents and against which alternative?

### Limits

- It does not identify the individual who benefits, promise one benefit in every group of 50, or summarize harms and burdens.

### Evidence and sources

- supports: Risk and NNT summaries depend on the comparison, population and observation period. — cochrane-results-interpretation. Claim basis: technical_definition. Claim limit: An NNT is an expected comparative frequency, not a prediction of which individual benefits.
Source access: selected_html_passages; accessed on: 2026-09-17; inspected sections: ["15.2", "15.3.1", "15.4"].
Source note: The separate ASA statement is used for p-value interpretation. No probability-of-the-null interpretation is adopted from loose explanatory language.
Agent scope review: Cochrane 15.4.2 links NNT to the comparator, outcome and follow-up. The fictional arithmetic is correct and the boundary rules out individual predictions and guarantees. (15.2; 15.4)
- cochrane-results-interpretation: Chapter 15: Interpreting results and drawing conclusions — https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-15

Review: agent · Agent editorial and scoped source review · 2026-11-12T17:42:00Z
Published: 2026-11-12T17:42:00Z

---

## Find out which way the scale points

ID: MHC-D-RESEARCH-0063 · Version: 0.1.0 · Kind: checklist
Source: https://vedokrok.com/knowledge/find-out-which-way-the-scale-points

Some scales reward going down.

Knowledge source release: MHC-RPUB-20261112-07d31a69
Source corpus commit: 07d31a6904d38f9fe2189aee7b26bb065724b120
Source manifest SHA-256: d676e87a17b9210d8037fd74c3f1ea735debd384fa069ba127a87ad441aa4f20
Source projection SHA-256: 7e7900a14145b4d025c2279ceb005ff3633adb88263038886b09282e4dafb04b

### Use when

- A change score is described as better simply because its number increased.

### Avoid when

- Do not invent a meaningful-change threshold from the scale range. That requires instrument- and context-specific evidence.

### Explanation

Identify the instrument, units and meaning of a higher score.
For a change score, check which time point was subtracted. For a group difference, check which group was subtracted.
Preserve that direction when summarizing or combining results.

### Checklist

- Identify the instrument, units and meaning of a higher score.
- For a change score, check which time point was subtracted. For a group difference, check which group was subtracted.
- Preserve that direction when summarizing or combining results.

### Example

A higher satisfaction score and a higher difficulty score can move together while meaning opposite things for the learner.

### Check

Does the sign of the number match the words 'better' or 'worse' in your summary?

### Limits

- Do not invent a meaningful-change threshold from the scale range. That requires instrument- and context-specific evidence.

### Evidence and sources

- supports: Outcome scales can point in opposite directions; standardization does not automatically fix that. — cochrane-effect-measures. Claim basis: technical_definition. Claim limit: Scale interpretation requires the specific instrument, not its name alone.
Source access: selected_html_passages; accessed on: 2026-09-17; inspected sections: ["6.1-6.2", "6.4.1", "6.5.1 and MECIR C61"].
Source note: Handbook methodological guidance, not a recent intervention trial. Numerical examples in the elements are invented teaching examples.
Agent scope review: Cochrane 6.5.1 and MECIR C61 require aligning scale directions before combining measurements. The checklist also preserves the separate issue of a meaningful-change threshold. (6.5.1; MECIR C61)
- cochrane-effect-measures: Chapter 6: Choosing effect measures and computing estimates of effect — https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-06

Review: agent · Agent editorial and scoped source review · 2026-11-12T17:42:00Z
Published: 2026-11-12T17:42:00Z
Related (use_before): https://vedokrok.com/knowledge/look-behind-the-pooled-average

---

## Look behind the pooled average

ID: MHC-D-RESEARCH-0064 · Version: 0.1.0 · Kind: question
Source: https://vedokrok.com/knowledge/look-behind-the-pooled-average

A summary is not a promise that all its ingredients agree.

Knowledge source release: MHC-RPUB-20261112-07d31a69
Source corpus commit: 07d31a6904d38f9fe2189aee7b26bb065724b120
Source manifest SHA-256: d676e87a17b9210d8037fd74c3f1ea735debd384fa069ba127a87ad441aa4f20
Source projection SHA-256: 7e7900a14145b4d025c2279ceb005ff3633adb88263038886b09282e4dafb04b

### Use when

- A meta-analysis is presented as one effect that should occur in every setting.

### Avoid when

- Do not select the study you like best or delete inconvenient outliers. A few studies may not support a reliable explanation of variation.

### Explanation

How do the included populations, methods and effects differ?

Is a common summary meaningful, and how does the review describe uncertainty and between-study variation?

### Question

How do the included populations, methods and effects differ?

Is a common summary meaningful, and how does the review describe uncertainty and between-study variation?

### Example

Studies of intensive supported courses are pooled with brief self-guided lessons. The overall average does not tell you which delivery you would actually receive.

### Check

Can you describe the differences the review found and any uncertainty about what explains them?

### Limits

- Do not select the study you like best or delete inconvenient outliers. A few studies may not support a reliable explanation of variation.

### Evidence and sources

- supports: A pooled average can conceal important between-study variation; random-effects analysis does not erase it. — cochrane-meta-analysis. Claim basis: guidance. Claim limit: Explaining variation reliably may be impossible with a few small studies.
Source access: selected_html_passages; accessed on: 2026-10-01; inspected sections: ["10.2", "10.10.1-10.10.4", "10.11.3", "10.11.1-10.11.3.1", "MECIR Box 10.11.a", "10.11.5.2-10.11.5.6", "10.11.6"].
Source note: The chapter reports a November 2024 update. No single heterogeneity threshold is promoted as decisive. Separate source-access audit (2026-10-01): Chapter 10 still identifies November 2024 and version 6.5. The 2026-10-01 check covers its citation and sections 10.11.1-10.11.3.1, 10.11.5.2-10.11.5.6 and 10.11.6. The older 10.2/10.10 reading history is retained, not newly appraised. Overall Handbook updates do not change this chapter’s stated revision. No single heterogeneity threshold is promoted as decisive.
Agent scope review: Cochrane 10.10.1–10.10.4 describe between-study variation and the limits of a random-effects summary. The example concerns the transfer of training formats. (10.10.1-10.10.4)
- cochrane-meta-analysis: Chapter 10: Analysing data and undertaking meta-analyses — https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-10

Review: agent · Agent editorial and scoped source review · 2026-11-12T17:42:00Z
Published: 2026-11-12T17:42:00Z

---

## Write the applicability gap before adopting the result

ID: MHC-D-RESEARCH-0065 · Version: 0.1.0 · Kind: template
Source: https://vedokrok.com/knowledge/write-the-applicability-gap-before-adopting-the-result

Relevant is not the same as identical, and neither word should be guessed.

Knowledge source release: MHC-RPUB-20261112-07d31a69
Source corpus commit: 07d31a6904d38f9fe2189aee7b26bb065724b120
Source manifest SHA-256: d676e87a17b9210d8037fd74c3f1ea735debd384fa069ba127a87ad441aa4f20
Source projection SHA-256: 7e7900a14145b4d025c2279ceb005ff3633adb88263038886b09282e4dafb04b

### Use when

- A study concerns people or conditions that differ from the decision in front of you.

### Avoid when

- Differences need interpretation, not automatic rejection. Clinical applicability should be discussed with an appropriate professional.

### Explanation

The study tested [intervention] against [comparison] in [population] for [outcome]. My situation differs in [feature]. That difference may matter because [reason or uncertainty].

### Template

The study tested [intervention] against [comparison] in [population] for [outcome]. My situation differs in [feature]. That difference may matter because [reason or uncertainty].

### Example

Expert-supervised training may not transfer unchanged to a self-guided app. Identify the missing support instead of borrowing the study's result for the app.

### Check

Is the transfer assumption visible and open to challenge?

### Limits

- Differences need interpretation, not automatic rejection. Clinical applicability should be discussed with an appropriate professional.

### Evidence and sources

- supports: Applicability can differ with the population, intervention, comparator and outcomes studied. — cochrane-evidence-certainty. Claim basis: guidance. Claim limit: A difference requires interpretation; it does not automatically invalidate all transfer.
Source access: selected_html_passages; accessed on: 2026-09-17; inspected sections: ["14.1.1-14.1.2", "14.2.1", "14.2.2 indirectness and imprecision"].
Source note: GRADE ratings belong to bodies of evidence for specified outcomes; the corpus does not assign new GRADE ratings through these cards.
Agent scope review: Cochrane 14.2.2 Table 14.2.b supports checking population, intervention, comparator and outcome before applying findings. The card sends clinical applicability to a professional. (14.2.2, indirectness)
- cochrane-evidence-certainty: Chapter 14: Completing Summary of findings tables and grading the certainty of the evidence — https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-14

Review: agent · Agent editorial and scoped source review · 2026-11-12T17:42:00Z
Published: 2026-11-12T17:42:00Z

---

## A changed biomarker is not automatically a changed life

ID: MHC-D-RESEARCH-0066 · Version: 0.1.0 · Kind: concept
Source: https://vedokrok.com/knowledge/a-changed-biomarker-is-not-automatically-a-changed-life

The dashboard can move before the destination is established.

Knowledge source release: MHC-RPUB-20261112-07d31a69
Source corpus commit: 07d31a6904d38f9fe2189aee7b26bb065724b120
Source manifest SHA-256: d676e87a17b9210d8037fd74c3f1ea735debd384fa069ba127a87ad441aa4f20
Source projection SHA-256: 7e7900a14145b4d025c2279ceb005ff3633adb88263038886b09282e4dafb04b

### Use when

- A product's benefit is inferred from a laboratory measure rather than how people feel, function or survive.

### Avoid when

- Some surrogate endpoints are well validated and useful. Do not reject all biomarkers or change care because a direct long-term endpoint was not measured.

### Explanation

A surrogate stands in for an outcome of interest. Ask what evidence validates that substitution in this context; an attractive mechanism alone does not finish the argument.

### Example

A fictional intervention changes a measured molecule. Without suitable validation or outcome evidence, that is not yet proof that users feel better or live longer.

### Check

Can you distinguish what was measured from the benefit being promised?

### Limits

- Some surrogate endpoints are well validated and useful. Do not reject all biomarkers or change care because a direct long-term endpoint was not measured.

### Evidence and sources

- supports: A surrogate endpoint substitutes for a clinical outcome and requires context-specific evidence of its predictive value. — fda-surrogate-endpoints. Claim basis: technical_definition. Claim limit: Some surrogates are well validated; others remain candidates.
Source access: selected_html_passages; accessed on: 2026-09-17; inspected sections: ["What are clinical trial endpoints?", "How can biomarkers be used as surrogate endpoints?"].
Agent scope review: FDA defines a surrogate as a substitute for a direct measure of how a patient feels, functions or survives and explains that its validation depends on context. The card gives a fictional example and explicitly rules out changing care on this basis. (Clinical trial endpoints; biomarkers as surrogate endpoints)
- fda-surrogate-endpoints: Surrogate Endpoint Resources for Drug and Biologic Development — https://www.fda.gov/drugs/development-resources/surrogate-endpoint-resources-drug-and-biologic-development

Review: agent · Agent editorial and scoped source review · 2026-11-12T17:42:00Z
Published: 2026-11-12T17:42:00Z
Related (useful_with): https://vedokrok.com/knowledge/keep-evidence-certainty-attached-to-its-outcome

---

## Was safety measured, or merely not mentioned?

ID: MHC-D-RESEARCH-0067 · Version: 0.1.0 · Kind: question
Source: https://vedokrok.com/knowledge/was-safety-measured-or-merely-not-mentioned

Silence has no sample size.

Knowledge source release: MHC-RPUB-20261112-07d31a69
Source corpus commit: 07d31a6904d38f9fe2189aee7b26bb065724b120
Source manifest SHA-256: d676e87a17b9210d8037fd74c3f1ea735debd384fa069ba127a87ad441aa4f20
Source projection SHA-256: 7e7900a14145b4d025c2279ceb005ff3633adb88263038886b09282e4dafb04b

### Use when

- A study with no prominent harm result is described as proof of complete safety.

### Avoid when

- Do not infer that an unreported harm occurred. Separate missing evidence, observed events and a demonstrated causal risk.

### Explanation

Which adverse outcomes were actively collected, for how long and in how many people?

Could this design detect uncommon, delayed or population-specific problems?

### Question

Which adverse outcomes were actively collected, for how long and in how many people?

Could this design detect uncommon, delayed or population-specific problems?

### Example

A short, small trial reports task performance but says little about unwanted effects. Its duration cannot establish what happens after years of use.

### Check

Does the safety claim stay within what was observed and could reasonably be detected?

### Limits

- Do not infer that an unreported harm occurred. Separate missing evidence, observed events and a demonstrated causal risk.

### Evidence and sources

- supports: Important harms may require evidence beyond short efficacy trials and should not disappear from the question. — cochrane-evidence-certainty. Claim basis: guidance. Claim limit: A missing harm report is neither proof of safety nor proof that a harm occurred.
Source access: selected_html_passages; accessed on: 2026-09-17; inspected sections: ["14.1.1-14.1.2", "14.2.1", "14.2.2 indirectness and imprecision"].
Source note: GRADE ratings belong to bodies of evidence for specified outcomes; the corpus does not assign new GRADE ratings through these cards.
Agent scope review: Cochrane 14.1.2 includes benefits and harms among outcomes important to readers. Its adverse-effects section also discusses insufficient detection in small trials and the need for separate evidence on uncommon or long-term harms. The example does not infer a harm from silence. (14.1.2, selecting benefit and harm outcomes)
- cochrane-evidence-certainty: Chapter 14: Completing Summary of findings tables and grading the certainty of the evidence — https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-14

Review: agent · Agent editorial and scoped source review · 2026-11-12T17:42:00Z
Published: 2026-11-12T17:42:00Z
Related (useful_with): https://vedokrok.com/knowledge/keep-evidence-certainty-attached-to-its-outcome

---

## Count studies, not publication wrappers

ID: MHC-D-RESEARCH-0068 · Version: 0.1.0 · Kind: protocol
Source: https://vedokrok.com/knowledge/count-studies-not-publication-wrappers

A second paper is not necessarily a second experiment.

Knowledge source release: MHC-RPUB-20261112-07d31a69
Source corpus commit: 07d31a6904d38f9fe2189aee7b26bb065724b120
Source manifest SHA-256: d676e87a17b9210d8037fd74c3f1ea735debd384fa069ba127a87ad441aa4f20
Source projection SHA-256: 7e7900a14145b4d025c2279ceb005ff3633adb88263038886b09282e4dafb04b

### Use when

- Several articles may describe the same participants and intervention.

### Avoid when

- Do not discard secondary reports automatically; they may correct or complete the main account. Overlap can require author clarification.

### Explanation

Compare trial identifiers, recruitment dates, sites and participant details across reports.
Link reports from the same study while preserving useful additional outcomes and follow-up.

### Steps

1. Compare trial identifiers, recruitment dates, sites and participant details across reports.
2. Link reports from the same study while preserving useful additional outcomes and follow-up.

### Example

A main result, a subgroup article and a later follow-up all use one trial cohort. They offer three reports, not three independent replications.

### Check

Can every claimed replication be traced to a genuinely separate study?

### Limits

- Do not discard secondary reports automatically; they may correct or complete the main account. Overlap can require author clarification.

### Evidence and sources

- supports: Several papers can describe one study and should not be counted as independent replications. — cochrane-study-reports. Claim basis: guidance. Claim limit: Secondary reports can still add useful details or later outcomes.
Source access: selected_html_passages; accessed on: 2026-09-17; inspected sections: ["5.2; MECIR C42; linking multiple reports"].
Agent scope review: Cochrane 5.2.1 and MECIR C42 require identifying reports of the same study without discarding unique information. The card gives a distinct provenance check for counts of replications. (5.2; MECIR C42)
- cochrane-study-reports: Chapter 5: Collecting data — https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-05

Review: agent · Agent editorial and scoped source review · 2026-11-12T17:42:00Z
Published: 2026-11-12T17:42:00Z
Related (use_before): https://vedokrok.com/knowledge/look-behind-the-pooled-average
