Checklist
Define the forecast event before assigning a probability
A probability cannot rescue an event nobody defined.
Before you use it
- Use when
- People are debating whether something is 'likely' without agreeing on exactly what would count as occurring.
Keep this limit in mind
- A precise event can still be based on poor evidence; resolution clarity improves evaluability, not forecast accuracy.
An idea to explore.
Try this idea.
Inspect the example, try the idea in your situation, then check your next move.
Checklist
- A third party can determine the outcome later without asking the forecaster what they originally meant.
Why it matters
Specify the event, observation window, resolution source and ambiguous edge cases before writing the probability. For continuous outcomes, define the threshold or interval being forecast. This makes later scoring possible and prevents forecasters from changing the meaning after seeing what happened.
An example
Replace 'the rollout will probably be stable' with 'there is a 70% chance that no Sev-1 rollback-triggering incident occurs during the first seven days after full rollout.'
Check your result
A third party can determine the outcome later without asking the forecaster what they originally meant.
Save and practise this idea
My learning
Read. Recall. Repeat.
Save this idea and return to it on any device.
Keep exploring
Connected ideas
You are readingDefine the forecast event before assigning a probability
Evidence and sources
Strictly proper scoring rules are designed so that a forecaster minimizes expected loss by reporting their actual probability belief rather than strategically distorting it.
A proper score rewards honest probabilistic reporting; it does not guarantee that the underlying belief is well informed.
Proper Scoring Rules for Estimation and Forecast Evaluation · Definition and motivation of proper scoring rules
All sources (1)
Edition and review
Version 0.1.0
No review details are supplied for this edition.