predictor-hand-skill

Forecasting Expert Knowledge

Superforecasting Principles

Based on research by Philip Tetlock and the Good Judgment Project:

1. Triage: Focus on questions that are hard enough to be interesting but not so hard they're unknowable
2. Break problems apart: Decompose big questions into smaller, researchable sub-questions (Fermi estimation)
3. Balance inside and outside views: Use both specific evidence AND base rates from reference classes
4. Update incrementally: Adjust predictions in small steps as new evidence arrives (Bayesian updating)
5. Look for clashing forces: Identify factors pulling in opposite directions
6. Distinguish signal from noise: Weight signals by their reliability and relevance
7. Calibrate: Your 70% predictions should come true ~70% of the time
8. Post-mortem: Analyze why predictions went wrong, not just celebrate the right ones
9. Avoid the narrative trap: A compelling story is not the same as a likely outcome
10. Collaborate: Aggregate views from diverse perspectives

Signal Taxonomy

Signal Types

Signal Strength Assessment

STRONG signal (high predictive value):
  - Multiple independent sources confirm
  - Quantitative data (not just opinions)
  - Leading indicator with historical track record
  - Structural change with clear causal mechanism

MODERATE signal (some predictive value):
  - Single authoritative source
  - Expert opinion from domain specialist
  - Historical pattern that may or may not repeat
  - Lagging indicator (confirms direction)

WEAK signal (limited predictive value):
  - Social media buzz without substance
  - Single anecdote or case study
  - Rumor or unconfirmed report
  - Opinion from non-specialist