BooksLessons

Superforecasting

Philip E. Tetlock and Dan Gardner

Science & Clear Thinking

Superforecasting

Philip E. Tetlock and Dan Gardner · 2015

Best for: Analysts, investors, leaders and anyone who wants to predict better.

1 min read · 3 lessons

The core thesis

Good forecasting is a learnable skill, not a gift.

Summary

Philip Tetlock spent years studying expert predictions and found many experts did little better than chance. Then he ran the Good Judgment Project, part of a forecasting tournament funded by a US intelligence research agency.

A small group of ordinary volunteers, which he called superforecasters, consistently beat others, including some intelligence analysts with access to classified information.

With journalist Dan Gardner, Tetlock explains what made them so good: their habits of mind, not genius.

Why it matters

Superforecasting showed that good judgment can be measured and improved. It is widely used by analysts, investors and policymakers.

Key takeaways

01

Forecasting is a skill.

Good forecasters improve through practice and feedback. Tetlock found that people who kept score and studied their misses improved year after year.

02

Break problems down.

Split big questions into smaller, answerable parts. Fermi-style estimates turn a scary unknown into a chain of smaller guesses you can reason about.

03

Start with base rates.

Begin with how often similar events happen, then adjust for specifics. This is the outside view: what usually happens to cases like this one, before you look at the details.

04

Update incrementally.

Change your mind often, but in small steps, as new information comes in. Big swings in opinion are a warning sign. Steady, small corrections track reality better.

05

Be precise.

Use specific probabilities rather than vague words like likely. Saying 70 percent instead of 'probably' lets you be checked, and being checked is how you get better.

06

Keep score and reflect.

Track forecasts, measure accuracy and learn from mistakes. Superforecasters ran honest postmortems on their hits and misses alike, asking what they got right for the wrong reasons.

Action plan

Lessons

  1. 01

    Break big questions into small ones.

    Superforecasters split impossible questions into pieces they can estimate, the way physicist Enrico Fermi did.

  2. 02

    Update often and in small steps.

    The best forecasters change their minds frequently, but by small amounts as evidence arrives.

  3. 03

    Keep score.

    You cannot improve what you do not measure. Write predictions down with probabilities and check them later.

Key terms and mental models

Superforecaster
A person who consistently makes highly accurate predictions.
Brier score
A measure of forecast accuracy.
Fermi estimate
Breaking a big question into smaller guesses.

Who should read it / Who should skip it

Who should read it

Analysts, investors, leaders and anyone who wants to predict better.

Who should skip it

You want simple answers about the future.

Criticisms and counterpoints

The tournament questions were mostly short-term, so the methods are harder to apply to rare or long-range events.

Compare

Compare related books
Book

Superforecasting

Superforecasting

Thinking, Fast and Slow

Cover of Thinking, Fast and Slow

Thinking, Fast and Slow

The Demon-Haunted World

The Demon-Haunted World

Core thesisGood forecasting is a learnable skill, not a gift.Human judgment has predictable flaws you can learn to catch.Science is a candle in the dark against superstition and nonsense.
Best forAnalysts, investors, leaders and anyone who wants to predict better.Anyone who makes decisions, which is everyone, especially leaders and investors.Anyone who wants to think more critically about claims they hear.
Problems it solvesDecision MakingDecision Making, OverthinkingDecision Making
Lessons344
Year201520111995