Write Your Predictions Down With a Number, or Hindsight Rewrites Them

Write each prediction down before the outcome is known, with a probability, an absolute date and a rule for what counts as a yes. Without that line, memory quietly drags your old forecast toward whatever happened. Baruch Fischhoff and Ruth Beyth measured that drift around Nixon's 1972 trips, and in 1974 Fischhoff gave the core of this advice.

In October 2014, Kevin Kelly wrote in Wired: "My prediction: By 2024, Google's main product will not be search but AI."¹ His 2016 book The Inevitable prints the same sentence with one change: "By 2026, Google's main product will not be search but AI."² Both versions sit beside the same phrase, "With another 10 years". Nobody moved a goalpost. Another ten years is a relative horizon, and it re-anchored to each publication date.

On 12 October 2026 the book excerpt reached Hacker News, and readers began grading it. One reader called the Google line "Nah. Not by any common-sense measure at least." and scored the set three out of five. Another split the list into 16 claims and counted about eight hits.³

Same printed text, different grades.

Kelly's forecast was written down — which already puts it ahead of most predictions anyone makes. It still lacked three things a fair grade needs: a fixed date, a test for what counts, and a number. Each gap has a fix, and the fixes are older than the forecast.

What does hindsight do to a prediction you never wrote down?

It moves it. People who wrote down probabilities before Nixon's 1972 trips to China and the USSR later remembered those numbers as closer to what actually happened. Events they had called nearly impossible did occur, yet in memory their past held few surprises. The forecast in your head is a revised edition, and nobody marked the revisions.

Fischhoff described the study in his own 1974 report for the Office of Naval Research. Subjects were asked on the eve of the trips "to estimate the probability of various possible outcomes of the trips". Then: "Two weeks to six months after the trips' completion, these same subjects were asked to remember as best they could, their own original predictions."⁴

The result ran one way — toward the outcome. In his words, "subjects remembered having given higher probabilities than they actually had to events believed to have occurred" — and lower ones to events they believed had not.⁴

The surprises vanished too. "Their original predictions showed that they were too often surprised—many highly unlikely or impossible events (assigned probability = 0%) did occur." Remembered, the same past held "too few surprises".⁴

Nobody in the study was lying. They were remembering.

The full paper, with Beyth as co-author, appeared in 1975. A 2007 review in Social Cognition summarises it this way: "People exhibited memory distortions—that is, they thought they had made predictions that were closer to the actual outcomes than they in fact had been."⁵

Is hindsight bias one study, or a pattern?

A pattern. By 2007, more than 150 journal articles and book chapters, plus two meta-analyses, had studied it. A 2004 meta-analysis pooled 95 studies and found a clear overall effect, and studies built to reduce the bias did not shrink it. Resolving to remember honestly is not a fix. A written record is.

Hartmut Blank, Jochen Musch and Rüdiger Pohl counted the literature in their 2007 review: "more than 150 journal articles and book chapters", plus two meta-analyses.⁵ The more recent of those meta-analyses, by Erin Guilbault and colleagues, reported that "A total of 95 studies (83 published and 12 unpublished) were included, and 252 independent effect sizes". The pooled result: "the overall mean effect size was Md = .39 with a 95% confidence interval of .36 to .42".⁶

The line that matters for anyone keeping notes is this one: "studies that included manipulations to reduce hindsight bias did not produce lower effect sizes".⁶ The review agrees. Warnings "have met with only limited success", and "typically one has to go to great lengths to get a sizable reduction of the bias".⁵

The same review names the cost. "If we think that we knew it all along, we may not find anything wrong with our foregoing analyses and decisions."⁵

Willpower is not in the data.

Two honest limits apply. Guilbault's team found that "effect sizes of studies that used almanac questions were larger than effect sizes of studies that used real-world events or case histories".⁶ Trivia inflates the bias. The Nixon study did not use trivia — it used real political events and people's own written numbers.

And some researchers argue that hindsight bias "should actually be seen as a natural byproduct of adaptive learning".⁵ That is fair — updating what you believe about the world is useful. Updating what you believe you used to believe is what wrecks the grade.

What should a written prediction contain?

Four parts: the event, a probability written as a number, an absolute resolve-by date, and a test that says what counts as yes. Add the date you wrote it and a line of reasoning. Fischhoff's own advice asked for both the predictions and "the considerations which guided them" — the why belongs in the record.

His closing advice, from the 1974 report, is the whole method in one sentence: "If you have hopes of learning how to forecast better, it might well pay to keep a written record of your own predictions and the considerations which guided them—a record which can be evaluated in the light of reality (what eventually happens)."⁴

On the same page he imagines the question a reader cannot answer without one: "Was I really certain that Mao and Nixon would get together?"

Each part closes a specific gap:

1. A number, not a word. "Probably" can be graded generously after the fact. 70% cannot.
2. An absolute date. Kelly's "another 10 years" landed on 2024 in one printing and 2026 in the other. by 2027-06-30 does not move.
3. A resolution test. resolves YES if … names the observable fact before anyone knows the answer. The HN graders disagreed because "main product" had no such test.
4. The date of writing, plus the why. A forecast made a month out is easier than one made a year out. The reasons are what you study when it misses.

Dan Luu graded the published forecasts of several well-known futurists. He named what vagueness buys a forecaster — "it's much easier to make vague predictions sound roughly correct than to make precise predictions correct". Of forecasts that only said something "may" happen, he wrote: "Such a prediction is, of course, unfalsifiable because the statement is true whether or not the event happens."⁷

If the reasoning line is worth writing, so is its mirror: what would change your mind. That is the companion habit in writing down the fact that disagrees with you first.

How do you score your own predictions?

Use the Brier score. For each prediction, subtract the outcome (1 if it happened, 0 if not) from your probability, square the difference, then average across every prediction. Zero is perfect. Writing 50% every time scores 0.25, so an average below 0.25 means your numbers carried real information about the world.

Glenn Brier, of the U.S. Weather Bureau, published the score in Monthly Weather Review in 1950.⁸ His aim was a score that rewards honesty.

Some verification schemes, he wrote, "may lead the forecaster to forecast something other than what he thinks will occur". His does not. A forecaster who gambles on certainty "is fooling nobody but himself if he thinks he can beat the verification system by putting down only zeros and unities when his forecasting skill does not justify such statements of extreme confidence".⁸

One detail trips people up. Brier's original formula sums over both outcomes — rain and no rain — so it runs from 0 to 2. The common one-outcome form used in this post runs from 0 to 1. For a yes/no question, it is exactly half of Brier's number.

His own worked example used 10 real rain forecasts. It rained 3 out of 10 times, and the forecasts scored 0.19. A flat 0.3 every day — the right base rate, known only afterwards — would have scored 0.42; a climatological 0.2, 0.44.⁸ In this post's form, halve each: about 0.10 against 0.21 and 0.22. Skill is the gap between you and the lazy baseline.

Here is the same arithmetic on five personal forecasts:

The one miss at 70% costs more than the other four put together — 0.49 against 0.25. That is the design: confident misses are expensive, so honest numbers win over time.

Does writing predictions down make you better at them?

Not by itself. In a four-year forecasting tournament, under an hour of training improved accuracy by 6 to 11%, but simply having taken part the year before did not. The written record is the measuring tape, not the coach. Improvement tracked training and effortful practice — returning to a question and updating.

Welton Chang, Eva Chen, Barbara Mellers and Philip Tetlock studied forecasters in "a four-year series of geopolitical forecasting tournaments sponsored by the U.S. intelligence community". Their headline: "Although the training lasted less than one hour, it consistently improved accuracy (Brier scores) by 6 to 11% over the control condition."⁹

The contrary finding sits in the same paper: "a subject merely having prior experience in a year of the tournament did not yield significant improvements in Brier score", and "simply doing the minimum possible to remain in the experiment was not enough to boost accuracy."⁹

What did predict accuracy was what the authors called targeted practice, measured as the average number of forecasts per question. They put the general lesson plainly: "Although feedback is necessary for improvement, it is unlikely to be sufficient."⁹

Logging alone is attendance.

An earlier paper by Mellers and colleagues, on a two-year geopolitical forecasting tournament, adds detail on what the training did: "Probability training corrected cognitive biases, encouraged forecasters to use reference classes". The researchers also tracked people — "Tracking placed the highest performers (top 2% from Year 1) in elite teams that worked together." Training, teaming and tracking together "improved both calibration and resolution".¹⁰

Calibration is the part a personal log can measure — of everything you marked 70%, about 70% should come true. That check needs only your own lines and an honest tally.

How many predictions before your score means anything?

More than ten. In a simple simulation, a perfectly calibrated forecaster scored worse than a reckless one who only ever wrote 5% or 95% in 36% of ten-forecast samples. At 100 forecasts that fell to 12%, and at 300 to 2%. Judge your score after dozens of predictions, never after one bad month.

The simulation was written for this post — it is not a study. It draws 20,000 runs of forecasts at true probabilities between 10% and 90%. One forecaster writes the true probability. The other rounds everything to 5% or 95%. Over many forecasts the calibrated one averages 0.175 and the overconfident one about 0.23. Over ten forecasts, the calibrated forecaster's own score ranges from 0.077 to 0.292 across the middle 90% of runs.

A single score of 0.29 tells you almost nothing. Two hundred forecasts start to.

Comparisons between people are harder still. Jared Park, writing on the blog of Forecast, a social prediction-market app, put the problem in one question: "What about question difficulty? It is, for example, a lot easier to predict who will lead the senate majority in 2021 today compared with a month ago."¹¹ A score depends on how hard the questions were and when they were answered. That is why the line carries the date you wrote it.

The fair comparison is the one Brier made: you against the lazy baseline, on the same questions. On your own log, that baseline is 0.25.

What does a one-line prediction log look like?

One dated line per forecast, written before the outcome, holding the question, a number, a resolve-by date and a yes-test. When the date passes, add a new dated line with the result and the score. Never edit the original. Plain text is enough, and a year of lines fits in one file.

2026-10-13 | Q: Will the migration finish by 2027-03-31? | 70% | resolves YES if: old system switched off in production | why: two of five services left; one blocked on vendor
2026-10-13 | Q: Will we keep the current CI provider through 2027-12-31? | 90% | resolves YES if: no contract change announced | why: three-year contract, no complaints logged

2027-04-01 | resolved: migration — NO (two services still live) | score: (0.70 − 0)² = 0.49 | miss note: vendor block took 11 weeks, not 3

The resolution goes on its own line because the original must stay exactly as written. That rule is the argument of keeping a daily note as an append-only log. It applies here with more force — an edited forecast is exactly the revision hindsight wants to make.

The miss note answers Chang's warning. A score is feedback, and feedback alone is not enough; the note records what the reasoning got wrong, while the reasons are still on the page beside it.

A decision log records what was chosen. A prediction log records what you expected to happen, so it can be graded. Written forecasts do get graded, sometimes harshly — see how expert time-savings estimates compared with a measured trial in Your Minutes Are the Other Bill.

At the end of a year, search the file for resolved: and average the scores. Then read every line that scored above 0.25.

That reading is the practice.

Frequently asked questions

Why should I write down my predictions?

Because memory edits them. In Fischhoff and Beyth's study of Nixon's 1972 trips, people later remembered giving higher probabilities to events that happened than they actually had. A meta-analysis of 95 studies found that efforts to reduce this bias did not lower it. A dated line written before the outcome is the only version hindsight cannot revise.

How do I keep a prediction journal?

Write one dated line per forecast: the question, a probability as a number, an absolute resolve-by date, a test for what counts as yes, and a short reason. When the date passes, add a new dated line with the outcome and score. Never edit the original. Review the misses at least once a year.

What is a Brier score in plain English?

It is the average squared gap between your probability and what happened. Write 0.8 for an event that occurs and you score (0.8 − 1)² = 0.04; if it does not occur, 0.64. Zero is perfect. Glenn Brier's 1950 two-category version runs from 0 to 2; the common one-outcome form runs from 0 to 1.

What is a good Brier score?

Below 0.25 on your own questions, at minimum, since writing 50% for everything scores exactly 0.25. Beyond that there is no universal benchmark: scores depend on how hard the questions were and how far ahead you forecast. Compare yourself against that baseline over dozens of forecasts rather than against other people.

What is an example of hindsight bias?

Before Nixon's 1972 trips to China and the USSR, people wrote down probabilities for possible outcomes. Two weeks to six months later, asked to recall those numbers, they remembered giving higher probabilities to the events they believed had happened. Many events they had rated at 0% occurred, yet in memory they had been rarely surprised.

Does keeping a prediction log make you a better forecaster?

Not on its own. In a four-year forecasting tournament, prior experience alone did not significantly improve Brier scores, while under an hour of probability training improved them by 6 to 11%. Effortful practice, measured as more forecasts per question, also predicted accuracy. The log makes improvement measurable; reviewing its misses is the practice.

A prediction you did not write down is not a prediction. It is a memory, and memory works for the outcome.

The method is not new. It is Baruch Fischhoff's, written down in 1974.

mnmnote.com keeps each day as a dated plain-Markdown note on your own device, so a year of forecast lines is one search away.