Week 2 Reflective Journal

This paper is a reflective journal about superforecasting. Based on the readings in
chapters 3 and 4 of Superforecasting by (Tetlock & Gardner, 2015), I have gained new insight
into the application of superforecasting in daily life. The reading changed the way I think about
specificity. Forecasters who make judgments of other people’s forecasts need to have accurate
information. The information needs to be numerical and not clouded by ambiguous words. This
way, it can be tested through calibration. The essence of calibration is to increase accuracy by
collecting information over time to make accurate judgments. I also learned that forecasters need
embrace more testing and experimenting through sourcing the wisdom of the crowd and using
regression to mean to identify instances where occurrences are affected by chance or luck.
Forecasters need to be able to analyze and judge forecasts, but that cannot happen if the
forecasts are ambiguous. Forecasts need to be as specific as possible to be well analyzed. Tetlock
& Gardner (2015) give an example of a statement made by a Microsoft official about the iPhone
that there was no chance that the iPhone would ever get any significant market share. AT first
glance, this statement may seem misplaced, considering the size of Apple and the revenues and
market share that the iPhone currently generates. The statement was ambiguous because it bore
no context of the specific market in question. Other forecasts made about time also need to be
specific. Forecasters operate in an environment controlled by wealth. It is hard to judge forecasts
that are not limited by time because forecasts may be right or wrong, and the truth may be
relative if it is not specified in their own words; Tetlock & Gardner (2015) state that a forecast
without a time frame is absurd. There is a certain degree of certainty that cannot be achieved
regardless of the strengths in numbers. For example, if there is a 70% chance that an event
happens and it happens, then the statistic could be viewed as accurate. However, accuracy is

3
relative because someone would argue that the statistic is misleading if the event does not
happen—a 70% chance of happening means a 30% chance that the event does not happen. There
will always be figures and probabilities in forecasting. To overcome this challenge, forecasters
need to collect a lot of information for calibration. In the example mentioned earlier, if
continuous analysis of the 70% probability shows that the forecasted event happens as often as
predicted, the forecast can be judged to be accurate. If the forecasted event happens only a few
times despite being forecasted with a 70% probability, the forecast can be judged as inaccurate.
Accurate forecasting requires accommodating and testing outliers. There are endless
probabilities in research; any limitations that forecasters have are a product of their imaginations
(Tetlock & Gardner 2015). Tetlock & Gardner (2015) further gives an example of a large group
of forecasters where 40 are better than the rest. By taking their data and analyzing it further, one
might develop a more accurate forecast through a process known as the wisdom of the crowd.
This accuracy could be better because when you increase the numbers, you multiply the sources
of information. The wisdom of the crowd method collects information dispersed within a crowd,
but none of the crowd has access to all that information.
Accurate predictions also identify the probability of luck in results. If 50 people are given
a test and 20, a mean score for the test results is done that shows 75%, it could mean that the
majority of the people who took the test understood the concepts being tested. In the previous
chapter, this is defined as system 1 reasoning. Assuming a top percentile of the performers, such
as the 20 best performing individuals, are given a subsequent test, through system 1 reasoning,
one could expect that they perform as well. If a number of the individuals perform dismally on
the second test, this should not be a surprising result to a forecaster.

4
System 1 could be inaccurate, and there could be a probability that a few of the best-performing
people had taken the test previously hence their good performance in the first test. If they
perform well as expected at the second test, a third test could be administered, and so on. This
concept is known as regression to mean. Regression may be useful in identifying outliers caused
by luck. The speed of regression is key as slow regression is observed in activities dominated by
skill, and faster regression is more associated with chance or luck (Tetlock & Gardner, 2015).

5

References

Tetlock, P.E & Gardner, D. (2015). Superforecasting: The art and science of prediction.
Retrieved from https://www.redshelf.com