# Superforecasting - The Art and Science of Prediction Author: [[Philip Tetlock]] ## Summary This has become a must-read, and although I've put it off for a while I am glad I got around to it eventually. We all make forecasts. It's unavoidable. In almost all decisions we make there are implicit and explicit beliefs about what will or won't happen in the world. This book tells us how we can forecast better. Unfortunately it also has a lot of overlap with other recent content on decision-making. The author wrote his first book about the forecasting performance of pundits which showed that they were very good at creating high confidence narratives, but not good at predicting outcomes. The second book is the result of the good judgment project, in which people were invited to forecast sociopolitical and economic events. Surprisingly there were a group of forecasters (Superforecasters) that did much better than other methods of forecasting, including intelligence analysts with classified information. Although there are limits to how far and how accurately we can predict things, the project demonstrates that it is possible to get (much) better at forecasting. The book describes how we can get better. In order to improve we need feedback on how we are doing. To get feedback we need to measure our forecasting performance and understand what went wrong in our forecast by investigating our analysis. The first step to improving our analysis is to begin recording the thought process that goes into the analysis: the data we are looking at, the models we are using, and the conclusions we came to. Only by elucidating the analysis can we look back and discover our error. One common obstacle is the use of vague language (maybe, probably, etc) because it obfuscates what we mean and how well we predicted things. Being precise, for example using probabilities in estimating outcomes, can help us determine our performance, which can then help us improve. Forecasts can be improved by structuring the analysis better. Break the problem down into smaller problems that can be more easily answered and build the answer back up. Similar to how fermi questions should be answered. You should also start with an outside view based on a generalization. This can often be the frequency of occurrence of a comparable set of instances, but can also be a simple model such as mean reversion or 'nothing changes'. After that, start to incorporate the details of your particular question and how it differs from the generalization. Keep up to date on new information. In particular information related to the variables that are critical to your (mental) model, or information that informs you that your model is misconstructed. Staying open-minded is important when looking at new information. Don't let your personal desires get in the way of your analysis. You have to find the middle ground between overreacting to noise, and under-reacting to information that creates significant change in outcome or that demonstrates your model is wrong. Forecasters working in groups tended to outperform those working alone. When working in groups try to avoid groupthink, which happens when a common narrative is shared by the group and negatively impacts the quality of analysis. Challenging each other's assumptions and analysis is necessary but only when it does not erode trust. Trust is important within the group in order to encourage sharing and constructive criticism for the good of the analytical outcome. Asking for clarifications on specific issues is better than loaded questions that can hurt egos. At the end of the book, the author summarises the characteristics that are shared by Superforecasters. They: - see the world as stochastic where many things are possible. - are open-minded and willing to challenge their beliefs. - are curious, analytical and like to solve puzzles. - have above average intelligence. - are introspective and reflective. - value a diverse set of views. - thoughtfully update beliefs. - are aware of their cognitive and emotional biases. - have humility in knowing what they do and don't know. - have a growth mindset and look to improve themselves. ## Related - [[Psychology of Intelligence Analysis]] - [[Resources/Kindle Quotes/Market Mind Games]] - [[The Behavioral Investor]] - [[Superforecasting Fundamentals Course]] - [[Interviews with Superforecasters]]