Reading Superforecasting by Philip Tetlock and Dan Gardner got me thinking about, well, forecasting. The book builds upon Tetlock’s previous work on measuring the accuracy of expert predictions of the future. In a study he ran from 1984 to 2003 covering 284 experts he found that on average expert opinions were no more accurate at predicting the future than a “dart-throwing chimp.”Being an economics nerd the experts I think of when it comes to forecasting are, naturally, economists. Twice a year economists at the IMF publish forecasts of economic growth for nearly every country in the world, and they also keep track of their historical forecasts, making it easy to judge their accuracy, which is what the graph below does.
- Sort from underestimated to overestimated
This graph measures the average difference between actual growth and what the IMF predicted it would be one year in the future for each country. Countries in red grew slower on average than the IMF predicted and countries in green grew faster on average. For example, on average Iraq grew 3.84 percentage points slower than the IMF predicted. The map below shows the same data.Many countries in East Asia seem to perform much better than expected, while there are some darker red patches in Sub-Saharan Africa, West Africa, and Eastern Europe. It might seem obvious why some countries are performing better than expected (China) or worse (Iraq), but if some of these countries are performing better or worse than expected year after year, why hasn't knowledge of that history improved the accuracy of IMF forecasts for these countries? It could be that the economies of these countries are simply more unstable and thus harder to forecast. Or it could be unforeseen conflict or fortune. There's a lot of information that can be gathered from this data, and this graph and map only touch upon a small part of it. More to come.About the data and graphs:The historical IMF forecast data used was from the fall meeting before the year forecasted. For example, the fall 2010 forecast of 2011 GDP growth. This data can be downloaded on the left side of this page by clicking "Historical WEO Forecast Database". The actual values used are from the most recent IMF World Economic Outlook Database, which is on the same page as the previous link. The errors between the forecasts and actual growth values were averaged for each country to obtain the final numbers.All data analysis was done with R, the map was made with R too. The graph was make with D3. An example graph made by slnader served as the backbone of the graph in this post, in fact, it's almost the same, so thank you slnader.