Ever wonder why economists, in general, seem to always “get it wrong?” Did the thought cross your mind that they seem not to be improving in their economic forecasting but get lucky once in awhile? How about the national weather forecasters – they seem to be definitely improving even though they drive us crazy showing us storm landfall models and giving us “percent chance of precipitation” numbers. Ever wonder how reasonable people can look at exactly the same data and come up with conclusions 180 degrees apart? The answers to these and many more questions are in Nathaniel Read (Nate) Silver’s excellent book, The Signal and The Noise.
Silver introduces us to Bayes’s theorem. He does so in his now familiar comfortable style which allows those readers who are not also math wizards to understand the practical use of this probabilistic formulation. What’s important to understand is that Bayes theorem requires that we take new information and initial conditions (prior beliefs) into account so that we continuously improve the forecast or prediction. This is something that, regrettably, many of us do not do. “Bayes’s theorem requires us to state— explicitly— how likely we believe an event is to occur before we begin to weigh the evidence. It calls this estimate a prior belief.”
Armed with the knowledge of how Bayesian thinking can and should be employed whenever we attempt to predict events in nature or human affairs, Silver takes us through many examples ranging from baseball to weather forecasting to earthquake prediction to demonstrate how practical (and difficult) all of this is for the forecaster. The most interesting example for me, and the reason I came to know of Silver in the first place, was his work on political forecasts. Silver has consistently forecasted correctly all 50 states in the last two elections using his methodology. And perhaps the most vivid example of how one can go wrong by ignoring Bayes, was when one prominent political hack on a conservative news channel refused to believe the election call his station made for which way Ohio would go early on election night. He did not consider his own prior beliefs when it came to his predications. That coupled with the fact that he had much on the line in terms of donors’ money and his own reputation blinded him to the “new information” from the exit polls. He preferred to believe there was something wrong with the data. Had he been following Silver’s forecasting in the NYT, he would have known that there was only a 35% chance that his candidate would win Ohio. As Silver noted: “The most calamitous failures of prediction usually have a lot in common. We focus on those signals that tell a story about the world as we would like it to be, not how it really is. We ignore the risks that are hardest to measure, even when they pose the greatest threats to our well-being. We make approximations and assumptions about the world that are much cruder than we realize. We abhor uncertainty, even when it is an irreducible part of the problem we are trying to solve.”
My conclusion is, after reading Silver’s excellent book, that I have to do a much better job making sure I am seeking out the real signal in the noise we all endure today. I will also have to make sure I question others who claim to have found the signal in the noise. Anyone who is required to provide forecasts (sales, budgets, returns on investments, market acceptance of products, etc.) must learn how to think in Bayesian terms and separate the real signal(s) from the noise. Nate Silver can help you do just that.
Click here to see the book on Amazon.