There are a couple of authors who I like to “follow.” By that I mean, I usually get any new book they publish. Michael Lewis is one of those authors. It all started with Moneyball, even though I’m a dyed-in-the-wool NON sports fan. And so The Undoing Project was a must on my reading list.
As is almost always the case with Lewis, The Undoing Project is a set of stories leading the reader to draw his or her own conclusions. I also kept wondering the whole time — “What are we undoing?”
From what I read, what is getting undone is the idea that we can rely solely on data analysis. Turns out that some good old human intuition and interpretation is often useful. In other words, getting back to the understanding of the imperfections and unpredictability of the human mind.
I think the point is that we can never underestimate the way human beings can “screw up the data.” Not even on purpose! So as we do the analysis of the “big data” we collect, we have to be prepared to allow for human foibles.
Undoing Fixed Mind
Trying to insert human foibles back into analysis of cold data is difficult. As Lewis explained in the coda to his book, the data folks (economists) and the human mind folks (psychologists) didn’t like each other. So collaboration was problematic at best.
By the early 1990s a lot of people thought it was a good idea to bring together psychologists and economists, to allow them to get to know each other better. But as it turned out, they didn’t particularly want to know each other better. Economists were brash and self-assured. Psychologists were nuanced and doubtful. “Psychologists as a rule will only interrupt a presentation for clarification,” says psychologist Dan Gilbert. “Economists will interrupt to show how smart they are.” “In economics it is completely normal to be rude,” says economist George Loewenstein. “We tried to create a psychology and economics seminar at Yale. We had our first meeting. The psychologists came out completely bruised. We never had a second meeting.”
It turns out that the human mind is just horrible at seeing what it doesn’t expect to see. The psychologists Daniel Kahneman and Amos Tversky were what Miles Shore, Professor of Psychiatry at the Harvard Medical school, referred to as a fertile pair. Together they were exceptional according to Kahneman.
What their work showed, and what Lewis is pointing out, is that we have to undo our dependence only on cold data analysis and include human nature in our analysis. By doing so, we will get much closer to accurate predictions and understanding of the real probabilities.