In today’s New York Times, Robert Shiller argues that the recent crisis should precipitate a movement to collect better data that could help in predicting similar events in the future. He likens the problem to predicting hurricanes; something that was impossible at one time, but which we can now do accurately because of better data. Shiller’s point is a sharp contrast to the perspective of Duncan Watts espoused in his recent book, Everything is Obvious (Once You know the Answer). Watts argues that no matter how good the data, predicting many things is just impossible, in part because its impossible to even know what it is you should be predicting. Shiller’s opinion on this matter has an extra sense of authority of course, because he is one of the few economists who saw the housing bubble/credit crunch/financial crisis coming and said something about it. He also forecasted the stock market collapse of 2001.
So, who’s right? Is Shiller just another example of luck and large numbers? After all, a lot of people are making a lot of predictions out there, so we shouldn’t be surprised that someone got it right. And Watts’ argument that in many cases we can’t know what we should have been predicting until we put the outcome in a context has merit. Both are right to an extent. I do believe the crisis was predictable, and that Shiller wasn’t just lucky in foreseeing it, but I also agree with Watts that more data isn’t the full solution. We also need to know where to look — how to put that data together. We can forecast hurricanes today not just because we have better data, but because we have better models and a better understanding of how the weather works. It’s a little surprising that Shiller doesn’t point this out, because he is also well-known for arguing that economists should in part be looking in a different direction than they have in the past. Namely, our economic models need to incorporate more social and psychological effects (see his recent book with George Akerlof, Animal Spirits).