Savage, The Flaw of Averages

I missed Sam L. Savage’s The Flaw of Averages: Why We Underestimate Risk in the Face of Uncertainty when it was published in hard cover in 2009, so I was delighted to see that it had been reissued in paperback (Wiley, 2012). There’s something in this book for almost everyone—well, at least for almost everyone who thinks about how to make decisions in an uncertain world. Its potential audience encompasses the statistically savvy as well as those who never cracked open a statistics book. The left-brained analyst might be attracted to it, but he’ll be challenged to be a right-brained thinker.

The recurring theme throughout the book is that using single, average numbers to plan for uncertain outcomes leads to systematic errors. Or, as Harry Markowitz writes in his foreword, “plans based on average assumptions are wrong on average.”

Take, for instance, a diversified portfolio of mortgages spread across various housing markets. “Suppose that property values are expected to rise in some of these markets but fall in others, remaining the same on average. What do you suppose the profit graph looks like with respect to property values? In locations where values increase, defaults drop slightly, increasing profit slightly. But where values fall, defaults go up. In some cases values will fall to the extent that the equity in the houses drops below the amount owed. At that point defaults increase dramatically, with owners just dropping off the house keys at the bank and moving into Motel 6.” Savage uses some hypothetical numbers to illustrate his point. “In this example, an 8 percent increase in value in one location improves profit by less than 5 percent, whereas an 8 percent devaluation in another location decreases profit by a whopping 40 percent. Thus the profit of a mortgage portfolio based on what are expected to be average property values will overestimate average profit.” (p. 95)

Savage, whose father wrote the groundbreaking The Foundations of Statistics in 1954, eschews the “Steam Era” anachronistic vocabulary of statistics in favor of plain English and computer simulations. But he can’t quite break with the past; he briefly describes key statistical “red words” and then suggests that we forget them. For example, in place of “hypothesis testing,” we should use “Did it happen by chance?” Similarly, we should forget about correlation and covariance and use scatter plots instead.

Statisticians will not, of course, be persuaded to abandon their formulas. Nor should they. But someone has to be on hand to ask the right kinds of questions and to understand the pitfalls inherent in certain ways of thinking. Black boxes should not be allowed to rule the world unchecked.

Savage writes in an entertaining style, as some of the chapter titles indicate: “When Fischer and Myron Met Bob: Option Theory,” “Some Gratuitous Inflammatory Remarks on the Accounting Industry,” and “Sex and the Central Limit Theorem.” He clearly explains such issues as how much a person should pay for information to reduce uncertainty and why the common corporate practice of sandbagging can be costly for the company.

The Flaw of Averages will not replace Statistics 101, but it’s a lot more fun.