Silver, The Signal and the Noise

Nate Silver, the author of the popular political forecasting blog FiveThirtyEight, now part of The New York Times stable, is out with his first book, The Signal and the Noise: Why So Many Predictions Fail—but Some Don’t (Penguin Press, 2012). Although Silver covers a broad spectrum of topics, from weather forecasting to sports, with a terrific chapter on poker from the vantage point of a Bayesian, I am going to focus on a couple of general ideas that are relevant for investors and traders and then turn to his discussion of the financial markets.

We live in an age of information glut. A few years back retail investors opened up their daily newspapers to see how their investments were doing; now they can follow their holdings in real time, tick by tick. In the past they got stock recommendations from their broker, read (or didn’t read) annual reports, and perhaps watched Wall $treet Week with Louis Rukeyser. Now they are deluged with the constant chatter of pundits, twitter feeds, webinars—you name it. But are investors better off with all this information? Silver doesn’t think so: “We face danger whenever information growth outpaces our understanding of how to process it.” (p. 7)

The challenge is to separate the signal from the noise, to construct a predictive model that rises above the fanciful level of a child finding animal patterns in clouds. “Finding patterns is easy in any kind of data-rich environment…. The key is in determining whether the patterns represent noise or signal.” (p. 240) If the NYSE has closed higher on Mondays 59% of the time over the past year (a figure I invented) but over the last five months has been up only twice and down a whopping 17 (information I gleaned a while back from Bespoke: it is not current), does this information offer a tradeable signal? Or is it just noise? Even if it has statistical significance, does it have practical significance? That is, could an investor profit from this pattern? Silver offers his own example to suggest a negative outcome: the “Manic Momentum” strategy, that over a ten-year period outperformed the market handily without transaction costs but lost almost 99% of the trader’s original capital with a 0.25% per trade transaction cost.

Technical traders have another problem: in trying to find signals amid the noise they are prone to overfitting. They devise a complex function that “chases down every outlying data point, weaving up and down implausibly as it tries to connect the dots. This moves us further away from the true relationship,” if there is in fact any true relationship in price action, “and will lead to worse predictions.” (p. 166)

Simply trying to parse data in search of a predictive signal is a fool’s errand, Silver believes. He illustrates this point when he takes ECRI to task for its September 2011 prediction of the near certainty of a double dip recession. In explaining its reasoning ECRI invoked “dozens of specialized leading indexes.” “Theirs,” Silver writes, “was a story about data—as though data itself caused recessions—and not a story about the economy. ECRI actually seems quite proud of this approach. ‘Just as you do not need to know exactly how a car engine works in order to drive safely,’ it advised its clients in a 2004 book, ‘You do not need to understand all the intricacies of the economy to accurately read those gauges.’ This kind of statement is becoming more common in the age of Big Data. Who needs theory when you have so much information? But this is categorically the wrong attitude to take toward forecasting, especially in a field like economics where the data is so noisy. Statistical inferences are much stronger when backed up by theory or at least some deeper thinking about their root causes.” (p. 197)

We make predictions every day, most of them quite mindless. But when predictions are important, mindlessness has no place. Not only should we theorize about causes and relationships, we should also couch our conclusions probabilistically. And yet “most of us—including most of us who invest for a living—are [very poor] at estimating probabilities.” The exceptions are the skilled options traders “who make bets on probabilistic assessments of how much a share price might move.” (Silver is quick to point out, lest the reader miss the qualifier ‘skilled’, that “You should not rush out and become an options trader. … [M]ost options traders receive a poor return.”) (p. 364)

The Signal and the Noise is a very rich book, one that I highly recommend. It takes a technical topic and makes it not only accessible to the statistically unwashed but engrossing. And does so with vividly portrayed illustrations. Let me close with one of my favorites: the two-track market.

“There is the signal track, the stock market of the 1950s that we read about in textbooks. This is the market that prevails in the long run, with investors making relatively few trades, and prices well tied down to fundamentals. … Then there is the fast track, the noise track, which is full of momentum trading, positive feedbacks, skewed incentives and herding behavior. Usually it is just a rock-paper-scissors game that does no real good to the broader economy—but also perhaps no real harm. It’s just a bunch of sweaty traders passing money around. However, these tracks happen to run along the same road, as though some city decided to hold a Formula 1 race but by some bureaucratic oversight forgot to close one lane to commuter traffic. Sometimes, like during the financial crisis, there is a big accident, and regular investors get run over.” (p. 368)