Surowiecki, The Wisdom of Crowds

When I started my series of posts on crowds I said that although I had the necessary material to write about the madness and the stupidity of crowds, I was dependent on the kindness of the inter-library loan system to look into the wisdom of crowds. In general I’m not prone to excess but, when it comes to books, my attitude is: why request just one when I can ask for three? Then, of course, came the decision which book to read first. Perhaps I implicitly relied on the wisdom of crowds in opting for James Surowiecki’s book of the same name (Doubleday, 2004) because I’m familiar with his New Yorker financial columns and knew that I would be treated to a combination of literary style and easy accessibility. I was not disappointed.

Surowiecki quickly glosses over Mackay and dismisses Le Bon out of hand, claiming that he had things “exactly backward.” In its place Surowiecki promotes the thesis, borne out by many behavioral experiments, that the decisions of a diverse crowd on balance trump the decisions of the individual, no matter how smart he is. Moreover, the decisions of a crowd made up of people spanning a reasonable range of the intellectual spectrum are better than those of a crowd made up exclusively of very bright people.

A crowd can make better decisions than its individual members only if the people in the group are relatively independent of each other. This means, first, that their mistakes are not correlated and, second, that more information is brought into the mix. By contrast, in a glaring example, when mutual fund managers, following Keynes’ piece of “worldly wisdom” that “it is better for reputation to fail conventionally than to succeed unconventionally,” imitate each other, they shrink the overall intelligence of the market. They don’t contribute any new information, and their mistakes become correlated. As a result, between 1983 and 1999 almost 90% of them underperformed the Wilshire 5000.

The problem is that we tend to be imitators. Indeed, “imitation is a kind of rational response to our own cognitive limits.” (p. 58) So how can we reconcile the wisdom of crowds with a bunch of “me-tooers”? Behavioral researchers quickly fine-tuned the class of imitators into two sub-classes: the rational imitators and the slavish or herding imitators. When there is rational imitation in crowds, its members initially have a wide array of options and information. Moreover, to prevent herding there have to be some people in the crowd who buck the trend. These mavericks are often the overly confident; “they overestimate their ability, their level of knowledge, and their decision-making prowess.” (p. 61) Left to their own devices they’re prone to bad decisions, but as members of crowds they serve as circuit breakers; they can sometimes, for instance, stop a negative information cascade.

Although Surowiecki draws examples from such diverse sources as football and traffic, let me zero in on his discussion of the stock market. He contrasts buying an apple with buying stock. Your decision whether to buy an apple depends on how much you like apples, whether this particular apple looks good, and what the grocer is charging for it. Your decision is effectively independent of what other shoppers for apples are deciding. “By contrast, the price of a stock often reflects a series of dependent decisions, because when many people calculate what a stock is worth, their evaluation depends, at least in part, on what everyone else believes the stock to be worth.” (p. 247) In brief, it echoes Keynes’s beauty contest model. “What Keynes recognized is that what makes the stock market especially strange is that often investors are concerned not just with what the average investor thinks but with what the average investor thinks the average investor thinks.” (pp. 247-48) Admittedly, not every investor follows the Keynesian model; some actually try to pick the prettiest girl, not what they think other investors will find the prettiest. Most of the time, Surowiecki contends, “the stock market is an ever-changing but relatively stable mix of independent and dependent decision making.” (p. 248) Problems arise when everybody starts piggybacking on the wisdom of the crowd and no one is adding to the wisdom of the crowd.

Surowiecki argues that CNBC exacerbated this problem; it “magnified the dependent nature of the stock market because it bombarded investors with news about what other investors were thinking.” (p. 253) Not only does this encourage herding, it is not conducive to good decision making. A series of experiments at MIT showed that “more news does not always translate into better information.” Two groups of students selected an initial portfolio of stocks, after which they could buy or sell at will. The first group saw only the price changes in the stocks in their portfolio; the second group had both price information and a constant stream of financial news that purported to explain what was happening. The first group’s performance far surpassed that of the “better-informed” group.

There’s an obvious moral here.