Plight of the Fortune Tellers: two views of probability

It might seem that I’m going off the deep end. First a foray into admittedly phony astrology, and now a book entitled Plight of the Fortune Tellers. But this book is serious stuff. It’s written by Riccardo Rebonato, I assume still global head of market risk and global head of the quantitative research team at RBS, a visiting lecturer at Oxford, and adjunct professor at Imperial College; it’s subtitled Why We Need to Manage Financial Risk Differently; and it was published by Princeton University Press (2007). The fact that the Royal Bank of Scotland subsequently had to be bailed out to the tune of some $74 billion might make you snicker (yes, I guess RBS should have managed its financial risk differently). But I’m not reading this book as a call for reform, despite its subtitle. Rather I’m interested in Rebonato’s analysis of concepts we can all use in managing our trades and our portfolios. I’ll share insights from this book in dribs and drabs over time. Fortune tellers might seem like a sexy subject; risk management, I know, is not.

Today’s topic is probability. Tomes have been written about probability, but here I’m getting down to basics. There are two general ways to look at probability. The first is the frequentist view, best illustrated by coin tosses. The second is the Bayesian (or subjective) view, “often seen as a measure of belief, susceptible of being changed by new evidence.” (p. 19) The frequentist view underlies most current risk management: “We estimate the probabilities, and from these we determine the actions.” Rebonato suggests that the opposite should apply: “We observe the actions, and from these we impute the probabilities.” (p. 18)

The frequentist view requires an event such as the tossing of a coin that can, in principle, be repeated many times under virtually identical conditions. The findings can be precise, down to many decimal points. We can say, for instance, that the chance of a coin coming up heads is 0.5000001, that the probability of a newborn baby being female is 0.52345, or that the probability of a randomly chosen individual in a well-specified population having red hair is 0.1221.

But not all probabilistic statements have these characteristics. If, for instance, you are speculating about the probability of a Democrat being elected president (and are just as willing to wager on the outcome of the election as you would be on the outcome of a coin toss or the sex of the next newborn), you’ve moved outside the world of the repeatable. You have to rely on beliefs about the likelihood of a president being elected if the economy is soft, if we’re involved in war, and so on. You can look back on previous presidential elections for guidance, but they did not take place under identical conditions. Moreover, you would never make such a bizarre statement as that the probability of a Democrat being elected president is 0.520032.

A fairly strong case can be made that the frequentist view is a subset or a special case of Bayesian statistics. Rebonato illustrates this with a story about a Martian and a coin toss. You meet the Martian and, “not knowing how to strike up polite conversation, you take a coin out of your pocket.” (p. 50) The Martian has never seen a coin, so he obviously has never taken part in a coin flipping experiment. Your coin is run of the mill, nothing special about it. You toss it four times and each time it comes up heads. Rebonato then asks what you would conclude about the fairness of the coin and what the Martian is likely to conclude. You would probably just soldier on, assuming it was fair, and require odds of 50-50. The rational Martian, however, should not accept these odds; for him heads is much more likely than tails.

Rebonato continues: “You and your Martian friend have observed the same experiment, yet you reach very different conclusions. How can that be? Because, Bayesian statisticians say, we almost never approach an experiment with random outcomes without some prior ideas about the setting and the likelihood of the possible outcomes. New evidence modifies (should modify) our prior beliefs and transforms them into our posterior beliefs. The stronger the prior belief, the more difficult it will be to change it. If we really and truly have no prior beliefs about the situation at hand, as was the case with the Martian when he observed the coin flips, then and only then will we be totally guided by the evidence. This is the situation that is dealt with with infinite care and precision in ‘traditional’ statistics books. Yet it remains a rather special situation.” (p. 52)