The best books on Statistics and Risk

Before we discuss your book choices on statistics and risk, I want to ask: What do you mean by “risk literacy” and why do you think it is so important for people to have it?

I am a statistician and I know that people find probability and statistics quite difficult to understand, and not intuitive. And after years and years of careful research I have finally concluded that it is because probability and statistics really are difficult to understand and unintuitive.

I think knowing something about how chance works in the world is a basic skill that people should have, along with reading, writing and basic numeracy. Otherwise you can be subject to all sorts of manipulations, and that will come out in some of my book choices.

Let’s have a look. First up is The Drunkard’s Walk by Leonard Mlodinow, which looks at how the mathematical laws of randomness affect our lives.

This is a general introduction to the history of probability and the way it comes into everyday life. It intersperses the historical development with modern applications, and looks at finance, sport, gambling, lotteries and coincidences.

It starts off with quotes from Cicero feeling that people were being misled by thinking that the gods influenced the throw of a die. Then it carries on through the early development of probability in the 16th century with mathematician Gerolamo Cardano. He threw two dice and looked at the distribution of the sum of the two faces. There was an incredibly popular game called Hazard where you threw two dice and betted on what the total would be. Amazingly, people had been gambling for centuries and had never realised you could do maths on gambling. Probability – which used to be known by the wonderful term “the doctrine of chances” – grew out of this.

What other examples does he give?

The book illustrates various issues in ordinary life. For example it looks at the gamblers fallacy, where if you are betting on something like roulette and red has come up more than four times in a row, the fallacy is that black must be due to come up. That isn’t true, as long as the wheel is fair. You get that in lotteries as well, where if numbers haven’t come up for a while people feel that they must be due – which isn’t the case as there is no memory in the system.

He also covers coincidences, for example the fact that people do win freak lottery results, such as winning twice in a short period. Actually, these things are not unexpected if you take into account the number of opportunities for odd coincidences like this to happen. So he explores the different ways that probability can do strange things, which he illustrates very nicely with a whole lot of historical characters.

As a final example he talks about Francis Galton, a rather strange but brilliant man who came up with the idea of “regression to the mean”. This is the fact that if you pick things when they are on a high for some reason, just by chance they are likely to come down. He uses the example of the supposed jinx of Sports Illustrated, which is the idea that after anyone appears on the cover of Sports Illustrated magazine, inevitably their career seems to suffer a terrible blow. He points out that this is really regression to the mean. The fact that you are on the cover means that you have had a run of good luck, but luck doesn’t last and eventually things average out.

Your next book, Stephen Senn’s Dicing with Death, has been described by reviewers as very entertaining and funny – which is not something you would normally expect from a book on statistics.

I know Stephen, and he is full of very entertaining jokes and rude remarks about people. This book is somewhat similar to The Drunkard’s Walk in that it covers the history of probability. But since Stephen’s background is in medical statistics, he looks at clinical trials and other medical studies.

He is particularly interested in different ideas on statistical inference. It is not generally known that within the statistical world, for decades if not centuries, there have been arguments about different ways of thinking about inferences. This is the way that you draw conclusions from data about the underlying states of the world. Some of these arguments involve quite deep philosophical issues. For example, “What is probability?” Stephen Senn covers the different schools of statistical inference, including the so-called Bayesian inference, of which I am an enthusiast, as well as that based on [Ronald] Fisher, [Jerzy] Neyman and [Karl] Pearson.

Stephen Senn tends to follow the Fisherian approach, and he can be very rude about the Bayesian school. He accuses them of having meetings where they sing Bayesian songs which he says are of “mind-numbing puerility”. As someone who has sung some of these songs at meetings, I think he might have a point.

He also looks at some of the common misconceptions people have towards health risks.

Yes, he deals with the need for randomisation when evaluating a medical treatment – how you should assign patients at random to treatment or a control group, in order to make sure that the real differences in the outcome are due to what you have done, rather than just how you selected your patients.

He also puts in some maths, and even tries to explain quite technical issues such as the Cox Proportional Hazard Regression Model for survival analysis. To put this into a popular book is very bold and I admire his courage. For those who don’t know, this is an incredibly valuable way to compare, for example, how long people live after different medical treatments. It was developed in cancer therapy, when people were trying new cancer drugs and then watching the patients to see what happened. That is known as survival analysis, which had previously been applied to things like the length of time that a light bulb survived before it expired. Then this theory was moved over to medical statistics, and the statistician David Cox developed a mechanism for comparing groups of patients at the same time as seeing how other factors might effect their survival. The major international cancer prize was given to him for his work on this, and it has been cited by about 25,000 people.

Your next book is more about perceptions than hard data. This is David Ropeik’s book How Risky Is It, Really?

Yes, this is a completely non-technical book, which is looking at things from the point of view of psychology. It is essentially a review of the recent research on risk perception. He is looking at how dangerous it is if we get our interpretations wrong. He uses our response to 9/11 quite a lot as an example, but he has also recently written about the response to Fukushima and nuclear risk.

He starts with neuroscience, pointing out that we have a fear response which has evolved to react intuitively to threats, and this has been very useful. But sometimes this shortcut can lead us to make the wrong decisions – our gut feelings can make us too fearful sometimes, and not fearful enough at other times.

What does he think we are too fearful of?

The usual things that tick the fear-factor box are often not natural, and things that we aren’t in control of radiation from nuclear plants is a prime example. They could be unfamiliar risks, or come from sources that we don’t trust.

So what should we really be worrying about?

Issues like sustainability and biodiversity, rather than pesticides and vaccines which people often get concerned about. He argues that homeland security after 9/11 was a massive waste of resources, and how by building up people’s fears it allows particular political agendas to be imposed.

Next up is Damned Lies and Statistics by Joel Best, which relates some of the potentially dangerous stories which have come out of false statistics.

This is written by a sociologist, so we have moved on from psychology to sociology. I like this book because one of the things that really irritates me as a statistician is when numbers are bandied around as though they are God-given truths and can’t be argued with. There is a nice quote from Joel Best that “all statistics are social products, the results of people’s efforts”. He says you should always ask, “Why was this statistic created?” Certainly statistics are constructed from things that people have chosen to measure and define, and the numbers that come out of those studies often take on a life of their own.

Best uses the lovely phrase “number laundering”, when the origins of numbers are forgotten. They get so bandied around that nobody knows where they came from. In the BBC Radio Four programme More or Less these are known as “zombie statistics”, numbers that refuse to die. However much people criticise them, they keep getting wheeled out.

There is a good example in the book, about priests. A psychologist estimated that perhaps 6% of priests were at some point sexually attracted to young people. This was transformed through a chain of errors into the “fact” that 6% of priests were paedophiles,

which is obviously incredibly dangerous.

Exactly, numbers get distorted. What happens is that the statistic starts to take on a new interpretation, as the original definition is replaced by an emotive example of a particular case. For example, less than half of violent crimes in the UK actually involve any form of violence. In the other cases the victim isn’t actually touched or harmed at all. But the popular image of a violent crime is based on extreme examples covered in the media. Of course, if I took a swing at you with an axe and missed it is still a violent crime, but it does mean that we have to be really careful about definitions.

So Joel Best has lots of good examples of how numbers get misused by people, including politicians, and how we should be wary of that. There is a sociological idea of “deconstruction”, and what I have really learnt from all these books is the importance of deconstructing statistics. We need to be able to view them critically.

Your final choice also helps us do that. Do You Feel Lucky? by Kjartan Poskitt.

This is part of the Murderous Maths series, and is my favourite book. It is great. It is a book for secondary school kids full of cartoons and lovely characters like this filthy boy called Pongo McWhiffy, or the one I like best is an upper-class idiot called Binky Smallbrains who falls for every trick. It is a book about chance, luck and probability, and these are difficult things that people have to learn about at school. This book really cracks through it. It starts with probability trees and then goes on to look at things like the gambler’s fallacy, Pascal’s triangle and so on.

Is it training the next generation of bankers?

Possibly. What is nice is that it is not ashamed to face up to using gambling as a way of teaching, because games of gambling are often the best way to explain some of these concepts.

So should I continue to buy my lottery tickets every week or am I just deluding myself?

You are just deluding yourself if you are going for a gamble that is likely to pay off. But psychology books would say there is a reason that people do this, because it gives them a small possibility of changing their lives. Psychologically, it might be quite a reasonable thing to do. Financially, it really isn’t worth it.

What is a better way statistically to get some free money?

It depends on what your knowledge is, and whether you can use your knowledge to beat the odds. I like premium bonds because you can keep the stake.

You have four rules of risk to help people tackle statistics. What are they?

The first one is “stuff happens”. Which means we cannot predict exactly how every event will turn out, but we can often predict the overall pattern of events surprisingly well.

The next is that it is good to compare like with like. So if you want to show that speed cameras reduce road traffic accident rates, don’t just put them in places that have just had a run of accidents, as regression to the mean states that there will be fewer accidents by chance alone.

Another is to think “What am I not being told?” For example, a person may have got better after she took this wonder treatment, but how many other people’s stories are not being featured?

And finally, twice not very much is still not very much, which means that increasing a tiny risk may not be so important. Almost everything interesting might help, and it might also harm. The trick is working out the balance for you.

But in terms of tackling statistics in the news, I think things are changing. For example, here in the UK the head of the statistics authority, Sir Michael Scholar, spends his time publicly telling government ministers off for their use of statistics. So it is a very interesting time where I think things are gradually improving.