I knew I would like this book. It’s the same sort of thing as the book Risk, which I loved, and Freakonomics, which I loved. It’s not so much a book, as a toolbox. In that sense it is similar to Carl Sagan’s The Demon-haunted World: Science as a candle in the Dark.

A toolbox which will allow you to question the way science is presented to you in the media and in advertising. My own personal bugbear is pentapeptides. You know what they are, don’t you? No? Well, you know how they react chemically and biologically, don’t you? No? Well why the bloody hell should you assume that they are going to be beneficial, then? Just because they sound scientific?

There’s a lovely article in the Times which demolishes the scientific credentials of this sort of thing. And this is what Bad Science does on a book-scale.

Detox via a foot spa? The brown gunk comes out whether or not your feet are present. It therefore does not come from your feet. It is rust. The author, Ben Goldacre runs Bad Science.net and is on twitter as @bengoldacre. Even if I can’t persuade you to buy his book (hopefully from one of my nice amazon affiliate links) then at least visit the blog and read the tweets…

Part of the problem is that a lot of people — in the media, and in ‘real life’ — don’t understand the difference between correlation (two things tend to occur together) and causation (one thing makes the other thing happen). Also, even if you do have causation, it’s important to get the direction of causation right. Ambulances do not cause heart attacks, yet are frequently found around heart attack victims.

This is illustrated beautifully by a cartoon from XKCD.com called ‘correlation’…

Right. Back to the book. One of the things that I have learned is that I am a nutritionist. As in, I know something about nutrition (although that is in fact not required) as anyone can call themselves a nutritionist, as opposed to say a ‘dietitian’ which actually requires some knowledge. Of course, you don’t need to take Ben’s word for it — you can look it up yourself. Here’s one of the comparisons I found.

In many countries only people who have specified educational credentials can call themselves “dietitians” — the title is legally protected. The term “nutritionist” is also widely used; however, the term nutritionist is not regulated as dietitian is. People may call themselves nutritionists without the educational and professional requirements of registered dietitians.

Wikipedia: Dietitian

Ben demolishes a lot of the stuff that gets reported in the media — from “Doctor” Gillian McKeith (and some ridiculous statements that seem to have gone relatively unchallenged), and also some rather bizarre claims that Vitamin C is better against AIDS than the AZT anti-virals. The point of what Ben does — irrespective of whether or not you actually agree with his conclusions — is to show you how the scientific method works.

If claims are put forward which do not match up to this — randomisation, double blind tests, comparison against placebo (or comparative drug) — and so on, then the research doesn’t tell you very much. It might well tell you that taking a pill has a more significant effect than doing nothing, but it doesn’t tell you that the active ingredient in that pill has actually played any part.

Ben’s chapter on the placebo effect is magnificent, illustrating that even when given a medicine which should make a patient’s condition slightly worse, the placebo effect counteracts this to the extent that they may actually get slightly better. This chapter should be required reading for pretty much everyone with even a passing interest in testing medicine (whether normal or ‘alternative’). It would also dovetails very well with the chapter on the placebo effect in 13 things that don’t make sense, so feel free to read both of ‘em.

However, 13 Things… and Bad Science differ on homeopathy slightly. 13 Things… is sceptical of homeopathy but seems to suggest that some claims, and some meta-analyses, have produced positive results which are statistically significant in a pro-homeopathy way. Bad Science is a lot more scathing. It also references a meta-analysis which looked specifically at experiments deemed to be experimentally sound, and this results in nothing better than the placebo effect (possibly a slightly enhanced one, as the placebo effect is very complex and surprising — but you’ll have to read that for yourself).

At this point, I wondered which experiments were deemed sound, because earlier on Ben had mentioned some results which seemed very experimentally sound but nonetheless had resulted in a pro-homeopathy result, and had seemed to dismiss these experiments…

[...] show homeopathic remedies are pure placebos. The fact, however, that the average result of the 10 trials scoring 5 points on the Jadad scale [meaning that they appear the most experimentally sound] is consistent with the hypothesis that some (by no means all) methodologically astute and highly convinced homeopaths have produced results that look convincing but are, in fact, not credible

Professor Ernst, quoted in Bad Science

So, if you dismiss some results because you are suspicious of them, even though they appear experimentally sound, and then your results show no effect better than a placebo, that wouldn’t be surprising, because you’ve fixed the result somewhat (a bit like the cranial capacity measurements criticised in Stephen Jay Gould’s The Mismeasure of Man).

So I was a little wary of the results Ben was quoting here: it’s fair enough to assess the tests based on which appear the most experimentally sound, but not if you then arbitrarily dismiss some of them. So I thought clarification was needed — were these results dismissed in the experiment?

So I took the time to contact Ben, and ask him. And he took the time to reply (thanks, Ben). These experimentally sound seemingly pro-homeopathy results, while they may have been deemed ’suspicious’ were still included in the meta-analysis. Which meant that when the meta-analysis (basically, a compilation of all the results from good experiments) showed that homeopathy was no better than a placebo, it was doing so even with data deemed suspiciously pro-homeopathy.

However, homeopathy might be a better than usual placebo effect, it might actually have an enhanced placebo effect — because of the strange way the placebo effect works (look, just buy the book and read that chapter, okay?).

And there’s more. Such a lot more — from ‘the doctor will sue you now’ (a chapter available online under creative commons because owing to an ongoing lawsuit it couldn’t be included in the first version of the book), ‘bad stats’, ‘health scares’ and ‘how the media promote the public misunderstanding of science’, to ‘the media’s MMR hoax’ and ‘why clever people believe stupid things’.

MMR

I’m going to sidetrack a little here into something which is more of a personal account than a close-up look at what Ben Goldacre has to say about the whole MMR affair…

As a parent, I remember the MMR scare well. I remember debating with my wife whether we should look into paying for three separate jabs, or go for the free MMR. Because, as a parent, if you aren’t convinced something is safe (or at least safer than the alternative), you don’t want to subject your child to it, and when the media are repeatedly telling you that some people are saying that the MMR vaccine causes autism, and others are telling you that scientists disagree on it, the waters become very muddied and it can be difficult to know what is best.

What certainly didn’t help at all was the fact that then Prime Minister Tony Blair refused to reveal whether or not he’d given the vaccine to his son. Now, on the one hand, I agreed with this stance morally: I don’t think it’s appropriate for a politician to use their family to illustrate a political point. But on the other hand, it didn’t help clarify this issue when it appeared, presumably having easy access to the best medical advice, that the Blairs had been reluctant to give their son the MMR.

The other problem was that “officials” kept using the phrase MMR is safe. Now, we all knew there was no way it was going to be 100% safe in absolutely every case, and so to be presented blandly with this by the media instead of actually being presented with statistics and results made it look as though the government was just offering a load of flannel. As Ben points out, they weren’t; the Government were actually providing this information… only you wouldn’t have known it from the media, because that wasn’t the story they wanted to be selling.

I remember actually trying to find out the facts. It wasn’t easy — particularly online — but I remember arriving at the following conclusions:

• measles is very, very dangerous (although I already knew this)
• if the MMR uptake drops (and it had), more people will get measles as ‘herd immunity’ will diminish
• the initial research is being questioned
• further research seems to show no link between MMR and autism

As a parent, you’re not entirely sure. All you can do is to make the decision which you think best for your children. If I had just gone from the media alone, from the controversy alone, we might have given our children the MMR. Because we actually looked into it a bit, because we were sceptical of the controversy, we were able to arrive at the conclusion that even if there was something in it, it was safer and better to give our kid the vaccine.

Vaccine uptake has now dropped to around 73%. This is too low to provide any form of ‘herd immunity’ (i.e. too many people being immune to it for it to be effectively carried through a population), so measles is now a genuine risk in a way that it wasn’t before this whole scare started.

But would I have been sufficiently sceptical of the controversy to do that further reading around if I wasn’t a scientist [I did Biological Sciences at Uni]? How many lives have been damaged by measles because people were convinced by the media shitstorm? And that’s why I would urge everyone — whether or not they buy this particular book — to understand something of the scientific method, and to not just take someone’s word for it because they are a celebrity or have a TV programme, or whatever.

I was struck by something in the book which reminded me of something I had forgotten:

And in the 1970s — since the past is another country too — there was a widespread concern in the UK, driven again by a single doctor, that whooping-cough vaccine was causing neurological damage

Bad Science

I was one of those children who didn’t have the whooping cough vaccine. My parents were (and indeed still are, for that matter), educated and intelligent people. Not only that, there were specifically trained in the healthcare profession — both were qualified pharmacists. They were obviously sufficiently concerned about the risk of the vaccine that they deemed the risk to me to be greater to have the vaccine, than not having the vaccine.

They were very nearly seriously wrong. I contracted whooping cough. I recovered from it, but, just like with measles, not everyone does. But that was the difficulty of being a parent in the 1970s: you frequently have to make a judgement call from incomplete information, and then hope for the best — as Ben points out, advice that babies should sleep on their tummies has probably led to thousands of deaths.

But now, with the internet at your fingertips, you have no excuse. If it’s something important, don’t hand over your decisions to the voice on the telly. Find out about it yourself. Understand the difference between science and pseudo-scientific mumbo-jumbo designed to sound impressive and shift a few boxes of the latest product.

And if you don’t know how to question these things, or you want a few extra tools in your toolbox to help you differentiate the science from the bollocks-with-big-words-in, then Bad Science is a handy thing to have in your armoury.