Sep 05 2008

Statistically insignificant amount of truth in science reporting

Published by Dougal at 3:47 pm under Bad Science, Society

Recently there was a fluff science story on the BBC website about the happiest place in Britain. You know the type. What was unusual about this particular article was the disclaimer buried in the middle:

However, the researchers stress that the variations between different places in Britain are not statistically significant.

That’s right. After breathlessly telling us how much sadder the inhabitants of Edinburgh are than the people of Powys in Wales, they “let slip” that it’s all cobblers.

I mentioned this on the Bad Science forum but took it no further. Happily, others did. You can see Gimpy’s letter to the BBC and their response elsewhere, but I’ll quote the relevant bit from the BBC’s response here:

We felt that as the story was of a light-hearted nature, and that as the conclusions were not of great importance, or significance, it made for a light and entertaining read.

I agree that the story was both unimportant and insignificant. But it’s the fact they pretended it did have importance and significance that is so disturbing. They reported exactly the opposite of the research. They even dispatched a reporter with camera crew to Hay-on-Wye in Powys to really drive home the “insignificance” of the story.

BBC Most Read
BBC Most Read
© Betsy

I wonder what the criteria are for a “light and entertaining” story, as determined by the BBC? Off the top of my head I would guess it would include comic stories involving pets and record-breaking attempts to snog for 48 hours straight. I didn’t think it would include using a researcher’s name to assert the opposite of what they claim.

The BBC would like to claim that because there is no real importance to the story that they are not under any obligation to report accurately. Somehow they’ve decided that stories which are not Serious News™ can be treated with a freeform approach to reality. And that this slapdash journalism is okay if you admit somewhere in the article, “oh by the way, we made all this up”.

The “entertainment” excuse begins to wear very thin after a while. What kind of an impression does this reporting actually have on people? Are there less naive readers up and down the country who look at this story and think, “oh, those kidders at the BBC! what a lark they are!”. I doubt it. Instead, they’re more likely to do as Terry Wogan apparently did after reading out a letter from a listener:

“…apparently the second happiest place in the UK is Manchester, which is confusing because this very week began with more research which concluded that Manchester was number one when it came to self harm.”

I think possibly academics could keep things to themselves for a bit. We’re getting contradictory messages.

It seems even Terry Wogan (isn’t he employed by the BBC?) wasn’t aware that it was all just a joke guys! Get with the programme, Wogan! (Thanks to Bad Science forum user Allo V Psycho for the Wogan transcription and iPlayer link.)

I personally think that “entertainment” is a pretty pathetic excuse in this case. Which particular part of the story was entertaining, exactly? Francis Wheen mentions similar excuses in Mumbo Jumbo, given by newspapers for continuing to print horoscopes. “Entertainment” appears to be the last, nebulous excuse available when there is no good reason to do something.

Another Bad Science blogger recently suggested that The Media Think You Are Morons, or more accurately:

What I’ve found is that not only do the media not bother to give references to the academic work they are writing about, but they don’t bother to respond promptly to enquiries about their policy of keeping this information from us. The most likely conclusion is, I think, that they consider the general public to be morons incapable of understanding references — and they don’t think they need to explain themselves to their readers either.

The lack of references in news stories really annoys me. I’ve probably even written about it before. It can be pretty difficult to track down the source of a statement when all we have to go on is “scientists said” or “experts have found”. If these same news stories can now be relied upon to state the opposite conclusions to the source, what use is science reporting at all? It fails both to inform and to direct you to further data if needed. What service do the relevant media think they are offering here — “entertainment”?


Update: I am pleased as punch to note that Ben Goldacre wrote about this very topic in the weekend Guardian. I like to think it was because of the thread I started on the Bad Science forum. A boy can dream…

8 Responses to “Statistically insignificant amount of truth in science reporting”

  1. jdcon 05 Sep 2008 at 4:53 pm

    “If these same news stories can now be relied upon to state the opposite conclusions to the source, what use is science reporting at all?” That’s a very good question. Given that the media presumably expect us to trust their reports and that they don’t link to the original research (or cite it), I think we have a right to expect accurate reporting. I’m very disappointed that not only do the media seem to disagree with me on this, but they don’t feel that they owe me a response to any questions I ask on the subject. I don’t even find their stories entertaining, so I’m not sure why they consider them to be ‘entertainment’.

  2. […] through the piece and entirely scuppers it as a story.  A few ‘bad science’ types have spotted this - notably gimpy - and let the BBC know what they […]

  3. […] mislead, for a la…BBC editor concedes … on The BBC mislead, for a la…Looking Out To Sea … on The BBC mislead, for a la…gimpy on The BBC mislead, for a la…apgaylard […]

  4. […] up by ‘Ithika’, posting in the Bad Science forum, and it has also been blogged about at dougalstanton.net and […]

  5. […] over at the blog “Looking Out to Sea“, wrote a very entertaining and informative piece on the BBC’s own attempts at […]

  6. Mark Libermanon 10 Sep 2008 at 3:41 pm

    The lack of references in news stories really annoys me. I’ve probably even written about it before. It can be pretty difficult to track down the source of a statement when all we have to go on is “scientists said” or “experts have found”.

    In many (most?) cases, the journalist will have written the story from a press release, and perhaps an interview, without ever having seen the original source(s).

  7. Dougalon 10 Sep 2008 at 5:28 pm

    Mark, I would guess you’re absolutely right. Do press releases typically include all the references needed, or are they not even mentioned by university press offices? Maybe press offices should be encouraged to focus their blurbs on this information so that it doesn’t get omitted from newspapers. Even mentioning the researcher’s name and university would be a massive help in some cases. “Experts believe,” indeed!

  8. […] number of bloggers picked up on this facile churnalism (here, here and here).  Ben Goldacre covered it in his Guardian column.  I entered into a dialogue with one of the […]

Trackback URI | Comments RSS

Leave a Reply