Can You Really Trust Health Journalism? Think Like A Scientist.
Newspapers often give completely contradictory health advice. It’s difficult enough to keep track of what is supposed to be healthy, without papers changing their minds the whole time. Unless you have training in Science or access to the original source material, you often must take a newspapers interpretation at face value. But this can often be wrong, and in some cases, very dangerous. How dangerous? Well, read on to find out how you can spot the sensational and separate the wheat from the chaff.
Health And Contradiction
One example of a newspaper that is often sensational in its reporting is The Daily Mail. And, like any other news source, it has to keep up with new information, which may change within hours. In just a few examples, we see The Daily Mail appear to contradict itself on a variety of subjects. “Aspirin causes cancer” but also “Aspirin prevents cancer”. They tell us “Beer causes cancer” but then “Beer prevents cancer”. Also “Coffee causes cancer” except in those times when “Coffee prevents cancer”. “Eggs cause cancer” but, you guessed it, “Eggs prevent cancer”. “Soya causes cancer” but also “Soya prevents cancer”. And if lifestyle advice rather than dietary advice is your thing, don’t worry, they’ve got you covered. “Stress causes cancer” but, in an unexpected turn of events, also “Stress prevents cancer”.
If you were to go by this reporting, which in itself seems well sourced, would you be confident to have eggs on toast and a soy latte? Why such contradictory advice? What has gone wrong here? Is it the newspaper’s fault? Or is it more complicated than that?
Contradiction Is Natural in Science
Knowledge about a subject changes over time. We all remember adverts where doctors recommended cigarettes, but you’d struggle to find many nowadays claiming cigarettes aren’t harmful. And those that do require some education. But that’s only part of the story. To understand more about it, let’s look at how medical research gets published, and how newspapers access this research.
If you’re a researcher wanting to get your work published, you submit it to a professional journal. There’s a kind of pecking order of these, with the top ones being well known (Nature, The British Medical Journal, The Lancet etc). At the top of the journal food-chain, journal editors are able to employ lots of independent experts who can “peer-review” articles before they are published, checking that everything looks accurate. It’s the journal’s reputation on the line too. These journals make their money by being a trusted resource that people are keen to subscribe to and keep up to date with cutting-edge research. Because you often have to pay to access these articles, it’s less likely that the general public or the press have access to them.
Here lies a conundrum. The very nature of profit means that the public may have to rely on second-hand accounts.
A bit down the chain are the less big-name journals that provide articles free to access and make their money through advertising. These journals still peer-review articles and often play an important role in publishing articles that bigger journals might not take on, but are still important: for example articles about treatments that have been tried but don’t work.
Further down the food-chain are journals that rely on charging researchers to publish in them, just for the kudos of having their research published. The majority of these journals are honest, but a few are just money-making schemes, that will accept any article as their business depends on it. They will often actively go looking for researchers to submit articles to them. While they claim to be “peer-reviewed” often these articles aren’t.
So how do you distinguish between verified and peer-reviewed science, and what is just money-making? And how can you tell what you should believe in when there is so much out there?
The Wheat from the Chaff
One famous example of science shenanigans is David Mazieres and Eddie Kohler of New York University and the University of California, Los Angeles. They were contacted by a journal that sent hundreds of emails out to researchers, suggesting that they submit work to their journal. They wrote a paper called “Get Me Off Your F*****g Mailing List” as a joke. Surprisingly, their paper was “peer-reviewed” and accepted for publication by the impressively named “International Journal of Advanced Computer Technology”
Now, that’s an extreme case, but it shows that we need to be cautious about some of these medical articles that are the easiest for newspapers to access. Just because something has been published in a “journal” doesn’t mean it’s good research, and expert peer-review is essential. Unfortunately, it’s often the very articles that are less rigorously checked that are easiest to access by newspapers.
Why is this a problem? Well, proper research is often really difficult to do, but misinterpreting results is easy. As any statistician worth their salt will tell you, 86.7% of statistics are made up. So if an article hasn’t been “peer-reviewed”, you’re going to have to review it yourself. Luckily, there are certain clues in good research that help you think like a Scientist.
Think like a Scientist
In Science, nothing is confirmed. Only refuted, and conclusions drawn in very narrow margins. Researchers have to be careful that their findings are “statistically significant”, and they use various ways to do this. They often quote a factor in their articles called a “p value” which is a measure of how likely it is the results they found are due to chance. For example, a p-value of 0.05 suggests that there’s only a 5 in 100 chance that the results of the research are just due to chance. That sounds impressive, right? Well, yeah, but if your research is looking at 100 things that might cause cancer, 5 things might show up as positive just by chance.
This is incredibly common, and a natural limitation of science. It is akin to ‘we are confident of this relationship in most cases, but cannot account for all.’
Also, you need to be sure that the things you are looking at really are the cause, and aren’t just “confounding factors” meaning that the thing you are looking at isn’t really the cause, but is associated with something that is. Let’s say you’re looking at what causes obesity, and you find that Diet drinks are linked to it. It might be that they really are – but it’s unlikely considering the number of calories in them is often zero. So a simple confounding factor is that the already overweight may drink more diet drinks to try and cut down. Correlation does not imply causation.
Erm, So Now What?
So I should just ignore everything I read right? Well, no. Research is essential to better understand the world we live in, but the bottom line is we shouldn’t just trust something because it’s been published in a journal with an impressive name, and then reported on in a newspaper. The Daily Mail has an excellent health writer, but without the opportunity to review the work in the context of everything else you cannot make a truly informed choice.
Furthermore, often the relationship between a variable (coffee) and an outcome (stress) is not something simple, linear and dose-related. You can have too much or too little. So if you claim ‘Coffee causes stress’ and ‘Coffee helps you destress’ you may be right, but about different doses, lifestyles and context.
Check the sources, ask yourself a few simple questions about it, and if you’re interested look into it more. If you see something that doesn’t make sense, ask about it, write a comment below the article, because unless we highlight bad reporting of poor research, things are never going to change. Also, there are some very good sources out there for health, such as ‘Cochrane‘ which creates reliable information about health research by reviewing all the data at regular intervals.
But nothing is better, or safer, than seeing your GP. So if you are concerned about your health, or the impact of changing diet, behaviour or medication, consult with your local provider before hand.
So, with that in mind, cast a keener eye on the headlines. And if you want to know a bit more about how to approach an article critically, check out our Big 5 questions. And, as always, help us out by sharing our work.
- Learn more about why learning to learn may help you.
- Follow Ben on Twitter so you never miss an article from drbenjanaway.com
- Give this a share if you found it interesting.
- Let me know what you think in the comments below or on social media.
The opinions expressed in this article are those of Dr Rod Kelly and Dr Janaway alone and may not represent those of their affiliates. Original article was written by Dr Kelly and edited for publication by Dr Janaway. If you wish to discuss the article, please contact us. Image courtesy of Jon S. No alteration was made to the image except resizing to fit banner (640x378p.) We pass on our thanks to the creator of the Image.
If you have concerns about your health, always consult a Physician.