UPDATE: After you read this story, make sure you check out the follow up piece. Editors at Embargo Watch have found evidence that The Sustainable Food Trust manipulated the media to prevent public criticism of this paper.
Yesterday, in an aside to a post criticizing an astroturf political campaign in California, Mark mentioned a new study that supposedly found GM corn causes tumors in rats. As Mark said in an update to that post, this study is severely flawed, but I wanted to follow-up on that with some discussion about why it's flawed.
After all, the study was peer-reviewed, right? Doesn't that mean we can trust it?
Here's the thing. Peer review is not perfect. It's not a panacea. It's simply the basic level of due diligence. By submitting work for peer review, a scientist has allowed people outside her own team to critique her work. And the journal might require some changes to the paper based on the critique — anything from edits for clarity to requesting that the scientist perform another experiment in a different way. If a paper hasn't gone through peer review, you should be more skeptical of it. Avoiding peer review means that the researcher decided to show the public her results before allowing those results to be critiqued by independent experts.
But, at the same time, just because something has gone through peer review doesn't mean it's been certified to be accurate. It just means that roughly three other experts have looked at the paper before publication. There's still a lot of room for things to go wrong. Peer review is like the bouncer at the door. The bouncer doesn't guarantee that every person in the bar would be a good person for you to date. Even if a paper gets through, you still have to think about it critically and evaluate it on its own merits. This recent paper on GM corn and rat tumors is an excellent example of that …
Over at Discovery News, Emily Sohn has a great breakdown of everything that's wrong with the GM corn and rat tumors study. And there's a lot that's wrong with it. In fact, the laboratory that did this research — a French team led by Gilles-Eric Séralini — has been heavily criticized for the poor quality of their research into GM food on multiple occasions.
It's not Monsanto spokespeople doing the criticizing, either, Sohn writes.
One immediate problem, Newell-McGloughlin said, is that the line of rodents used in the study, known as Sprague-Dawley rats, are frequently used in cancer research because a large majority of them naturally develop tumors at a high rate, regardless of what they eat or how they're raised. What's more, the rats were allowed to eat an unlimited amount of food, which increases their chances of developing tumors. And two is a very old age for these rats, which could account for the large rate of cancer seen across all groups, including the controls.
The small size of the control group also raised red flags. Even experienced scientists in the field had trouble interpreting data in the study, as seen in comments collected by the UK's Science Media Center, but it appears that the study included just 10 or 20 control animals. That means there were at least nine times more test animals than control animals. If anything, studies of this kind usually include two or three times more controls than experimental animals.
The results don't make a lot of sense, either. No matter how much of either herbicide-laden or genetically modified maize the rats ate in proportion to their other food, rates of cancer and premature death remained the same. However, to be meaningful, toxicology studies like this should show a dose-dependent response, which means that if something is toxic, more of it should be more toxic.
Looking at the data, it appears that the study authors never tested their results to see if the numbers they turned up could have occurred by random chance, said David Tribe, a microbiologist at the University of Melbourne in Australia. And given the small numbers of animals used in the study, that's a real possibility.
Notably, the authors of the paper never responded to Sohn's request for an interview.
• Read Emily Sohn's report on the problems with the GM corn and rat tumors study at Discovery News
• Read the full study online
• Read an earlier BoingBoing post explaining how peer review works, and some of the flaws in the system. (Be sure to check out the extensive comments, which include further context from scientists.)
Image: Corn Maze, a Creative Commons Attribution (2.0) image from pinkmoose's photostream
