What topic can embroil one of Britain's leading scientific journals, various newspaper, TV and radio commentators, as well as the Royal Society British counterpart to our National Academy of Sciences in heated controversy? The culprit, at least at first glance, is genetically engineered potatoes. But the real question underlying this latest skirmish in the bioengineering wars is 'what constitutes good science?'
The furor first arose in 1998 when Dr. Arpad Pusztai announced to the world via the media that his research on rats showed that genetically engineered plants could be dangerous to human health. Pusztai, a researcher at the Rowett Institute in Scotland, and colleagues had genetically modified potatoes by inserting a gene from small flowers called snowdrops.
This gene caused the potatoes to produce a lectin a protein that snowdrops normally make, that has insecticidal properties. The idea, of course, was to get the potatoes to produce this natural insecticide themselves. The technique worked, and the modified potatoes did indeed produce the snowdrop lectin insecticide.
This transfer of genetic material was not the basis of the debates, however. The controversy erupted in part because Dr. Pusztai used the public media to announce that such genetically engineered potatoes made the rats that ate them ill before other scientists had a chance to critically review his experiments.
Opponents of genetic engineering heralded the announcement as bolstering their arguments that genetically engineered foods are unsafe for human consumption. But many scientists objected.
In fact, the prestigious Royal Society reviewed Pusztai's data. Their conclusions, released in May, 1999, stated that the reported work was "...flawed in many aspects of design, execution and analysis and that no conclusions should be drawn from it."
In spite of this, the October 16, 1999 issue of The Lancet, one of Britian's most prominent and respected scientific journals, published a brief report of Pusztai's work. The paper had been reviewed by six different experts in fields pertaining to the science under consideration. They asked for several revisions to the paper before recommending publication, and even then not all agreed that the paper should be published. And the editor of The Lancet, Richard Horton, found himself under attack for proceeding with publication of a paper that eminent scientists had disapproved.
Was the research so seriously flawed? Unfortunately, it was, for a number of reasons. First, the investigators fed one group of rats raw potatoes. Researchers who do such work know that rats don't digest raw potatoes or any kind of raw starch very well. It makes them ill. Second, people don't typically eat raw potatoes, so this part of the experiment is not even a good model for humans. Even more important, the study was performed without the appropriate control group.
In any sort of feeding experiment, when a new ingredient is being tested, investigators set up their experimental groups in such a way that the results of the study will clearly indicate whether the new ingredient had the expected effect. Puzstai and his colleagues did not do this.
Further, their work was criticized on the grounds that since the genetically modified potatoes contained lower than usual amounts of protein, the animals may have been suffering from protein deprivation. The Royal Society, in a comment on the paper published in the Lancet stated that "...the diets used were incompletely controlled; no control group of rats was fed a reduced protein diet. This omission is important because the [genetically modified] potatoes contained less protein than normal potatoes."
Normally, a study as flawed in basic experimental design as Dr. Pusztai's would likely not have been accepted for publication. At best, reviewers would critique the study and suggest improvements in design and/or execution. The journal editor might then encourage the authors of the paper to address the criticisms, and do further experimental studies or analyses; or he or she might refuse to publish it at all. Puzstai and colleagues, by announcing their work via the popular media, short-circuited this process. By doing so, they reported unreliable conclusions to the public conclusions that were not firmly grounded in a sound scientific study.
So was this bad science or junk science as some pundits have called it? Not really it was a bad study. But one study does not make or break a scientific theory. One canon of science is that the results of one study no matter how well-designed or logical they may seem do not really prove anything. Experimental results must be reproduced, preferably by other researchers, before they are considered reliable.
By publishing this incomplete and badly designed study, The Lancet did nothing to advance our understanding of potential advantages or disadvantages in genetically engineered crops. What the journal did do, was indicate that if you make enough noise even with a dearth of reliable data you can get even the scientific establishment to pay attention. But we knew that anti-biotech activists have been doing it for years.