On Meta-Research and the STAP Fiasco
Between them, the two papers seemed to demonstrate that a physical perturbation could do what had previously been achieved only by genetic manipulation: transform adult cells into pluripotent stem cells able to differentiate into almost any other cell type. The acronym STAP (stimulus-triggered acquisition of pluripotency) became instantly famous.
The errors, some of them identified during the institutional misconduct investigation (pdfs, 1 & 2), others by the authors of the papers (the retractions are combined), include several misrepresentations of Figures, sloppy handling of data that may have been deliberate, reuse of material from an earlier thesis that used a different process, switched samples and plagiarism (which might have been due to a simple omission of citation). In the court of public opinion, however, the crucial fact is that no one has been able to duplicate the results.
The official retractions are linked from the papers (it is Nature's policy to annotate rather than delete retracted material). Three articles by David Cyranoski (1, 2, 3) in Nature's News section, which is editorially independent, provide more details, and Paul Knoepfler has compiled a linked timeline covering the five-month controversy. (See also Science, The New York Times, Time, etc.) Knoepfler also obtained answers to half a dozen questions he posed to the journal, which provide some more detail about the process that, clearly, failed.
The two principal scientists, however, are maintaining that their work is fundamentally sound. From the retractions (signed by the eight and eleven co-authors):
We apologize for the mistakes included in the Article and Letter. These multiple errors impair the credibility of the study as a whole and we are unable to say without doubt whether the STAP-SC phenomenon is real. Ongoing studies are investigating this phenomenon afresh, but given the extensive nature of the errors currently found, we consider it appropriate to retract both papers.
Charles Vacanti, who first had the idea, issued a statement saying "that he still believed the concept would be proven right." Haruko Obokata, who developed it, "will attempt to recreate the widely-trumpeted findings" under video surveillance over the next five months.
Other scientists don't give them much of a chance, according to the Boston Globe. Rudolf Jaenisch considers the question "finally settled" and criticizes Harvard for its "deafening" official silence. Yoshiki Sasai, one of the co-authors of both papers, says that "it has become increasingly difficult to call the STAP phenomenon even a promising hypothesis." Harvard's Leonard Zon is even blunter:
"I don't think there's any shred of hope for these cells."
This is a rapid about-face for Nature, which is the butt of deserved criticism for missing some of the issues that post-publication review revealed. Most such retractions used to take much longer; the Science retraction of Hwang Woo-suk's two human-stem-cell papers took almost two years from the publication of the first, though the second was only eight months old. Social media made a difference, as Knoepfler has noted, and we can expect similar speedy scrutiny in future.
For it's almost certain that something like this will happen again. Nature and Science and presumably other journals, are tightening their review processes, but it remains true that, as a Washington Post survey of mishaps and worse said:
Science is open to error, misinterpretation and even fraud
Indeed, almost a decade ago John Ioannidis published what became the most-accessed article in the history of Public Library of Science, with over a million hits:
Why Most Published Research Findings Are False
Now, he hopes to do something about that. In April, Ioannidis and Steven Goodman launched a new center at Stanford:
Scholars at the Meta-Research Innovation Center, or METRICS, will focus on conducting research about research.
The effort is timely, and Ioannidis seems to be both smart and appropriately cynical, according to The Economist:
Dr Ioannidis plans to run tests on the methods of meta-research itself, to make sure he and his colleagues do not fall foul of the very criticisms they make of others. "I don't want", he says, "to take for granted any type of meta-research is ideal and efficient and nice. I don't want to promise that we can change the world, although this is probably what everybody has to promise to get funded nowadays."
Previously on Biopolitical Times:
- Hwang's Patented Fraud and New Questions about STAP Cells
- Are STAP Stem Cells a Paradigm Changer? Hopes and Fears
- Induced Pluripotent Stem Cells and Germline Engineering
- Cloning-Derived Stem Cells Raise Policy Questions