To Maintain Integrity of Science, Kellis and Birney Should be Investigated
A few months back, I was talking to a bioinformatician in Hong Kong and asked about his group’s choice of software programs for analyzing human sequence data of medical origin. He mentioned that people in China are very conservative, but they consider US science with highest regards. Therefore, he had to either use programs known as standards in USA or benchmark against the same programs to show that theirs can accomplish the same (or do better).
I have seen the same respect toward US science by talking to researchers from other Asian countries, but all that is going to change, if we do not maintain any semblance of standards in conducting our science. We can learn from how various Asian countries took steps to remove bad apples.
Taiwan: In 2013, we wrote a blog post - ‘Nanopore Sequencing by Divine Intervention?’ to report about a very unexceptional claim from a Taiwanese research group. Other blogs in Taiwan started making noise after that. As a result, the University started an investigation (Bizarro Details Coming out on Nanopore Sequencing by Divine Intervention Story) and fired the professor and student. Their paper got retracted.
Japan: Last year, a Japanese group reported about turning somatic cells into pluripotent cells by acid bath. The claim got widely disputed, because nobody could reproduce the experiment elsewhere. Moreover, the bloggers reported about finding many inconsistency in the figures of Haruko Obokata, the first author of the papers. RIKEN opened an investigation and as a result, the papers got retracted and Obokata resigned from RIKEN.
Korea: In another examples, Koreans took care of fraudulent claims from Hwang Woo-suk.
-————————————-
Only when it comes to USA/UK and especially the NHGRI-funded major research groups from USA/UK, no allegation seems to stick. Such an affair is unfortunate given that these days the entire world can follow the scientific conducts of researchers from other countries including USA through social media. Therefore, we sincerely request our scientific communities to investigate and resolve the following specific high-profile cases. We do not accuse anyone of wrongdoing but an investigation is definitely warranted here given the evidence collected so far. Depending on what the investigators find, we can outline the lessons to be learned by the scientific communities around the world.
1. Dan Graur, Ewan Birney and ENCODE Hype:
In 2012, NHGRI-funded ENCODE project reported about a major discovery, where they found 80% of human genome to be functional. ENCODE and Nature went on a major media campaign, and their ‘discovery’ is being widely reported as a major ‘breakthrough’ from the human genome project. For example, if you google ‘human genome achievement’, this page comes among the top hit.
In 2013, evolutionary biologist Dan Graur and co-authors strongly criticized ENCODE’s claim (check “On the immortality of television sets: function in the human genome according to the evolution-free gospel of ENCODE”). Initially, Graur has been called all kinds of names, but those criticisms turned out to be about his ‘tone’ rather than content. Given that Graur has not retracted his paper and more importantly, nobody found any real criticism to his claims, it clearly looks like Graur is on very solid ground.
Yet, nobody investigated ENCODE’s conduct despite Graur’s serious accusations. The individual, who tried to take most credit for ENCODE’s success in 2012, should be the first person to be investigated for his conduct after ENCODE’s fall, and that person is Ewan Birney.
In 2012, Birney was being promoted as the topmost leader and achiever of ENCODE (see ‘Genomics Big Talker’ in Nature) and he himself wrote a Nature article teaching others about how to make ‘big science’ successful.
The making of ENCODE: Lessons for big-data projects
I have been involved in consortia at various levels since 1999. In 2004, I became the coordinator of the ENCODE analysis. I have learned that consortia are difficult to make successful, because they involve people who might be competing with one another in another context. Getting competitors to work openly together towards a shared goal is not trivial. It relies on the good will of all.
Does ‘shared goal’ mean knowing the conclusions even before doing the experiment? Here is an interesting recent comment at Lior Pachter’s blog -
I was involved in one of the early ENCODE estimates of biochemical function. Ive seen it change from 5% to 80% of the genome with no one even mentioning one of the old papers, and bemoaning the inconsistency. It had to be 80%, it had to support Ohno. My PI once told me, when I showed him our confidence intervals included his theory, the competing one and a few more: then statistics must be wrong. I dont think he was downright dishonest: I think he was blinded by ambition and culturally impervious to mathematical reasoning, like many lab biologists. p-values were just one more obstacle, like publication fees, pesky reviewers, faulty reagents. Give me more data, make those equations disappear. Its all arbitrary anyway, bayesian, frequentist, you can even swap the null and the alternative He had no belief that statistics was the only way to make our statements stand the test of time. And if you look at the crisis of reproducibility across the biomedical sciences, this is not a Kellis or MIT or ENCODE problem. Who cares if some people believe in Ohno for another decade? I do, but it actually hurts more when it affects preclinical studies. Make no mistake, unless purifying selection is brought to bear on striking, irreproducible science on a urgent, radical basis, our departments will be so stocked with people just playing the game within a generation that only the appearance of Science will be left. Like ancient Roma senators who convened until the fall of the empire to support the illusion of a republic, we can have journals, grants, professors, editors, conferences and not have any Science, but only a shadow of it.
Getting back to Birney’s article, he wrote -
The ENCODE consortium had an internal structure that I believe was instrumental to its success. It had a spine of leadership comprising: scientifically aware project officers in the primary funding agency, the National Human Genome Research Institute at the US National Institutes of Health; a few leading scientists with goals aligned to the consortium; and one or two scientific project managers hired inside the consortium who had a detailed understanding of all the tasks and people involved. ENCODEs two key project coordinators (Ian Dunham and Anshul Kundaje) were funded for the lifetime of the project through a grant for which I was the principal investigator. Successful consortia tend to have similar core structures, suggesting that this is a natural and effective way to organize such projects.
Given that ENCODE was a failure and scientific disgrace, the investigation needs to start with Birney. The scientific communities should determine whether ENCODE’s leadership structure caused its failure. Moreover, we need to also know, to what extent the publicly funded researchers can mislead public about what they discovered using public money.
-——————————————–
2. Lior Pachter, Manolis Kellis and the Accusation of Fraud:
The case against Kellis is also serious. In a series of three blog posts, Berkeley mathematician Lior Pachter accused Manolis Kellis of scientific fraud. For old-fashioned scientists like me from the previous century, that is a serious charge to make in a public forum. Pachter did not make his claim lightly and continues to stand behind it as he reiterated in a recent comment.
You mock my accusation of fraud (I assume you are referring to my post on the Network Nonsense of Manolis Kellis) but in fact, it is a very very serious accusation that I made carefully and after much consideration. I dont think it, and the other criticisms Ive posted constitute minor errors. If you think so then all I can suggest is please read the posts. Otherwise all you have done is slander.
Yet, nobody bothers to investigate the conducts of Birney and Kellis to put these matter to rest. What kind of lessons are the young scientists going to learn? What are the scientists from around the world going to think of US science?
