The Original Purpose of Peer Review
After Pachter’s tarring and feathering of bioinformatics ‘leaders’ Manolis Kellis and Eric Lander, many researchers are talking about the value of ‘post-publication peer review’ (see comments here or at the Eisen’s blog). I find those comments misguided, because peer review was never meant to be similar to a professor grading student’s homework. Real peers always judged papers after publication and at times many years after.
The original system was built a long time back, when scientists followed a different culture. I left the world of theoretical physics years ago, and back in those days the physicists cared a lot about reputation. Reputation was like a currency they carried, and it increased or decreased based on the quality of publications. The funding system was peripheral to reputation. An University wanted to hire reputed scientists, no matter how little money he brought to the organization. A system close to that is still followed in mathematics, where most would find Grigori Perelman as a reputed mathematician.
In those days, the reputation currencies could be lost very easily by poor publications (or misconduct, which was very rare). My professor often mentioned about something like a ‘two strikes’ rule. He said that the physics community would excuse someone’s first incorrect paper, but if the same researcher published another low-quality paper in her career, he would be filing for ‘reputational bankruptcy’. That makes sense in the real world. If I announced about proving Fermat’s last theorem in 1990 and Poincare conjecture in 1991, and both of those proofs were wrong, I doubt any mathematician would have cared to hear my 1992 announcement. Sadly, the bioinformaticians do not treat Kellis and Birney in the same way, and not doing so would in turn disrepute the entire field of bioinformatics.
In the old system, the role of peer review was to help fellow scientists catch obvious errors before the paper went on print so that they did not lose reputation. However, ultimately the full responsibility rested on the author(s) of the paper. To be doubly sure about not making mistake in a proof, we often circulated the preprints among friendly scientists prior to submission (friendly defined by one who wont scoop my results), and that preprint circulation system morphed into the physics arxiv in early 90s, when I was a graduate student.
Why doesn’t reputation matter any more? The answer is hugely expanded role of NIH. It is encouraging that Dan Graur and I come to find the same reason for destruction of an effective scientific culture.
My only explanation for their continuing existence is that the wannabe ignoramuses, self-promoting bureaucrats, and ol fashion crooks of ENCODE are protected from criticism and penalties for cheating by the person who gives them the money. Thus, they can continue to take as much money from the public as their pockets would hold, and in return they will continue to produce large piles of excrement that are hungrily consumed by gullible journalists who double as Science editors.
Therefore, the only way to begin fixing the system would be to shut down NHGRI and send Eric Green home. All other changes, including the stupid DORA pledge, post-publication peer review, etc. are ineffective attempts to fix the symptoms without attacking the real problem.
Lets Discuss Is it Time to Shut Down NHGRI?
The Genome Institute and Its Role Ken Weiss
Despite above appeals, I do not personally believe the mainstream science in USA can be fixed. The system is heading into collapse, because the crooks with vested interest captured the system and have no interest in fixes going against their interests. One can see many similarities with the banking system prior to 2008 breakdown, and interestingly many ‘outside’ blogs that pointed out the banking problems in 2007-2008 are starting to do the same for mainstream science in 2015. The following comment was posted in the linked blog post -
Very enjoyable. As a molecular biologist, your paragraph about science particularly hit home.
It is hard to convey how wasteful and useless a lot of scientific work is to people outside the field. The 3 main requirements for successful career scientists - to publish, publish, and publish - do not select for careful or thoughtful inquiry. The day-to-day process of science as I have observed in multiple institutions is so wasteful of reagents and consumables that it would shock you. We scientists sit at the top of a pyramid, demanding raw materials and factories and specialised chemical manufacturies and chemical processing plants, and it is noticeable how swiftly the costs of research have increased as every step of that pyramid suddenly gains additional costs as scarcity of raw materials and costs of transportation increases. The companies supplying the biomedical field merge or go quietly bust, remove seemingly profitable product lines without explanation, and yet few of my peers seem to recognise or acknowledge the underlying processes at work.
I once sat in a conference and listened to a giant of my field talk of what she saw as the future of research - more and bigger sequencing projects! It is not enough to sequence a single cancer to inform a patients treatment(in theory) - 10 clones, 50 clones, 100 clones can be sequenced! The faint echo you hear is the heavily diminishing return in such an egregious waste of resources. There are genetic diseases caused by single gene disorders discovered in the 80s which have no viable treatment - we know exactly why they occur, but not how to change them back to health without making things worse.
Do not look to any scientist in the hope of saving the current status quo - science is great at converting grant funding into mortgage payments, and coffee into published papers - but it is largely yet another shell game like fracking, albeit one which at least has a chance of VERY occasionally discovering something “useful” - usually something hideously expensive to produce, and of some small measureable benefit at a population level. The truth is, the money spent on that wonder drug to give you 6 more months of life would probably been better spent helping ten or a hundred people survive, ie. using an ad campaign telling you to check any odd looking moles out with your GP. But one is ‘progress’ and ‘r and d spending’ and can build someone’s career, and the other isn’t, and doesn’t.
I continue to convert my own wage into resilient personal skills. Thank you for your blog, it has helped.
