A Silver Lining in Mozilla's Plan to Review 'Scientific Code'
The following Nature story covered by Erika Check Hayden is a good example of how much science has degraded in recent years.
Mozilla plan seeks to debug scientific code
An offshoot of the Internet non-profit organization Mozilla has entered the debate, aiming to discover whether a review process could improve the quality of researcher-built software that is used in myriad fields today, ranging from ecology and biology to social science. In an experiment being run by the Mozilla Science Lab, software engineers have reviewed selected pieces of code from published papers in computational biology. Scientific code does not have that comprehensive, off-the-shelf nature that we want to be associated with the way science is published and presented, and this is our attempt to poke at that issue, says Mozilla Science Lab director Kaitlin Thaney.
In the not-too-distant past, scientists used to be very concerned about their honor and reputation, and that concern alone was the balancing factor against going all willy-nilly in scientific publications. In those days, a reputed scientist (an endangered species deserving UN protection today) confirmed his results in ten different ways before putting them in a paper. The purpose was not to somehow get past the reviewer, but to uncover laws of nature that would be accepted 100 or 500 years after his death. Errors made by incorrectly written code was not a reviewer’s headache in that era, because the blame ultimately fell on the scientist conducting the research. Getting past the reviewers was not the primary objective either. If at all, reviewers helped many scientists from serious embarrassment.
The following tragic story by Jonathan Eisen about his dad is a good example of how concerned scientists were about their reputation.
A day to think, to pause, to ponder
The person I refer to was my father. On this day, February 7, 1987, my father Howard J. Eisen took his own life. I was a freshman in college then. Enjoying life on my own at Harvard. Exploring the world of new friends, academic pursuits, and the usual college antics. And then it all exploded. The details are a bit of a blur and most are not really important for what I write about here. But suffice it to say I was devastated.
I flew home to Maryland with my brother and slowly the details emerged. My father was a researcher at the NIH. A paper was being prepared for publication by a post doc who worked for a colleague / boss of my father and who my father also worked with. My father was apparently asked to look at the paper and some “discrepancies” were noted and my father helped launch an investigation into the work. The NIH panel that was brought in to investigate the work of this post doc was very aggressive - very unpleasant - and even though no accusations of wrong doing were made against my father - the style and tone of the investigation pushed him over the edge. And he could not dig himself out.
Having a ‘regulatory body’ to ‘check’ software code is as ridiculous as having a regulatory body to check every kit, instrument and chemical in a biology lab prior to publication, but there is a silver lining in the story.
The time and skill involved may justify paying reviewers, just as statistical reviewers of large clinical trials are paid.
Mozilla ‘Science Lab’ found out a nice scheme to get paid, but that will raise the question - why not the other reviewers? We believe all reviewers should get paid in this new era of degraded science and the rate should be no lower than FBIs Criminal Investigative Division (CID).
