Mapping God Found 'Scientifically Dishonest' by Anonymous Peer Reviewers

Cross-posted here (also see Heng’s comment)

-—————————————————————————-

The whole process of sales and marketing of papers (also known as ‘peer reviewed publication’) is so broken in biology that one can only laugh by watching the devolution of the system. We noticed the problem as soon as we started submitting papers in this field. One of our friends always had better luck than us in getting his papers accepted for review process at Science and he even managed to get one of our ‘do not like to review, because not interesting for our great journal’ paper go through the review process. Being curious (a 17th century practice), we checked how he did it, and discovered that he always started his conversation with Science editors with ‘I am the student of Nobel Laureate so and so’.

That method works very well, as found out by a study of papers published in top psychology journals. h/t: @DanGraur

The present investigation was an attempt to study the peer-review process directly, in the natural setting of actual journal referee evaluations of submitted manuscripts. As test materials we selected 12 already published research articles by investigators from prestigious and highly productive American psychology departments, one article from each of 12 highly regarded and widely read American psychology journals with high rejection rates (80%) and nonblind refereeing practices.

With fictitious names and institutions substituted for the original ones (e.g., Tri-Valley Center for Human Potential), the altered manuscripts were formally resubmitted to the journals that had originally refereed and published them 18 to 32 months earlier. Of the sample of 38 editors and reviewers, only three (8%) detected the resubmissions. This result allowed nine of the 12 articles to continue through the review process to receive an actual evaluation: eight of the nine were rejected. Sixteen of the 18 referees (89%) recommended against publication and the editors concurred. The grounds for rejection were in many cases described as serious methodological flaws. A number of possible interpretations of these data are reviewed and evaluated.

With that background, let us get to the main story. We just learned that Heng Li’s BWA-Mem paper got rejected by a journal.

Hi bwa users,

The bwa-mem manuscript has been rejected. Interestingly, the first reviewer only raised a couple of minor concerns and then accepted the manuscript in the second round of the review. The second reviewer made quite a few mistakes on some basic concepts and was hostile from the beginning. The third reviewer gave fair and good review in the first round, all of which have been addressed, but he then tried hard to argue one particular mapper to be the best in accuracy that on the contrary is inferior to most others in my view. I admit that my responses were not appropriate, either, when I was in rage under the charge of being “scientifically dishonest” - because I did not show that particular mapper in the main figure given its absence of mapping quality.

Anyway, this is the past and you cannot really tell if I am the biased one when myself is describing the review process. As to the manuscript, I do not have plan to submit it to another peer-reviewed journal before I get time from the problems that interest me more. The manuscript is fairly short and simple. It should not be hard for you to judge how bwa-mem is compared to those peer- reviewed mappers. If you decide to use bwa-mem, please cite:

“Li H. (2013) Aligning sequence reads, clone sequences and assembly contigs with BWA-MEM. arXiv:1303.3997v1 [q-bio.GN].”

I would greatly appreciate. In these years, I sometimes did not pay enough attention to users’ requests and sometimes even lost patience in answering questions. I deeply apologize for these. Just remember: when I calm down, I am always grateful to every user, really.

Thank you very much,

Heng

PS: I regard this review process as an isolated event. I don’t mean to criticize the journal or the peer-review process in general with this example. As I said, I was not doing everything in the best way, either.

Here are the problems with the objections of the reviewers, and moreover the big picture of existing approach to publish in so-called peer reviewed journals.

1. Heng Li has a large electronic footprint, which is one’s real ‘publication record’ in the internet era. Anyone, who reads Heng Li’s comments at Biostars or Seqanswers, knows that he tries to do very thorough job. So, throwing big words like ‘scientifically dishonest’ disgraces the peer review process rather than saying anything about the quality of his submitted work.

2. Only reason for his sending the paper to a traditional journal is possibly to be able to fill up ‘publication’ column in grant review, because the github, biostars, seqanswers, blogs and arxiv are not considered as ‘publications’ by NIH.

3. Even the mapping God can be wrong, as he surely is in the last part of his comment :)

I regard this review process as an isolated event. I don’t mean to criticize the journal or the peer-review process in general with this example.

The does not agree with the mentioned study on psychology papers in the top part of the commentary. Unless Heng Li revises the last part of his opinion, we will start calling him all kinds of nasty names like reviewer 3.