DORA Pledge - YASI from Government-funded Scientists
Before continuing further,
DORA = Declaration on Research Assessment
YASI = Yet Another Stupid Idea
Rage against impact factor has become the new fashion. The following note has been making the rounds on twitter and other places.
An ad hoc coalition of unlikely insurgentsscientists, journal editors and publishers, scholarly societies, and research funders across many scientific disciplinestoday posted an international declaration calling on the world scientific community to eliminate the role of the journal impact factor (JIF) in evaluating research for funding, hiring, promotion, or institutional effectiveness.
The San Francisco Declaration on Research Assessment, or DORA, was framed by a group of journal editors, publishers, and others convened by the American Society for Cell Biology (ASCB) last December in San Francisco, during the Societys Annual Meeting. The San Francisco group agreed that the JIF, which ranks scholarly journals by the average number of citations their articles attract in a set period, has become an obsession in world science. Impact factors warp the way that research is conducted, reported, and funded. Over five months of discussion, the San Francisco declaration group moved from an insurrection, in the words of one publisher, against the use of the prominent two-year JIF to a wider reconsideration of scientific assessment. The DORA statement posted today makes 18 recommendations for change in the scientific culture at all levelsindividual scientists, publishers, institutions, funding agencies, and the bibliometric services themselvesto reduce the dominant role of the JIF in evaluating research and researchers and instead to focus on the content of primary research papers, regardless of publication venue. The DORA coalition calls on all individuals and organizations engaged in scientific research to sign the San Francisco declaration.
The stupidity of these central planners knows no bound !! If signing a declaration can solve any problem, we will start collecting signatures for DONL (declaration on not lying) for politicians, DONBTG (declaration on not bribing the government) for big companies and DOTI for all DORA signers. Yes, DOTI stands for ‘declaration for thinking intelligently’, if you have not figured it out.
Impact factor is only a symptom, but the real disease is large centrally- funded science. Prior to government taking over science, scientists used to argue about nature, truth, plagiarism (defined as stealing of ideas, not paragraphs). There were only a few new ‘fields’ and even fewer scientists, but most of them were solid scientists and not bull-shitters (BSers). Science thankfully did not have enough money to feed too many BS-ers, and those were the happy days.
After government took over and decided to pour billions of dollars, it had to create simple measures to ‘evaluate’ productivity. Simplification and creating rules is what the government does, because it has to give directions to create uniform rules for processing large number of applications. At first, paper count became the simple measure of choice. In the early to mid-90s, paper count was in vogue. Those, who graduated in that period, gamed paper count game heavily to get grants and positions. There were many ‘you scratch my back, I scratch your back’ arrangements. Eventually, the central planners decided to do something new and brought in ‘citation count’ and ‘impact factor’. Some people still have not adjusted to the new mode as you can see in this 2009 paper titled Estimates of global research productivity in gynecologic oncology -
Research production and international cooperative teamwork in the 2 main journals of gynecologic oncology increased within the 10 last years; 65.3% of all published articles dealt either with epithelial ovarian cancer, cervical cancer, or endometrial cancer. Endometrial cancer had the worst ratio number of publications to estimated national incidence (United States, 2007). The United States (41.15%) and Europe (29.72%) make up a striking 70.87% of the world’s research production in the field of gynecologic oncology. However, the highest rate of increase shows in Turkey (22.5), the People’s Republic of China (6.87), and South Korea (5.83). Adjusted to the national GDP per capita and population for the year 2006, research productivity seems best in Israel, Austria, and Turkey.
Other planners want to be more cute and measure both ‘quantity’ and ‘quality’. Check “Worldwide research productivity in critical care medicine” for example.
Introduction
The number of publications and the impact factor of journals are accepted estimates of the quantity and quality of research productivity. The objective of the present study was to assess the worldwide scientific contribution in the field of critical care medicine.
Method
All research studies published between 1995 and 2003 in medical journals that were listed in the 2003 Science Citation Index (SCI) of Journal Citation Reports under the subheading ‘critical care’ and also indexed in the PubMed database were reviewed in order to identify their geographical origin.
Results
Of 22,976 critical care publications in 14 medical journals, 17,630 originated from Western Europe and the USA (76.7%). A significant increase in the number of publications originated from Western European countries during the last 5 years of the study period was noticed. Scientific publications in critical care medicine increased significantly (25%) from 1995 to 2003, which was accompanied by an increase in the impact factor of the corresponding journals (47.4%). Canada and Japan had the better performance, based on the impact factor of journals.
Conclusion
Significant scientific progress in critical care research took place during the period of study (19952003). Leaders of research productivity (in terms of absolute numbers) were Western Europe and the USA. Publications originating from Western European countries increased significantly in quantity and quality over the study period. Articles originating from Canada, Japan, and the USA had the highest mean impact factor.. Canada was the leader in productivity when adjustments for gross domestic product and population were made.
If central planners measure how well they ‘cured’ cancer by counting the number of papers or ‘high impact’ papers on cancer, then ‘paper count’ and ‘impact factor’ will be maximized. Therein lies the problem. Moreover, due to simplistic nature of central planning, central planners will have to shift to a new measure to evaluate efficiency of central planning in post-(impact factor) era. So, DORA declaration, if adopted properly, will merely shift the problem to something else.
Support of large projects and ‘center grants’ is another bad effect of central planning. Central planners can only juggle with few balls in the air, and that is why they want huge ‘successful’ projects like ENCODE in their portfolio.
What is the solution then?
In 1971, John Cowperthwaite became the governor of Hong Kong. He decided to stop collection of all kinds of government statistics. People of Hong Kong never lived better !!
Asked what is the key thing poor countries should do, Cowperthwaite once remarked: “They should abolish the Office of National Statistics.” In Hong Kong, he refused to collect all but the most superficial statistics, believing that statistics were dangerous: they would led the state to fiddle about remedying perceived ills, simultaneously hindering the ability of the market economy to work. This caused consternation in Whitehall: a delegation of civil servants were sent to Hong Kong to find out why employment statistics were not being collected; Cowperthwaite literally sent them home on the next plane back.
We believe impact factor and other anomalies will go away, if scientists shun central planning agencies (NSF, NIH, etc.) creating measures for their effectiveness. The next step should be complete dismantling or decentralization of those central planning agencies. We do not see small software companies worried about ‘impact factor’ of their libraries before releasing new code. Why scientists?
