Tweet or Perish?

The thinking of academic community is moving fast in the direction that we envision. We expect that, in not too distant future, researchers will post their papers in a web-repository like arxiv.org, write a blog post or two describing it in simple language, and then tweet a small link to those familiar with his work. Also, they will continue to write their daily or weekly thoughts in a blog, so that the paper ‘published’ in arxiv.org will be more like stringing together of many blog posts than all new discoveries. We have not seen any academic researcher go that far yet (except one or two), but a post in Rafael Irizarry’s blog shows that people are starting to think that way.

Should we stop publishing peer-reviewed papers?

Nate Silver, everyones favorite statistician made good, just gave an interview where he said he thinks many journal articles should be blog posts. I have been thinking about this same issue for a while now, and Im not the only one. This is a really interesting post suggesting that although scientific journals once facilitated dissemination of ideas, they now impede the flow of information and make it more expensive.

Two recent examples really drove this message home for me. In the first example, I posted a quick idea called the Leekasso, which led to some discussion on the blog, has nearly 2,000 page views (a pretty recent number of downloads for a paper), and has been implemented in software by someone other than me. If this were one of my papers, it would be one of the more reasonably high impact papers. The second example is a post I put up about a recent Nature paper. The authors (who are really good sports) ended up writing to me to get my critiques. I wrote them out, and they responded. All of this happened after peer review and informally. All of the interaction also occurred in email, where no one can see but us.

Pierre Vandergheynst elaborates on publication system that fits world of not too distant future (aka present :) ) -

[don’t miss the comment section]

First we need a paper repository, with full proof back-up and always on capacity.On top of that repository we will implement a collaborative reviewing layer. This really is first and foremost a platform enabling users to comment on papers in the repository.

First about the comments: they should not be text only. We are talking about rich comments, with pictures, support for latex formula ( la mathjax, something really easy to use, maybe even offering a GUI of symbols for those who do not master latex). I believe comments should also include code, data. A simple way would be to allow users to upload material attached to their comments. Code is particularly important: it should be treated just like a paper. Reviewed, tested. Comments can be rated and the most highly rated comments will be promoted as reviews.There should be a robust system to clean offending comments, spams, robots. This could actually be managed via user authentication.

Also from Yann LeCun:

I am proposing a new publication model that dissociates dissemination from evaluation. Its main characteristics are as follows:

Authors post their papers on a repository as soon as they deem them acceptable for public consumption. Papers can be revised and are given version numbers. They become immediately citable (examples of such repositaries in math, physics and social sciences already exist: arXiv.org SSRN,….).

The repository provides a web-based infrastructure with which people can organize themselves into “Reviewing Entities” (RE), similar to existing journal editorial boards, conference program committees, special interest groups, etc. Using this infrastructure, REs can assign reviewers to papers (or simply give a weight to a review of a paper). An RE may consist of a single individual. or may be an informal group of people with a common interest (what we call an “area”), or may be a more traditional editorial board.

Any RE can choose to review any paper at any time (out of the author’s control). The reviews are published and accessible with the paper. Reviews include the name of the RE. Individual reviewers that are part of an RE may choose to reveal their identity or not. REs may choose to give a rating or “seal of approval” to papers they review, so as to bring them to the attention of the communities they cater to. REs do not “own” papers exclusively, as traditional journals do, so that multiple REs can give ratings to a single paper.

However, authors may formally request that a particular RE review their paper. They may make only one such formal request at a time. The RE is given a short time to accept or refuse to review (e.g. 2 weeks) and a slightly longer time before the author is allowed to submit a request to another RE (whether the requested RE has produced a review or not). RE will have an incentive to review good papers, as their reputation will increase with the quality of the papers which they are the first to rate highly.

Reviews are published and are themselves citable documents with the same status as regular publications. Reviewers may choose to keep their names hidden or not. Reviews can be given a rating by readers (who are themselves REs), hence reviewrs will get objectively evaluated and will be assigned a kind of “karma” that indicates the average usefulness of their reviews (many sites such as Slashdot and Amazon have such a feature). This will give reviewers an incentive to do a good job. Eventually, people will indicate their reviewing karma on their CV.

Papers are under a revision control system. Papers that are revised as a consequence of a comment or review should cite that comment or review, thereby giving credit to the reviewer (whether the reviewer was anonymous or not. as this can be tracked by the system). The citation identifier of a paper includes its revision number.

Users can set up alerts for all papers approved by their favorite REs.

One can imagine sophisticated credit assignment systems for reviewers and authors that:

propagate credit down the citation graph, so that you get credit if you paper is cited by highly-cited papers.

gives karma to reviewers whose evaluation was predictive of the ultimates success of the paper.

In the meanwhile, academic journals cannot catch a break. Sarah Kendzior came up with another hard-hitting piece - Academic paywalls mean publish and peris h.

Academic publishing is structured on exclusivity, and to read them people must shell out an average of $19 per article.

On July 19, 2011, Aaron Swartz, a computer programmer and activist, was arrested for downloading 4.8 million academic articles. The articles constituted nearly the entire catalogue of JSTOR, a scholarly research database. Universities that want to use JSTOR are charged as much as $50,000 in annual subscription fees.

Individuals who want to use JSTOR must shell out an average of $19 per article. The academics who write the articles are not paid for their work, nor are the academics who review it. The only people who profit are the 211 employees of JSTOR.

Swartz thought this was wrong. The paywall, he argued, constituted “private theft of public culture”. It hurt not only the greater public, but also academics who must “pay money to read the work of their colleagues”.

For attempting to make scholarship accessible to people who cannot afford it, Swartz is facing a $1 million fine and up to 35 years in prison. The severity of the charges shocked activists fighting for open access publication. But it shocked academics too, for different reasons.

“Can you imagine if JSTOR was public?” one of my friends in academia wondered. “That means someone might actually read my article.”

The model of journal publishing is broken, because it is impossible to fight with instant internet access by locking all doors. Also, given that the cost of electronic publishing is nearly zero with internet technology, the $1000 or so fee charged by ‘open access’ journals will not be sustainable. Why not? Well, there will be ‘Chinese competition’ to start with !! A new open-access journal called GigaScience started with publication fee of what it should be - $0.

Readers may also look at a blog post titled ‘Is the Internet the new Bell Labs?’ in C. Titus Brown’s blog along the same line.

Now, I know I’m a bit of an idealist, but to me this sounds like the way I and other scientists are using the Internet. I post ideas, they post ideas, and we interact on those ideas. I think things like arXiv and blogs like Haldane’s Sieve are moving it in the right direction: free exchange of often rather deep scientific ideas.

Just to follow up this idea, as well as pimp it to those darned Open Access fanatics… it seems to me that further developing this kind of free exchange of ideas is what we should be striving for as science moves forward. Not a new thought, but it does require that we be prepared to engage with what other people post – something I’m trying to do more – and develop good sites like Haldane’s Sieve where we can build fluid communities around our own little research focus of the moment.

Blog, email, twitter, arxiv.org, etc. are the tools of this new Bell lab, and ‘access by fee’ journals are not admitted there. Given a chance between joining the Bell lab and working at an insignificant research facility, which one will you pick? Or in the language of Steve Jobs (to woo John Scully from Pepsi):

Do you want to sell sugared water for the rest of your life? Or do you want to come with me and change the world?