OUseful Info

Peer Review 2.0

Seeing this post just now - O'Reilly Radar > Digging The Madness of Crowds - about how an unchecked, largely untrue story rose to prominence on Digg, resonated with one aspect of something I've been mulling around for a few days with respect to the question of Are Blogs Academic Publications? having originally doodled around the idea here.

The quality of the great majority of even the best blog posts still falls short, in word count at least, and work effort, I suspect, of most academic journal (and perhaps conference) publications. After all, why put the effort in if there is no payoff in terms of academic brownie points, or a jolly to a conference!

I'm not sure where in the quality scheme of things blog posts should come, but I suspect that in some cases it's not so far off parity with a workshop poster presentation.

Now publications as minor as this won't boost an academic CV too much, (although they may help to provide a bit of bulk!), except insofar as they demonstrate that an academic is participating in that form of discourse that is available through the blogosphere.

However, if blog posts do have a place at the bottom of the academic publication chain, then it would perhaps be interesting if the academic publishers started exploring ways of introducing academic quality measures such as some form of peer review.

The payoffs that immediately come to my mind for the publishers are exposure, a testbed for novel peer review mechanisms, and the ability to spot upcoming issues that might make for a good special issue, for example.

So - how might the academic publishers get into the blogging community (or perhaps they already are? Do any journals have 'blogclips' reviews of relevant news items pulled from the blogosphrere?)

Well - perhaps Digg provides a useful model (although as the link at the top of this post suggests, there are potentially problems with that approach).

The idea with Digg is that someone posts a story link, which is associated with a topic and a voting button. Other readers who 'digg' the story can vote for it. Stories with the most (recent) diggsfrloat to the top of the Dig new story list. Sort of - you can read the actual details here.

So here's where one possible academic publishing twist might come in. A social bookmarking system such as Connotea, which has academic credibility, set up the ability to post at least a link to a blog story, if not the blog post itself, to a particular journal's bookmarking/news area (a Digg site dedicated to, or topic partitioned for, that journal). Queued stories (i.e. ones that have been submitted but not 'reviewed') are listed in a hlding area. Reviewers from the journal are allowed to 'digg-yes' or 'digg-no' the post if they think it is useful/interesting/relevant, or not. This is a minimal form of lightweight review, and shouldn't impose too much on the reviewer, particularly if the digg-act has a side effect of adding the link (or not, for a digg-no vote) to the user's preferred social bookmarking system, if they use one. Stories that have enough positive votes are promoted to the Journals front page, and perhaps featured as a 'blog letter', maybe with blogosphere comments/commentary attached (perhaps public 'offcial' reviewer comments to the blog post could be made to, and hosted on, the journal's site - a bit like a social commentary in a
social bookmarking system... Shadows supports this sort of feature, I think, as does Connotea.)

Now I know this system relies on the journal having 1) a relatively large number of reviewers, who 2) are active web users, but if there is a growing tide of Academic 2.0 types, then perhaps Peer Review 2.0 models will become more likely. (ENOUGH of the 2.0, -Ed!)

Posted by ajh59 at January 10, 2006 01:16 PM