Who watches the Watchperson?

Nature's letters section recently included a piece entitled "Reviewers' reports should in turn be peer reviewed" (Nature, July 6th 2006, pp26). The letter explains how peer-review of reviewers' comments would hold reviewers more accountable and result in a fairer process.

At first, this might sound like another example of over-regulation: an endless regress over "Who watches the Watchperson?" ...until, finally, researchers have no time left to do research.

But I have recently come to wonder whether peer review, the 350-year-old foundation of research publishing, is in need of a health check.

I recently read the peer-review comments returned on a paper submitted by a former colleague. Having been out of the business for a couple of years, reading the comments gave me the impression that I had landed on Mars.

It was not just the menacing tone I found alien, it was also the lack of any meaningful review commentary on how the paper could be improved, or how the methods could be refined; experimental controls added and so on.

Indeed, within the space of 4 lines the reviewer had opined that the paper would damage my colleague's reputation forever, and moreover, had proposed totally new experiments and recommended that, in all reasonability, the paper should apply itself to a rather different question. This was not peer-review as I recalled it.

Would the scrutiny of peers have moderated this tirade, or at least encouraged the reviewer, perhaps, to address the paper rather than focus on his/her own research agenda?

I can only hope so. Somehow, extra checks and balances in peer-review doesn't seem like such a bad idea....