Re: The religion of peer review

["Lisa Dittrich" <lrdittrich@aamc.org>]

I agree 100%.  I am amazed by how suspicious people are of peer 
review, especially the assumption that peer-reviewers are all 
"biased"  (usually against).  The peer reviewers I work with--and 
they number in the 100s--are almost uniformly courteous, 
thoughtful, and respectful.  And the bias they most often show is 
bias FOR an area of study, not against.

And this is where the editor must come in.  No quality journal 
chooses articles based on peer review alone--it is one component. 
The second is editorial judgement.  A good editor will spot the 
biases, can tell when a tentative "reject--option to revise" 
really means a resounding "reject," etc.  And yes, there are bad 
eggs in the lot, as in all walks of life.  Also, copy and 
substantive editors are there to catch the finer errors that 
reviewers will almost always miss--errors that rarely effect the 
overall findings of a paper but that still need to be corrected.

So many of these discussions--about publishers (esp. 
publishers!!), peer-reviewers, etc.--seem to operate on the 
assumption that we're all out to take advantage of one another. 
Publishers are out there just dreaming up ways to gouge libraries 
and cheat authors.  Peer reviewers are trying to steal work, or 
just help their friends.  Authors are trying to publish 
fraudulent results.  Of course, there are people doing that--many 
make the news.  The rest of us plugging away day-to-day doing our 
best to publish our journals, responsibly review mss., and 
conduct and publish ethical research are, of course, rarely 
front-page news.

Lisa

Lisa Dittrich
Managing Editor
Academic Medicine
2450 N Street NW
Washington,D.C. 20037
lrdittrich@aamc.org (e-mail)
202-828-0590 (phone)
202-828-4798 (fax)
Academic Medicine's Web site: www.academicmedicine.org

>>> harnad@ecs.soton.ac.uk 2/22/2006 3:50 PM >>>

Peer review is just qualified specialists vetting the work of 
their fellow-specialists before further specialists risk the time 
and effort of trying to build on it. Sometimes it's about 
protecting the public from health risk.

A religion - Anyone have a better idea? No vetting? Unqualified 
vetting? Opinion polls? Pot luck?

No one who has had to sit for a quarter century in a journal 
editorial office dealing with raw, unfiltered submissions has any 
doubt about the value, indeed the necessity, of qualified, 
answerable vetting, to protect researchers time and effort; but 
armchair speculation about it will no doubt proceed apace...

      Harnad, Stevan (1998/2000/2004) The invisible
      hand of peer review. Nature [online] (5 Nov. 1998)
      http://helix.nature.com/webmatters/invisible/invisible.html
      Longer version in Exploit Interactive 5 (2000):
      http://www.exploit-lib.org/issue5/peer-review/ and
      in Shatz, B. (2004) (ed.) Peer Review: A Critical
      Inquiry. Rowland & Littlefield. Pp. 235-242.
      http://cogprints.org/1646/

> "THE RELIGION OF PEER REVIEW
>
> Despite a lack of evidence that peer review works, most
> scientists (by nature a skeptical lot) appear to believe in
> peer review. It's something that's held "absolutely sacred" in
> a field where people rarely accept anything with "blind faith,"
> says Richard Smith, former editor of the BMJ and now CEO of
> UnitedHealth Europe and board member of PLoS. "It's very
> unscientific, really." This from a very interesting article -
> worth reading through:
>
> Alison McCook.  Is Peer Review Broken?  The Scientist:  Magazine of
> the Life Sciences 20:2, page 26.  at:
> http://www.the-scientist.com/2006/2/1/26/1/