REPORT: Little Evidence for Effectiveness of Scientific ReerReview

["Gerry Mckiernan" <gerrymck@iastate.edu>]

Little Evidence for Effectiveness of Scientific Peer Review

A Most Revealing (and Perhaps Disturbing[?]) Report on the Effectiveness
of Scientific Peer Review

[Thanks, Ben Toth, NHS Information Authority (UK) for informing me of this
major report!!!]

The report focuses on biomedical journals. I'd be interested in Any and
All similar studies for *other* disciplines.

/Gerry

Gerry McKiernan 
Iowa State University
Ames IA 50011

gerrymck@iastate.edu 

***********************************************************

Little Evidence for Effectiveness of Scientific Peer Review
by  Caroline White  / BMJ 2003;326:241 ( 1 February )
[http://bmj.com/cgi/content/full/326/7383/241/a ]
[ http://bmj.com/cgi/reprint/326/7383/241/a.pdf ]
DateLine: London

Despite its widespread use and costs, little hard evidence exists that
peer review improves the quality of published biomedical research,
concludes a systematic review from the international Cochrane
Collaboration.

[ http://www.update-software.com/Cochrane/MR000016.pdf ]

Yet the system, which has been used for at least 200 years, has only
recently come under scrutiny, with its assumptions about fairness and
objectivity rarely tested, say the review authors. With few exceptions,
journal editors-and clinicians-around the world continue to see it as the
hallmark of serious scientific endeavour. Published last week, the review
is the third in a series from the Cochrane Collaboration Methods Group.  
...

Only the latter escapes a drubbing, with the reviewers concluding that
technical editing does improve the readability, accuracy, and overall
quality of published research.The Cochrane reviewers based their findings
on 21 studies of the peer review process from an original trawl of only
135.

[snip]

On the basis of the current evidence, "the practice of peer review is
based on faith in its effects, rather than on facts," state the authors,
who call forlarge, government funded research programmes to test the
effectiveness of the system and investigate possible alternatives. "As the
information revolution gathers pace, an empirically proven method of
quality assurance is of paramount importance," they contend. Professor Tom
Jefferson, who led the Cochrane review, suggested that further research
might prove that peer review, or an evolved form of it, worked. At the
very least, it needed to be more open and accountable. But he said that
there had never even been any consensus on its aims and that it would be
more appropriate to refer to it as "competitive review."

[snip]

REPORT FULL-TEXT AVAILABLE 

Editorial peer-review for improving the quality of reports of biomedical
studies

[ http://www.update-software.com/Cochrane/MR000016.pdf ]

Jefferson TO, Alderson P, Davidoff F, Wager E

This is a reprint of a Cochrane methodology review, prepared and
maintained by the Cochrane Collaboration and published in The Cochrane
Library 2003, Issue 1