Publish and Be Wrong

["James J. O'Donnell" <jod@georgetown.edu>]

Fromthe Economist, 10/9/08, with full text at
http://www.economist.com/science/displaystory.cfm?story_id=3D12376658

Publish and Be Wrong

IN ECONOMIC theory the winner's curse refers to the idea that
someone who places the winning bid in an auction may have paid
too much. Consider, for example, bids to develop an oil field.
Most of the offers are likely to cluster around the true value of
the resource, so the highest bidder probably paid too much.

The same thing may be happening in scientific publishing,
according to a new analysis. With so many scientific papers
chasing so few pages in the most prestigious journals, the
winners could be the ones most likely to oversell themselves--to
trumpet dramatic or important results that later turn out to be
false. This would produce a distorted picture of scientific
knowledge, with less dramatic (but more accurate) results either
relegated to obscure journals or left unpublished.

In Public Library of Science (PloS) Medicine, an online journal,
John Ioannidis, an epidemiologist at Ioannina School of Medicine,
Greece, and his colleagues, suggest that a variety of economic
conditions, such as oligopolies, artificial scarcities and the
winner's curse, may have analogies in scientific publishing.

. . .

The assumption is that, as a result, such journals publish only
the best scientific work. But Dr Ioannidis and his colleagues
argue that the reputations of the journals are pumped up by an
artificial scarcity of the kind that keeps diamonds expensive.
And such a scarcity, they suggest, can make it more likely that
the leading journals will publish dramatic, but what may
ultimately turn out to be incorrect, research.

[snip]

****