Re: Publish and Be Wrong

[Stevan Harnad <harnad@ecs.soton.ac.uk>]

On Fri, 10 Oct 2008, James J. O'Donnell wrote:

> Fromthe Economist, 10/9/08, with full text at
> http://www.economist.com/science/displaystory.cfm?story_id=3D12376658

> With so many scientific papers chasing so few pages in the most 
> prestigious journals, the winners could be the ones most likely 
> to oversell themselves=97to trumpet dramatic or important 
> results that later turn out to be false. This would produce a 
> distorted picture of scientific knowledge, with less dramatic 
> (but more accurate) results either relegated to obscure 
> journals or left unpublished.
>
> The assumption is that, as a result, such journals publish only 
> the best scientific work. But Dr Ioannidis and his colleagues 
> argue that the reputations of the journals are pumped up by an 
> artificial scarcity of the kind that keeps diamonds expensive. 
> And such a scarcity, they suggest, can make it more likely that 
> the leading journals will publish dramatic, but what may 
> ultimately turn out to be incorrect, research.

Whether it is true that there is a higher proportion of error in 
the higher quality journals is an empirical question, but there 
are reasons to be skeptical about the conclusions of this PLoS 
article. It says that science is compromised by insufficient 
"high impact" journals to publish in. The truth is that just 
about everything gets published somewhere among the planet's 
25,000 peer reviewed journals, just not all in the top journals, 
which are, by definition, reserved for the top articles -- and 
not all articles can be top articles.

The triage (peer review) is not perfect, so sometimes an article 
will appear lower (or higher) in the journal quality hierarchy 
than it ought to. But now that funders and universities are 
mandating Open Access, all research, top, middle and low will be 
accessible to everyone. This will correct any access inequities 
and it will also help remedy quality misassignment (inasmuch as 
lower quality journals may have fewer subscribers, and users may 
be less likely to consult lower quality journals). But it will 
not change the fact that 80% of citations (and presumably usage) 
goes to the top 20% of articles, though it may flatten this 
"skewness of science" (Seglen 1992) somewhat.

Seglen PO (1992) The skewness of science. Journal of the American 
Society for Information Science 43:628-38