Re: Randomized controlled study of OA publishing

[Stevan Harnad <harnad@ecs.soton.ac.uk>]

On Wed, 13 Dec 2006, Phil Davis wrote:

> We are in the process of conducting a randomized controlled 
> study of Open Access publishing to ascertain if free-access to 
> scholarly articles increases readership and citation impact.
>
> To date, the limited numbers of empirical studies have employed 
> methodologies that do not control for potential biases and 
> competing explanations.  A citation advantage may be the result 
> of increased access, but may equally be the result of higher 
> quality articles being published as OA.  By using a randomized 
> controlled methodology, we will be in a stronger position to 
> attribute a citation advantage [if discovered] to increased 
> access.
>
> During the feasibility stage of our study, we will be 
> partnering with the American Physiological Society and 
> experimenting with eleven of their journals.  Another of their 
> journals allows author-supported OA publishing will be used as 
> a control.  We will be studying the performance of these 
> articles, in terms of article downloads and citations, for the 
> next four years.

OA advocates and sceptics alike welcome this study.

But for those who would like to hear results sooner than four 
years from now (so as to be able to act on them sooner!) I will 
be reporting some preliminary findings along the same lines in a 
few days (in the context of a reply to Henk Moed), but based on 
comparing citation counts of articles in 2004 that were 
self-archived in Institutional Repositories with and without 
self-archiving mandates. The comparison will be: 
self-archived/self-selected (Ss), self-archived/mandated (Sm), 
non-archived/self-selected (Ns), non-archived/mandated (Nm -- 
i.e., noncompliant with mandate), comparing, always, within the 
same journal.

The sample is small (only 5 mandatory IRs so far), and the 
mandates are still young (so the compliance is still spotty), but 
the trends are interesting. The impatient can already see the 
data summary at:

     http://www.ecs.soton.ac.uk/~harnad/Temp/moedrep.ppt

Stevan Harnad