Craig et al.'s review of the OA citation advantage

[Stevan Harnad <harnad@ecs.soton.ac.uk>]

                     ** Cross-Posted **

     Craig, Ian; Andrew Plume, Marie McVeigh, James Pringle & Mayur
     Amin (2007) Do Open Access Articles Have Greater Citation Impact?
     A critical review of the literature. Journal of Informetrics.
     http://www.publishingresearch.net/Citations-SummaryPaper3_000.pdf.pdf

I've read Craig et al.'s critical review ("proposed by the 
Publishing Research Consortium") concerning the OA citation 
Impact effect and will shortly write a short, mild review. But 
first here is a commentary from Bruce Royan, followed by a few 
remarks from me. -- Stevan Harnad

>    ---------- Forwarded message ----------
>    Date: Sun, 20 May 2007 08:00:02 +0100
>    From: Bruce Royan <bruce.royan-- concurrentcomputing.co.uk>
>    To: diglib--infoserv.inist.fr
>    Subject: RE: [DIGLIB] Recent research tempers citation advantage of open
>           access
>
>    Sally claims that according to this article "the relationship between
>    open access and citation, once thought to be almost self-evident,
>    has almost disappeared."

>    Now I'm no Informetrician, but my reading of the article is that
>    the authors reluctantly acknowledge that Open Access articles do
>    have greater citation impact, but claim that this is less because
>    they are Open Access per se, and more because:

>               -they are available sooner than more conventionally
>               published articles, or

>               -they tend to be better articles, by more prestigious
>               authors

>    Sally's point of view is understandable, since she is employed by a
>    consortium of conventional publishers. It's interesting to note that
>    the employers of the authors of this article are Wiley-Blackwell,
>    Thomson Scientific, and Elsevier.

>    Even more interesting is that, though this article has been accepted
>    for publication in the conventional "Journal of Informetrics", a pdf
>    of it (described as a summary, but there are 20 pages in JOI format,
>    complete with diagrams, references etc) has already been mounted on
>    the web for free download, in what might be mistaken for an example
>    of green route open access.

>    Could this possibly be in order to improve the article's impact?
>
>    Professor Bruce Royan   http://www.linkedin.com/in/bruceroyan
>    Concurrent Computing Limited.     Registered Office:
>    Wellington House, Aylesbury Rd, Princes Risborough, Bucks HP27 0JP

It is notoriously tricky (at least since David Hume) to "prove" 
causality empirically. The thrust of the Craig et al. critique is 
that despite the fact that virtually all studies comparing the 
citation counts for OA and non-OA articles keep finding the OA 
citation counts to be higher, it has not been proven beyond a 
reasonable doubt that the relationship is causal.

I agree: It is merely highly probable, not proven beyond a 
reasonable doubt, that articles are more cited because they are 
OA, rather than OA merely because they are more cited (or both OA 
and more cited merely because of a third factor).

And I also agree that not one of the studies done so far is 
without some methodological flaw that could be corrected.

But it is also highly probable that the results of the 
methodologically flawless versions of all those studies will be 
much the same as the results of the current studies. That's what 
happens when you have a robust major effect, detected by 
virtually every study, and only ad hoc methodological cavils and 
special pleading to rebut each of them with.

But I am sure those methodological flaws will not be corrected by 
these authors, because -- OJ Simpson's "Dream Team" of Defense 
Attorneys comes to mind -- Craig et al's only interest is 
evidently in finding flaws and alternative explanations, not in 
finding out the truth -- if it goes against their client's 
interests...

   Iain D.Craig: Wiley-Blackwell
   Andrew M.Plume, Mayur Amin: Elsevier
   Marie E.McVeigh, James Pringle: Thomson Scientific

Here is a preview of my rebuttal. It is mostly just common sense, 
if one has no conflict of interest, hence no reason for special 
pleading and strained interpretations:

(1) Research quality is a necessary, but not a sufficient 
condition for citation impact: The research must also be 
accessible to be cited.

(2) Research accessibility is a necessary but not a sufficient 
condition for citation impact: The research must also be of 
sufficient quality to be cited.

(3) The OA impact effect is the finding that an article's 
citation counts are positively correlated with the probability 
that that article has been made OA: The more an article's 
citations, the more likely that that article has been made OA.

(4) This correlation has at least three causal interpretations:

     (4a) OA articles are more likely to be cited.

     (4b) More-cited articles are more likely to be made OA.

     (4c) A third factor makes it more likely that some articles will be
     both made OA and more cited.

(5) Each of these causal interpretations is correct, and hence a
contributor to the OA impact effect:

     (5a) The better the article, the more likely it is to be cited,
     hence the more citations it gains if it is made more accessible
     (3a). (OA Article Quality Advantage, QA)

     (5b) The better the article, the more likely it is to be made OA
     (3b). (OA Article Quality Bias, QB)

     (5c) 10% of articles (and authors) receive 90% of citations. The
     authors of the better articles know they are better, and hence are
     more likely both to be cited and to make their articles OA, so as
     to maximize their visibility, accessibility and citations (3c). (OA
     Author QB and QA)

(6) In addition to QB and QA, there is an OA Early Access effect (EA):
providing access earlier increases citations.

(7) The OA citation studies have not yet isolated and estimated the
relative sizes of each of these (and other) contributing components
(OA also increases downloads, and downloads are correlated with later
citations).

(8) But the handwriting is on the wall as to the benefits of making
articles OA, for those with eyes to see, and no conflicting interests
to blind them.

I do agree completely, however, with erstwhile Princetonian Bob May's call
for "an evidence-based approach to the scholarly communications debate."

Stevan Harnad
http://www.princeton.edu/~harnad/
http://amsci-forum.amsci.org/archives/American-Scientist-Open-Access-Forum.html