RE: Open access to research worth A3 1.5bn a year

["David Goodman" <David.Goodman@liu.edu>]

Dear Phil,

since I know you to be a better statistiician than I, what is your
evaluation of the two separate studies:

I.  Jonathan Wren's study in BMJ. 2005 May 14; 330(7500): 1128. "Open
access and openly accessible: a study of scientific publications shared
via the internet" available as
www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3D557897

The results are totally independent of the Soton studies, but use a
similar computerized search for finding the OA articles. They are however
limited to a single search engine (Google), a single type of OA file
(PDF), and a single subject (medicine).

II.  Kristin Antleman's paper in CREL Sept. 2004 65(5), p.372 -382 "Do
Open-Access Articles Have a Greater Research Impact? available at
eprints.rclis.org/archive/00002309/

This data is completely independent of any of the Soton work, uses a
manual rather than a computerized method, and covers several different
fields.

Both of them seem convincing to me, but an analysis similar to that on the
Anderson study would be much appreciated by many of us. Together with what
you have already said, it will inform us about the state of the art.

Dr. David Goodman
Associate Professor
Palmer School of Library and Information Science
Long Island University
dgoodman@liu.edu


-----Original Message-----
From: owner-liblicense-l@lists.yale.edu on behalf of Phil Davis
Sent: Thu 9/29/2005 9:17 PM
To: liblicense-l@lists.yale.edu
Subject: Re: Open access to research worth A3 1.5bn a year

I just read the JEP article (referred to by Peter Banks) comparing
articles printed in Pediatrics with other articles only appearing in the
online addition.  The authors' main findings suggest that despite wider
potential audience for articles published freely online, articles
appearing in print received more citations:

"The difference between the mean citation levels for print and online was
3.09 =B10.93 in favor of print (95% CI), meaning that an online article
could expect to receive 2.16 to 4.02 fewer citations in the literature
than if it had been printed."

Or in other words, their data do not support the hypothesis that full OA
journals receive more citations than non-full OA journals.

Yet it is methodologically difficult to rigorously test this hypothesis,
and the use of inferential statistics in this study suggests that they are
trying to generalize beyond their own journal.  In this study, the authors
compared two different sets of articles: 1) those that were selected for
inclusion in the main journal, and 2) those that were not.  Selection bias
alone may explain the different results, or at least interject a large
enough bias where the results may not accurately reflect their research
question.  In other words, it would be difficult to understand whether
their results are a reflection of accessibility, or selection bias.

Still, this article fails to support the unstated hypothesis that full OA
journal articles receive more citations than non-full OA journal articles.

For that conclusion alone, we would be wise to stay with the null
hypothesis (that is, no significant difference) unless we start seeing
compelling evidence the other way.

The other conclusion that we may come to is that it may be impossible to
come up with universal statements about Open Access publishing (i.e. it
can provide 50 - 25% more citations).  Methodology problems in designing
rigorous studies may only permit us to make anecdotal statements about
particular journals or publishing models that have very narrow parameters
for generalization.

--Phil Davis


At 06:33 PM 9/28/2005, you wrote:
>I did find one peer-reviewed study on the impact of open access on
>citation rate: "Publishing Online-Only Peer-Reviewed Biomedical
>Literature: Three Years of Citation, Author Perception, and Usage
>Experience," by Kent Anderson and his colleagues.
>
>It is a study of online-only vs. print articles in the journal =
Pediatrics.
>It does not find the same citation advantage for online publications
>claimed by Harnad and his colleagues.
>
>See http://www.press.umich.edu/jep/06-03/anderson.html
>
>Peter Banks