Re: Does the arXiv lead to higher citations and reduced publisher downloads?

[Stevan Harnad <harnad@ecs.soton.ac.uk>]

On Thu, 23 Mar 2006, Phil Davis wrote:

> 1) You call the citation differential an "Open Access Advantage",
> thus affirming that access is both the cause and effect of the
> phenomena.  This is a circular argument.  But more importantly,
>
> 2) Your theory is unrefutable.

There's nothing circular here (though there's plenty that's 
getting repetitious)! If Phil doesn't like to call the OAA the 
Open Access Advantage, call it the Open Access Numerator 
Advantage (OANA): Either way, it means, operationally, that when 
you divide the average citation counts for self-archived articles 
by the average citation counts for non-self-archived articles in 
the same journal/issue, the numerator is always bigger. That's 
the OANA.

The rest is about the *cause* of that OANA. Phil likes QB: an 
author bias toward self-archiving higher-quality, higher-citation 
articles. I also like QA: increased usage and citation of freely 
available articles, with greater OANA the higher the quality of 
the article. No circularity for either inference, but no direct 
test of causality either, for either inference.

Testing causality requires a more direct control and analysis of 
the timing. For example, if we have deposit dates as well as 
publication dates, we can look at articles that are 
self-archived, say, two years after their publication date, and 
we can equate articles for their citation counts up to the date 
they were self-archived. If Phil is right, there should be no 
difference at year three for the citation counts of self-archived 
articles, compared to control non-self-archived articles that had 
the same citation counts at year two. If I am right, there should 
still be a OANA at year three.

That is just one of many ways one could test causation.

So it's not about irrefutable hypotheses, it's about unreflective
interpretations.

Stevan Harnad

PS Since Arxiv authors tend to deposit preprints before publication
and postprints at publication, but rarely postprints much later than
publication, we cannot do this particular study in Arxiv; it could be
done in CogPrints, but the N's would be small. Once IRs with date-stamps
grow, we will be able to do it there.

On Thu, 23 Mar 2006, Phil Davis wrote:

> Stevan Harnad wrote:
>
> "What your data show is that the OA Advantage (which everyone
> confirms) is stronger on the high-end, and this could either be
> because people tend to self-archive high-end articles more (QB),
> or because the OA Advantage is stronger on the high end (QA).
> Either way it's a quality effect. One way it's a Quality Bias
> (QB), the other way it's a Quality Advantage (QA).  You think
> it's mostly QB, I think it's mostly QA. The data are compatible
> with both. More fine-tuned causal tests are needed to decide."
>
> Phil Responds:
>
> Stevan, you seem to be referring repeatedly to your web blog,
> "Open Access Archivangelism", that
>
> OA Impact Advantage = EA + (AA) + (QB) + QA + (CA) + UA
> http://openaccess.eprints.org/index.php?/archives/29-guid.html
>
> While I certainly agree that a citation advantage can have
> multiple and conflating causes, you call the result of all of
> these causes the "Open Access Advantage". In your equation, ALL
> possible causes (early view, arXiv, quality bias, quality
> advantage, competitive advantage, and usage advantage) can
> support your OAA theory.  This is problematic on a fundamental
> scientific level for two reasons:
>
> 1) You call the citation differential an "Open Access Advantage",
> thus affirming that access is both the cause and effect of the
> phenomena.  This is a circular argument.  But more importantly,
>
> 2) Your theory is unrefutable.  When everything confirms the Open
> Access Advantage, OAA becomes meaningless as a scientific theory.
> The philosopher of science Karl Popper would have called OAA
> "unfalsifiable" and therefore unscientific.
>
> So if you insist on describing a possible quality differential
> between articles deposited in the arXiv and those that are not an
> "Open Access Advantage", then I have merely reaffirmed that our
> results support OAA.  But so does the weather, the stock market,
> and my horoscope for today (which by the way, says that I should
> avoid Sagittarius)
>
> --Phil Davis