Re: RECENT MANUAL MEASUREMENTS OF OA AND OAA

[Phil Davis <pmd8@cornell.edu>]

It is common for those who don't understand research to discredit 
results on irrational grounds, especially when they do not 
support one's own opinion -- or worse, when they challenge one's 
dogma.  It was surprising to see an established researcher and 
journal editor make such claims.  The Goodman et al. article is 
based on a sample of nearly 1,200 articles.  This is no "tiny" 
sample when one realizes that this study was done manually.  In 
addition, the study used defendable and reproducible inferential 
statistical tests -- it was not necessary to do a complete census 
of all OA articles to back up their claims.

Goodman et al. bring up serious issues with regard to validity 
(measuring what you say you are measuring), and reliability 
(being able to reproduce results).  It would be much more 
constructive if Stevan spent time trying to find problems in 
their methodology and analysis rather than discrediting their 
research on emotional grounds.  The shrillness does not appear to 
come from Goodman's side.

--Phil Davis


At 08:41 PM 1/22/2006, you wrote:
> Before anyone gets too excited about the tiny Goodman et al. test
> result, may I suggest waiting a couple of weeks, when we will be
> reporting the results of a far bigger and more accurate test of
> the robot's accuracy?
>
> Those who (for some reason) were hoping that the robot would
> prove too inaccurate and that the findings on the OA advantage
> would prove invalid may be disappointed with the outcome. I can
> already say that overinterpretations of the tiny Goodman et al.
> test as showing that the OA/OAA findings to date are "worthless"
> are rather overstated even on the meagre evidence to date,
> especially since two thirds of the published findings on the OA
> citation advantage are not even robot-based!.
>
> (This shrillness also seems to me to be trying to make rather
> much out of having actually done rather little!)
>
> As to the separate issue of how to treat the OA journal article
> counts (as opposed to the counts for the self-archived non-OA
> journal articles): We count it all, of course, but only use the
> non-OA journal article counts in calculating the OA advantage,
> because those are (necessarily) within-journal ratios, and
> citation ratios of zero and infinity are meaningless. Think about
> it.
[SNIP]

> Stevan Harnad