[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Alma Swan: The OA citation advantage: Studies and results to date
- To: email@example.com
- Subject: Re: Alma Swan: The OA citation advantage: Studies and results to date
- From: Stevan Harnad <firstname.lastname@example.org>
- Date: Fri, 12 Mar 2010 20:21:30 EST
- Reply-to: email@example.com
- Sender: firstname.lastname@example.org
On Thu, Mar 11, 2010 at 11:15 PM, Philip Davis <pmd8 -- cornell.edu> wrote on the liblicense list: > Stevan, > > In my critique of this review today, I commented on the > inappropriate use of meta-analysis to the empirical OA citation > studies: > > "Meta-analysis is set of powerful statistical techniques for > analyzing the literature. Its main function is to increase the > statistical power of observation by combining separate empirical > studies into one uber-analysis. It's assumed, however, that the > studies are comparable (for instance, the same drug given to a > random group of patients with multiple myeloma), but conducted at > different times in different locales. > > This is not the case with the empirical literature on open access > and citations. Most of the studies to date are observational > (simply observing the citation performance of two sets of > articles), and most of these use no statistical controls to > adjust for confounding variables. Some of the studies have > focused on the effect of OA publishing, while others on OA > self-archiving. To date, there is still only one published > randomized controlled trial. > > Conducting a meta-analysis on this disparate collection of > studies is like taking a Veg-O-Matic to a seven-course dinner. > Not only does it homogenize the context (and limitations) of each > study into a brown and unseemly mess, but it assumes that > homogenization of disparate studies somehow results in a clearer > picture of scientific truth." > > see: > Rewriting the History of the Open Access Debate > http://j.mp/d91Jk2 > > --Phil Davis Phil, Thanks for the helpful feedback. I'm afraid you're mistaken about meta-analysis. It can be a perfectly appropriate statistical technique for analyzing a large number of studies, with positive and negative outcomes, varying in methodological rigor, sample size and effect size. It is a way of estimating whether or not there is a significant underlying effect. I think you may be inadvertently mixing up the criteria for (1) eligibility and comparability for a meta-analysis with the criteria for (2) a clinical drug trial (for which there rightly tends to be an insistence on randomized control trials in biomedical research). Now I would again like to take the opportunity of receiving this helpful feedback from you to remind you about some feedback I have given you repeatedly http://bit.ly/dkieVi on your own 2008 study -- the randomized control trial that you suggest has been the only methodologically sound test of the OA Advantage so far: You forgot to do a self-selection control condition. That would be rather like doing a randomized control trial on a drug -- to show that the nonrandom control trials that have reported a positive benefit for that drug were really just self-selection artifacts -- but neglecting to include a replication of the self-selection artifact in your own sample, as a control. For, you see, if your own sample was too small and/or too brief (e.g., you didn't administer the drug for as long an interval, or to as many patients, as the nonrandom studies reporting the positive effects had done), then your own null effect with a randomized trial would be just that: a null effect, not a demonstration that randomizing eliminates the nonrandomized drug effect. (This is the kind of methodological weakness, for example, that multiple studies can be weighted for, in a meta-analysis of positive, negative and null effects.) [I am responding to your public feedback here, on the liblicense and SERIALST lists, but not also on your SSP Blog, where you likewise publicly posted this same feedback (along with other, rather shriller remarks) http://j.mp/d91Jk2 because I am assuming that you will again decline to post my response on your blog, as you did the previous time that you publicly posted your feedback on my work both there http://bit.ly/8LK57u and elsewhere -- refusing my response on your blog on the grounds that it had already been publicly posted elsewhere!...] -- Stevan Harnad PS The idea of doing a meta-analysis came from me, not from Dr. Swan.
- Prev by Date: Re: Georgia State University electronic course reserves copyright case nearing resolution?
- Next by Date: University of Chicago Press Joins Current Scholarship Program
- Previous by thread: Re: Alma Swan: The OA citation advantage: Studies and results to date
- Next by thread: Does "free" lead to "paid"?