[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

University Cross-Check on Thomson ISI Citation Metrics

         ** Cross-Posted **

On Wed, 19 Dec 2007, Armbruster, Chris wrote:


Subject: [SIGMETRICS] FW: GENERAL: accuracy of Thomson data
Incorrect journal abbreviations and non-ISI sources Citations
http://users.fmg.uva.nl/lleydesdorff/list.htm
Calls for an audit of WoS data.

 Would you trust the situation to improve if digital repositories
 (institutional, disciplinary and/or national) were to provide data
 in future?
Of course -- and particularly institutional repositories (IRs), since the universities and research institutions themselves are the primary content-providers!

http://roar.eprints.org/

 One would possibly expect that a decentralised solution
 would provide more comprehensive (types of publication,
 languages etc.) and more accurate coverage,
Not because it was "decentralized" but because the authors' institutions (not their journals!) are the primary content-providers and have a direct stake in the discoverability, validity and attribution of their own research output.

but one might also worry that the corpus will be less well defined....
How will it be less well defined? All journal articles -- their full texts and metadata, *including their cited references* -- will be deposited, tagged, harvestable, harvested, indexed and analyzed by (open and transparent) software, globally. The reference lists of each article will provide a redundant, distributed cross-check on all the articles they cite, many times over. Central indexes of journals and their contents (like Thomson ISI) will provide further cross-checks on validity, and will be able to correct their own data against the primary OA database.

But the prerequisite for all of this is that the primary content must be provided in the author's own institution's Open Access (OA) IR.

http://openaccess.eprints.org/index.php?/archives/338-guid.html

Hence, what would you think if repositories developed a system of author registration (unique identifier, institutional affiliation) and provided data?
It is an obvious and natural solution -- once all the primary content is being systematically self-archived in the author's own OA IR. (Not while only 15% of it is being haphazardly deposited willy-nilly -- in IRs, Central Repositories, and on arbitrary websites.)

The way to ensure that all of this is systematically and reliably done is for researchers' own institutions (and funders) to mandate the self-archiving of their own published research output:

http://openaccess.eprints.org/index.php?/archives/71-guid.html

 What is the scope for delivering scientometrics to the digital
 workbench of scientists?  I have anecdotal evidence that review
 panels (for major grants, tenure etc. - often very senior scientists)
 routinely use software and search engines to look up the citation data
 and indices of applicants and candidates.
All that is need is for research institutions and funders to mandate that the all-important primary data itself be provided (by mandating self-archiving). The rest (the harvesting and the software) will take care of itself, many times over. It is that primary distributed institutional OA database itself that is still missing today, and urgently overdue.

If we were not to dismiss this simply as evaluation mania, but to say that all scientists (senior and junior) now need tools for metric research evaluation to reduce complexity on an everyday basis (and develop strategies for research, teaching, publishing and networking) - is scientometrics developed enough to be a reliable tool?
What is not "developed enough" is university and research-funder policy for exposing and managing their own research assets online -- for which the essential component is each researcher's institution's own OA IR, reliably filled with each institution's own research article output. Scientometrics is waiting to data-mine that OA corpus, once universities (and funders) get around to doing the obvious (and already overdue) thing in the online era: to mandate the deposit of their research output in the researcher's OA IR.

http://openaccess.eprints.org/index.php?/archives/136-guid.html
http://www.eprints.org/openaccess/policysignup/

 Context: for the Max Planck Digital Library I am looking into the
 potential of digital libraries and repositories for the generation,
 collection and evaluation of scientometric data.
Splendid! And are the Max-Planck Institutes at long last getting around to implementing their "Berlin Declaration" by mandating the deposit of their own research output in their own IR (and making the IR OA)?

http://www.dlib.org/dlib/march05/harnad/03harnad.html
http://www.eprints.org/events/berlin3/outcomes.html

For some idea of how long this has been taking at the MPIs, Google:
site:users.ecs.soton.ac.uk amsci ("max planck" OR mpi)

Brody, T., Carr, L., Gingras, Y., Hajjem, C., Harnad, S. and Swan, A.
(2007) Incentivizing the Open Access Research Web:
Publication-Archiving, Data-Archiving and Scientometrics. CTWatch
Quarterly 3(3). http://eprints.ecs.soton.ac.uk/14418/

Harnad, S. (2007) Open Access Scientometrics and the UK Research
Assessment Exercise. Proceedings of 11th Annual Meeting of the
International Society for Scientometrics and Informetrics 11(1) : 27-33,
Madrid, Spain. Torres-Salinas, D. and Moed, H. F., Eds.
http://eprints.ecs.soton.ac.uk/13804/

Shadbolt, N., Brody, T., Carr, L. and Harnad, S. (2006) The Open
Research Web: A Preview of the Optimal and the Inevitable, in Jacobs,
N., Eds. Open Access: Key Strategic, Technical and Economic Aspects.
Chandos. http://eprints.ecs.soton.ac.uk/12453/

Stevan Harnad
AMERICAN SCIENTIST OPEN ACCESS FORUM:
http://amsci-forum.amsci.org/archives/American-Scientist-Open-Access-Forum.html
http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/