RE: citations as indicators of quality

["Rhonda Oliver" <rhonda.oliver@portlandpress.com>]

There is an excellent article related to this topic that was publihed in
ASBMB today (July 2007) by Vincent Hascall, Johan Bollen and Richard
Hanson entitled "Impact Factor Rankled" and those of you with a
mathematical bent might try the original paper by Bollen et al. (2006)
Scientometrics 69, 669-687, that applies the "notions of popularity and
prestige" to the domain of scholarly assessment. Their thesis is that by
counting just the amount of citations and disregarding the prestige of
the citing journals, the ISI IF is a metric of popularity, not prestige.
They demonstrated "how how a weighted version of the popular PageRank
algorithm can be used to obtian a metric that reflects prestige" and
give us new metric the "Y" factor to get our heads around (a combination
of the ISI IF and the weighted page rank)for a better understanding of
journal status.

Rhonda Oliver

-----Original Message-----
From: owner-liblicense-l@lists.yale.edu
[mailto:owner-liblicense-l@lists.yale.edu] On Behalf Of B.G. Sloan
Sent: 20 November 2007 15:49
To: liblicense-l@lists.yale.edu
Subject: Re: citations as indicators of quality

Sandy Thatcher said:

    "It begins by noting one fundamental flaw in any citation analysis by
quoting another author thus: 'if [Journal X] published an execrable
paper that attracted a million critical citations as an example of
appalling practice, all other papers previously and later published in
that journal would suddenly be much more highly ranked.'"

    This reminds me of something I asked about a couple of years ago in
another forum...

    Most of the citation analysis studies I see nowadays involve
quantitative analyses for the most part. Just wondering if many people
are into studying citations from a qualitative standpoint?
For example, in a lot of studies a citation is a citation is a citation,
with little concern for how a given paper was cited qualitatively within
the context of the citing paper. For example, an author could cite a
paper very positively, or the citation could be pretty much
value-neutral, or, as Sandy notes, the citation could be negative. But
in a quantitative analysis these various types of citations pretty much
all carry the same weight.

    When I looked into this several years ago, a number of people alerted
me to some qualitative citation studies. The interesting thing is that
most of these studies were maybe 20 years old, at least. It almost
seemed like people got away from doing qualitative citation analyses as
it got easier to do quantitative analyses, i.e., as databases such as
the ISI indices became available in electronic form.

    Anyway, I am interested in hearing about relatively recent
qualitative citation analysis.

    Thanks,

    Bernie Sloan