RE: Impact Factor, Open Access & Other Statistics-Based Quality Models

["David Goodman" <David.Goodman@liu.edu>]

There are many yet more subtle measurements than can be made. COUNTER user
statistics distinguish between articles and full text, and this is a very
relevant ratio--it says something about a journal if all the users go as
far as the abstract, and no further .  Eventually, COUNTER will have the
option of distinguishing the referring page, and it will be possible to
see if people link from databases, other journals, or direct entry of the
article. We would very much like to know what measures of this sort are
relevant to the users. By that point the data will be in XML format, whixh
will make automatic generation of such ratios easier.

We would also like to know if there were any feasible data segments that
would be of interest, as there's no point in devloping something that no
one cares about. Suggestions should me sent to Peter Shepherd, the project
director for COUNTER at: pshepherd@projectCounter.org

Many of these analyses depend on institutions and publishers cooperating
to permit the use of their statistics, at least in aggregated form. Both
sometimes regard this data as private, if not absolutely secret.

For examples of what can already be done with these data, see the papers
at http://people.cornell.edu/pages/pmd8/

It is difficult to manipulate usage statistics on an article by article
basis. The bibliographic references of different articles in a journal are
usually located in the same way. A publisher can (and should)  develop a
article display that makes cross linking from its pages easier, just as it
can (and should) develop an interface that makes finding articles easier.
Many of the tricks of web designers are not applicable to something as
standardized as a scientific journal.

Log analysis can discriminate multiple requests at the same time. Since it
includes the ip address from which the request is made, excessive requests
of apaper or from a single source are detectable. In fact, since such
methods are used in the theft of electronic material, all providers watch
for them very carefully, and make their site immune to automatic
harvesters. When we have open access, and publishers need no longer need
worry about theft, controls of this sort will still be part of any prudent
system management.

Yes, you can ask all our students to read all your papers, just as you can
ask them to cite all your papers in their own works. These would not add
much to the citation or the use counts for major items or journals, but
they might in either case increase a count that would otherwise be
approximately zero.

However, there are bigger errors that cannot necessarily be controlled. A
classic example is someone printing multiple copies of something
interesting for the laboratory group. As the practice goes of saving
papers on private machines (permitted by almost all licenses) knowing
their subsequent use will be difficult. Some things previously
unmeasurable will be measureable--for example, log analysis can distinguis
an individual's repeated use of the same peper--some will find it more
efficient to store them this way than on a disk or in paper

Dr. David Goodman
dgoodman@liu.edu

Member of the Executive Committee of COUNTER, but writing unofficially.

-----Original Message-----
From:owner-liblicense-l@lists.yale.edu on behalf of Gillikind@aol.com
Sent:  Fri 5/28/2004 3:17 PM
To:  liblicense-l@lists.yale.edu
Subject:  Re: Impact Factor, Open Access & Other Statistics-Based Quality  
Models

In a message dated 5/27/2004 8:21:03 PM Eastern Daylight Time,
J.F.Rowland@lboro.ac.uk writes:

> Impact factors (whether based on ISI databases or any other), calculated
> from citations or weblinking, and usage statistics (whther COUNTER-
> compliant or not) measure somewhat different things.
> 
> Impact factors measure the perceived usefulness of an article - someone
> thinks it is worth citing, or worth putting in a web link to.  Usage stats
> measure the fact that someone thought it was worth looking at (cursorily
> or more), perhaps a rather lower standard of usefulness.
> 
> Time will tell which of these measures is regarded as the more meaningful.  
> If both parameters could be adequately standardised, it might be
> interesting to look at the ratio between the two.  A high "impact factor
> per use" would imply that of those who looked at the paper, a large
> proportion found it useful enough to cite it.  A low "impact factor per
> use" would imply that lots of people looked at it but few cited it - the
> sort of fate that might befall one of the less impressive 
> papers from a well-know author, perhaps?

I think this discussion is overlooking one major difference between the
ideas of impact factor based on citation vs impact factor based on usage.
Citation means that one author is citing another for some reason - this is
takes some degree of 'effort' or interest and is not that easy to
manipulate.  It could be done, but manipulation would take some effort.

However, impact factor based on web usage is a very easy number to change
and influence. The location for a link to a paper on web page alone can
change the usage rate (higher placement on a web page has a greater chance
of being used, especially if the link is not below the scrolling area),
not to mention single users making multiple requests for one paper or the
vast variations of spiders/robots that could be developed.

David Gillikin