Evaluation and open access & timing

[Heather Morrison <heatherm@eln.bc.ca>]

greetings all,

There appears to be a trend towards evaluation of open access
publications, which is, of course, welcome and necessary.  It is very
encouraging to see early indications that open access journals and
articles have good impact, which only makes sense.

While evaluation is essential, I would suggest that timing can be a
critical factor in evaluation, from either a scientific or business
viewpoint.

When conducting a scientific study, evaluation of results does not
conclude until the effects of what one is studying take place.  For
example, in a pharmaceutical study, one might evaluate impact of a new
drug shortly after starting a treatment session, but if you anticipate
that it might take a month for the drug to take effect, you do not draw
any conclusions until measurements are taken after this time.

New businesses generally take some time to become profitable.  Imagine
what would happen to the restaurant business, for example (which already
has a very high failure rate), if bankers formed their conclusions about
profitability two weeks after opening.

For an example closer to my own profession, if a library designs a new
comprehensive information literacy program, involving every department,
program, and faculty member in a university, it would be a very good idea
to evaluate that very first actual session, and adjust the program
accordingly; but the full impact of the program could not be known until
the first year students that enter when the program is begun have finally
graduated.

Picture premature evaluation of the switch from leaded to unleaded
gasoline:  all those full unleaded gas tanks, so few cars able to use
them.  How awful from an economic point of view, and how misleading when
we look back from this moment in history.

Shifting to open access involves inventing and implementing new ways of
publishing, as well as authors and others changing the way we do things.  
This takes time.  The proper time to begin evaluating open access journals
will be in about 5 years time or so.  Until then, some preliminary
evaluation - to adjust models, that sort of thing - is appropriate.  
Conclusions, however, are not.

As always, this is purely my personal opinion as a professional librarian,
not an official position of ELN.

Heather G. Morrison
BC Electronic Library Network
Email:  heatherm@eln.bc.ca