Re: Another Poynder Eye-Opener on Open Access

[Stevan Harnad <harnad@ecs.soton.ac.uk>]

On 2011-05-11, at 8:35 PM, <jean.claude.guedon@umontreal.ca> 
wrote:
>> SH:
>> to deposit everything as unrefereed preprints in an IR
>> [instead of submitting to a journal for peer review] and
>> then wait for the better stuff to be picked up by an "overlay
>> journal". (I actually think that's utter nonsense.)
>
> JCG:
> If overlay journals (or any equivalent scheme) were to be as
> passive as Stevan describes, I would fully agree with him,
> However, it is not ridiculous to imagine consortia of
> repositories forming to promote their content, and, on top of
> that, establish a new layer of active judgement that would create
> new forms of value for these articles. The tyranny of citation
> impacts and the misuse of citation impacts must be, to say the
> least, diluted to bring back some sanity to the evaluation
> procedures presently in force in various scientific communities.

More metrics are always welcome, but in addition to -- not in 
place of -- peer review.

>> SH: The frequently mooted notion... of postpublication "peer review"
>> ... is like a kind of "evolutionarily unstable strategy" that
>> could be dipped into experimentally to test what scholarly
>> quality, sustainability, and scaleability it would yield -- until
>> (as I would predict) the consequences become evident enough to
>> induce everyone to draw back.
>
> JCG:
> What I just wrote above may partially correspond to this
> postpublication "peer review"; but then it may not. In any case,
> I would see this effort as one aiming at buiding well-defined
> quality levels, rather than ranking systems.

(1) How do "levels" differ from "ranks"?

(2) And post hoc means post hoc: prepublication means that papers 
need to meet peer review standards in order to be accepted for 
publication (and hence certified as having met the quality 
standards of the journal that accepted it). This often means 
modification of the submitted draft, not just a "grade" attached 
to it.

> JCG:
> In short, beside
> ranking everybody in ways that are sometimes difficult to justify
> (three decimals with impact factor measurements, for example...),
> it might be interesting to provide A, B, C, D grades to articles
> after publication.

(a) If it hasn't met quality standards before publication, it's 
not publication but vanity-press self-publication.

(b) Competent referees hardly have the time to review what 
reputable editors of reputable journals ask them to review:  Why 
would they do it voluntarily, or randomly, for unfiltered 
self-publications that are not even answerable to their 
recommendations? Cloud-tagging?

> JCG:
> Who would do that? Juries established by the
> consortia of repositories I mentioned earlier. Why would they do
> that? To promote their content and make it more useful
> (especially if the metadata included an extension incorporating
> these grades).

The "juries" are the referees, today. And they referee 
unpublished material that is answerable to their judgements, just 
are their judgments are answerable to the editor's judgments. The 
resultant published quality is then the journal's quality 
standard, for which it is in turn answerable.

Consortia "promoting" their own content? Sounds like 
vanity-press-squared!

>> SH:
>> Richard replied that the reason he did not dwell on Green OA,
>> which he too favors, is that he thinks Green OA progress is still
>> too slow (I agree!) and that it's important to point out that the
>> fault in the system is at the publisher end -- whether non-OA
>> publisher or OA. I continue to think the fault is at the
>> researcher end, and will be remedied by Green OA self-archiving
>> by researchers, and Green OA self-archiving mandates by research
>> institutions and funders
>
> JCG:
> If publishers did not constantly muddy the waters and create all
> kinds of variations on what one can self-archive (for example no
> publisher pdf)

PDF is irrelevant. The only relevant policy statement desired 
from the publisher is an endorsement of OA self-archiving of the 
refereed draft immediately upon acceptance for publication (and 
even that isn't necessary, just helpful, as the refereed draft 
can and should be self-archived whether or not it is made 
immediately OA):

Sale, A., Couture, M., Rodrigues, E., Carr, L. and Harnad, S. 
(2010) Open Access Mandates and the "Fair Dealing" Button. In: 
Dynamic Fair Dealing: Creating Canadian Culture Online (Rosemary 
J. Coombe & Darren Wershler, Eds.) 
http://eprints.ecs.soton.ac.uk/18511/

> researchers would not feel that there is too much
> complexity and uncertainty in self-archiving.
> So, yes, researchers are ultimately responsible, but they cannot be held
> completely responsible if the rules are made very complex and
> subtle.

The remedy for researcher passivity is deposit mandates from 
their institutions and funders (preferably making the deposit the 
mechanism for submitting publications for annual performance 
review).

>  A third stake holder deserves being mentioned in this
> context: research administrators. If their evaluation procedures
> does not include some bonus for self-archiving, why should the
> regular researcher self-archive?

Because it is a mandatory condition for research funding and the 
mandatory means of submitting publications for institutional 
performance review.

> Stevan would probably answer
> that they should because of the oA advantage, and, on the whole,
> he is right, but are they aware of that? Is it so important for
> their careers at the local and institutional level.

No, researchers are not aware enough of the OA Advantage -- yet.

And yes, impact metrics are important for careers.

> If you are a
> regular research Joe whose impact is, at bestvery limited, yet
> does his basic job all right, what does it matter to him/her that
> his impact is doubled if it is very small to start with?

He still needs to comply with institutional deposit mandates, 
still needs to submit publications for performance review, and 
still needs to meet funders' requirements (if he is funded).

> At some
> point in the scientific pyramid, competition ceases to be very
> crucial, and I suspect that a majority of "researchers" are in
> that category. In other words,the realities on the ground of
> scientific practise should be examined very carefully before
> assigning responsibility only to one category of stakeholders.

All academics need to submit to institutional performance 
evaluation. Funded researchers need to meet funders' 
requirements.

The "stakeholder" most responsible for a researcher's not 
self-archiving is the researcher. Next is the researcher's 
employer (for not mandating it) and the researcher's funder (for 
not mandating it).

Publishers who are not green on self-arciving are lowest in the 
responsibility chain. Proof: over 60% of journals are green on 
author self-archiving (and over 90% of the top journals are 
green), but only about 15% of articles are being self-archived. 
Culprit? The authors, institutions and funders, not the journals.

Chrs, Stevan
>
>>
>> Harnad, S. (2010) No-Fault Peer Review Charges: The Price of
>> Selectivity Need Not Be Access Denied or Delayed. D-Lib Magazine
>> 16 (7/8).
>
> Thank you for pointing out this article. I had not seen it.
>
> [snip]
>
> Jean-Claude Guedon