REALISM AND IDEALISM IN INSTRUCTIONS TO REFEREES

Sheila M McNab
Buys Ballot Laboratory, Universiteit Utrecht, Princetonplein 5, Postbus 80.000, 3508 TA Utrecht, The Netherlands

Objective: By analyzing instructions to referees for 5 scientific journals to assess whether the demands made on referees are reasonable (realistic) or unreasonable (idealistic) and thereby to influence journals to modify their instructions.

Design: Five sets of instructions (4 in physics, 1 in geophysics) were compared by an experienced authors' editor. The journals originated in the US(2), the UK (1), and Sweden (1). The 3 selection criteria were: importance of journal, instructions covered at least 9 items, and were available from referees working the same department as the authors editor.

Results: All journals request timely return of reviews (range: 1-3 weeks). Other items cover: clarity of tables and diagrams (5 journals), originality and soundness of science (4), adequacy of title and abstract (4, only 1 referring to suitability for inclusion in an abstracting service), overlap with published work (3, 1 requesting appropriate references), need to shorten/lengthen certain parts (3, 1 asking the referee to specify what should be omitted), suitability for journal (3). All 5 journals ask that proper credit be given to related work, 1 stating specifically that authors (not referees) are responsible for awareness of published research and for deficiencies in references. All 5 journals ask about readability and acceptability of the language; only 1 requests the referee not to correct grammar and punctuation unless the science needs clarification. Although 5 journals stress confidentiality, 4 permit referees to pass the paper to a colleague.

Conclusions: Reviews should be tackled promptly, but in academia reuesting return of the review within a week is unrealistic. It is reasonable to ask a referee to assess the soundness of the science and the adequacy of the title and abstract. Advising the author to shorten/extend parts of the paper should also be part of the referee's remit. However, it is questionable whether the average referee is really capable of judging originality and identifying overlaps with published work. Since the editor is responsible for journal policy, it is the editor's task to decide on paper's suitability for a particular journal. Checking that credit is given to related work and that references are balanced is more realistic than being held responsible for missing references. The interpretation of readability will depend on the reviewer's mother tongue. Reviewers who speak English as a second language should not be required to correct the English in a manuscript; a paper in poor English should be submitted to an Anglophone for a language review. Confidentiality could be breached if a referee is allowed to pass on a paper to a colleague. A reference to the minimum time to be spent on a review would increase the credibility of comprehensive instructions.

Return to Session Information