Peer Review in Journals Indexed in Index
Medicus
(JAMA. 1994;272:156-158)
Lois Ann Colaianni, MLS
Objective.--To determine whether peer review policies
are published in English-language journals indexed in Index
Medicus and, secondarily, to obtain information on the peer review
practices of such journals.
Design.--Examined one issue of a sample of all journal
titles written in English and indexed in Index Medicus, and
all indexed English-language journals in four subject fields. A
questionnaire was sent to the editors of journals in the subject fields
requesting information on their peer review practices.
Setting.--Journals received at the National Library of
Medicine.
Participants.--Editors of journals in four subject fields.
Main Outcome Measure.--Existence of a printed statement of
the peer review process for manuscripts.
Results.--Although the editors queried in the four subject
fields indicated that overall, 56% to 65% of the articles were peer
reviewed, clear statements about their peer review practices were not
found in half of their journals or in the overall sample.
Conclusions.--Editors should publish clear statements of the
peer review process followed for each type of article published in
their journals.
(JAMA. 1994;272:156-158)
A PEER-REVIEWED journal is defined by the
International Committee of Medical Journal Editors as "one that has
submitted most of its published articles for review by experts who are
not part of the editorial staff."[1] Peer review is the
accepted method for ensuring that the science reported in the
biomedical literature is of the highest quality. Increasing attention
has been focused on the peer review process as evidenced by journal
articles, editorials, and the two Peer Review Congresses sponsored by
the American Medical Association. Addeane S. Caelleigh, editor of
Academic Medicine, urges editors to publish their peer review
policy clearly, stating the rules for publishing in their journal. She
states that the policy must be public and easily available and should
"cover both the background requirements for submitting a paper for
consideration and the journal's procedures for handling the paper, the
review and publication process, and problems or allegations that may
arise during review and publication."[2]
To what extent do editors publish their peer review policies? Can
readers tell if an article was peer reviewed? This study was designed
(1) to determine whether peer review policies are published in
English-language journals indexed in Index Medicus and (2)
to obtain information on the peer review practices of such journals.
Index Medicus is a monthly bibliographic index to 3058
basic science and clinical journals in the biomedical and health
sciences. The citations to the articles, editorials, and letters in
Index Medicus journals plus 686 dental and nursing journals
comprise MEDLINE, the world's most heavily searched biomedical
database. Although peer review is not a requirement for a journal title
to be selected to be indexed in Index Medicus and MEDLINE, the
Literature Selection Technical Review Committee, a National Institutes
of Health advisory committee that reviews all new biomedical and health
sciences titles and recommends those to be indexed, examines journals
for evidence of peer review and considers this in making its
recommendations.
METHODS
The study was conducted by examining (1) the first issue in 1992
of a sample of English-language journals indexed in Index Medicus
and (2) all such titles in four subject areas. The sample was drawn
by selecting every 25th title and deleting titles without
English-language content. The four subject fields for which all
English-language-indexed journals were examined were dermatology,
neurology, orthopedics, and otolaryngology, selected because the titles
in each field published a variety of types of editorial content.
For both groups, the "Instructions to Authors" section in the first
issue published in the calendar year was examined for statements
describing the peer review process or references to an issue containing
such a statement.
In addition to examining one issue of each title, the editors of the
journals in the four subject fields were asked to complete a
questionnaire with 12 questions about their editorial and peer review
practices. The questionnaire elicited information about whether the
editors were full- or part-time, the number of editors on the internal
editorial staff and the external editorial board, the location of the
peer review statement in the journal, and the process used to review
five types of articles.
The five types of articles (definitions available on request)
were research, review, case report, general, and invited, and another
category included editorials, book reviews, and the like. Editors were
given the choice of acceptance without review and four review
processes: (1) review by scientists who have expertise in the subject
area and are not officially part of the journal's editorial staff or
board; (2) review by scientists who are members of the editorial board
but have the subject knowledge and geographic representation
appropriate to the scope of the journal; (3) review by members of the
editorial staff; and (4) review by the chief editor of the journal.
Multiple responses were allowed. For this study, a journal was
considered to be peer reviewed if the editor indicated that articles
had been reviewed by external reviewers or members of the external
editorial board.
RESULTS
A total of 293 titles were examined, 85 in the overall sample and
208 in the four subject areas. There was an overall response rate of
73% to the questionnaire. Differences between responders and
nonresponders, such as commercial or noncommercial publisher,
sponsorship by a society, type of articles published, and the presence
of a peer review statement in the journal were examined. An average of
72% of the editors responding to the questionnaire were associated
with commercial publishers; 86% of the nonresponders were also
associated with commercial publishers. Fifty-nine percent of the
editors who responded and 42% of the nonresponders edited a
society-sponsored publication. Evidence of peer review was found in
55% of the journals of the editors who responded and in 38% of
journals for which there was no response.
Table 1 lists the types of articles that the editors
reported reviewed by referees or the external editorial board. As was
expected, almost 100% of the research articles were peer reviewed.
Ninety-two percent of the case reports, 86% of the review articles,
82% of the general articles, and irrespective of the type of content,
67% of the editors peer reviewed invited articles. A few editors
reported reviewing manuscripts by themselves. Although this practice
was never reported for research articles, it was reported for the
remaining types of articles. Invited articles were reported reviewed by
only the editor in 15% of the titles; unfortunately, a substantial
number of titles contain invited articles and a reader cannot usually
identify them.
Although the editors reported that a large percentage of the articles,
especially research articles, published in the journals in these four
subject areas are peer reviewed, in 28% (20% to 35%) of the cases,
the editors reported that they did not print a statement of peer
review. A statement implying peer review was found in only 47% of the
titles in the overall sample and in 49% of the journals in the subject
fields (Table 2); however, few editors described the
process sufficiently to determine if their journals adhered to the
International Committee of Medical Journal Editors' definition.
Examples of statements included the following: "subject to peer
review," "sent to two specialists," "are reviewed," "to aid
refereeing," "rigorously appraised by at least two referees," and
"rapid peer review." None of the editors described variations in
the peer review process depending on the type of manuscript. In most
cases the discrepancy between the findings on examination and the
practice reported by the editors can be explained by the ambiguous
wording in printed statements.
If printed, the statement is most likely to be located in the
"Instructions to Authors" section (54%) or in a section describing
the journal's aims (14%). Not all journals publish instructions to
authors; unfortunately, the study did not record this frequency.
CONCLUSIONS
If peer review is important, editors should clearly indicate in
the journal the review process followed for each of type of article
published, including any supplements. The preferred location would be
the "Instructions to Authors" or "Notes to Contributors" when
this is published or in prefatory material when it is not.
From Library Operations, National Library of Medicine, National
Institutes of Health, Bethesda, Md.
Presented at the Second International Congress on Peer Review in
Biomedical Publication, Chicago, Ill, September 10, 1993.
The questionnaires described in the "Methods" section
were sent to the editors of the dermatology journals by Philip C.
Anderson, MD, Professor and Director, Division of Dermatology,
University of Missouri Columbia School of Medicine; to the editors of
the neurology journals by Robert Joynt, MD, Vice President and Vice
Provost for Health Affairs, University of Rochester (NY) Medical
Center; and to the editors of the orthopedics journals by Frank C.
Wilson, MD, Kenan Professor of Surgery and Chief, Division of
Orthopaedics, the University of North Carolina at Chapel Hill School of
Medicine.
Address correspondence to Library Operations, National Library of
Medicine, 8600 Rockville Pike, Bethesda, MD 20894 (Ms Colaianni).
References
1. Uniform Requirements for Manuscripts
Submitted to Biomedical Journals and Supplemental Statements From the
International Committee of Medical Journal Editors. Philadelphia, Pa:
American College of Physicians; 1993.
2. Caelleigh AS. Role of the journal editor in
sustaining integrity in research. Acad Med. 1993;68(September
suppl):S23-S29.
Table of Contents
