Editorials represent the opinions of the authors and The Journal and not those of the American Medical Association.
The Second International Congress on Peer Review in
Biomedical Publication
(JAMA. 1994;272:91)
In this issue of The
Journal, we publish 26 of the papers presented at the Second
International Congress on Peer Review in Biomedical Publication, held
in Chicago, Ill, September 9-11, 1993. Of these 26 papers, two, those
by Kassirer and Campion[1] and
Judson,[2] were
invited. The others were submitted in response to a concerted effort we
have made during the past 8 years to stimulate research into editorial
peer review.[3]
[4]
Given the subject, it is appropriate
to comment on the review process used to help select these papers. We
received 110 abstracts describing work to be presented at the congress.
These were sent to seven independent reviewers (from the United States,
Canada, and the United Kingdom) who allotted scores to each abstract;
selection was made according to these scores. Thirty-three abstracts
(30%) were selected for oral presentation, and 21 abstracts (19%)
were presented as posters. Subsequently, the presentations were
thoroughly revised for consideration for inclusion in this issue.
Thirty-four manuscripts were submitted. Each was sent for review to a
wide variety of reviewers (overall, 63 reviewers contributed to this
effort, several of whom had not attended the congress). Eight of these
papers were rejected after review, and the others were again revised
into their current form.
We at JAMA have already
commented on the vigorous debates that followed each presentation
during the congress,[5]
[6] and others have written about it in
the scientific press.[7]
[8] Our purpose, then, is to tell the
readers something about who attended and what they thought of this
initiative.
Two hundred seventy-five editors, researchers, and
others interested in biomedical publication from 20 countries attended
the congress, which was similar to the 283 from 22 countries who
attended the first congress in 1989. We have heard from some of our
more vocal colleagues that we have been preaching to the choir, and
that a self-appointed clique has conducted, presented, and discussed
all the research and generated all the controversies that have come out
of these two congresses. Irked and intrigued by this possibility, we
reviewed the attendance records for each congress. If any such groups
had in fact formed, they could only have been drawn from at most a
quarter of those who attended the second congress; only 26% of the
1993 participants had attended the first congress.
As we did during the first
congress,[9] we asked the participants
of the second congress to give us their opinions on a number of
subjects related to peer review and the integrity of the publication
process by completing precongress and postcongress attitude surveys.
Since we did not count the heads of those specifically attending the
first and last days of the meeting, we do not know how many registrants
actually participated in the congress on each day of the survey.
However, from the total number of registrants (275), we calculated
minimum response rates of 85% for the initial survey and 68% for the
final survey. Despite the limitations of this approach, we can still
glean a few trends from the replies to the congress surveys. For
example, the vast majority of respondents (95% precongress and 92%
postcongress) agreed that peer review improves the quality of
manuscripts for publication. No other question, except the one asking
whether or not to hold another congress, received such high accord.
Two additional questions generated agreement among the respondents
before the meeting that became stronger afterward. From the precongress
survey, 68% thought journals should adopt more uniform standards for
peer review. After the congress, this number increased to 81%.
Similarly, the majority of respondents concurred that editors should
assist in or encourage research to establish baseline data on the
prevalence of scientific fraud (75% precongress and 85%
postcongress). While reflecting only moderate agreement, responses to
two other questions changed during the meeting. Before the congress,
49% of respondents reported that journals should publish retractions
if the research institution, but not the authors, requested it; this
number increased to 58% after the congress. Demonstrating a change in
the opposite direction, 56% of respondents initially agreed that
journals should establish a section for speculative and unconventional
work; after the congress, this number decreased to 48%.
As alluded to earlier, 95% of postcongress survey respondents said that
we should hold future, regular conferences on editorial peer review. In
order to hold another congress, say in 4 years, we need
prospective participants to begin thinking of innovative ways to assess
peer review and to set up study protocols, since the whole point of
this initiative is to stimulate and present research.
Many people
contributed to the success of the congress. We are grateful to the Peer
Review Congress Advisory Board for their advice and assistance,
especially Jane Smith, MSc, John Bailar, MD, PhD, Suzanne Fletcher, MD,
Brian Haynes, MD, Elizabeth Knoll, PhD, Stephen Lock, MD, FRCP, and
George Lundberg, MD. We thank Sharon Iverson for coordinating the
manuscripts for this issue of The Journal. We also wish to
acknowledge the work of everyone who submitted abstracts for
consideration. Their efforts have collectively advanced the science of
editorial peer review.
Drummond Rennie, MD
Annette Flanagin, RN, MA
Dr Rennie is Deputy Editor (West),
JAMA, and Ms Flanagin is
Associate Senior Editor,
JAMA.
Reprints not available.
References
1. Kassirer JP, Campion EW. Peer review: crude and
understudied, but indispensable. JAMA. 1994;272:96-97.
2. Judson HF. Structural transformation of the sciences and
the end of peer review. JAMA. 1994;272:92-94.
3. Rennie D. Guarding the guardians: a conference on
editorial peer review. JAMA. 1986;256:2391-2392.
4. Rennie D. Editorial peer review in biomedical
publication: the first international congress. JAMA.
1990;263:1317.
5. Rennie D. More peering into peer review. JAMA.
1993;270:2856-2858.
6. Cotton P. Flaws documented, reforms debated at congress
on journal peer review. JAMA. 1993;270:2775-2778.
7. Taubes G. Peer review goes under the microscope.
Science. 1993;262:25-26.
8. Vanchieri C. Peer review out to the test: credibility at
stake. J Natl Cancer Inst. 1993;85:1632-1633.
9. Flanagin A, Rennie D, Lundberg G. Attitudes of Peer
Review Congress attend-ees. In: Peer Review in Scientific
Publishing. Chicago, Ill: Council of Biology Editors;
1991:260-263.
Table of Contents
