Effect of Institutional Prestige on Reviewers'
Recommendations and Editorial Decisions
(JAMA. 1994;272:137-138)
Joseph M. Garfunkel, MD; Martin H. Ulshen, MD; Harvey J. Hamrick,
MD; Edward E. Lawson, MD
Objective.--To determine whether manuscripts from
institutions with greater prestige are more likely to be recommended
for publication by reviewers and to be accepted for publication.
Design.--Retrospective study of reviewers' recommendations
and editorial decisions for manuscripts from the United States received
at the Journal of Pediatrics between January 1 and July 31,
1992. Manuscripts were classified as major papers or as brief reports.
Institutions were ranked in quintiles according to the monetary value
of grants funded by the National Institutes of Health. Reviewers'
recommendations were classified as reject, reconsider, or accept, and
editorial decisions as accept or reject, without regard to qualifying
recommendations.
Results.--For the 147 brief reports, lower institutional
rank was associated with lower rates of recommendation for acceptance
and of selection for publication. For the 258 major papers, however,
there was no significant relationship between institutional rank and
either the reviewers' recommendations or the acceptance rate. Similar
results were found when the manuscripts were divided into five
numerically equal groups according to institutional rank.
Conclusions.--Major manuscripts from institutions with
greater prestige were no more likely to be recommended or accepted for
publication than those from institutions with lesser prestige. In
contrast, the likelihood of recommendation for acceptance and of
selection for publication of brief reports appeared to correlate with
the prestige of the institution.
(JAMA. 1994;272:137-138)
RESEARCHERS, particularly those with relatively
little experience, have long suspected that papers are more likely to
be recommended by reviewers for publication, and accepted by editors,
if they originate from larger, better-known institutions. As an
example, this suspicion has been supported by the oft-quoted study of
Peters and Ceci,[1] who changed the authors' names and
institutions on previously accepted papers and had them rereviewed as
if they were new submissions. Most of the reviewers recommended
rejection, not having recognized that the papers had already been
published. Others have found less evidence of bias (reviewed by
Lock[2]). Blinding of reviewers to the origins of a paper has
been recommended[3] and is currently the normal practice for
some journals. However, blinding is not always successful, and some
reviewers believe that they are denied important information regarding
the validity of the findings if they do not know the identity of the
authors and where the work was done.[4]
We therefore decided to determine whether manuscripts received at the
Journal of Pediatrics from institutions with greater prestige
are more likely to be recommended for publication by reviewers, and
more likely to be accepted for publication, than are manuscripts from
institutions of lower prestige.
MATERIALS AND METHODS
Institutional prestige was determined according to the monetary value
of research and training grants and contracts funded by the National
Institutes of Health (determined according to the Summary of
National Institutes of Health Fiscal Year 1991 Extramural Awards to
Medical Schools by Rank of Institution and Activity). One hundred
twenty-five medical schools were ranked into quintiles, each containing
25 institutions. Manuscripts received from hospitals with an
institutional affiliation noted on the letterhead, or from authors who
cited an academic appointment at a medical school, were included with
the manuscripts from the parent institution. Those from the National
Institutes of Health were ranked in the top 25, and those from
unaffiliated institutions in the last 25. Manuscripts were received
from 64 medical schools and affiliates, the National Institutes of
Health, and six unaffiliated institutions.
The study was done retrospectively so as to avoid the potential
for bias in the selection of papers for publication by an editorial
staff aware that a study was in progress. We included all manuscripts
from the United States that were received between January 1 and July
31, 1992. Manuscripts received from outside the United States (43%
of the total) were not included because we had no objective way to
assess institutional prestige. Manuscripts were classified as either
major papers (case series, reports of clinical or laboratory research,
epidemiologic studies, and the like) or brief reports (case reports and
similar short papers submitted for our Clinical and Laboratory
Observations section, which are restricted to 1000 words and one or two
tables and figures). Reviewers' recommendations were classified as
"reject," "reconsider with revision," or "accept," and
editorial decisions as "accept" or "reject." Manuscripts
recommended for publication or accepted subject to revision were
included in the "accept" categories.
The association between institutional rank and editorial decision was
assessed with a Mantel-Haenszel correlation chi2 statistic.
Fisher's Exact Test was used for comparisons with small sample sizes.
We also examined the power of the study to identify a significant
difference. We used SAS for all analyses.
RESULTS
For the 147 brief reports, lower institutional rank was associated with
lower rates of recommendation for acceptance by reviewers
(P<.001; Figure 1). The
acceptance rate for the
highest quintile was 32%, compared with 6% and 11% for the fourth
and fifth quintiles, respectively (P=.004). For the 258 major
papers, however, there was no significant relationship between
institutional rank and either the reviewers' recommendations
(P=.409) or the acceptance rate (P=.508; Figure 2). Reviewers had opposing recommendations
for 15 brief reports and 27 major papers. These numbers were too small for
meaningful analysis, but the editorial decisions showed no evidence of
prejudice in favor of the larger institutions.
For the editor's decision to accept or reject, we examined the power
of the study to detect a significant difference by comparing the
top-ranked group with the last three groups, providing a sample size of
approximately 100 in each group. The power to detect a difference of
20% between the two groups was approximately 85%, with a significance
level of .05.
Institutions in the highest quintile submitted the greatest number of
manuscripts and, therefore, had the greatest number of manuscripts
accepted (Figure 3). Because more manuscripts are received
from institutions of higher rank, we repeated the analyses by dividing
the number of manuscripts into five approximately equal groups. The
results of these analyses were the same (data not shown); for the brief
reports there was a significant association between institutional rank
and reviewers' recommendations (P<.001) and between
institutional rank and editorial decision (P=.008). By
contrast, for the major papers there was no relationship between
institutional prestige and either reviewers' recommendations or
editorial decisions (P=.771).
COMMENT
A quarter of a century ago, Merton[5] noted that allocation
of research resources was often uneven and that investigators with
successful records were more likely to receive grants, get adequate
research space, and otherwise receive support that made further success
more likely. He also noted several examples indicating that
better-known scientists were more likely to receive recognition for
work similar to that of other researchers. Because of their high
standards and productivity, these individuals had less need to publish
all of their work, but could confine their writing to their better
efforts. Because of their prestige and their facilities, they attracted
bright young colleagues, who then produced work of their own, often
including the name of the senior investigator among the authors--a
practice that young investigators in smaller institutions believe
enhances the chance for acceptance of a manuscript.
Prestigious investigators and their teams tend to congregate in large,
well-funded research universities, and prestigious departments of
pediatrics tend to be associated with these institutions. We chose to
base our study on the institution of origin because there was available
a reasonably objective measure of prestige--the allocation of research
and training grants and contracts by the National Institutes of Health.
When we applied that criterion and evaluated reviewers'
recommendations and editorial decisions with respect to substantive
research work, we found that these did not correlate with institutional
rank. For brief papers, such as case reports, however, we did find a
significant relationship between institutional rank and both
recommendations for acceptance and the editorial decision to accept.
One possible explanation for this observation is that there is a lack
of pressure to publish minor work from major institutions and,
therefore, brief papers submitted from them are likely to be of better
quality. In contrast, investigators in the smaller schools, finding it
necessary to "publish or perish" but without access to research
facilities, and with limited opportunities to work with established
investigators, are more likely to submit brief reports of lesser
quality.
Although we found no tendency at our journal to accept a greater
percentage of major papers from more prestigious institutions than from
less prestigious institutions, we cannot state unequivocally that the
manuscripts were treated equally, as we did not evaluate the quality of
individual papers. Nevertheless, if our experience can be confirmed by
similar studies at other journals, young investigators may feel less
threatened by what they currently perceive as a bias of the peer review
system in favor of manuscripts submitted from "name" institutions.
From the Department of Pediatrics, University of North Carolina
School of Medicine, and
Journal of Pediatrics, Chapel Hill.
Presented at the Second International Congress on Peer Review in
Biomedical Publication, Chicago, Ill, September 11, 1993.
Address correspondence to Journal of Pediatrics, CBN 7230,
Medical School Wing C, University of North Carolina, Chapel Hill, NC
27599-7230 (Dr Garfunkel).
References
1. Peters DP, Ceci SJ. Peer-review practices of
psychological journals: the fate of published articles, submitted
again. Behav Brain Sci. 1982;5:187-255.
2. Lock S: A Difficult Balance. Philadelphia, Pa:
ISI Press; 1986:26-30.
3. McNutt RA, Evans AT, Fletcher RH, Fletcher SW. The
effects of blinding on the quality of peer review: a randomized trial.
JAMA. 1990;263:1371-1376.
4. Yankauer A. How blind is blind review? Am J Public
Health. 1991;81:843-845.
5. Merton RK. The Matthew effect in science.
Science. 1968;159:56-63.
Table of Contents
