MASKING AUTHOR IDENTITY IN PEER REVIEW:
DOES IT IMPROVE PEER-REVIEW QUALITY?

Amy C Justice,¹ Mildred K Cho, Margaret A Winker, Jesse A Berlin, Drummond Rennie, Mike Berkwits, Michael Callaham, Phil Fontanarosa, Erica Frank, David Goldman, Steven Goodman, Roy Pitkin, Rohit Varma, and Joseph Waeckerle
¹Case Western Reserve University, School of Medicine, WB-29, 10900 Euclid Avenue, Cleveland, OH 44106-4961, USA

Objective: To determine whether masking reviewers to author identity was associated with higher review quality at several biomedical journals and to determine the success of masking at these journals.

Design: A randomized trial at: Annals of Emergency Medicine, Annals of Internal Medicine, JAMA, Obstetrics & Gynecology, and Ophthalmology. Two reviewers reviewed each manuscript, 1 randomized to masked and the other to unmasked review. Editors (masked to reviewer randomization) scored review quality on a 5-point Likert scale. Quality differences between unmasked and masked review scores for each manuscript were tested using paired t-tests. A difference of >0.5 was considered important.

Results: Fifty-eight percent 74/128 of manuscripts had both reviews returned. Editors perceived no significant difference in quality between masked and unmasked reviews (mean difference 0.1, 95% CI: -0.3-0.4). Differences did not vary significantly by journal (in order listed above, -0.2, 0.1, -0.2, 0.6, and 0.1). However, masking was not universally successful. Masking success varied by journal (in order listed above): 88% (70%-98%), 59% (33%-82%), 50% (29%-71%), 71% (42%-92%), and 62% (32%-86%) of (P=.05). Analyzing only pairs for which masking was successful (59%) yielded similar results (0.0, 95% CI: -0.4-0.4).

Conclusions: This multi-journal study did not detect an improvement in peer review quality of greater than 0.4 on a 5-point Likert scale. Poor masking success at most journals may have compromised potential beneficial effects of masking if manuscripts most likely to benefit from masking were more likely to be unmasked.

Return to Session Information