EXTERNAL REFEREEING OF PROTOCOLS FOR SYSTEMATIC REVIEWS
Lisa A Bero,1 Roberto Grilli,2 Jeremy Grimshaw,3 Emma Harvey,4 and Andrew D Oxman5
1Institute for Health Policy Studies, University of California, San Francisco, 1338 Sutter St, 11th Floor, San Francisco, CA 94109, USA; 2Mario Negri Institute, Italy; 3University of Aberdeen, UK; 4University of York, UK; 5National Institute of Public Health, Norway
Objective: To determine whether external peer refereeing of protocols for systematic reviews decreases the time necessary for peer review and provides useful feedback to authors.
Design: Randomized controlled trial. All protocols for systematic reviews submitted to the Cochrane Collaboration on effective professional practice were randomized to receive
internal peer review (IPR) by the editorial team or external peer review (EPR) by 3 referees. After completion of the protocol, all reviews were externally refereed. We measured
1) time for refereeing protocol and completed review and 2) number of "major" comments (those that would be difficult or impossible to correct after the review is completed) and
"minor" comments made by external referees. We also determined whether the suggestions made during protocol refereeing were followed.
Results: External peer-review protocols were refereed in 56 ± 21 days (n=9) compared to 31 ± 37 days for IPR protocols (n=9). To date, 7 protocols have been completed as reviews.
The time for refereeing of the completed reviews was 28 ± 20 for those with EPR protocols (n=2) and 72 ± 22 for those with IPR protocols (n=5). During protocol refereeing, 38% (10/26) of
external referees contributed major comments. During review refereeing, 33% (2/6) of referees for reviews with EPR protocols made major comments, compared to 53% (8/15) of referees for
reviews with IPR protocols. In 1 case, a major suggestion given by an external referee at the protocol stage was not followed.
Conclusions: External refereeing of protocols does not increase the time necessary for peer review and provides authors with substantive suggestions for improving their reviews.
Return to Session Information
