THE USE OF SYSTEMATIC REVIEWS FOR PEER-REVIEWING
Tom Jefferson,1 Vittorio Demicheli, and John Hutton
1Army Medical Directorate, Keogh Barracks, Ash Vale, Hants GU12 5RR, UK
Objective: The best introduction to a topic of research is represented by a systematic review of that topic. If the results of the systematic review are included in the research protocol, readers, researchers, editors, and peer reviewers can then use the evidence presented to judge the importance of the topic and the contribution of the "new" paper to understanding of the subject (Chalmers, 1991). Equally, a peer reviewer can use systematic reviews to reinforce subjective knowledge of the background to an individual candidate paper and to assess the candidate paper against a population of its peers, thus highlighting the contribution and general soundness of the new comer to the existing body of knowledge. This study was undertaken to test the feasibility of this novel approach to the use of systematic reviews in peer review.
Design: The pilot study was based on 2 examples involving systematic reviews of hepatitis B vaccines. In the first example, a new cost-benefit analysis of hepatitis B vaccine was tested for methodological consistency against similar papers in a review of economic evaluations of hepatitis B vaccine. In the second example, a Cochrane review of hepatitis B vaccines was used to check candidate papers for duplicate publication.
Results: In the first example, the new study was found to be an outlier in terms of the relationship between incidence of hepatitis B and the benefit-cost ratio, as it produced a high benefit-cost ratio with a low incidence rate. The results of the second exercise revealed that of 59 identified trials, 15 showed evidence of multiple publication, 4 of which were fraudulent (the duplicate was not cross-referenced) and 3 non-fraudulent.
Conclusions: The use of systematic reviews for peer review has limitations because of their format and availability. However, this pilot study showed that important issues can be brought to the attention of peer reviewers by comparison of candidate papers with the results of systematic reviews. The use of systematic reviews to test methodological robustness of candidate papers and prevent multiple publication is worth further testing.
Return to Session Information
