THE COCHRANE COLLABORATION: ITS IMPACT ON THE QUALITY
OF SYSTEMATIC REVIEWS
Alejandro R. Jadad,1,2,3 Deborah J Cook,3,4 Alison Jones,5 Terry Klassen,6 Michael Moher,7 Peter Tugwell,5,8 and David Moher5
1Health Information Research Unit, McMaster University, 1200 Main St, W, Hamilton, Ontario L8N 3Z5, Canada; 2Canadian Cochrane Centre, McMaster University, Hamilton, Ontario, Canada; 3Department of Clinical Epidemiology & Biostatistics, McMaster University, Hamilton, Ontario, Canada; 4Department of Medicine, McMaster University, Hamilton, Canada; 5Clinical Epidemiology Unit, Loeb Medical Research Institute, Ottawa, Canada; 6Department of Pediatrics, University of Ottawa, Canada; 7General Practice Research Group, University of Oxford, Oxford, UK; 8Department of Medicine, University of Ottawa, Ottawa, Canada
Objective: To compare the methodologic and reporting aspects of meta-analyses published by the Cochrane Coflaboration with those published in paper-based journals indexed in MEDLINE.
Design: Thirty-six completed reviews published in the Cochrane Database of Systematic Reviews (CDSR) in 1995 were compared with a random sample of 38 meta-analyses published in
1995 in 32 different journals and found by a refined MEDLINE search strategy. Using criteria defined a priori, the following information was extracted in duplicate from each report: number of authors,
trials and patients included in each review, description of the sources of trials, inclusion and exclusion criteria, language restrictions, quality assessment of priary trials, heterogeneity testing,
and quantitative effect estimates.
Results: Meta-analyses found in MEDLINE included more authors (median: 3 vs 2, P<.001), more trials (median: 13.5 vs 5,P<.00l) and more patients (median: 1,280 vs 528, P<.00l) than those in CDSR.
More reviews in CDSR than in MEDLINE reported the inclusion and exclusion criteria(35/36 vs 18/39,P<.001) and assessed trial quality (36/36 vs 12/39, P<.00l). Fewer reviews in CDSR had language
restrictions (0/36 vs 7/39, P=.012). There were no differences in the number of sources of trials, in the frequency of heterogeneity testing or in the description of the quantitative estimates.
MEDLINE was the source of trials in less than two thirds of the meta-analyses.
Conclusions: Cochrane reviews appear to have greater methodological rigor than meta-analyses published in paper-based journals. The limitations of this study (including possible confounding factors)
and future research directions will be discussed in relation to the Cochrane Collaboration's peer review system as compared with that used by journals.
Return to Session Information
