DOES PEER REVIEW FAVOR
THE CONCEPTUAL FRAMEWORK OF ORTHODOX MEDICINE?
A RANDOMIZED CONTROLLED STUDY
Karl-Ludwig Resch,1 Edzard Ernst,2 and John Garrow3
1Forschungsinstitut für Balneologie und Kurortwissenschaft, Lindenstr 5, 08645 Bad Elster, Germany; 2Department of Complementary Medicine, Postgraduate Medical School, University of Exeter, Exeter, UK;
3European Journal of Clinical Nutrition, Hertfordshire, UK
Objective: To test the hypothesis that experts who review papers for publication are prejudiced against complementary medicine.
Design: A randomized, controlled, double-blind study. Two versions of a fake short report on treatment of obesity, identical except for the nature of the intervention (orthodox: hydroxycitrate vs homoeopathic remedy: sulphur C50) were produced. According to a randomization list, 1 of the 2 versions was sent to each of a list of experts (n=398; identified by means of a MEDLlNE search) using the letterhead and evaluation sheet of an established journal. Main outcome parameters were the reviewers' rating of importance on a scale of 1-5 (trivial to major contribution) and a visual analog scale (with rating ranging from "reject outright" to "definitively accept"). Reviewers were unaware of the fact that they participated in a study, but are currently being debriefed.
Results: Overall response rate was 41.7% with 141 assessment forms being suitable for statistical evaluation. After dichotomization of the rating scale (ratings of 1 and 2 = negative; ratings of 4 and 5 = positive), a significant difference in favor of the orthodox version was observed (P=.03, chi square test) with an odds ratio of 3.01 (95% CI, 1.03-8.25). This observation was confirmed by the visual analogue scale (P=.052, Mann-Whitney U-test). The respective medians and interquartile ranges were 67% (51%-78.5%) for the orthodox version and 57% (29.7%-72.6%) of maximal acceptance for the unconventional version.
Conclusions: Despite a remarkably large within-group variation in both groups, there seems to be a relevant reviewer bias against papers dealing with unconventional medical concepts.
Return to Session Information
