PEER REVIEW: RECOMMENDATION IS BASED ON FEW ASPECTS
Karl-Ludwig Resch,1 Annegret Franke,1 and Edzard Ernst2
1Forschungsinstitut für Balneologie und Kurortwissenschaft, Lindenstr 5, 08645, Bad Elster, Germany; 2Dept of Complementary Medicine, University of Exeter, Exeter, UK
Objective: To determine which aspects a peer reviewer is commonly asked to comment on are vital, and which are possibly redundant.
Design: As part of a study into peer review, a bogus letter to the editor was sent to 291 medically trained participants of an interdisciplinary international conference, who were debriefed only after the study. They were asked to comment on the following quality criteria by indicating their position on visual analog scales, ie, horizontal lines in between 2 given extremes (poor to excellent): relevance of subject, formulation of hypothesis, randomization, inclusion/exclusion criteria, sample size, statistical evaluation, choice of main outcome measures, follow-up, clarity of description, linguistic quality, overall quality of the study, and overall quality of the manuscript. By means of a multiple regression analysis, we tried to identify the optimal set of items (best prediction with least items) to predict overall quality of manuscript.
Results:Response rate was 635 (183/291); 127 responses were suitable for evaluation. Significant but moderate bivariate correlations were observed between overall quality of manuscript and relevance of subject (r=0.51), formulation of hypothesis (r=0.71), statistical evaluation (r=0.54), choice of main outcome measures (r=0.66), clarity of description (r=0.71), and linguistic quality (r=0.52). Multiple regression analysis, however, revealed a multiple adjusted l:l2 of over 075 with a set of only 3 items (ie, clarity of description, choice of main outcome measures, and formulation of hypothesis). Stepwise inclusion of further items resulted in an only minute increase of R2.
Conclusions: In this setting, the peer reviewer's recommendation was well predicted by a limited set of criteria. In other settings this may be different. Nevertheless, it seems promising to direct more research into the development of a standardized (and validated) assessment form.
Return to Session Information
