THE EFFECT OF INFORMING REFEREES THAT THEIR COMMENTS WOULD BE EXCHANGED ON THE QUALITY OF THEIR REVIEWS
Savitri Das Sinha, Peush Sahni, and Samiran Nundy
The National Medical Journal of India, AIIMS, Ansari Nagar, New Delhi 100 029, India
Objective: The quality of manuscript refereeing in developing countries is thought to be poor. As we send each of the original and review articles submitted to The National Medical Journal of India to an Indian as well as a non-Indian (usually Western) referee, we tested whether informing a pair of referees that their comments would be exchanged improved the quality of their reviews.
Design: In a prospective, randomized, blinded study, we sent 100 manuscripts to pairs of referees of which 78 pairs of replies were suitable for analysis (the others were incomplete). Thirty-eight pairs of reviews were exchanged and 40 were not. The quality of the reviews was assessed by 2 editors who were unaware of the referees' nationality and whether they had been given this information. Quality was scored out of 100 (based on a predesigned evaluation proforma) according to whether the review examined the importance of the question, targeted the key issues, assessed the validity of the methods, assessed the quality of presentation, and provided an overall assessment.
Results: Overall non-Indian referees scored higher than Indians (mean scores, non-Indians first, 56.7 vs 48.6, P<.00l) especially those in the nonexchanged group (58.4 vs 47.3, P<.001) but not the exchanged group (54.8 vs 50.0, P<.06). Being informed that reviews would be exchanged did not affect the quality of reviews by non-Indians (54.8 exchanged vs 58.4 nonexchanged) and of reviews by Indians (50.0 exchanged vs 47.3 nonexchanged). The assessments of the 2 editors matched well (r=0.59, P<.001).
Conclusions: In this study, non-Indian referees are better reviewers than Indians and telling referees that their views would be exchanged does not seem to make much difference.
Return to Session Information
