IS PEER REVIEW TECHNOLOGICALLY OBSOLETE?
Susan Feigenbaum1 and David M Levy2
1Department of Economics, University of Missouri-St Louis, St Louis, MO 63121, USA; 2Center for Study of Public Choice, George Mason University, Fairfax, VA, USA
Objective: To document trends in the statistical practices of researchers over the past 3 decades; link these trends to changes in the "price" of computing technology; and explore the implications for traditional peer-review processes as guarantors of research integrity and robustness.
Design: A retrospective study was conducted based on all articles published in the 1960 through 1989 issues of The Review of Economics and Statistics. For each article, data were gathered on statistical technique, size of data set, number of control (explanatory) variables and results reported, number of coauthors, and research funding sources.
Results: From 1960 through 1989, the median number of observations per article increased from 123 to 385; the median number of control (explanatory) variables increased from l to 11; the median number of reported results decreased from 7.5 to 6. In contrast, the maximum "frontier" values for data set size and number of reported results peaked (at 349,060 and 150, respectively) and fell during the period, while the maximum number of control variables (91) rose throughout. The percent of articles using a linear estimation technique fell from 100% to 22%; the percent of articles using nonlinear maximum likelihood estimation rose from 0% to 55%.
Conclusions: Statistical practices in published research have become more complex, thereby compromising the ability of peer reviewers to assess the integrity of the work at low cost. At the same time, a low-cost signal of research quality-the reputation of the author-has been diluted by the increasing frequency of multiple authorship. Peer review must develop new strategies to more finely match reviewers and submissions if it is to continue as a primary mechanism for assuring journal research quality.
Return to Session Information
