USE OF BIBLIOMETRIC MEASURES TO ASSIST THE
PEER-REVIEW PROCESS IN NEUROSCIENCES

Grant Lewison and Wendy Ewart
The Wellcome Trust, 183 Euston Road, London NW1 2BE, UK

Objective. To determine whether detailed bibliometric indicators of applicants' publication track records would assist a grants panel in making judgements on major awards in neurosciences, and if so, which indicators would be most useful. This panel, of up to 20 experts in mental health and neurosciences, also sees at least 3 referees' reports on each application.

Design. The publications in the Science Citation Index (or the Social Sciences Citation Index) of some 30 applicants for long-term grants were analysed to show their number, potential impact (from the rating of the journal), actual impact (from counts of citations, compared with those in the applicant's specific field) and the fractional contribution of the applicant, taking account of the overall number of his/her co-authors. They were compared with similar indicators for about 5 world leading scientists in the field, "peers," selected by the program manager but usually taken from a list suggested by the applicant.

Results. After 1 year of trialling, some 13 panel members were surveyed and felt that the provision of bibliometric data, especially citation data, put peer-review on a sounder basis. There was a strong correlation between the average scores on several different indicators and combinations thereof and the results of the panel's deliberations, but there were some exceptions, such as when a proposal was rejected because of its poor design despite the applicant's reputation.

Conclusions. Bibliometric data assist award panels and multiple indicators are particularly helpful, but only for candidates with substantial numbers of publications (about 20 or more). The use of bibliometric indicators has continued and been extended to other committees of award within the Wellcome Trust.

Return to Session Information