Skills test moving forward; pilot studies prove effective

An assessment test of medical students' clinical skills by the National Board of Medical Examiners is set to start in 2004, but cost and venues are still under discussion.

By Jay Greene, amednews staff. Oct. 8, 2001.

Medical students and schools across the nation are gearing up for the start of a clinical skills assessment examination likely to begin the summer of 2004.

Public demand that physicians possess greater clinical skills led the National Board of Medical Examiners and the Federation of State Medical Boards in 1999 to approve a plan to require the test for licensure.

The licensing boards also felt lack of clinical skills, including poor communication between patient and doctor, can contribute to medical errors.

"There are two sets of competencies that doctors must have -- cognitive and clinical skills," said Peter Scoles, MD, NBME's senior vice president of assessment programs. "We do well testing cognitive skills, but we needed a way to test clinical skills."

No one objects to the need for standardized clinical training, but students and school officials are concerned about the projected per test cost of $1,000. Another concern is the limited number of regional testing locations. Preliminary plans call for the test to be offered at four to six fixed sites, Dr. Scoles said.

The NBME also is studying the possibility of rotating locations at medical schools. Those schools wishing to participate most likely would be required to absorb an additional $600 to $700 per test in administrative costs above the $1,000 student fee.

"This test will just add to the overall cost of medical education, which ranges between $80,000 to $120,000," said Dwight Johnson, the AMA's medical student section representative to the NBME. "We are pushing for students not having to travel by car any more than 450 miles."

The proposed clinical skills exam will feature 10 standardized cases taking about 30 minutes each. Using standardized patients, students are tested on their ability to gather and interpret clinical patient data, communicate effectively and render a differential diagnosis.

Failure rate on the test is projected at 5% to 7% for first-time takers and 1% to 2% for repeat takers, Dr. Scoles said. "We believe that roughly the same number of students who fail the cognitive portions of the [U.S. Medical Licensing Examination] will fail this exam," he said.

Pilot studies conducted over the last three years have confirmed the effectiveness of the test, Dr. Scoles said. During the next year, NBME will conduct additional field tests at several medical school sites.

One pilot site has been at the University of Arkansas for Medical Sciences College of Medicine in Little Rock. "We have substituted the NBME pilot exam for our objective structured clinical exams (OSCE) the last 3 years," said Richard Wheeler, MD, Arkansas' dean of students. "These tests help us identify weak areas students have but also tell us how we are doing in teaching. We have picked up a lot of problems and made adjustments in our curriculum."

In 1991, the Liaison Committee on Medical Education, which accredits the nation's 125 allopathic medical schools, began to require performance-based evaluations of clinical skills. But LCME data show 25% of medical schools do not use standardized patient exams or OSCEs in either clerkships or formal courses, according to a report in the Sept. 5 issue of JAMA.

"Effective communication and interpersonal skills are a big part of good medicine," said Carol Clothier, FSMB's assistant vice president of examination services. "Right now there is no effective tool for testing individuals and to assure medical boards that individuals meet a certain standard."

While almost all schools are beefing up clinical skills training, Gregory Vercellotti, MD, senior associate dean for education at the University of Minnesota Medical School in Minneapolis, does not support a national examination.

"We have been using our OSCE for years and it is a very powerful examination tool," Dr. Vercellotti said. "You can pick up skills, attitudes and behaviors that are hard to detect."

But Dr. Vercellotti said a national exam may not be a better predictor of performance than OSCEs administered by medical schools. "I would rather have the LCME require clinical assessment exams through their accreditation process," he said.

AMA policy supports clinical skills evaluations for licensure, "when the methods have been demonstrated to be valid, reliable and practical."