Evidence doesn't support push for clinical skills exam

AMA Leader Commentary. By William G. Plested III, MD, Dec. 1, 2003.

AMA LEADER COMMENTARY

A few words about current events in medical practice

SEE ARCHIVES

A message to all physicians from William G. Plested III, MD, chair of the AMA Board of Trustees.

Today in the United States our profession of medicine is under siege from every direction. It seems like everyone knows more about medicine and its practice than physicians -- and all are doing their best to impose their views on the men and women who do the daily work of caring for patients.

We have insurers braying that they will pay for "quality," which they will define -- without bothering to share that definition with us. This is nothing more than a pitiful excuse to underpay a majority of physicians and stuff even more millions of dollars into the pockets of third-party payers. On the other extreme, we have business people forming groups designed to bring "business practices and ethics" to medicine.

That's all we need: to have the Enron-Global Crossing bunch teach physicians about ethics!

In the midst of this frenzy, we have our own institutions leaping on the bandwagon with new schemes to "ensure" physician competence. One of these schemes is the clinical skills examination, scheduled for implementation next summer.

For years the National Board of Medical Examiners and the Federation of State Medical Boards have administered the U.S. Medical Licensing Examination. This examination is given in three parts for U.S. graduates. Step 1 is usually given during the second year of medical school, Step 2 in the fourth year, and Step 3 in the first or second year of residency training. Successful completion of all three steps is a requirement for licensure in all 50 states.

Some have clamored that someone must ensure that physicians possess "basic clinical skills" before being allowed to practice medicine. At first glance, that doesn't sound unreasonable. In fact, the AMA has a spectacular record of achievement in this arena. As the prime motivational force behind the Flexner Report, the AMA was pivotal in the development and maintenance of the greatest medical education system in history.

The question is: How can one "prove" that medical students have an acceptable level of clinical skills?

On the one hand is the argument that four years of daily observation by faculty residents, fellows and the aggregate opinion of the extensive and diverse collection of professors, clinical instructors and mentors should be decisive. On the other hand, there are always those who believe only a test will prove the matter beyond question.

Thus we have the proposed clinical skills examination. As currently defined, the exam will involve one-on-one encounters with actors who are taught signs and symptoms of a particular disease or malady.

These encounters will be recorded and videotaped, and each student will be evaluated on three separate criteria: the quality and thoroughness of the physical examination and the written findings; the ability to establish rapport and communicate effectively with the actor; and proficiency in spoken English.

The marked subjectivity of the exam is an obvious, if not fatal, shortcoming. Actors are, at best, actors and vary widely in capability and effectiveness. In addition, the actors' jobs are demanding, poorly compensated and difficult to keep filled. Therefore, the performance and affability of each actor could vary widely throughout a long and exhausting day, week or month of repeating lines with which he or she may have no firsthand experience. And yet, some actors may feel so comfortable with their roles that they might embellish.

Another problem is that the exam has been arbitrarily assigned by the National Board of Medical Examiners and the Federation of State Medical Boards as part of Step 2 of the USMLE.

This designation is worrisome for several reasons.

First, more than 75 U.S. medical schools will not allow a student to graduate without passing Step 2 and most residency programs desire passage of Step 2 for acceptance. Furthermore, passage of Step 2 is a requirement for registration for Step 3.

The timing of the examination is a paramount consideration for students. Taking the test earlier allows the opportunity for a re-exam, since retesting is not allowed within 60 days of a failed exam. However, the earlier a student takes the test, the fewer opportunities he or she has to get the clinical experience necessary to pass. Does the medical school have any responsibility to remediate this student? Whose responsibility is it?

There are downsides to taking the exam later, too. The paucity of testing sites, combined with the 60-day retest rule, means there will be no possibility of a retest prior to graduation, should a student fail. For those students, this will delay graduation and cancel their residency match.

Why force this dilemma by making an arbitrary decision about timing? We can make a strong argument for making this test a requirement for residency, not for medical school.

In addition to these concerns about when to take the examination, the high cost of the exam and the expense of reaching one of the test sites must be borne by each student. This just adds to the crushing debt that already burdens our medical students.

Finally, and most importantly, there is no evidence that this exam will produce the results that are desired, i.e., fewer state license actions for misconduct, negligence and incompetence. These are legitimate interests of the state, but overwhelmingly take place after many years of practice. Ironically, in this day of evidence-based medicine, those who would tell us how to practice the profession, for which we have trained extensively and exhaustively, have yet to consider any evidence for the requirements they so pompously heap upon us.

The AMA continues to voice our firm opposition and has requested NBME to increase the number of test sites immediately; consider rotating actors throughout the sites while centralizing the scoring of videos; provide a list of recommended texts to prepare students, including NBME products and others; and address and carefully plan for the remediation and retesting issues that may arise.

We have also requested medical schools not to require passage of the clinical skills examination for graduation and we have encouraged residency program directors not to require passage of the exam for entering into the residency.

The AMA member medical students, residents, medical school faculty members and physicians have spent hundreds, if not thousands, of hours examining this problem -- and communicating it to the powers that be.

Despite repeated accountings of our concerns, the NBME and FSMB have paid little if any attention and have recently affirmed their recalcitrant position. I urge you to contact the NBME (www.nbme.org), your state licensing board and your medical school.

Give voice to your opposition.

Dr. Plested, a thoracic and cardiovascular surgeon from Brentwood, Calif., was chair of the AMA Board of Trustees during 2003-04 and served as AMA president during 2006-07.