CSAE Position Statement

Prepared by:
David C. Buck, MD
Past Chair, AMA-Medical Student Section
Resident Physician, Orthopaedic Surgery
University of Nebraska Medical Center/Creighton University Hospital

Background

The American Medical Association strongly supports clinical skills training for medical students in all Liaison Committee on Medical Education (LCME) accredited programs in our country. With a majority of these programs providing clinical skills training and assessment as a component of their curriculum, the need for a standardized national examination must be carefully scrutinized. The Medical Student Section of the AMA is committed to working with the LCME, the AMA Section on Medical Schools, and other national medical student organizations to assist and develop those programs that need to improve their evaluation and assessment of clinical skills in order to ensure superior training and patient safety.

The National Board of Medical Examiners (NBME) has developed a proposal for a national standardized patient exam designed to assess data gathering and communication skills as part of Step 2 of the United States Medical Licensing Examination (USMLE). The examination will consist of a minimum of 10 scored cases each lasting 30 minutes. The proposal calls for the examination to first occur in 2004 for those students graduating in 2005.

The NBME proposal for a clinical skills assessment examination (CSAE) has many drawbacks and limitations. Although the AMA-MSS supports having students learn and demonstrate appropriate clinical skills, the examination proposed by the NBME has failed to address key concerns outlined below of medical students nationwide.

• The research and data on the proposed examination in regards to the validity and cost breakdown of the test has yet to be presented to the AMA-MSS student representatives to the NBME.
• Currently, over 75% of LCME accredited medical schools provide a mechanism for clinical skills assessment and evaluation.⁽¹⁾
• Because information detailing how many students have failed the pilot studies has not been determined, the necessity and cost-benefit of such a standardized CSAE administered by the NBME has not been established.
• One factor in the cost-benefit analysis of the proposed examination is that students will have to pay an estimated $975 to sit for the examination, and must travel to one of five proposed national testing sites nationwide, i.e., incurring further costs for travel and lodging expenses.

The burden is on the NBME to demonstrate that the benefit of this exam outweighs its projected costs, and that the exam provides a method of identifying those medical students deficient in clinical skills that cannot be accomplished at each individual medical school. Until this data can support the use of a nationally administered CSAE as a more effective means of assessing clinical skills than assessments conducted at individual medical schools, the AMA-MSS will remain strongly opposed to the current proposal.

Research

Possible alternatives to the NBME’s proposal for a national examination of clinical skills as part of the USMLE are outlined below. Research conducted over the last twenty years finds that using standardized patients and Objective Structured Clinical Examination (OSCE) are successful at assessing a medical student’s clinical skills. Because these programs can be successful at each individual medical school, the need for objective data supporting a national examination must be studied using a cost-benefit analysis.

• The use of a structured clinical observation (SCO) with guidelines and observation sheets for history taking, physical examination, and data gathering skills has been used successfully to assess those clinical skills intended to be evaluated via the costly national examination.⁽²⁾ The SCO has been highly rated by faculty and students alike as a positive tool in medical education and is an inexpensive and qualitative alternative to the proposed NBME examination
• The standardized patient examination (SPEx) is a component of the majority of medical school curricula as a means to assess clinical skills. This examination often utilizes both quantitative and qualitative comments from faculty, patients, and observers. The authors of this study suggest that in addition to utilizing SP numerical ratings, medical educators utilize the qualitative feedback to examine a medical students development of clinical competence.⁽³⁾ This method of clinical skills assessment can easily be introduced at the medical school level rather than as a component of a national examination.
• The NBME is using data from 1986 to report that some residents display unsatisfactory clinical and communication skills, which was easily remedied by using standardized patients at their home institution.⁽⁴⁾ Since the time of this paper, many medical schools have improved the curricula surrounding clinical skills assessment, with over 75% of schools utilizing a similar method of testing as proposed by the NBME.
• The NBME quotes a follow-up study in 1990 that found the performance of a small number ^* of students fell below standards set by the faculty. In addition, the article stated that using standardized patients at the medical school level as part of an organized clinical skills curriculum can provide faculty with important information about student’s performance of essential clinical activities and the levels of their clinical skills.⁽⁵⁾
• With appropriate attention to design, acceptable reliability and validity can be achieved for the Objective Structured Clinical Examination (OSCE). The authors of this study concluded that a combination of the OSCE, standardized board examinations for cognitive skill evaluation, and direct observation in the clinical setting has the potential to become the "gold standard" for measuring physician competence.⁽⁶⁾
  *-(italics introduced by author of this position paper)

The AMA-MSS will be working with its members, AMA Councils, and other leaders to make focused and critical clinical skills assessment a priority in medical school programs. We are committed to promoting implementation of school-based alternatives to ensure that students graduate with the clinical skills necessary to ensure patient safety and comfort.

NBME Arguments in Support of a National Examination

NBME*: There is evidence that poor communication, interpersonal and general clinical skills are related to a higher incidence of malpractice suits, lower treatment compliance as well as lower overall patient satisfaction. ^7,8,9 The adoption of a common, national examination of clinical skills will reinforce efforts to teach and evaluate communication and interpersonal skills in medical schools. In addition, the exam may screen out physicians who lack competence in this area, and who would be prone to related problems if allowed to enter practice.

Response: The measurements of interpersonal skills and use of English as quoted in the citations prepared by NBME do not address the more important problem of economic pressures on physicians to spend less time with patients as a major component of malpractice and patient compliance. The "screening-out" of physicians who lack clinical skill competence is of vital importance to both the NBME and the AMA. The problem is that there is no objective data that supports a national examination versus clinical skills training during medical school. Until that data exists, it would be irresponsible to institute another costly examination that may or may not do a better job of assessing a medical student’s clinical skills. Improvement of clinical skills training for medical students should be linked to changes in the curriculum and this is more appropriately the focus of the medical school accreditation of process.

NBME*: The NBME plans to provide diagnostic feedback to all candidates. This may take the form of the diagnostic feedback that is currently provided to unsuccessful candidates for the ECFMG clinical skills examination.

Response: The proposed diagnostic report is an objective method of measuring a medical student’s clinical skills. The main argument for a national examination is standardization of the model patient being interviewed. There is no data that supports that having a "standardized" standardized patient will result in a more valid assessment of clinical skills. A majority of LCME programs already utilize standardized patients or OSCE’s to evaluate a students clinical skills. Having these standardizations occur at the local level as a part of the curriculum will remove the cost burden from medical students and improve clinical skills year-round rather than as a single test day.

*- NBME statements taken from a February 2002 response to questions and concerns raised by medical student liaisons in November 2001.

Action Items for the Future

The AMA Medical Student Section will encourage the AMA to:

• Continue to support development of a standardized curricula surrounding clinical skill assessment as a component of LCME accreditation at each individual medical school. The data supports using standardized patients and OSCE’s at medical schools in combination with the USMLE licensing exams (Steps 1 and 2) to successfully prepare medical students for residency training.
• Oppose the current NBME proposal until two studies can provide objective data in support of a national standardized patient exam.
• First, there must be data showing that a national standardized exam will do a better job of assessing clinical skills than those programs already in place at the medical school.
• Second, a detailed cost-analysis should be completed regarding the number of students found to be deficient in clinical skills versus the heavy cost burden on students prior to implementation of a national examination.
• Encourage full disclosure of the NBME’s pilot study data to determine the percentage of students who fail the CSAE in order to begin to analyze the issue from a cost-benefit perspective. Upon completion of this analysis, the community of medical educators will be better prepared to determine whether testing of clinical skills should occur as a component of medical school curricula or as part of the USMLE licensing requirements. Without this data showing the yield of the proposed exam, a critical evaluation regarding the best avenue to both ensure patient safety and minimize time, financial, and geographical burdens for medical students will not be attained.
• Work with the LCME to ensure that proper clinical skills assessment occurs at the small percentage of medical schools whose programs are currently deficient so that patient safety is assured.
• Adopt explicit policy stating that, for graduates of an LCME accredited program, the AMA endorses that clinical skill requirements have been met and the FSMB should not require completion of a national clinical skills examination as a requirement for licensure.

In addition to supporting these activities, the AMA-MSS will:

• Work with other student organizations to promote making clinical skills assessment in medical schools a high priority.
• Work with the AMA Section on Medical Schools to encourage medical school deans to make clinical skills assessment a high priority in their programs.
• Encourage individual members to work with their medical school faculty and administration to facilitate discussions about clinical skills training and assessment, and identify ways to better prepare and evaluate students in this area.

References

1. US Medical Education Programs 2000 - 2001, JAMA, 286(9): 1053-1054.
2. Lane JL (2000) Structured clinical observations: a method to teach clinical skills with limited time and financial resources. Pediatrics, 105(4): 973-7.
3. Rose M (2001) Widening the lens on standardized patient assessment: what the encounter can reveal about the development of clinical competence. Academic Medicine, 76(8): 856-9
4. Stillman PL (1986) Assessing clinical skills of residents with standardized patients. Annals of Internal Medicine, 105(5): 762-71
5. Stillman PL (1990) An assessment of the clinical skills of fourth-year students at four New England medical schools. Academic Medicine, 65(5): 320-6.
6. Carraccio C, Englander R (2000) The objective structured clinical examination: a step in the direction of competency-based evaluation. Archives of Pediatric and Adolescent Medicine, 154: 736-741.
7. Beckman, HB, Markakis, KM, Suchman, AL, and Frankel, RM (1995). The doctor-patient relationship and malpractice: Lessons from plaintiff depositions. Archives of Internal Medicine, 155(5): 543.
8. Moore PJ and Adler NE, and Robertson PA (2000). Medical malpractice: the effect of doctor-patient relations on medical patient perceptions and malpractice intentions. Western Medical Journal, 173(4): 244.
9. Vincent C, Young M, Phillips A (1994). Why do people sue doctors? Lancet, 343: 1609.