As a key assessment of how students understand concepts covered during their clinical clerkships, subject matter exams—frequently referred to as shelf exams—are a major milestone for medical students. As is the case with much of undergraduate medical training, the COVID-19 pandemic has forced schools and students to adapt in the administration of them.
With bodies overseeing medical education recommending students not have direct patient contact during the pandemic, clinical training has moved online. At many schools, so have shelf exams.
“When COVID-19 broke out, we had been evaluating how to support the incredible change medical schools were facing due to social distancing guidelines,” said Aggie Butler, vice president of academic and professional programs at the National Board of Medical Examiners (NBME), the organization that oversees Subject Exams. “We reached out to a panel of medical educators, faculty, and assessment administrators to identify and quickly create a short-term solution under the circumstances.”
Many forms of assessment have been put on hold during the pandemic. At present all levels of the United States Medical Licensure Examinations have been postponed.
Challenges of disrupted clinical training
When the Association of American Medical Colleges (AAMC) recommended that students refrain from direct patient contact in mid-March, trainees at the Warren Alpert Medical School of Brown University were involved in clerkships in five different specialties.
Brown has followed guidance from the AAMC and Liaison Committee on Medical Education on how to safely augment medical student training. That meant moving aspects of clinical training online, using video cases as learning materials to help make up for at least some portion of what was missed. At Brown the plan is to have students go back on the wards to complete some of the direct-patient interaction that is necessary in clerkship once the suspensions has been lifted.
“The students will eventually have to make up some time in those clerkships when we can place them back in clinical settings,” said Paul George, MD, MHPE, Brown’s associate dean for medical education. “We are watching AAMC guidelines around that. But the supplemental material we gave students we think will count for about two weeks of credit.”
With medical schools being forced to adapt to potential disruptions, the NBME offered web-based conferencing as a remote proctoring option for the administration of Subject Exams. Brown adopted it a week ago, in spite of the fact that students haven’t been seeing patients.
“The shelf exams focus on some of the bread and butter stuff students see on rotations, but a lot of it is the more esoteric stuff students aren’t going to see on their clerkship,” Dr. George said. “Students don’t need the clinical experience to pass a shelf exam. They need book knowledge to pass their shelf exam.” To get that knowledge, Dr. George said, students are drawing from question banks in addition to the video case studies they see.
The remote testing experience
As far as how the exam is proctored, Brown is doing it over the video-conference platform Zoom. Subject Exams are computer-based so the administration requires students to have two devices. Proctors are employed by Brown, and they observe 10 students taking the exam at a time. The NBME’s guidance on remote proctoring does acknowledge concerns about exam security, having the test taken remotely. Dr. George isn’t concerned about any potential lapses in student honesty and expects that students will follow Brown University’s Honor Code.
“These are adult learners,” he said. “They need to know this information. If a student used an unfair advantage to cheat on the exam, they are potentially harming their future patients. I think there’s a different level of how students approach exams in medicine than anything else because it’s a profession with life and death consequences on the other end.
Other potential challenges include variance in exam setting and technology failures.
Devin Morris, a third-year medical student at Brown, recently took her internal medicine shelf exam remotely.
“You have to understand there is an individual staring at you through a camera, even though you are unable to see them,” Morris said. “That’s the strangest part. That’s the only portion that makes a remote exam more nerve-wracking. Once you get through your first 10 or so questions you realize [the test-taking] environment is essentially the same and you forget about it.”
The AMA has curated a selection of resources to assist residents and medical students during the COVID-19 pandemic to help manage the shifting timelines, cancellations and adjustments to testing, rotations and other events.
