When it comes to practicing medicine, there are tangible qualities and knowledge bases that make a physician effective. Then there are the intangibles. A new program is attempting to measure the latter, and it doesn’t require a visit to a testing center.
The Standardized Video Interview (SVI), a pilot program being conducted among Accreditation Council for Graduate Medical Education-accredited emergency medicine residency programs for the 2018 application cycle, is a tool aimed at allowing students to share their skills that might not be as evident through their academic metrics.
Launched by the Association of American Medical Colleges (AAMC) following a one-year research study in 2016, the program aims to measure two ACGME competencies: knowledge of professional behaviors and interpersonal skills and communication.
“We started to realize there were a lot of pain points around that whole transition to residency and the application process,” said AAMC Executive Vice President Atul Grover, MD, PhD. “What we found is applicants were applying to more and more programs. Programs were getting more applications and applicants felt like they weren’t able to give a more holistic view of themselves beyond what was there on paper in terms of academic records.”
Search for effective metrics
A number of factors—United States Medical Licensing Examination (USMLE) scores, letters of reference, grades in basic science and core clerkship classes, a personal statement—go into evaluating medical students applying to residency programs. A 2012 National Resident Matching Program survey of residency program directors found that 82 percent of programs use USMLE scores as a key factor in selecting applicants to interview. Conversely, 55 percent of programs cited interpersonal skills as a factor in selecting applicants to interview.
“The only objective data that has been available at this point has been USMLE scores, test scores and similar standardized exams ,” said Fiona Gallahue, MD, the program director for University of Washington’s emergency medicine residency program. But, she added, that is a shortcoming the new interview process may help correct.
“[USMLE scores] are certainly not predictive of who program directors and faculty view as great residents. There’s very little in the applications that predicts great residents.”
In doing a 2016, internal research study of the SVI in which candidates for emergency medicine, obstetrics and gynecology, pediatrics, and surgery programs participated on a voluntary basis, the AAMC found that there was a small correlation between scores on the video interview and other tools used to evaluate residents such as the USMLE Step 1 or Alpha Omega Alpha Honor Medical Society membership. The finding suggests the video interview may provide new information about applicants.
Scoring key attributes
The video interview consists of six questions and applicants have three minutes to answer each one. Applicants are asked to draw on personal experiences in answering questions designed to probe their professionalism and interpersonal skills.For example: How have you handled dealing with some of your more challenging patients? What specific actions did you take? What was the outcome?
These answers are then scored by AAMC-trained raters on a scale of one (rudimentary) to five (exemplary). Submissions for the 2018 residency cycle were due at the end of July. In September, the scores will be made available to applicants and residency programs.
It is hard to identify professionalism and interpersonal skills during a single personal interview day, Dr. Gallahue said, yet those are among the most essential traits successful residents must possess.
“Those are the skills that make people effective on the first day. Those are skills that are difficult to teach and absolutely critical to being a great doctor,” she said.
While the standardized video interview, or SVI, adds another step to an already rigorous application process, Dr. Gallahue argues that it may be beneficial to applicants in the long term.
Medical students enter medicine “for all the right reasons,” she said. “They want to help people, they want to be part of a community that works professionally and communicates well. We’re very optimistic that the SVI will actually predict residents who are professional and have good interpersonal and communication skills. If that’s the case, that would be very exciting for all of us.”
The AAMC is in the early stage of looking at the effectiveness of the standardized video interview. The association will evaluate the pilot program in emergency medicine, assessing whether the SVI adds value to the selection process, its impact on group differences, and how SVI scores translate to success, before offering it to other specialties.
“We are trying to figure out how we can get applicants the best information about programs and how we can get programs the best information about the people applying so they can both do this as strategically as possible,” Dr. Grover said. “This is just one set of tools and data we’re looking at, but that’s the goal.”
- Medical students have the tools, tenacity to be effective leaders
- What if you don’t match? 3 things you should do
- Match ranking advice: Consider goals beyond clinical training
