The medical field is keeping a close eye on ChatGPT (Generative Pretrained Transformer), a large language model developed by OpenAI that leverages huge amounts of data to mimic human conversation and assess language patterns.
ChatGPT could potentially be used as a physician’s digital assistant or to enhance clinical decision support systems. A recently published study has spotlighted its ability to pass well-known licensing exams, suggesting a useful role in medical education.
Kimberly Lomis, MD, the AMA’s vice president for medical education innovation, is hoping the attention around ChatGPT will elevate the broader issue of AI, not just how it applies to health care delivery but to education of all health professionals across disciplines.
“We have a group of innovators across health professions that's associated with the National Academy of Medicine. We’ve been trying to encourage the medical education community to get more broadly up to speed on AI,” said Dr. Lomis, co-author of a National Academy of Medicine discussion paper that addresses AI’s potential to supplement health professions education.
“There’s honestly been some hesitance to engage with it,” she added.
AI passes the USMLE
A study published in PLoS Digital Health that evaluated ChatGPT’s performance of the three exams associated with the United States Medical Licensing Exam (USMLE), has drawn new interest in the technology.
Investigators assessed its clinical reasoning abilities based on how well it answered questions from the USMLE Step 1, Step 2CK and Step 3 exams. The technology proved it could pass the exams at 60% accuracy without prior reinforcement or training.
The results suggest “that large language models such as ChatGPT may potentially assist human learners in a medical education setting, as a prelude to future integration into clinical decision-making,” wrote the study’s authors.
The AMA House of Delegates uses the term augmented intelligence (AI) as a conceptualization of artificial intelligence that focuses on AI’s assistive role, emphasizing that its design enhances human intelligence rather than replaces it.
Test is just one metric
Some may look at these study results and assume that AI could do what a doctor does, or that the USMLE exams are perhaps too simplistic.
The USMLE tests are just one metric of baseline medical knowledge, said Dr. Lomis. “There are other factors and skills that come into play with actually caring for patients.”
Ideally, the goal with AI is to combine physicians' or other health professionals' training and judgment with the power of a tool like AI to get the ultimate outcome.
Knowledge is exploding in the medical field, she noted. “We have many more factors available to us on any given patient decision, and it is starting to get to the point where there’s more information than the human mind can readily process. So how do we use AI to complement our own tool set?”
The cheat factor
When people first heard about ChatGPT in the context of education, questions about cheating arose. Some thought that students could cheat by getting ChatGPT to write their essays for them, said Dr. Lomis.
Instead of trying to control for cheating, educators could use ChatGPT to illustrate to learners the concept of augmented intelligence: that the powerful combination of human capacity and judgment assisted by AI will lead to the best care of patients, she added.
For example, medical students could critique an essay that ChatGPT writes for them, see what’s missing or what isn’t expressed that well, and learn from the experience.
The AMA has been very active in providing AI training and circulating new content, said Dr. Lomis.
“Everybody's going to need training on this,” she said. The AMA’s medical education innovation unit has already begun some work on AI foundational modules. It’s also collaborating with the National Academy of Medicine to host a workshop on AI in the health professions education this spring. The “Global Forum on Innovation in Health Professional Education” builds upon the paper co-written b y Dr. Lomis to focus on the urgent need to address AI and move this work forward.
Exchanges will take place on the implications of introducing AI in training—the topics, the expectations and how AI would apply across the different professions.
“We will be talking about the potential for tools like ChatGPT to be incorporated into the education process,” said Dr. Lomis.
