Should AI be used in health care? Risks, regulations, ethics and benefits of AI in medicine

. 13 MIN READ

AMA Update covers a range of health care topics affecting the lives of physicians, residents, medical students and patients. From private practice and health system leaders to scientists and public health officials, hear from the experts in medicine on COVID-19, medical education, advocacy issues, burnout, vaccines and more.

How is AI used in the medical field? Who regulates AI in health care? In this episode: how AI is disrupting health care and patient data privacy.

Our guest is Stephen Parodi, MD, executive vice president of external affairs, communications and brand at The Permanente Federation and associate executive director for The Permanente Medical Group. AMA Chief Experience Officer Todd Unger hosts.

Speaker

  • Stephen Parodi, MD, associate executive director, The Permanente Medical Group 

AMA Recovery Plan for America’s Physicians

After fighting for physicians during the pandemic, the AMA is taking on the next extraordinary challenge: Renewing the nation’s commitment to physicians.

Unger: Hello and welcome to the AMA Update video and podcast. We're back with another episode in our new What Keeps Me Up series, where we talk with health care leaders about one thing that's on their minds right now and what they're doing about it.

My guest today is Dr. Stephen Parodi, the executive vice president of external affairs, communications and brand at The Permanente Federation and associate executive director for The Permanente Medical Group in Oakland, California. I'm Todd Unger, AMA's chief experience officer in Chicago. Dr. Parodi, it is always a pleasure to have you on The Update.

Dr. Parodi: And it's good to be with you, Todd, today.

Unger: Well, let's get started. What is keeping you up these days? And why?

Dr. Parodi: You know, I would say that number one on my list here is technology. And within technology, it's artificial intelligence. And there's both the sort of promise, hope with it and then concerns.

And for me, I think that AI is going to be significantly transformative. It already is within our clinical practices and then actually how we're going to be able to conduct care and how the health care sort of writ large organizations are going to be structured. So I think all of that is on my mind. Keeps me up.

Unger: Well, you're not alone. This is kind of technology week on the AMA Update. And we're talking about many of these same issues. But before we dig into your specific concerns, I'd love for you to talk about some of the benefits that you see for the care team with AI.

Dr. Parodi: Yeah, so I think that there are a couple of things. One is that, from a population health perspective, we've already implemented algorithms and some machine learning and AI into our practice. So let me give you an example. We use AI for advanced alerts, both in the hospital and outside of the hospital.

So it actually gives us insight into patients who are at risk for clinical deterioration. And what's important is that it's not AI acting on its own. It's really augmenting our practice. So it's alerting a care team, in some cases, a hospital team, in some cases, an ambulatory team, who then can act on those, you know, sort of alerts and information that we're getting from the AI.

And then we've been able to show mortality benefit both in the hospital and reductions in emergency visits and hospital visits if we use it in the ambulatory setting. So that's number one. Let me give you a second example. And actually, we've published on this, which is looking at AI and radiology, and using it to augment our ability to detect cancer and breast cancer in particular with mammography.

That's one simple example where I think you're going to see that expand into multiple applications in radiology. So it's not replacing a radiologist. It's enhancing their ability to actually provide an accurate diagnosis, help someone like me who's on the receiving end of looking at a radiology film in making a quicker diagnosis.

You know, so I think that those are what hold promise when it comes to AI. And of course, the question is, how do we make sure it's augmenting and integrating into practice not replacing clinical judgment and practice?

Unger: Well, those are two great examples that you just gave. And again, I want to just stress the magic word that you're using there because we hear this, certainly, here at the AMA and from a lot of physicians out there is that word augment, in terms of how the vision for AI can fit in here. So that's an excellent perspective.

Of course, as you mentioned, there are issues with AI and the technology too. What do you think are your personal top concerns with the technology?

Dr. Parodi: So number one is acceptance of it. So let me give you one other example that's also a benefit. And we're still learning. So we actually recently, about two and half months ago, introduced an ambient AI to listen in on a clinical visit.

And what it's able to do is actually from the patient and from the physician who's in the patient interaction able to generate a note. So you can imagine from a clinical documentation perspective, way more efficient. In fact, we're seeing improvements in, you know, documentation time by an hour or two in a given day for a clinician. Huge win, right?

Unger: That's a stunning amount of time. Yeah.

Dr. Parodi: It raises a whole bunch of questions.

Unger: Yeah, absolutely.

Dr. Parodi: It's amazing. Yeah. But on the other hand, it raises questions around privacy, acceptance. You know, what, from a regulatory standpoint, is required to make sure there's accurate documentation and coding with an AI? And so we're going to have to make sure that that's monitored.

So again, it's not as simple as just introducing it into the milieu. You actually have to then make sure that you're monitoring it. Meaning the humans are monitoring the AI and the application of it.

When it comes to, what federal agencies are looking into AI? And which ones are regulating it? And are they talking to doctors and getting input on how the AI should be implemented and how it should be monitored? I think that keeps me up at night.

Third thing is, how are we going to make sure that it's equitably used, that we don't have inadvertent sort of redlining of patients by the application of AI? We want to make sure that we don't introduce unintended biases into our practice because of predisposing assumptions that are in an electronic health record that AI is scraping. So we've got to have processes in place for making sure that we do have equity and that, if we see inequities, that we correct for them. And again, I think that's going to be critically important that physicians with our clinical lens, with our operational lens are involved with this conversation.

Membership Moves Medicine™

  • Free access to JAMA Network™ and CME
  • Save hundreds on insurance
  • Fight for physicians and patient rights

Unger: You've hit on several really big topics there. The first one is just the potential benefits of AI to relieve the burdens that are getting in the way of patient care. So a score of one to two hours a day is that would be unbelievable.

But of course, you point out the flip side of that being other concerns around the implementation and privacy. The second big point that you talked about is equity with the use of algorithms and making sure we don't introduce biases through those. These are big issues. What steps are you taking right now at The Permanente Medical Group to address concerns like these?

Dr. Parodi: So one of the things is having an open conversation, making sure that everyone is aware, you know, whether that's the physicians or patients that we're being transparent that we are using AI and how we're using it. The second is actually setting up some governance, you know, within the medical group and then with our hospital and health plan partners that we work with. So that we're actively dialoguing about what's being implemented, how it's being implemented.

Is it effective? Is it not? What does—you know, involving bioethicists, you know, so having the ethical considerations of implementation in addition to the operational considerations. This is not just an IT thing, OK? Let me just be real clear.

I mean, this is about how we actually practice. And so you need the people that are involved with the actual clinical care, the people that are involved with back end operations coming together, just as we would with any other initiative. So think of AI as a tool, not as a replacement. It's critically important, from my perspective. And those are really sort of the actions that we're taking within the medical group.

And then I think there's a second piece here, Todd, which is engagement in the broader medical community, right? This is bigger than just The Permanente Medical Group. And so making sure that industry players, whether that's technology players, other health systems, other physician practices, and I mean all ilk of physician practices, large groups, small groups, solos, all need to be involved in this conversation.

So we've been engaging with our medical society within California. And I'm glad to be engaging with you, Todd. Because I think this is very much a national issue.

Unger: Absolutely. And I love that you pointed out this is not an IT issue. As I had a discussion with your colleague Dr. Maria Ansari at The Permanente Medical Group, co-CEO of the Federation, and I loved her point about how cultural issues are just so important. We know with any kind of change that it's not just technology. It's about culture.

And the impact of your work, obviously, going to affect the broader landscape of medicine. And so tell me what you think about what needs to happen, so that AI can be safely used by the care team.

Dr. Parodi: So I think there's a couple of components here. So number one is making sure that we've got sufficient engagement of leadership within the medical group. And actually, I'm going to say within society all the way from the top to the people that are actually in the exam room. And having that connection is critically important.

Second is that we know that regulation is going to be coming when it comes to AI. And how we write the rules of the road are going to be critically important to ensuring, again, digital equity and to ensure that it's applied in a way that's effective. So we need to have measures. So there actually is going to have to be a strategy for what those measures look like.

And so you know, much of the journey that we went on when I think about the quality journey back in 2000 and setting up the rules of the road there, we need to have that same kind of infrastructure for AI. Except, we need to do it fast. Because, Todd, it's already here. It's already embedded in society.

And in fact, in many ways, we don't even know that it's running. So I think where we have an advantage in health care is that we're maybe a few steps farther behind than the rest of society. So we have an opportunity to set those—set the table, if you will, now. But I really think it's now. We can't wait.

And like I said, I think the benefits here are tremendous. But we have to have measures for what those benefits look like and what actually we want to avoid. We need to be able to define that.

Unger: You know what's really encouraging about your leadership in the field, here is it gets to a theme that we've been talking about this week, which is making sure that technology works for physicians and care teams, not the other way around. And having physicians kind of drive that conversation, get ahead of the rules, as you're pointing out, is so important. How do you make sure that continues to occur?

Dr. Parodi: So I think there are a couple of opportunities here. So one is that, within the AMA, being able to define those policies through our House of Delegates and through Interim meetings, so that we've got clear policy objectives as an AMA and then being able to go to Capitol Hill to inform and educate in addition to speaking specifically to policy.

And so what I'm excited about is that there are meetings that are occurring with policy makers and with their staffs. And I think that's critically important to do this year in 2024. I encourage everyone to also engage with their state medical societies. And you know, there are many regulations that are going to be formed at the state level as well.

So being able to provide those real-world stories, some of the ones we were talking about earlier, Todd, about what good looks like, what good governance looks like within an organization to show that actually we in health care are aware of the issues and know what they need to be in terms of what good looks like, that's going to be really important.

So I would say that those are sort of steps one and two for me when it comes to addressing the needs that, really, our patients are looking for. They're looking for us as physicians to lead in this space.

Unger: Thank you so much for the important point that you made there, which is, if you really want to ensure that physicians have a say in how this technology is rolled out, how it's regulated, then participate in the AMA, participate in your state medical societies. That's where you can really have a voice in this matter.

Dr. Parodi, it is always so great to have you. I love to hear your perspective on the issues, in particularly, technology right now. The AMA also has a wealth of resources on AI. And we're going to include a link to some of our latest in the description of this particular episode.

That wraps up today's AMA Update. Be sure to subscribe for new episodes and find all our videos and podcasts at ama-assn.org/podcasts. Thanks for joining us today. Please take care.


Disclaimer: The viewpoints expressed in this video are those of the participants and/or do not necessarily reflect the views and policies of the AMA.

Subscribe to AMA Update

Get videos with expert opinions from the AMA on the most important health care topics affecting physicians, residents, medical students and patients—delivered to your inbox.

AMA Update podcast logo

FEATURED STORIES