The Food and Drug Administration (FDA) has authorized for market some 1,000 medical devices powered by augmented intelligence (AI), even as a “whole government” policy on its use has yet to be offered by lawmakers or regulators in Washington.
This gap in government guidance was referenced in an AMA Board of Trustees report (PDF) on “Augmented Intelligence Development, Deployment and Use in Health Care” that included policy recommendations that were adopted last November at the 2024 AMA Interim Meeting in Lake Buena Vista, Florida.
“New policy and guidance are needed to ensure that they [AI-enabled health care tools] are designed, developed and deployed in a manner that is ethical, equitable, responsible, accurate and transparent,” the report says.
In an AMA webinar discussing the current and future landscape of health care AI policy, AMA President Bruce A. Scott, MD, said that “voluntary standards are not going to be enough—we need to make sure that the principles of AI implementation are regulated.”
Dr. Scott also explained why the AMA refers to AI in the manner that it does, and how the word choice represents an underlying principle concerning the use of AI in health care.
“At the AMA, we like to refer to AI not as ‘artificial intelligence,’ but rather as ’augmented intelligence‘ to emphasize the human component of this new resource and technology so patients know that, whatever the future holds, that there will be a physician at the heart of their health care and their decisions,” Dr. Scott said.
He added that AMA policy is “pushing the fact that physicians should be involved in the development and the implementation of AI technology so that we will know that it will work at the bedside, in our clinics, in our offices, in the hospitals, or the emergency departments, wherever it is that we practice.”
According to AMA survey results published this year, physicians’ use of health AI for certain tasks nearly doubled in just one year, and enthusiasm for the technology is growing even if some doubts still linger.
From AI implementation to EHR adoption and usability, the AMA is fighting to make technology work for physicians, ensuring that it is an asset to doctors—not a burden.
States step up
Webinar panelist Jared Augenstein, senior managing director at the consulting firm Manatt Health, noted that state and federal lawmakers and regulators are seeking to strike a balance. The aim is to enact rules that address concerns about accuracy, bias and privacy—but not at the expense of stifling the promised benefits that AI innovation is expected to produce.
Because there has been little AI legislation coming out of Congress, state legislatures are stepping up—as evidenced by 250 health AI-related bills being introduced this year across 34 states, Augenstein said.
Generally, the state bills cover these four basic topics.
Transparency. Typically, these bills outline disclosure or information requirements between those who develop AI systems, those who deploy them, and the end user.
Consumer protection. These are focused on ensuring that AI systems don't unfairly discriminate, that they abide by disclosure requirements, and that there's a way for end users of AI systems to contest AI decisions.
Payer use of AI. “We've seen a lot of activity in that space,” Augenstein said, adding that these bills generally establish what oversight measures are necessary when payers use AI tools to support clinical decision-making and utilization management.
Clinical use. Bills are being introduced relating to the use of health AI tools by physicians and nonphysician clinicians.
The most meaningful legislation has been passed by California, Colorado and Utah, Augenstein said, but he added that there is some movement toward postponing implementation of Colorado’s law until differences between the governor and the legislature can be ironed out.
Panelist Kimberly Horvath, senior attorney for the AMA Advocacy Resource Center, noted the important role states play in the development of national laws and policies.
“States pass a lot of bills in a lot of different areas—a lot more than you see at the federal level,” Horvath said. “They move much faster, and they're often viewed as laboratories for these potential policy solutions, and a lot of things that trickle up to the federal level start at the state level.”
Stay up to date on AI
Follow the latest news on AI and its applications and effects for health care—delivered to your inbox.
Federal regulations in flux
The beginnings of a comprehensive AI policy at the federal level started to form as the Biden administration drew to a close. The January 2024 release of a final rule from the Assistant Secretary for Technology Policy (formerly the Office of the National Coordinator for Health IT) on health data, technology and interoperability covered updates to the EHR certification program and regulations on algorithm transparency and information sharing.
“We were very supportive of those efforts,” said Shannon Curtis, a webinar panelist and the AMA’s assistant director for federal affairs. “It was really the first federal effort at mandating any kind of transparency from an EHR vendor—which we thought was a really critical step toward what has been a really high priority for us: transparency requirements through federal regulation.”
The Centers for Medicare & Medicaid Services (CMS) also released important and welcome guidance regarding Medicare Advantage plans’ use of algorithms for prior authorization and claims review. Included was a stipulation that, if using AI to assist in coverage determinations, plans must consider a patient’s medical history and physician’s recommendations.
“However, that was just in a [FAQ] memo—it was guidance, and there's not formal law,” Curtis said.
In January, the Biden administration released a proposal for FDA guidance to device manufacturers developing AI-powered tools.
“It was the first time we'd seen any guidance from FDA that was recommending how companies should approach describing the intended use of a product, what their performance validation should look like, and how that should be communicated,” Curtis said.
This included “things like user-interface design, cybersecurity and—really important for us—new recommendations for how these products should be labeled,” she added. “We were strongly encouraging of that draft guidance (PDF) and hope they do move towards finalization with this new administration.”
The Trump administration has shown interest in developing its own AI policies and, in May, put out a request for information on the health technology ecosystem. The new administration is soliciting input on “the market of digital health products for Medicare beneficiaries as well as the state of data interoperability and broader health technology infrastructure.”
Regarding AI specifically, the Trump administration requested information about “how can key themes and technologies such as artificial intelligence, population health analytics, risk stratification, care coordination, usability, quality measurement, and patient engagement be better integrated into APM [alternative payment model] requirements?”
“We're clearly moving towards an administration—and potentially a Congress—that is much more interested in deregulation than looking at higher levels of regulation or more appropriate regulation in the AI space,” Curtis said.
Curtis expressed concern about the proposed budget bill before Congress that includes a provision to put a 10-year moratorium on state-based regulation of AI.
“If that were to pass, no states would be able to pass new AI laws or new AI regs for 10 years,” she said. “We were very, very concerned to see that given the fact that the federal government has been so slow in moving forward on any more appropriate regulatory schemes for AI and particularly for health care AI.”
