How Will the FDA Regulate Artificial Intelligence?

This past April, we reported from the World Medical Innovation Forum on the future of Artificial Intelligence (AI) in orthopedics.

We noted that AI is already in orthopedics in the form of diagnostic programs, robot assist programs and patient treatment algorithms. There is big money at stake. Morris Panner, CEO of Ambra Health, has written that funding for companies that provide AI-based health care solutions surpassed $400 million in 2017 and is breaking investment records.

Alan Reznick, M.D., M.B.A., F.A.A.O.S. and Ken Urish, M.D., Ph.D. write in AAOS NOW that AAOS is currently developing big data through registries like the American Joint Replacement Registry (AJRR), “and as we look to find a useful home for AI in orthopaedics, AJRR could be one place to start.”

“AI could be used to screen radiographs for subtle abnormalities, back up an emergency department doctor’s nighttime fracture readings with a machine-learning-based second opinion or follow a bone tumor’s response to chemotherapy.”

Urish is currently working on AI as a way to evaluate MRI data to detect osteoarthritis and track cartilage loss over time. “This approach may have many implications as we evaluate the usefulness of treatments like lubricants, platelet-rich plasma, and stem cells, as well as medical treatments for inflammatory arthropathies.”

Proposed Regulatory Framework

But how is the FDA going to assure the safety and effectiveness of the software associated with artificial intelligence—also known as software-as-a-medical-device (SaMD)— whose algorithms can change without human intervention and possibly affect people in ways for which they were not approved or cleared.

We didn’t have to wait long to find out how the FDA is thinking about this.

In April the agency issued a Discussion Paper titled: “Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD)“.

Then-Commissioner Scott Gottlieb, M.D., said this was an effort to “consider a new regulatory framework specifically tailored to promote the development of safe and effective medical devices that use advanced artificial intelligence algorithms.”

Avoid Returning for FDA Permission

At issue is how the agency will regulate AI/ML used in software and how it will keep track of ongoing changes to such software to ensure that any changes are safe and effective, without requiring an entirely new submission for each iteration of the software.

Currently, the FDA does not take into consideration the continued adaptation of SaMDs using AI/ML.

Simply put, AI algorithms are software that can learn from and act on data. But what if that data changes the safety profile of the device? Does the manufacturer have to go back for another clearance or approval?

The agency wants to base its regulatory framework on the internationally harmonized International Medical Device Regulators Forum (IMDRF) risk categorization principles, FDA’s benefit-risk framework, risk management principles in the software modifications guidance, and the organization-based TPLC approach as envisioned in the Digital Health Software Precertification (Pre-Cert) Program.

Let’s start with some definitions. The FDA defines SaMD is a software “intended to be used for one or more medical purposes that perform these purposes without being part of a hardware medical device.”

The agency wants to figure out how to regulate SaMDs that “learn” by using real-world evidence to continuously adapt and improve and, therefore, may need to be re-submitted for a new premarket approval.

For instance, a SaMD may need to undergo maintenance throughout its lifecycle. Maintenance can be adaptive (e.g., adjust to the use environment), perfective (e.g., improve performance), corrective (e.g., correct errors) or preventative (e.g., adjust any errors before they become issues). In the U.S., how and when manufacturers present these changes to FDA to allow the agency to reassess the safety and effectiveness of the SaMD, is a “work in progress.”

Gottlieb said the artificial intelligence technologies granted marketing authorization and cleared by the agency so far are generally called “locked” algorithms that don’t continually adapt or learn every time the algorithm is used.

These locked algorithms are modified by the manufacturer at intervals, which includes “training” of the algorithm using new data, followed by manual verification and validation of the updated algorithm.

“But there’s a great deal of promise beyond locked algorithms that’s ripe for application in the health care space, and which requires careful oversight to ensure the benefits of these advanced technologies outweigh the risks to patients,” said Gottlieb.

“…machine learning algorithms that continually evolve, often called “adaptive” or “continuously learning” algorithms, don’t need manual modification to incorporate learning or updates. Adaptive algorithms can learn from new user data presented to the algorithm through real-world use.”

For example, as we recently reported, the FDA cleared an algorithm to identify cervical fractures from CT scans which can compare scans to other CT scans and “learn” the images that correlate with fractures, and eventually, be more accurate than a human in detecting cervical fractures from those image.

For traditional software as a medical device, when modifications are made that could significantly affect the safety or effectiveness of the device, a sponsor must make a submission demonstrating the safety and effectiveness of the modifications.

The Proposed Framework

The FDA papers says “the goal of the framework is to assure that ongoing algorithm changes follow pre-specified performance objectives and change control plans, use a validation process that ensures improvements to the performance, safety and effectiveness of the artificial intelligence software, and includes real-world monitoring of performance once the device is on the market to ensure safety and effectiveness are maintained.”

Gottlieb said the agency is considering how an approach that “enables the evaluation and monitoring of a software product from its premarket development to post-market performance could provide reasonable assurance of safety and effectiveness and allow the FDA’s regulatory oversight to embrace the iterative nature of these artificial intelligence products….”

He said the “first step in developing our approach outlines information specific to devices that include artificial intelligence algorithms that make real-world modifications that the agency might require for premarket review. They include the algorithm’s performance, the manufacturer’s plan for modifications and the ability of the manufacturer to manage and control risks of the modifications.”

The agency may also intend to review what’s referred to as software’s “predetermined change control plan.”

“The predetermined change control plan would provide detailed information to the agency about the types of anticipated modifications based on the algorithm’s re-training and update strategy, and the associated methodology being used to implement those changes in a controlled manner that manages risks to patients.”

“Consistent with our existing quality systems regulation, the agency expects software developers to have an established quality system that is geared towards developing, delivering and maintaining high-quality products throughout the lifecycle that conforms to the agency’s standards and regulations.”

AI/ML SaMD Improvements

According to the paper, if the AI/ML SaMD has a modification related to an algorithm design and requires re-training with a new data set, a new premarket submission may not be necessary. There are three types of modifications that fit this description:

• Performance — to improve the performance of the SaMD, such as increasing its sensitivity, while not altering the intended use of the device or the input type
• Inputs — increasing the compatibility of the SaMD and increasing the input data type, including analysis of oximetry data and heart rate data without changing the intended use.
• Intended Use Changes — changing the significance of the information, such as from driving clinical management to treating or diagnosing and/or changing in the healthcare situation or condition to from non-serious to serious, for example.

Traditionally, the above modifications would trigger the need for a new 510(k).

The paper highlights how anticipated performance and input changes could be described in a premarket submission by outlining the associated Algorithm Change Protocol (ACP). The ACP would detail the specific methods to be used in achieving the anticipated modifications and manage any new risk.

Industry and Physician Public Comments

The comment period on the discussion paper closed in early June. There were more than 100 comments, most of which are public.

AdvaMed and the Combination Products Coalition (CPC) praised the agency for developing a regulatory approach but offered a number of recommendations for the agency to consider as it works on a potential draft guidance.

The AMA said it does not want physicians penalized who do not use AI systems while regulatory oversight, standards, clinical validation, clinical usefulness, and standards of care are in flux. The group opposes mandates by payers, hospitals, health systems, or governmental entities mandating the use of health care AI systems as a condition of licensure, participation, payment, or coverage.

Next?

Gottlieb said the agency has more work to do to build out this initial set of ideas and will rely on comments and feedback from experts and stakeholders to help inform the agency as it continues to think about how it regulate artificial intelligence technologies to improve patient care.

“We anticipate several more steps in the future, including issuing draft guidance that’ll be informed by the feedback on [the] discussion paper.”