On October 27, 2021, the U.S. Food and Drug Administration (FDA), Health Canada and the United Kingdom’s Medicines and Healthcare products Regulatory Agency (MHRA) issued a set of ten guiding principles meant to aid the development of Good Machine Learning Practice (GMLP).

Artificial intelligence and machine learning (AI/ML) offers the potential to analyze the vast amount of real-world data generated from health care every day to provide transformative insights. These insights can not only help improve individual product design and performance, but also hold the promise of transforming health care.

However, AI/ML technology has unique complexities and considerations. The goal of these guiding principles is to help promote safe, effective, and high-quality medical devices that use AI/ML to best cultivate the future of this rapidly progressing field.

Although not formal or binding, as companies continue to leverage AI/ML in their medical devices, they should remain mindful of each of the ten guiding principles:

1. Leveraging Multi-Disciplinary Expertise Throughout the Total Product Life Cycle

Companies should leverage internal and external multi-disciplinary expertise to ensure they have a thorough understanding of the model’s integration into the clinical workflow, and the desired benefits and associated patient risks, to ensure the safety and effectiveness of the device while serving clinically meaningful needs throughout the product lifecycle.

2. Implementing Good Software Engineering and Security Practices

Companies should implement as part of model design data quality assurance, data management, good software engineering practices, and robust cybersecurity practices.

3. Utilizing Clinical Study Participants and Data Sets that Are Representative of the Intended Patient Population

Companies should ensure that their data collection protocols have sufficient representation of relevant characteristics of the intended patient population, use, and measurement inputs in an adequate sample size in their clinical study and training and test datasets so that results can reasonably be generalized to the population of interest.  Data collection protocols appropriate for the intended patient population may help to identify where the model may underperform and may mitigate bias.

4. Keeping Training Sets and Test Sets Independent

Companies should consider and address all sources of dependence between the training and test datasets, including patient, data acquisition, and site factors to guarantee independence.

5. Selecting Reference Datasets Based Upon Best Available Methods

Companies should use accepted, best available methods for developing a reference dataset, i.e., a reference standard, to ensure clinically relevant and well characterized data are collected (and that the reference’s limitations are understood).  Where available, companies should use accepted reference datasets in model development and testing that promote and demonstrate model robustness and generalizability across the target population.

6. Tailoring Model Design to the Available Data and Reflecting the Intended Use of the Device

Companies should have a solid understanding of the clinical benefits and risks related to the product and utilize this understanding to create clinically meaningful performance goals.  Additionally, companies should ensure the model design is suited to the available data and supports active mitigation of the known risks.

7. Focusing on the Performance of the Human-AI Team

Where the model has a human element, companies should consider human factors and human interpretability of the model outputs.

8. Testing Demonstrates Device Performance during Clinically Relevant Conditions

Companies should develop statistically sound tests and execute them to assess device performance data independent of the training data set. Such assessment should be conducted in clinically relevant conditions with consideration given to the intended use population, important subgroups, clinical environment and use by the Human AI-Team, measurement inputs, and potential confounding factors.

9. Providing Users Clear, Essential Information

Companies should provide users ready access to clear, contextually relevant information that is appropriate for the target audience. Such information includes not only information pertaining to the product’s intended use and indications for use, performance of the model for appropriate subgroups, characteristics of the data used to train and test the model, acceptable inputs, known limitations, user interface interpretation, and clinical workflow integration of the model, but also users should be made aware of device modifications, updates from real-world performance monitoring, the basis for decision-making (when available), and a way to communicate product concerns to the company.

10. Monitoring Deployed Models for Performance and Managing Re-Training Risks

Companies should deploy models that are capable of being monitored in real-world usage with a focus on maintaining or improving safety and performance. Further, when models are trained after deployment, companies should ensure there are appropriate controls in place to manage risks that may impact the safety and performance of the model.

FDA’s expectations with respect to GMLP will continue to advance and become more granular as additional stakeholder input is considered.  The docket for FDA’s GMLP Guiding Principles, FDA-2019-N-1185, is open for public comment.

Visit the Goodwin on Medtech hub to stay informed on important developments affecting medtech innovators and investors.

For emerging companies establishing their first supply chains, ensuring notification requirements in supply agreements for when commercial-stage manufacturing issues arise may not be top of mind. However, it is important for drug developers whose contracts enable continuation of a supply arrangement into the commercial-stage to be familiar with the U.S. Food and Drug Administration’s (FDA’s) field alert reporting (FAR) requirements for new drug application (NDA) and abbreviated new drug application (ANDA) holders to ensure adequate communication between developers and their suppliers.

By way of background, the FAR regulations at 21 C.F.R. §§ 314.81(b)(1) and 314.98(b) require NDA and ANDA holders to notify their FDA field office (using an Form FDA 3331a) within three business days of “receipt” of: (1) information concerning any incident that causes a distributed drug product or its labeling to be mistaken for, or applied to, another article; or (2) information concerning any bacteriological contamination, or any significant chemical, physical, or other change or deterioration in the distributed drug product, or any failure of one or more distributed batches of the drug product to meet the specification established for it in its approved application. In brief, timely notification by suppliers really does matter here and should not extend past one business day if at all possible.

This past summer, the FDA issued final guidance clarifying reporting timelines and the facts and circumstances that trigger submission of FARs. Amongst other things, the FDA clarified that the FAR requirements apply to all products marketed under an NDA or ANDA, including positron emission tomography drugs, designated medical gases, and combination products containing a drug constituent part. However, products that are only marketed abroad pursuant to a foreign approval with non-U.S. labeling are not subject to FDA’s FAR requirements. FDA also clarified that report-triggering events are not limited to active ingredient issues but can also include issues related to inactive ingredients, processing aids, and packaging.

Additional key takeaways include:

• FARs are required even when a problem is identified and corrected within the three business day reporting window.
• FARs are required even when a problem is identified beyond the three business day reporting window; however, a Form FDA 483 finding can result from the failure to submit timely FARs.
• Day “0” for calculation of the three business day reporting window is the day information triggering the report was received, even if received by a third-party contractor or supplier.
• Follow-up or final FARs are recommended but not required if significant new information is received.
• Separate initial FARs are required for a problem impacting drug products covered by multiple applications, but if conducting a single investigation into the issue after submitting the initial FARs, any follow-up can be provided in a single follow-up or final FAR.
• Investigations into issues identified with undistributed products should consider whether those issues may exist in distributed products, triggering a FAR.
• Possible changes or deterioration in distributed products triggering FARs include contamination by bacteria, yeast, mold, virus or other microorganisms.
• Issues leading to recalls do not release an NDA or ANDA holder from FAR reporting responsibility.

Overall, FDA’s FAR requirements necessitate prompt or immediate notification of any information discovered by suppliers that could trigger a FAR for NDA and ANDA holders. For supplier agreement negotiations, requiring prompt or immediate notification of issues in clinical-stage agreements positions a developer well to require the same in the commercial stage when FAR requirements apply. Additionally, in the commercial stage, FARs can prompt unannounced FDA for cause inspections and can also lead to expensive product recalls, so early notification, investigation, and remediation of issues warranting a FAR submission can help minimize potential liability and resource expenditure to remedy any issues that arise.

The U.S. Food and Drug Administration (FDA) now provides a list of Artificial Intelligence and Machine Learning (AI/ML)-Enabled Medical Devices that are legally marketed in the United States. These include devices (1) cleared via 510(k) premarket notifications, (2) authorized pursuant to De Novo requests, and (3) approved via premarket approval applications, or PMAs. FDA explains that the list, developed by FDA’s Digital Health Center of Excellence, while not exhaustive or comprehensive, is intended to increase transparency and access to information on these devices that span across medical disciplines.

Read the client alert.

The U.S. Food and Drug Administration’s (FDA’s) Office of Bioresearch Monitoring Operations (OBIMO) oversees domestic and foreign agency field inspections for clinical and non-clinical research. In particular, OBIMO manages the Bioresearch Monitoring (BIMO) Program which conducts onsite field inspections and data monitoring to ensure institution and industry compliance with FDA’s regulations relating to Good Clinical Practices (GCPs). These inspections can occur as a result of a marketing application submission, for general surveillance during an ongoing clinical trial, or as a result of a “for cause” reason. After an inspection, FDA investigators may issue a Form 483 to communicate any onsite findings of noncompliance with FDA’s regulations. BIMO also has authority to issue Warning Letters when the noncompliance FDA identifies is serious.

In the past 5 years, following are the three most common violations found in OBIMO Warning Letters:

1. Failure to ensure that the clinical trial was conducted according to the investigational plan. For example, in one Warning Letter, the FDA noted that a clinical investigator failed to adhere to the investigational plan because subjects took less than the required dosing of the study drug, and some subjects may have taken placebo rather than the required study drug, calling into question the validity of the study data.
2. Failure to maintain adequate and accurate study records, including the case histories of individual subjects, the disposition of the drug, or signed informed consent forms. For example, in one Warning Letter, the FDA found that a clinical investigator failed to complete diagnosis summary score sheets for multiple subjects, and the same clinical investigator also failed to accurately report the amount of drug dispensed versus the amount of drug taken by the subject.
3. Failure to ensure that proper informed consent was obtained. In several Warning Letters, the FDA determined that the investigators had failed to obtain proper informed consent from participants, including instances where exculpatory language was used waiving the participants’ legal rights, other necessary elements of informed consent were missing, and the form was not specific to the study or approved by the institutional review board.

Sponsors and sites should review FDA’s BIMO Compliance Program Guidance Manuals to better understand their responsibilities during clinical trials to ensure GCP compliance and to ensure readiness for future FDA BIMO inspections, should they occur. Anyone who has run a clinical trial will tell you that no trial is perfectly executed; deviations can and will occur, so preparedness is necessary. An effective monitoring program is critical to sponsors ultimately ensuring the integrity of their clinical trial records and data set. The Goodwin FDA Regulatory team works closely with sponsors on managing GCP issues when they arise during clinical trials.

Connect with our Goodwin FDA team to learn more.

*Madeline Fuller, a 2021 summer associate in Goodwin’s Washington, D.C. office, contributed to this post.

The Biden Administration’s withdrawal of the Trump Administration’s proposal to exempt 84 medical device types from the FDA’s premarket notification, or 510(k), requirement, underscores the promotional framework that developers and marketers of these devices are subject to.  The Trump Administration proposal included devices critical to combating the COVID-19 public health emergency, ranging from personal protective equipment and ventilators to remote patient monitoring and other types of digital health devices.

Read more about promotional considerations for these devices here.

Developing a new drug or biologic is a complex process. Based on our extensive experience advising early-stage and clinical-stage companies, the Goodwin FDA team created this “scorecard” for companies to use as a guide as they navigate the FDA drug development and approval process. The drug development scorecard (or checklist) can help companies keep track of progress, identify opportunities, and achieve milestones that are appropriate for each stage of development.

If you have product development or approval strategy questions, we encourage you to contact the Goodwin FDA team.

On May 19, 2021, the U.S. Food and Drug Administration (FDA) released an updated guidance in draft form on how to complete the Statement of Investigator form (Form FDA 1572). The guidance addresses frequently asked questions from sponsors, clinical investigators, and institutional review boards (IRBs), and it provides new information on waivers of the Form FDA 1572 signature requirement, which is particularly relevant for sponsors of clinical trials that include sites located outside the U.S.

Form 1572 is an agreement signed by an investigator to provide certain information to the sponsor and comply with FDA regulations on conducting a clinical investigation of an investigational drug or biologic, and under 21 CFR Part 312, an investigator must sign this agreement before participating in a trial. FDA’s previous guidance on the Form 1572 requirements and process, issued in 2010, touches briefly on the responsibilities of investigators conducting foreign studies under an investigational new drug application (IND) in the U.S., but it does not go into detail on how sponsors should proceed when an ex-U.S. investigator cannot or will not sign the 1572 (e.g., because the commitments for investigators on the Form 1572 extend beyond or conflict with what local law requires).

Under the updated guidance, FDA provides detailed steps for sponsors to request a waiver of the Form 1572 signature requirement for foreign investigators. A Form 1572 waiver allows a trial at a foreign site to take place under an IND even when the investigator cannot or will not sign the Form 1572, as noted above. When requesting a waiver, the sponsor should propose an alternative course of action to adequately satisfy the purpose of a signed Form 1572, and the sponsor must request and receive a 1572 waiver for an investigator before the study is initiated at the investigator’s site. Importantly, the guidance provides examples of sponsor and investigator commitment statements that would satisfy FDA’s guidelines for granting a waiver, and FDA recommends using these templates to enable FDA’s efficient review of a waiver request.

Overall, the guidance provides greater clarity on when a Form 1572 waiver would be needed and how a sponsor can obtain one. Sponsors planning to conduct a clinical study at a foreign site under an IND should review the updated guidance and, if a waiver is needed, factor in time for submission and FDA review of a waiver request before initiating the trial at a foreign site. Additionally, sponsors should ensure that clinical trial agreements with foreign sites contemplate Form 1572 completion and signatures and/or waivers when necessary.

Goodwin Life Sciences and Healthcare partner Roger Cohen and associate Anne Brendel along with Life Sciences and FDA associate Steven Tjoe kicked off Goodwin’s multi-part webinar series “Disrupt + Innovate + Transform: A Healthcare Webinar Series” with “Key Regulatory Issues for Digital Health Companies” discussing the key regulatory issues affecting digital health, telemedicine and healthcare IT companies.  The webinar series will be presented by a cross-disciplinary team of Goodwin lawyers exploring the topics that are most relevant for the healthcare industry today. From ever-changing regulatory guidelines to digital health, women’s health and privacy, Goodwin will take attendees through these topics and more and provide guidance to help you navigate the current healthcare landscape.

View the Video:

Given the breadth of clinical-stage companies that the Goodwin FDA and Healthcare teams advise, our regulatory attorneys together with our commercial contracting, products liability and insurance attorneys play an integral role in counseling clinical-stage companies on matters related to the conduct of clinical trials.

Learn more about our clinical trials service offering here.

On February 3, 2021 Goodwin FDA Regulatory partner, Julie Tibbets, Products Litigation + Counseling partner, Nilda Isidro, and Risk Management & Insurance counsel Brian Mukherjee discussed what drug and biologic companies with late-stage product candidates can do to best position their products to mitigate the risks that come with transitioning from clinical trials to marketing and sales.

Our speakers – leaders in life sciences regulatory compliance, product litigation preparedness, risk management, and insurance – highlighted best practices surrounding pharmaceutical promotion, preparing for risks inherent in the marketing and sale of prescription drugs, and the ways in which insurance can help mitigate those risks. This webinar identified key takeaways for companies that are nearing FDA approval and are poised to launch their commercial products.

Click here to view the slides and webinar recording.