NSF/FDA: Towards an active surveillance framework to detect AI/ML-enabled Software as a Medical Device (SaMD) data and performance drift in clinical flow

NSF/FDA：建立主动监测框架，以检测支持 AI/ML 的软件即医疗设备 (SaMD) 数据和临床流程中的性能漂移

• 批准号: 2326034
• 负责人: Yelena Yesha
• 金额: $ 19.98万
• 依托单位: University of Miami
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2326034&HistoricalAwards=false
• 关键词: NSF; FDA; Towards; active; surveillance

The increasing use of Clinical Artificial Intelligence/Machine Learning (AI/ML)-enabled Software as a Medical Device (SaMD) for healthcare applications, including medical imaging, is posing significant challenges for regulatory bodies in ensuring that these devices are valid, robust, transparent, explainable, fair, safe, and accurate. One of the major challenges is the phenomenon of data shift, which refers to a mismatch between the distribution of the data that was used for model training/testing and the distribution of the data to which the model was applied. This makes it difficult to generalize AI/ML-enabled SaMD across different healthcare institutions, different medical devices, and disease patterns, resulting in AI model performance deterioration, erroneous outputs, and adverse patient outcomes.This grant focuses on developing novel methodologies for detecting data shifts in AI/ML-enabled SaMDs in medical cyber-physical systems for healthcare, using lung cancer nodule prediction with research and commercially available AI tools in controlled experimental settings. The project's objective is to create a framework that allows SaMDs to adapt through real-world learning, enhancing their safety and effectiveness in detecting lung cancer nodules. The innovative data shift detection algorithms will advance AI/ML-enabled medical cyber-physical systems, improving model accuracy and reliability to address real-world challenges in the adoption of medical AI/ML applications. Moreover, this grant is committed to promote diversity, equity, and inclusion in STEM fields by providing opportunities for underrepresented minority groups and female scholars-in-residence to work as research scholars at the FDA.This research is supported by the Computer and Information Science and Engineering Directorate's Division of Computer and Network Systems (CISE/CNS) under the NSF Cyber-Physical Systems (CPS) program.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

越来越多地将支持临床人工智能/机器学习(AI/ML)的软件作为医疗设备(SaMD)用于医疗应用，包括医疗成像，这对监管机构在确保这些设备有效、稳健、透明、可解释、公平、安全和准确方面提出了重大挑战。主要挑战之一是数据转移现象，这是指用于模型训练/测试的数据分布与应用模型的数据分布之间的不匹配。这使得在不同的医疗机构、不同的医疗设备和疾病模式中推广启用AI/ML的SAMD变得困难，导致AI模型的性能恶化、错误的输出和不利的患者结果。这笔拨款专注于开发新的方法，用于在医疗保健的医疗网络物理系统中检测启用AI/ML的SAMD的数据移动，使用肺癌结节预测与受控实验设置中的研究和商业可用的AI工具。该项目的目标是创建一个框架，使SAMD能够通过现实世界的学习来适应，增强它们在检测肺癌结节方面的安全性和有效性。创新的数据移位检测算法将推进支持AI/ML的医疗网络物理系统，提高模型的准确性和可靠性，以应对采用医疗AI/ML应用程序的现实世界挑战。此外，这笔赠款致力于促进STEM领域的多样性、公平性和包容性，为未被充分代表的少数群体和女性常驻学者提供机会，在FDA担任研究学者。这项研究由NSF网络物理系统(CPS)计划下的计算机和信息科学与工程局计算机和网络系统部门(CEISE/CNS)支持。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。