NSF/FDA: Towards an active surveillance framework to detect AI/ML-enabled Software as a Medical Device (SaMD) data and performance drift in clinical flow

越来越多地将支持临床人工智能/机器学习（AI/ML）的软件作为医疗器械（SaMD）用于医疗保健应用（包括医学成像），这对监管机构提出了重大挑战，以确保这些器械有效、稳健、透明、可解释、公平、安全和准确。主要挑战之一是数据转移现象，这是指用于模型训练/测试的数据分布与模型所应用的数据分布之间的不匹配。这使得在不同的医疗机构、不同的医疗器械和疾病模式中推广AI/ML支持的SaMD变得困难，导致AI模型性能下降、错误输出和不良患者结局。该资助重点是开发新的方法，用于检测医疗保健医疗网络物理系统中AI/ML支持的SaMD的数据变化，使用肺癌结节预测与研究和商业可用的人工智能工具在受控的实验设置。该项目的目标是创建一个框架，允许SaMD通过现实世界的学习进行适应，提高其检测肺癌结节的安全性和有效性。创新的数据偏移检测算法将推动AI/ML医疗网络物理系统的发展，提高模型的准确性和可靠性，以应对采用医疗AI/ML应用程序的现实挑战。此外，这笔赠款致力于促进多样性，公平，通过为代表性不足的少数群体和女性常驻学者提供在FDA担任研究学者的机会，在STEM领域进行研究和包容。这项研究得到了计算机和信息科学与工程理事会计算机和网络系统部门的支持该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The increasing use of Clinical Artificial Intelligence/Machine Learning (AI/ML)-enabled Software as a Medical Device (SaMD) for healthcare applications, including medical imaging, is posing significant challenges for regulatory bodies in ensuring that these devices are valid, robust, transparent, explainable, fair, safe, and accurate. One of the major challenges is the phenomenon of data shift, which refers to a mismatch between the distribution of the data that was used for model training/testing and the distribution of the data to which the model was applied. This makes it difficult to generalize AI/ML-enabled SaMD across different healthcare institutions, different medical devices, and disease patterns, resulting in AI model performance deterioration, erroneous outputs, and adverse patient outcomes.This grant focuses on developing novel methodologies for detecting data shifts in AI/ML-enabled SaMDs in medical cyber-physical systems for healthcare, using lung cancer nodule prediction with research and commercially available AI tools in controlled experimental settings. The project's objective is to create a framework that allows SaMDs to adapt through real-world learning, enhancing their safety and effectiveness in detecting lung cancer nodules. The innovative data shift detection algorithms will advance AI/ML-enabled medical cyber-physical systems, improving model accuracy and reliability to address real-world challenges in the adoption of medical AI/ML applications. Moreover, this grant is committed to promote diversity, equity, and inclusion in STEM fields by providing opportunities for underrepresented minority groups and female scholars-in-residence to work as research scholars at the FDA.This research is supported by the Computer and Information Science and Engineering Directorate's Division of Computer and Network Systems (CISE/CNS) under the NSF Cyber-Physical Systems (CPS) program.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.