NSF/FDA: Towards an active surveillance framework to detect AI/ML-enabled Software as a Medical Device (SaMD) data and performance drift in clinical flow

支持临床人工智能/机器学习（AI/ML）的软件作为医疗设备（SaMD）越来越多地用于医疗保健应用（包括医疗成像），这给监管机构在确保这些设备有效、稳健、透明、可解释、公平、安全和准确方面带来了重大挑战。其中一个主要挑战是数据移位现象，这是指用于模型训练/测试的数据分布与应用模型的数据分布之间的不匹配。这使得很难在不同的医疗机构、不同的医疗设备和疾病模式中推广支持AI/ ml的SaMD，从而导致AI模型性能下降、错误输出和不良的患者结果。该资助的重点是开发新的方法，用于检测医疗保健网络物理系统中支持AI/ ml的samd中的数据变化，在受控实验环境中使用肺癌结节预测研究和商用AI工具。该项目的目标是创建一个框架，使samd能够通过现实世界的学习进行适应，提高其检测肺癌结节的安全性和有效性。创新的数据移位检测算法将推进支持AI/ML的医疗网络物理系统，提高模型的准确性和可靠性，以应对采用医疗AI/ML应用程序的现实挑战。此外，该资助致力于通过为代表性不足的少数群体和女性驻校学者提供在FDA担任研究学者的机会，促进STEM领域的多样性、公平性和包容性。该研究由计算机和信息科学与工程理事会的计算机和网络系统部门（CISE/CNS）在NSF网络物理系统（CPS）计划下支持。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The increasing use of Clinical Artificial Intelligence/Machine Learning (AI/ML)-enabled Software as a Medical Device (SaMD) for healthcare applications, including medical imaging, is posing significant challenges for regulatory bodies in ensuring that these devices are valid, robust, transparent, explainable, fair, safe, and accurate. One of the major challenges is the phenomenon of data shift, which refers to a mismatch between the distribution of the data that was used for model training/testing and the distribution of the data to which the model was applied. This makes it difficult to generalize AI/ML-enabled SaMD across different healthcare institutions, different medical devices, and disease patterns, resulting in AI model performance deterioration, erroneous outputs, and adverse patient outcomes.This grant focuses on developing novel methodologies for detecting data shifts in AI/ML-enabled SaMDs in medical cyber-physical systems for healthcare, using lung cancer nodule prediction with research and commercially available AI tools in controlled experimental settings. The project's objective is to create a framework that allows SaMDs to adapt through real-world learning, enhancing their safety and effectiveness in detecting lung cancer nodules. The innovative data shift detection algorithms will advance AI/ML-enabled medical cyber-physical systems, improving model accuracy and reliability to address real-world challenges in the adoption of medical AI/ML applications. Moreover, this grant is committed to promote diversity, equity, and inclusion in STEM fields by providing opportunities for underrepresented minority groups and female scholars-in-residence to work as research scholars at the FDA.This research is supported by the Computer and Information Science and Engineering Directorate's Division of Computer and Network Systems (CISE/CNS) under the NSF Cyber-Physical Systems (CPS) program.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.