Collaborative Research:SCH:Bimodal Interpretable Multi-Instance Medical-Image Classification

合作研究：SCH：双峰可解释多实例医学图像分类

• 批准号: 2306572
• 负责人: Tianbao Yang
• 金额: $ 85.5万
• 依托单位: Texas A&M Engineering Experiment Station
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-01 至 2027-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2306572&HistoricalAwards=false
• 关键词: Collaborative; Research; SCH; Bimodal; Interpretable

This project focuses on creating a smarter artificial-intelligence (AI) system to better understand and analyze complex medical images, such as those from multiple scans of a patient. Traditional methods have had some success but face challenges in dealing with rare diseases and in providing explanations that doctors and patients can easily understand. This project aims to develop a modern AI approach that overcomes these limitations by leveraging a vast collection of medical images and doctors' notes, regardless of the specific health conditions to which they pertain. The research team will tackle various challenges to make the AI system more scalable, interpretable, and robust. This innovative project will deliver trustworthy AI-driven diagnostic tools to medical workers, expediting the diagnostic process for complex medical images. The impact of this project will be felt broadly in AI research and beyond, as its foundational research is likely to have impact in various applications, and its use-inspired research will enable the accelerated transition of modern AI approaches into benefits for society. The approach to achieve the overarching goal is to develop a bimodal interpretable multi-instance medical image classification framework by a scalable pretraining and finetuning approach. The framework consists of bimodal prototype-based interpretable contrastive pretraining to learn paired image and text prototypes from imbalanced unlabeled data, and multi-instance learning by deep area-under-the-receiver-operator-curve (AUC) maximization methods to learn from imbalanced patient-level labeled data. To make contrastive pretraining scalable and robust to imbalanced data, the investigators will develop a unified framework based on partial AUC losses, which not only unifies the existing contrastive loss but also induces new advanced global contrastive losses. The team of researchers will leverage new optimization tools and develop improved stochastic algorithms with mathematical guarantee without dependence on the large batch size of existing methods. To make multi-instance learning scalable and robust to imbalanced data, the investigators propose efficient stochastic algorithms for multi-instance deep AUC maximization by developing stochastic pooling operations from the lens of multi-level compositional optimization. The investigators will not only employ standard performance metrics for evaluation but will also leverage the domain expertise from radiologists to evaluate model performance and interpretability. The investigators will disseminate results through publications, open-source software, tutorials, workshops, and course materials, additionally engaging in outreach initiatives to enhance STEM learning and foster greater interest in the field.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目的重点是创建更智能的人工智能 (AI) 系统，以更好地理解和分析复杂的医学图像，例如来自患者多次扫描的图像。传统方法取得了一些成功，但在处理罕见疾病以及提供医生和患者易于理解的解释方面面临挑战。该项目旨在开发一种现代人工智能方法，通过利用大量医学图像和医生笔记来克服这些限制，无论它们属于何种具体健康状况。研究团队将解决各种挑战，使人工智能系统更具可扩展性、可解释性和鲁棒性。这一创新项目将为医务人员提供值得信赖的人工智能驱动的诊断工具，加快复杂医学图像的诊断过程。该项目的影响将在人工智能研究及其他领域广泛感受到，因为其基础研究可能会对各种应用产生影响，而其受使用启发的研究将使现代人工智能方法加速转变为造福社会。 实现总体目标的方法是通过可扩展的预训练和微调方法开发双模可解释的多实例医学图像分类框架。该框架包括基于双模原型的可解释对比预训练，用于从不平衡的未标记数据中学习配对图像和文本原型，以及通过接收者操作曲线下深度面积 (AUC) 最大化方法进行多实例学习，以从不平衡的患者级标记数据中学习。为了使对比预训练具有可扩展性并且对不平衡数据具有鲁棒性，研究人员将开发一个基于部分 AUC 损失的统一框架，该框架不仅统一了现有的对比损失，而且还引入了新的高级全局对比损失。研究团队将利用新的优化工具，开发具有数学保证的改进随机算法，而不依赖于现有方法的大批量大小。为了使多实例学习具有可扩展性并且对不平衡数据具有鲁棒性，研究人员通过从多级组合优化的角度开发随机池操作，提出了用于多实例深度 AUC 最大化的有效随机算法。研究人员不仅将采用标准性能指标进行评估，还将利用放射科医生的领域专业知识来评估模型性能和可解释性。研究人员将通过出版物、开源软件、教程、研讨会和课程材料传播结果，此外还参与外展活动，以加强 STEM 学习并培养对该领域的更大兴趣。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。