Collaborative Research:SCH:Bimodal Interpretable Multi-Instance Medical-Image Classification

合作研究：SCH：双峰可解释多实例医学图像分类

• 批准号: 2306572
• 负责人: Tianbao Yang
• 金额: $ 85.5万
• 依托单位: Texas A&M Engineering Experiment Station
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-01 至 2027-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2306572&HistoricalAwards=false
• 关键词: Collaborative; Research; SCH; Bimodal; Interpretable

This project focuses on creating a smarter artificial-intelligence (AI) system to better understand and analyze complex medical images, such as those from multiple scans of a patient. Traditional methods have had some success but face challenges in dealing with rare diseases and in providing explanations that doctors and patients can easily understand. This project aims to develop a modern AI approach that overcomes these limitations by leveraging a vast collection of medical images and doctors' notes, regardless of the specific health conditions to which they pertain. The research team will tackle various challenges to make the AI system more scalable, interpretable, and robust. This innovative project will deliver trustworthy AI-driven diagnostic tools to medical workers, expediting the diagnostic process for complex medical images. The impact of this project will be felt broadly in AI research and beyond, as its foundational research is likely to have impact in various applications, and its use-inspired research will enable the accelerated transition of modern AI approaches into benefits for society. The approach to achieve the overarching goal is to develop a bimodal interpretable multi-instance medical image classification framework by a scalable pretraining and finetuning approach. The framework consists of bimodal prototype-based interpretable contrastive pretraining to learn paired image and text prototypes from imbalanced unlabeled data, and multi-instance learning by deep area-under-the-receiver-operator-curve (AUC) maximization methods to learn from imbalanced patient-level labeled data. To make contrastive pretraining scalable and robust to imbalanced data, the investigators will develop a unified framework based on partial AUC losses, which not only unifies the existing contrastive loss but also induces new advanced global contrastive losses. The team of researchers will leverage new optimization tools and develop improved stochastic algorithms with mathematical guarantee without dependence on the large batch size of existing methods. To make multi-instance learning scalable and robust to imbalanced data, the investigators propose efficient stochastic algorithms for multi-instance deep AUC maximization by developing stochastic pooling operations from the lens of multi-level compositional optimization. The investigators will not only employ standard performance metrics for evaluation but will also leverage the domain expertise from radiologists to evaluate model performance and interpretability. The investigators will disseminate results through publications, open-source software, tutorials, workshops, and course materials, additionally engaging in outreach initiatives to enhance STEM learning and foster greater interest in the field.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目专注于创建更智能的人工智能(AI)系统，以更好地理解和分析复杂的医学图像，例如来自患者多次扫描的图像。传统方法取得了一些成功，但在处理罕见疾病以及提供医生和患者容易理解的解释方面面临挑战。该项目旨在开发一种现代人工智能方法，通过利用大量的医学图像和医生笔记来克服这些限制，无论它们属于哪种特定的健康状况。研究团队将应对各种挑战，使人工智能系统更具可扩展性、可解释性和健壮性。这一创新项目将为医务人员提供值得信赖的人工智能驱动的诊断工具，加快复杂医学图像的诊断过程。该项目的影响将在人工智能研究和其他领域广泛感受到，因为其基础研究可能会在各种应用中产生影响，其受使用启发的研究将使现代人工智能方法加速转变为对社会的好处。实现这一总体目标的方法是通过可扩展的预训练和精调方法来开发一个双模式可解释的多实例医学图像分类框架。该框架包括：基于双峰原型的可解释对比预训练，用于从不平衡的未标记数据中学习成对的图文原型；多示例学习，通过深度接收器算子曲线下面积最大化(AUC)方法，从不平衡的患者级标记数据中学习。为了使对比预训练具有可伸缩性和对不平衡数据的健壮性，研究人员将开发一个基于部分AUC损失的统一框架，该框架不仅统一了现有的对比损失，而且还引入了新的高级全局对比损失。研究团队将利用新的优化工具，开发具有数学保证的改进随机算法，而不依赖于现有方法的大批量。为了使多实例学习具有可伸缩性和对不平衡数据的健壮性，研究人员从多层次组合优化的角度提出了多实例深度AUC最大化的高效随机算法。研究人员不仅将采用标准的性能指标进行评估，还将利用放射科医生的领域专业知识来评估模型的性能和可解释性。调查人员将通过出版物、开源软件、教程、研讨会和课程材料传播成果，此外还将参与促进STEM学习和培养对该领域更大兴趣的外展倡议。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。