RI: Small: Collaborative Research: Topology-Aware Image Understanding using Deep Variational Objectives

图像分割是计算机视觉的一项基本任务，它从给定的图像中提取感兴趣的对象。该项目开发了结合经典数学基础和现代深度神经网络的新型图像分割方法。特别是，所开发的方法将实现高质量的分割精细尺度的对象实例，以及它们的拓扑结构。正确分割细节和拓扑结构（如部件之间的连接）对于下游分析至关重要，例如推理对象的可供性-可以对它们进行什么操作-以及生物医学图像分析。该项目不仅弥合了原则性数学理论与实际深度图像分割框架之间的差距，还培养了下一代研究人员和教育工作者。通过精心设计的综合教育和推广计划，主要研究人员将吸引本科生，高中生，女性和其他代表性不足的学生参与研究活动。该项目研究分割问题的深度变分松弛，即将分割任务视为连续值预测问题，并使用变分泛函作为深度神经网络的训练损失函数。深度学习的引入允许估计高度非线性函数，并大大提高了变分方法的能力，如Mumford-Shah泛函和持久同源性，在分割具有尖锐边界和正确拓扑的实例时。机器人启示和影响预测以及医学成像的应用将改善这些领域的最新技术。所产生的技术和软件将在图像分割、启示和医学成像数据集上进行验证，以便对所提出的方法进行定量评估。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

Image segmentation, which extracts objects of interest from given images, is a fundamental computer vision task. This project develops novel image segmentation methodology combining classic mathematical foundations and modern deep neural networks. In particular, the developed methodology will achieve high quality in segmenting fine-scale object instances, as well as their topology. Correct segmentation of fine-details and topology such as connectivity between parts is critical for downstream analysis such as reasoning about affordance of objects - what actions can be made on them - and biomedical image analysis. This project not only bridges the gap between principled mathematical theory and the practical deep image segmentation framework, but also trains the next generation of researchers and educators. Through a carefully designed integrated educational and outreach plan, the principal investigators will engage undergraduate students, high school students, women, and other underrepresented students in the research activities.This project studies deep variational relaxations of segmentation problems, namely, consider the segmentation task as a continuous valued prediction problem and employ variational functionals as training loss functions for deep neural networks. The introduction of deep learning allows highly nonlinear functions to be estimated and greatly improves the capability of variational approaches such as the Mumford-Shah functional and the persistent homology, in segmenting instances with sharp boundaries and with correct topology. Applications on robotic affordance and influence prediction and medical imaging will improve state-of-the-arts in those areas. The resulting techniques and software will be validated on image segmentation, affordance and medical imaging datasets, in order to provide quantitative assessments of the proposed approaches.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.