RI: Small: Collaborative Research: Topology-Aware Image Understanding using Deep Variational Objectives

图像分割是一项基本的计算机视觉任务，它从给定的图像中提取感兴趣的对象。该项目开发了结合经典数学基础和现代深度神经网络的新的图像分割方法。特别是，所开发的方法将在分割精细对象实例及其拓扑时获得高质量。对细节和拓扑的正确分割(如部件之间的连接)对于下游分析至关重要，例如关于对象的承受能力的推理-可以对它们采取什么行动-以及生物医学图像分析。该项目不仅架起了原则性数学理论与实际深度图像分割框架之间的桥梁，也培养了下一代研究人员和教育工作者。通过精心设计的综合教育和推广计划，主要研究人员将邀请本科生、高中生、女性和其他代表性不足的学生参与研究活动。本项目研究分割问题的深度变分松弛问题，即将分割任务视为一个连续值预测问题，并使用变分泛函作为深度神经网络的训练损失函数。深度学习的引入使得可以估计高度非线性的函数，并极大地提高了变分方法(如Mumford-Shah泛函和持久同调)在分割具有尖锐边界和正确拓扑的实例方面的能力。在机器人负担和影响预测以及医学成像方面的应用将提高这些领域的最新水平。所产生的技术和软件将在图像分割、启示和医学成像数据集上进行验证，以提供对建议方法的定量评估。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Image segmentation, which extracts objects of interest from given images, is a fundamental computer vision task. This project develops novel image segmentation methodology combining classic mathematical foundations and modern deep neural networks. In particular, the developed methodology will achieve high quality in segmenting fine-scale object instances, as well as their topology. Correct segmentation of fine-details and topology such as connectivity between parts is critical for downstream analysis such as reasoning about affordance of objects - what actions can be made on them - and biomedical image analysis. This project not only bridges the gap between principled mathematical theory and the practical deep image segmentation framework, but also trains the next generation of researchers and educators. Through a carefully designed integrated educational and outreach plan, the principal investigators will engage undergraduate students, high school students, women, and other underrepresented students in the research activities.This project studies deep variational relaxations of segmentation problems, namely, consider the segmentation task as a continuous valued prediction problem and employ variational functionals as training loss functions for deep neural networks. The introduction of deep learning allows highly nonlinear functions to be estimated and greatly improves the capability of variational approaches such as the Mumford-Shah functional and the persistent homology, in segmenting instances with sharp boundaries and with correct topology. Applications on robotic affordance and influence prediction and medical imaging will improve state-of-the-arts in those areas. The resulting techniques and software will be validated on image segmentation, affordance and medical imaging datasets, in order to provide quantitative assessments of the proposed approaches.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.