RI: Small: Collaborative Research: Topology-Aware Image Understanding using Deep Variational Objectives

图像分割是一项基本的计算机视觉任务，它从给定的图像中提取感兴趣的对象。本课题将经典数学基础与现代深度神经网络相结合，开发新的图像分割方法。特别是，所开发的方法将在分割精细尺度对象实例及其拓扑方面实现高质量。对细节和拓扑结构（例如部件之间的连接）的正确分割对于下游分析（例如对物体的功能进行推理——可以对它们采取什么行动——以及生物医学图像分析）至关重要。该项目不仅在原理数学理论和实际深度图像分割框架之间架起了桥梁，而且还培养了下一代研究人员和教育工作者。通过精心设计的综合教育和推广计划，主要研究人员将吸引本科生、高中生、女性和其他代表性不足的学生参与研究活动。本课题研究分割问题的深度变分松弛，即将分割任务视为连续值预测问题，采用变分函数作为深度神经网络的训练损失函数。深度学习的引入允许高度非线性函数的估计，并极大地提高了变分方法的能力，如Mumford-Shah泛函和持久同调，在分割具有明确边界和正确拓扑的实例时。在机器人功能和影响预测以及医学成像方面的应用将改进这些领域的最先进技术。由此产生的技术和软件将在图像分割、可视性和医学成像数据集上进行验证，以便对提议的方法进行定量评估。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Image segmentation, which extracts objects of interest from given images, is a fundamental computer vision task. This project develops novel image segmentation methodology combining classic mathematical foundations and modern deep neural networks. In particular, the developed methodology will achieve high quality in segmenting fine-scale object instances, as well as their topology. Correct segmentation of fine-details and topology such as connectivity between parts is critical for downstream analysis such as reasoning about affordance of objects - what actions can be made on them - and biomedical image analysis. This project not only bridges the gap between principled mathematical theory and the practical deep image segmentation framework, but also trains the next generation of researchers and educators. Through a carefully designed integrated educational and outreach plan, the principal investigators will engage undergraduate students, high school students, women, and other underrepresented students in the research activities.This project studies deep variational relaxations of segmentation problems, namely, consider the segmentation task as a continuous valued prediction problem and employ variational functionals as training loss functions for deep neural networks. The introduction of deep learning allows highly nonlinear functions to be estimated and greatly improves the capability of variational approaches such as the Mumford-Shah functional and the persistent homology, in segmenting instances with sharp boundaries and with correct topology. Applications on robotic affordance and influence prediction and medical imaging will improve state-of-the-arts in those areas. The resulting techniques and software will be validated on image segmentation, affordance and medical imaging datasets, in order to provide quantitative assessments of the proposed approaches.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.