CAREER: Deep Neural Networks That Can See Shape From Images: Models, Algorithms, and Applications

过去十年见证了深度学习在图像处理和计算机视觉方面的成功。然而，越来越多的证据表明，深度神经网络强烈倾向于看到图像纹理，而不是几何形状。这个项目的目标是创建一个新的深度网络家族，可以从图像中分析形状，并利用这个角度来推进图像分析和机器学习，从几何、统计和优化的数学工具。这项研究的技术部分将融合几何形状建模、机器学习和图像分析等基本不相关的科学领域。这项研究的应用组件将对现实世界的广泛应用产生直接和持久的影响，包括用于早期神经退行性疾病检测的磁共振成像定量分析，用于解剖病理跟踪的计算机断层扫描，以及用于环境监测的卫星图像。此外，这项研究的多学科性质将为来自工程、数学和医学等不同背景的学生和研究人员提供独特的机会。目前的深度神经网络无法对图像中的形状进行量化和分析，其性能也受到训练数据高维的限制。本研究开发了新的算法基础：(i)为深度神经网络提供学习形状表征和量化形状变化的功能，以最好地支持图像分析；（ii）采用端到端方法将原始图像数据转换为形状特征空间，以便在个体、群体或时间序列之间轻松可靠地进行比较；（iii）为训练大规模和高维图像数据集提供鲁棒性、可扩展性和高效的推理。这项研究不仅将拓展基于深度学习的图像技术的前沿，而且将深刻地启发除图像分析和计算机视觉之外的更广泛的学术界，如计算解剖学和图形学。在这项研究中产生的结果和工具将紧密地集成到教育活动中，并将通过开放源代码存储库以及与会议、研讨会、讲习班和邀请演讲相结合的教程传播给一般社区。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The past decade has witnessed the success of deep learning in image processing and computer vision. However, increasing evidence has shown that deep neural networks are strongly biased towards seeing image textures rather than geometric shapes. The goal of this project is to create a new family of deep networks that can analyze shapes from images and leverage this perspective to advance image analysis and machine learning with mathematical tools from geometry, statistics, and optimization. The technical component of this research will merge the largely disconnected scientific areas of geometric shape modeling, machine learning, and image analysis. The applied components of this research will have an immediate and long-lasting impact on a wide range of real-world applications, including quantitative analysis of magnetic resonance imaging for early neurodegenerative disease detection, computed tomography for anatomic pathology tracking, and satellite images for environmental monitoring. Moreover, the multidisciplinary nature of this research will provide unique opportunities to engage students and researchers from diverse backgrounds broadly across engineering, mathematics, and medicine. Current deep neural networks are incapable of quantifying and analyzing shapes presented in images, and their performances are limited by the high dimensionality of the training data. This research develops new algorithmic foundations that (i) equip deep neural networks with the functionality of learning shape representations and quantifying shape changes to best support image analysis; (ii) enable end-to-end approaches to transform raw image data into spaces of shape features that are easily and reliably compared across individuals, groups, or time sequences; and (iii) provide robust, scalable, and efficient inferences for training large-scale and high-dimensional image datasets. This research will not only expand the frontier of deep-learning-based image technologies but also profoundly inspire broader academic communities beyond image analysis and computer vision, such as computational anatomy and graphics. Results and tools produced in this research will be tightly integrated into educational activities and will be disseminated to general communities through open-source repositories, as well as tutorials in conjunction with conferences, seminars, workshops, and invited talks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.