Deep Learning on Manifolds: New Architectures and Theoretical Foundations

流形深度学习：新架构和理论基础

• 批准号: 2113642
• 负责人: Zhiliang Xu
• 金额: $ 18万
• 依托单位: University of Notre Dame
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-07-01 至 2024-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2113642&HistoricalAwards=false
• 关键词: Deep; Learning; Manifolds; New; Architectures

Over the last couple of decades, deep learning approaches have achieved breakthrough performance in a broad range of learning problems from a variety of applications fields such as image recognition, speech recognition, natural language processing and others. Deep learning has also served as the main impetus for the advancement of recent artificial intelligence (AI) technologies. The main goal of this project is to develop new deep neural network (DNN) architectures, computational algorithms and foundational theory for deep learning in complex domains where the data are complex geometric objects. The underlying geometry of the space will be utilized and exploited for developing the DNNs. Such development will enable the application of deep learning models in medical imaging, computer vision, computer graphics, recommender systems and neuroscience to learning problems where the inputs are complex images, diffusion matrices, shapes or other geometric objects. In addition, results of the project will be integrated in a course on Geometry and Statistics for undergraduate and graduate students. The principal investigators will also be involved in the research training of students, possibly students from under-represented groups. The main goals of the research program are to develop general deep neural network architectures on manifolds and take some major steps toward understanding their theoretical foundations. Specifically, this project will (1) develop extrinsic deep neural networks (eDNNs) on manifolds by generalizing the popular feedforward neural networks in the Euclidean space to manifolds utilizing equivariant embeddings; (2) develop intrinsic deep neural networks (iDNNs) on manifolds employing a Riemannian structure of the manifold; (3) develop general retraction-based convolutional neural networks (RCNNs) on manifolds; (4) characterize theoretical properties of eDNNs, iDNNs and RCNNs on manifolds by studying their approximation properties as well as the estimation error for a class of empirical risk minimizers over the DNN class on manifolds; and (5) develop user-friendly software packages that can be readily used by the practitioners. The research program explores the interface among geometry, statistics and machine learning, and aims to broaden the paradigm of deep learning by extending geometric deep learning to manifolds.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在过去的几十年里，深度学习方法在图像识别、语音识别、自然语言处理等各种应用领域的广泛学习问题中取得了突破性的性能。深度学习也是最近人工智能（AI）技术进步的主要推动力。该项目的主要目标是开发新的深度神经网络（DNN）架构，计算算法和基础理论，用于复杂领域的深度学习，其中数据是复杂的几何对象。 空间的基本几何形状将被利用和开发用于开发DNN。 这样的发展将使深度学习模型在医学成像、计算机视觉、计算机图形学、推荐系统和神经科学中的应用成为可能，以学习输入是复杂图像、扩散矩阵、形状或其他几何对象的问题。此外，该项目的成果将纳入本科生和研究生的几何和统计课程。主要研究人员还将参与学生的研究培训，可能是代表性不足群体的学生。该研究计划的主要目标是在流形上开发通用深度神经网络架构，并采取一些重要步骤来理解其理论基础。具体而言，该项目将（1）通过将欧几里得空间中流行的前馈神经网络推广到利用等变嵌入的流形来开发流形上的外部深度神经网络（eDNN）;（2）采用流形的黎曼结构开发流形上的内部深度神经网络（iDNN）;（3）开发流形上的通用基于收缩的卷积神经网络（RCNN）;（4）通过研究流形上的eDNN、iDNN和RCNN的近似性质以及流形上的DNN类上的一类经验风险最小化器的估计误差来表征流形上的eDNN、iDNN和RCNN的理论性质;以及（5）开发用户友好的软件包，可以方便地被从业者使用。该研究项目探索几何学、统计学和机器学习之间的接口，旨在通过将几何深度学习扩展到流形来拓宽深度学习的范式。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Extrinsic Bayesian Optimization on Manifolds

• DOI: 10.3390/a16020117
• 发表时间: 2023-02
• 期刊: Algorithms
• 影响因子: 2.3
• 作者: Yi-Zheng Fang;Mu Niu;P. Cheung;Lizhen Lin

A likelihood approach to nonparametric estimation of a singular distribution using deep generative models

• 发表时间: 2021-05
• 期刊: J. Mach. Learn. Res.
• 作者: Minwoo Chae;Dongha Kim;Yongdai Kim;Lizhen Lin

Training Graph Neural Networks by Graphon Estimation

• DOI: 10.1109/bigdata52589.2021.9671996
• 发表时间: 2021-09
• 期刊: 2021 IEEE International Conference on Big Data (Big Data)
• 作者: Ziqing Hu-;Yihao Fang-;Lizhen Lin