Computational Methods for Hierarchical Manifold Learning

深度学习在实践中取得的巨大成功需要一个清晰而简洁的数学解释。尽管神经网络的组成部分和信息通过各层传播的规则非常简单，但迄今为止，人们对所涉及的各种机制的作用缺乏相应的深入理解。其次，缺乏透明度：虽然给定的一组网络权重可能适合样本内的科学或工程数据，甚至可以很好地推广样本外的数据，但使用神经网络来解释数据或产生数据的系统通常非常困难或不可能。在这个项目中，PI将利用应用和计算谐波分析在数学方法方面的最新进展，开发一种基于流形学习的分层算法，以复制深度学习的惊人成功特性，同时提高准确性、数据适应性、平滑先验和透明度。一系列计算项目计划开发一种基于拉普拉斯-贝尔特拉米算子表示流形的深度学习分层算法。提出的工作支持构建一个完整的算法，该算法使用多层流形学习核方法来表示深度流形学习基础结构中的数据。学习算法的开发包括三个部分：(1)构建分层流形学习架构，使用拉普拉斯-贝尔特拉米算子的特征函数来表示数据，并复制神经网络的共享和池化特征；(2)构建创新算法以优化识别问题的解决方案；(3)开发处理大数据集的重采样方法。

The enormous practical success of deep learning warrants a clear and concise mathematical explanation. Although the components of a neural network, and the rules for propagation of information through the layers, are extremely simple, there is to date a lack of a corresponding deep understanding of the roles of the various mechanisms involved. Second, there is a lack of transparency: While a given set of network weights may fit scientific or engineering data in-sample and even generalize well out-of-sample, using a neural net to explain the data or the system producing the data is usually very difficult or impossible. In this project, the PI will leverage recent progress in mathematical methods from applied and computational harmonic analysis to develop a hierarchical algorithm, based on manifold learning, to replicate the strikingly successful properties of deep learning while adding improved accuracy, adaptability to data, smoothness priors, and transparency.A sequence of computational projects is planned to develop a hierarchical algorithm for deep learning, based on representing manifolds by the Laplace-Beltrami operator. Proposed work supports the construction of a complete algorithm that uses layers of manifold learning kernel methods to represent data in a deep manifold learning infrastructure. The development of the learning algorithm consists of three parts: (1) the construction of a hierarchical manifold learning architecture, using eigenfunctions of the Laplace-Beltrami operator to represent data, and replicating the sharing and pooling features of neural networks, (2) building innovative algorithms for the purpose of optimizing the solution to identification problems, and (3) development of resampling methods to handle large data sets.