Collaborative Research: Algebraic Framework of Compositional Functions for New Structure, Training, and Explainability of Deep Learning

深度学习是一种受人脑启发的机器学习方法。数据被馈送到一个多层(深度)的可训练“神经元”网络，然后训练该网络来对复杂的关系和过程进行建模。深度学习在图像识别和自然语言处理等应用领域取得了令人印象深刻的成功。然而，对于这种方法，几乎没有理论上的保证，没有关于误差等性能特征的保证，也没有解释它的成功。这是深度学习更广泛应用的障碍，因为许多潜在的应用需要保证安全性、可靠性和准确性。该项目追求坚实的数学基础，以便更好地理解深度学习的可解释性，使更有效的神经网络设计和训练算法受益于广泛的应用。该项目还包括一个重要的教育部分，旨在通过吸收研究人员部门(应用数学、电气工程、计算机科学、数学)的本科生和研究生参与拟议的多学科研究，促进跨学科教育。该项目包括在加州大学圣克鲁斯分校和德克萨斯大学圣安东尼奥分校促进STEM教育的多样性、公平性和包容性的计划，这两所大学都是西班牙裔服务机构。该项目的总体目标是为深度神经网络开发一个统一的代数框架和近似理论，以便该框架适用于包括回归、求解微分方程、设计最优反馈控制和计算机视觉在内的广泛范围的问题。这项研究的动机是，现实世界中大多数复杂和高维的输入输出关系可以表示为简单的低维函数的组合。因此，组合函数，包括深度神经网络，是描述复杂高维函数的一种自然方式。该项目将组成函数表示为分层非循环图，将探索要通过机器学习解决的问题的组成特征；研究分层非循环图中的错误传播；并调查组成特征与机器学习基本问题之间的相互联系，如普遍逼近中的误差界、深度神经网络设计和训练以及验证和可解释性。在这个研究项目中开发的代数框架、近似理论和计算算法应该会促进深度学习的设计、培训和数学基础。他们还寻求直接适用于广泛的应用，包括反馈控制设计和计算机视觉，包括该项目中的反馈控制设计和计算机视觉，以及其他重要的机器学习应用，如回归和动态系统的数据驱动建模。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Deep learning is a method of machine learning inspired by the human brain. Data is fed to a multi-layered (deep) network of trainable 'neurons' and the network is then trained to model complex relations and processes. Deep learning has had impressive success in applications such as image recognition and natural language processing. And yet, there are few theoretical guarantees to the method, to provide assurances in regards to performance features such as error and to explain its success. This is an impediment to broader application of deep learning, as many potential applications require guarantees for safety, reliability, and accuracy. This project pursues a solid mathematical foundation for a better understanding of the explainability of deep learning, to enable more efficient neural network design and training algorithms that benefit a wide range of applications. The project also includes a significant educational component that is designed to foster interdisciplinary education by engaging undergraduate and graduate students from the investigators' departments (Applied Mathematics, Electrical Engineering, Computer Science, Mathematics) in the proposed multidisciplinary research. The project includes plans to promote diversity, equity and inclusion in STEM education at the University of California Santa Cruz and the University of Texas at San Antonio, which are both Hispanic Serving Institutions.The overarching goal of this project is to develop a unified algebraic framework and approximation theory for deep neural networks so that the framework is applicable to a wide spectrum of problems including regression, solving differential equations, designing optimal feedback control, and computer vision. The proposed research is motivated by the fact that most complicated and high dimensional input-output relations in real-world applications can be represented as compositions of simple low-dimensional functions. Thus, compositional functions, including deep neural networks, serve as a natural way to describe complex high dimensional functions. Representing compositional functions as layered acyclic graphs, the project will explore the compositional features of the problems to be solved by machine learning; study the error propagation in layered acyclic graphs; and investigate the interconnection between the compositional features and the fundamental issues of machine learning, such as the error bounds in universal approximation, deep neural network design and training, and validation and explainability. The algebraic framework, approximation theory, and computational algorithms to be developed in this research project should advance the design, training, and mathematical foundations of deep learning. They seek also to be directly applicable to a wide spectrum of applications including feedback control design and computer vision, which are included in this project, as well as other important machine learning applications, such as regression and data-driven modeling of dynamical systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.