Collaborative Research: CIF: Medium: Foundations of Robust Deep Learning via Data Geometry and Dyadic Structure

深度学习是机器学习的一个子领域，它使用人工神经网络从数据中学习模式。目前，深度学习方法在人脸识别、图像处理、语音识别和机器翻译等重要问题上取得了最好的性能，并已被用于空间气象、分子生物学、化学和健康科学等科学问题。尽管取得了这样的成功，深度学习方法仍然被广泛视为黑匣子，它们的决定很难解释。此外，对于如何设计一个有效的深度神经网络，人们在数学上知之甚少。这个项目将通过调查如何使用数据的几何形状来更好地理解和改进深度学习设计策略来促进这种理解。这些进展将提高医学、公共卫生、分子生物学和化学等具有重大社会影响的科学领域的深度学习表现。该研究项目还包括对博士生的指导，该项目产生的材料将被纳入多门关于数据科学的课程。在这个项目中，研究团队将开发一个全面的理论框架，利用数据几何、图形和不变性来提高对广泛深度学习问题的性能和可解释性。这将包括调查神经网络变换对数据几何的影响，这是通过在神经网络的不同层应用流形学习来捕获的。新的衡量标准将被设计成经验性地测试相对于不同层的对称群的不变性。这些指标将被用来驱动一个数据增强过程，该过程在训练期间最大化神经网络的学习不变性。还将开发检测数据中隐藏不变性的方法，可用于分析神经网络的训练过程。通过调查这些问题，该项目将提供深度神经网络设计指南，将导致超越启发式的新一代深度学习，并确保神经网络的设计能够确保学习任务的准确性、健壮性和可解释性。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Deep learning is a subfield of machine learning that uses artificial neural networks to learn patterns from data. Deep learning methods currently achieve the best performance in important problems such as facial recognition, image processing, speech recognition, and machine translation and have also been used in scientific problems in space weather, molecular biology, chemistry, and the health sciences. Despite this success, deep-learning methods are still widely viewed as black boxes, where their decisions are difficult to interpret. Furthermore, there is little mathematical understanding about how to design an effective deep neural network. This project will advance this understanding by investigating how the geometry of the data can be used to better understand and improve deep-learning design strategies. These advances will lead to improved performance in deep learning in scientific fields with large societal impact such as medicine, public health, molecular biology, and chemistry. The research project also involves the mentoring of doctoral students and the material generated from the project will be incorporated into multiple courses on data science. In this project, the team of researchers will develop a comprehensive theoretical framework for harnessing data geometry, graphs, and invariance to enhance performance and interpretability for a wide scope of deep learning problems. This will include investigating the effects of neural network transformations on the data geometry, which is captured by applying manifold learning at different layers in the neural network. New metrics will be designed to empirically test invariance relative to a symmetry group at different layers. These metrics will be used to drive a data-augmentation process that maximizes the learned invariance of the neural network during training. Methods for detecting hidden invariances in data will also be developed that can be used to analyze the neural network training process. By investigating these problems, this project will provide deep-neural-network design guidelines that will lead to a new generation of deep learning that moves beyond heuristics and ensures that a neural network is designed to ensure accuracy, robustness, and interpretability for the learning task.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.