Collaborative Research: CIF: Medium: Foundations of Robust Deep Learning via Data Geometry and Dyadic Structure

合作研究：CIF：媒介：通过数据几何和二元结构实现稳健深度学习的基础

• 批准号: 2212326
• 负责人: Qing Qu
• 金额: $ 47.2万
• 依托单位: Regents of the University of Michigan - Ann Arbor
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2026-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2212326&HistoricalAwards=false
• 关键词: Collaborative; Research; CIF; Medium; Foundations

Deep learning is a subfield of machine learning that uses artificial neural networks to learn patterns from data. Deep learning methods currently achieve the best performance in important problems such as facial recognition, image processing, speech recognition, and machine translation and have also been used in scientific problems in space weather, molecular biology, chemistry, and the health sciences. Despite this success, deep-learning methods are still widely viewed as black boxes, where their decisions are difficult to interpret. Furthermore, there is little mathematical understanding about how to design an effective deep neural network. This project will advance this understanding by investigating how the geometry of the data can be used to better understand and improve deep-learning design strategies. These advances will lead to improved performance in deep learning in scientific fields with large societal impact such as medicine, public health, molecular biology, and chemistry. The research project also involves the mentoring of doctoral students and the material generated from the project will be incorporated into multiple courses on data science. In this project, the team of researchers will develop a comprehensive theoretical framework for harnessing data geometry, graphs, and invariance to enhance performance and interpretability for a wide scope of deep learning problems. This will include investigating the effects of neural network transformations on the data geometry, which is captured by applying manifold learning at different layers in the neural network. New metrics will be designed to empirically test invariance relative to a symmetry group at different layers. These metrics will be used to drive a data-augmentation process that maximizes the learned invariance of the neural network during training. Methods for detecting hidden invariances in data will also be developed that can be used to analyze the neural network training process. By investigating these problems, this project will provide deep-neural-network design guidelines that will lead to a new generation of deep learning that moves beyond heuristics and ensures that a neural network is designed to ensure accuracy, robustness, and interpretability for the learning task.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度学习是机器学习的一个子领域，它使用人工神经网络从数据中学习模式。深度学习方法目前在面部识别、图像处理、语音识别和机器翻译等重要问题上表现最好，也已用于空间天气、分子生物学、化学和健康科学等科学问题。尽管取得了成功，但深度学习方法仍然被广泛视为黑箱，其决策难以解释。此外，关于如何设计有效的深度神经网络，几乎没有数学理解。该项目将通过研究如何使用数据的几何形状来更好地理解和改进深度学习设计策略来推进这种理解。这些进步将提高深度学习在医学、公共卫生、分子生物学和化学等具有重大社会影响的科学领域的性能。该研究项目还涉及对博士生的指导，该项目产生的材料将被纳入多个数据科学课程。在这个项目中，研究人员团队将开发一个全面的理论框架，用于利用数据几何形状、图形和不变性，以提高广泛深度学习问题的性能和可解释性。这将包括研究神经网络转换对数据几何形状的影响，这是通过在神经网络的不同层应用流形学习来捕获的。新的度量将被设计为经验性地测试相对于不同层的对称群的不变性。这些指标将用于驱动数据增强过程，最大限度地提高神经网络在训练过程中的学习不变性。还将开发用于检测数据中隐藏的不变性的方法，这些方法可用于分析神经网络训练过程。通过研究这些问题，该项目将提供深度神经网络设计指南，这将导致新一代深度学习超越数学，并确保神经网络的设计确保准确性，鲁棒性，该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响进行评估来支持审查标准。

项目成果

Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learning

• DOI: 10.48550/arxiv.2210.10041
• 发表时间: 2022-10
• 期刊: ArXiv
• 作者: Shuo Xie;Jiahao Qiu;Ankita Pasad;Li Du;Qing Qu;Hongyuan Mei

Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifold

• DOI: 10.48550/arxiv.2209.09211
• 发表时间: 2022-09
• 期刊: ArXiv
• 作者: Can Yaras;Peng Wang;Zhihui Zhu;L. Balzano;Qing Qu

Robust Training under Label Noise by Over-parameterization

• 发表时间: 2022-02
• 期刊: ArXiv
• 作者: Sheng Liu;Zhihui Zhu;Qing Qu;Chong You

On the Optimization Landscape of Neural Collapse under MSE Loss: Global Optimality with Unconstrained Features

• DOI: 10.48550/arxiv.2203.01238
• 发表时间: 2022-03
• 期刊: ArXiv
• 作者: Jinxin Zhou-;Xiao Li;Tian Ding;Chong You;Qing Qu;Zhihui Zhu

Are All Losses Created Equal: A Neural Collapse Perspective

• DOI: 10.48550/arxiv.2210.02192
• 发表时间: 2022-10
• 期刊: ArXiv
• 作者: Jinxin Zhou-;Chong You;Xiao Li;Kangning Liu;Sheng Liu;Qing Qu;Zhihui Zhu