CAREER: Towards Better Understanding, Robustness, and Efficiency of Deep Learning

职业：更好地理解深度学习、增强鲁棒性和效率

• 批准号: 2046710
• 负责人: Yingyu Liang
• 金额: $ 59.71万
• 依托单位: University of Wisconsin-Madison
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2026-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2046710&HistoricalAwards=false
• 关键词: CAREER; Towards; Better; Understanding; Robustness

Deep learning is a primary driving force behind many current intelligent decision-making systems and has achieved unprecedented success in various applications such as image processing, speech recognition, language translation, and game playing. However, the lack of adequate theoretical understanding is limiting the capacity to fully exploit the potential of deep learning in realistic environments, such as in security-sensitive or resource-constrained scenarios. This project aims to provide a thorough and systematic approach to understanding why practical deep-learning models succeed, under models of the data capturing the properties of real-world problems. This project will develop such an approach and employ the revealed principles in designing more robust and efficient deep-learning methods.The project will develop new theoretical models of properties of practical data leading to empirical success, and also provide frameworks for proving performance guarantees, including for learning in the presence of adversarial attacks or limited labeled data. It will also design new learning methods that are provably more robust and labeled-data efficient. This direction is still largely unexplored, despite significant recent research activities. The proposed theoretical and algorithmic solutions are possible through an interdisciplinary mix of tools from machine learning, statistics, and optimization. The proposed program is grounded in the investigator's prior work that includes both theoretical results and empirical validation. If successful, the proposed research can be transformational for modern intelligent systems by laying the foundations for further development. It will also help to solve new theoretical problems from practice that are not adequately addressed by current theory and will have lasting impacts on machine learning and optimization.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度学习是当前许多智能决策系统背后的主要驱动力，并在图像处理、语音识别、语言翻译和游戏等各种应用中取得了前所未有的成功。然而，缺乏足够的理论理解限制了在现实环境中充分利用深度学习潜力的能力，例如在安全敏感或资源受限的场景中。该项目旨在提供一个全面和系统的方法来理解为什么实际的深度学习模型在捕获现实世界问题属性的数据模型下成功。本项目将开发这样一种方法，并利用揭示的原则来设计更强大、更高效的深度学习方法。该项目将开发实际数据属性的新理论模型，从而获得经验上的成功，并提供证明性能保证的框架，包括在存在对抗性攻击或有限标记数据的情况下进行学习。它还将设计新的学习方法，这些方法被证明更健壮，标记数据更有效。尽管最近有重大的研究活动，但这个方向在很大程度上仍未被探索。通过机器学习、统计学和优化工具的跨学科组合，提出的理论和算法解决方案是可能的。提出的方案是在研究者的前期工作，包括理论结果和实证验证的基础上。如果成功，该研究将为现代智能系统的进一步发展奠定基础，从而具有变革性。它还将有助于解决当前理论没有充分解决的实践中的新理论问题，并将对机器学习和优化产生持久的影响。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Attentive Walk-Aggregating Graph Neural Networks

• 发表时间: 2021-10
• 期刊: Trans. Mach. Learn. Res.
• 作者: M. F. Demirel;Shengchao Liu;Siddhant Garg;Zhenmei Shi;Yingyu Liang

Stratified Adversarial Robustness with Rejection

• DOI: 10.48550/arxiv.2305.01139
• 发表时间: 2023-05
• 期刊: ArXiv
• 作者: Jiefeng Chen;Jayaram Raghuram;Jihye Choi;Xi Wu;Yingyu Liang;S. Jha

The Trade-off between Universality and Label Efficiency of Representations from Contrastive Learning

• DOI: 10.48550/arxiv.2303.00106
• 发表时间: 2023-02
• 期刊: ArXiv
• 作者: Zhenmei Shi;Jiefeng Chen;Kunyang Li;Jayaram Raghuram;Xi Wu;Yingyu Liang;S. Jha

When and How Does Known Class Help Discover Unknown Ones? Provable Understanding Through Spectral Analysis

• DOI: 10.48550/arxiv.2308.05017
• 发表时间: 2023-08
• 期刊: ArXiv
• 作者: Yiyou Sun;Zhenmei Shi;Yingyu Liang;Yixuan Li

Towards Evaluating the Robustness of Neural Networks Learned by Transduction

• 发表时间: 2021-10
• 期刊: ArXiv
• 作者: Jiefeng Chen;Xi Wu;Yang Guo;Yingyu Liang;S. Jha