CAREER: IIS: RI: Foundations of Deep Neural Network Robustness and Efficiency

深度神经网络在科学和工程领域取得了重大进展，并在现代机器学习的各种现实应用中发挥了重要作用，其中包括视觉、语音、模式识别和生物学等。在开发深度学习解决方案时，准确性或性能指标通常是重点。虽然性能很关键，但培训过程的计算负载和最终解决方案的安全性在真实环境中扮演着同样重要的角色。对抗性学习模型的最新进展在改进各种学习方法和防御威胁方面有很大的希望，但这些模型的基本方面仍然知之甚少，这限制了它们对高效和稳健决策的性能保证。考虑到这一点，这个项目同时研究了开发深层网络时需要解决的三个特性：1)性能、2)效率和3)健壮性。该项目还包括一项综合计划，通过资助研究生研究助理、高中学生和教师的暑期研究奖学金以及组织混合(在线和面对面)深度学习训练营，将研究成果整合到包容性、多样化和跨学科的教育多层次计划中。该研究计划的总体目标是通过利用概率论、信息论和统计学中的工具和概念，对深层网络的稳健性和计算方面有一个全面而基本的了解。该项目的目标是在1)适当的子网络对抗健壮性的公式，2)通过课程学习表征可转移性，以及3)开发有效的方法来降低训练中涉及的计算复杂性等方面取得关键进展。这个跨学科项目的理论和方法成果将拓宽深度学习的先验知识，并将改进机器学习模型的预测、探索和检测应用。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Deep neural networks have led to significant advances in science and engineering and play an important role in the success of modern machine learning in various real-world applications including vision, speech, pattern recognition, and biology to name a few. When developing deep-learning solutions, accuracy or performance metrics are often a key point of emphasis. While performance is critical, the computational load of the training process and security of the final solution play an equally important role in a real-world setting. Recent advances in adversarial learning models hold significant promise in improving various learning methods and defending against threats, but the fundamental aspects of these models are still poorly understood, which limits their performance guarantees for efficient and robust decisions. With this in mind, this project investigates simultaneously tackling three desirable properties when developing deep networks: 1) performance, 2) efficiency, and 3) robustness. This project also includes a comprehensive plan to integrate the research results into inclusive, diverse, and cross-disciplinary educational multilevel programs by funding graduate research assistants, summer research fellowship for high-school students and teachers, and organizing a hybrid (online and in-person) deep-learning boot camp. The overall goal of this research program is to develop a comprehensive and fundamental understanding of the robustness and computational aspects of deep networks by leveraging tools and concepts from probability, information theory, and statistics. This project aims to make critical advances in 1) proper formulations of subnetwork adversarial robustness, 2) characterizing transferability via curriculum learning, and 3) developing efficient approaches for reducing computational complexity involved in training, among others. The theoretical and methodological outcomes of this cross-disciplinary project will broaden the prior knowledge of deep learning and will improve prediction, exploration, and detection applications of machine-learning models.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.