CAREER: IIS: RI: Foundations of Deep Neural Network Robustness and Efficiency

深度神经网络已经在科学和工程领域取得了重大进展，并在现代机器学习在各种现实世界应用中的成功中发挥了重要作用，包括视觉，语音，模式识别和生物学等。 在开发深度学习解决方案时，准确性或性能指标通常是重点。虽然性能至关重要，但训练过程的计算负载和最终解决方案的安全性在现实世界中也扮演着同样重要的角色。对抗性学习模型的最新进展在改进各种学习方法和防御威胁方面具有重要的前景，但这些模型的基本方面仍然知之甚少，这限制了它们对有效和鲁棒决策的性能保证。考虑到这一点，该项目研究在开发深度网络时同时解决三个理想的属性：1）性能，2）效率和3）鲁棒性。该项目还包括一个全面的计划，通过资助研究生研究助理，高中学生和教师的夏季研究奖学金，以及组织混合（在线和面对面）深度学习靴子营地，将研究成果融入包容性，多样性和跨学科的教育多层次计划。该研究计划的总体目标是通过利用概率，信息论和统计学的工具和概念，对深度网络的鲁棒性和计算方面进行全面和基本的了解。该项目的目标是在以下方面取得关键进展：1）子网络对抗鲁棒性的正确公式，2）通过课程学习表征可转移性，以及3）开发有效的方法来降低训练中涉及的计算复杂性。该跨学科项目的理论和方法成果将拓宽深度学习的现有知识，并将改善机器学习模型的预测，探索和检测应用。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Deep neural networks have led to significant advances in science and engineering and play an important role in the success of modern machine learning in various real-world applications including vision, speech, pattern recognition, and biology to name a few. When developing deep-learning solutions, accuracy or performance metrics are often a key point of emphasis. While performance is critical, the computational load of the training process and security of the final solution play an equally important role in a real-world setting. Recent advances in adversarial learning models hold significant promise in improving various learning methods and defending against threats, but the fundamental aspects of these models are still poorly understood, which limits their performance guarantees for efficient and robust decisions. With this in mind, this project investigates simultaneously tackling three desirable properties when developing deep networks: 1) performance, 2) efficiency, and 3) robustness. This project also includes a comprehensive plan to integrate the research results into inclusive, diverse, and cross-disciplinary educational multilevel programs by funding graduate research assistants, summer research fellowship for high-school students and teachers, and organizing a hybrid (online and in-person) deep-learning boot camp. The overall goal of this research program is to develop a comprehensive and fundamental understanding of the robustness and computational aspects of deep networks by leveraging tools and concepts from probability, information theory, and statistics. This project aims to make critical advances in 1) proper formulations of subnetwork adversarial robustness, 2) characterizing transferability via curriculum learning, and 3) developing efficient approaches for reducing computational complexity involved in training, among others. The theoretical and methodological outcomes of this cross-disciplinary project will broaden the prior knowledge of deep learning and will improve prediction, exploration, and detection applications of machine-learning models.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.