权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Algorithms and Theory for Compressing Deep Neural Networks

压缩深度神经网络的算法和理论

基本信息

批准号：
2208126
负责人：
Penghang Yin
金额：
$ 27.66万
依托单位：
SUNY at Albany
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2022
资助国家：
美国
起止时间：
2022-09-01 至 2025-08-31
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2208126&HistoricalAwards=false
关键词：
Algorithms Theory Compressing Deep Neural

项目摘要

Deep neural networks (DNNs) have been the main driving force for recent advancements in artificial intelligence (AI) technology, profoundly impacting society in the areas of transportation, public safety, entertainment, health care, and other areas of public life. One of the biggest obstacles to AI's even broader impact on our daily lives is the typically enormous power consumption of DNNs upon deployment. The aim of this project is to develop mathematical and computational approaches for DNN compression to realize the fast and efficient deployment of AI systems on mobile platforms with low-power budgets such as smartphones. Results of this work will have a variety of applications which include video security systems, autopilot, smart robots, and face identification. The project will involve training of graduate students, development of data science courses, as well as collaboration with industry. The PI plans to (1) develop and analyze coarse gradient algorithms, featuring a biased first-order oracle, for the discretization of various neural architectures including transformer-based networks; (2) develop and analyze efficient thresholding-based algorithms for compressing networks via structured sparsity on both balanced and unbalanced data; (3) investigate the model capacity of compressed DNNs and establish universal finite-sample expressivity theory. The proposed research will also explore the applications of coarse gradient algorithms to other machine learning problems with discrete-valued loss functions and advance knowledge in discrete optimization.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度神经网络（DNN）一直是人工智能（AI）技术近期进步的主要驱动力，在交通、公共安全、娱乐、医疗保健和其他公共生活领域对社会产生了深远的影响。人工智能对我们日常生活产生更广泛影响的最大障碍之一是DNN在部署时通常会产生巨大的功耗。该项目的目的是开发DNN压缩的数学和计算方法，以实现人工智能系统在智能手机等低功耗预算的移动的平台上的快速高效部署。这项工作的结果将有各种各样的应用，包括视频安全系统，自动驾驶仪，智能机器人和人脸识别。该项目将涉及研究生的培训，数据科学课程的开发以及与行业的合作。PI计划（1）开发和分析粗梯度算法，具有有偏的一阶预言，用于各种神经架构的离散化，包括基于transformer的网络;（2）开发和分析有效的基于阈值的算法，用于通过平衡和不平衡数据的结构稀疏性压缩网络;（3）研究压缩DNN的建模能力，建立通用的有限样本表达理论。拟议的研究还将探索粗梯度算法在离散值损失函数和离散优化先进知识的其他机器学习问题中的应用。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。