权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

AitF: Collaborative Research: A Framework of Simultaneous Acceleration and Storage Reduction on Deep Neural Networks Using Structured Matrices

AitF：协作研究：使用结构化矩阵的深度神经网络同时加速和存储减少的框架

基本信息

批准号：
1733834
负责人：
Bo Yuan
金额：
$ 44.81万
依托单位：
CUNY City College
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2017
资助国家：
美国
起止时间：
2017-09-15 至 2018-11-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1733834&HistoricalAwards=false
关键词：
AitF Collaborative Research Framework Simultaneous

项目摘要

Deep neural networks (DNNs) have emerged as a class of powerful techniques for learning solutions in a number of challenging problem domains, including computer vision, natural language processing and bioinformatics. These solutions have been enabled mainly because we now have computational accelerators able to sift through the myriad of data required to train a neural network. As the size of DNN models continues to grow, computational and memory resource requirements for training will also grow, limiting deployment of deep learning in many practical applications. Leveraging the theory of structured matrices, this project will develop a general framework for efficient DNN training and inference, providing a significant reduction in algorithmic complexity measures in terms of both computation and storage. The project, if successful, should fundamentally impact a broad class of deep learning applications. It will explore accelerating this new structure for deep learning algorithms targeting emerging accelerator architectures, and will evaluate the benefits of these advances across a number of application domains, including big data analytics, cognitive systems, unmanned vehicles and aerial systems, and wearable devices. The interdisciplinary nature of this project bridges the areas of matrix theory, machine learning, and computer architecture, and will affect education at both Northeastern and CCNY, including the involvement of underrepresented and undergraduate students in the rich array of research tasks. The project will: (1) for the first time, develop a general theoretical framework for structured matrix-based DNN models and perform detailed analysis and investigation of error bounds, convergence, fast training algorithms, etc.; (2) develop low-space-cost and high-speed inference and training schemes for the fully connected layers of DNNs; (3) impose a weight tensor with structure and enable low computational and space cost convolutional layers; (4) develop high-performance and energy-efficient implementations of deep learning systems on high-performance parallel platforms, low-power embedded platforms, as well as emerging computing paradigms and devices; (5) perform a comprehensive evaluation of the proposed approaches on different performance metrics in a variety of platforms. The project will deliver tuned implementations targeting a range of computational platforms, including ASICs, FPGAs, GPUs and cloud servers. The hardware optimizations will focus on producing high-speed and low-cost implementations of deep learning systems.

深度神经网络（DNN）已经成为一类强大的技术，用于在许多具有挑战性的问题领域学习解决方案，包括计算机视觉，自然语言处理和生物信息学。这些解决方案之所以能够实现，主要是因为我们现在有了计算加速器，能够筛选训练神经网络所需的大量数据。随着DNN模型的规模不断增长，用于训练的计算和内存资源需求也将增长，这限制了深度学习在许多实际应用中的部署。利用结构化矩阵理论，该项目将开发一个有效DNN训练和推理的通用框架，在计算和存储方面显著降低算法复杂性。该项目如果成功，将从根本上影响广泛的深度学习应用。它将探索加速这种针对新兴加速器架构的深度学习算法的新结构，并将评估这些进步在许多应用领域的好处，包括大数据分析，认知系统，无人驾驶车辆和航空系统以及可穿戴设备。该项目的跨学科性质桥接了矩阵理论，机器学习和计算机体系结构的领域，并将影响东北大学和CCNY的教育，包括代表性不足和本科生参与丰富的研究任务。该项目将：（1）首次提出了基于结构矩阵的DNN模型的一般理论框架，并对误差界、收敛性、快速训练算法等进行了详细的分析和研究; (2)为DNN的全连接层开发低空间成本和高速推理和训练方案;（3）施加具有结构的权重张量，并实现低计算和空间成本的卷积层;（4）在高性能并行平台，低功耗嵌入式平台以及新兴计算范式和设备上开发高性能和节能的深度学习系统实现;（5）在不同的平台上对所提出的方法进行全面评估。该项目将针对一系列计算平台（包括ASIC、FPGA、GPU和云服务器）提供优化的实现。硬件优化将专注于深度学习系统的高速和低成本实现。

项目成果

期刊论文数量（5）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

C-LSTM: Enabling Efficient LSTM using Structured Compression Techniques on FPGAs

DOI：
10.1145/3174243.3174253
发表时间：
2018-02
期刊：
Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays
影响因子：
0
作者：
Shuo Wang;Zhe Li;Caiwen Ding;Bo Yuan;Qinru Qiu;Yanzhi Wang;Yun Liang
通讯作者：
Shuo Wang;Zhe Li;Caiwen Ding;Bo Yuan;Qinru Qiu;Yanzhi Wang;Yun Liang

Towards Ultra-High Performance and Energy Efficiency of Deep Learning Systems: An Algorithm-Hardware Co-Optimization Framework

DOI：
10.1609/aaai.v32i1.11653
发表时间：
2018-02
期刊：
ArXiv
影响因子：
0
作者：
Yanzhi Wang;Caiwen Ding;Zhe Li;Geng Yuan;Siyu Liao;Xiaolong Ma;Bo Yuan;Xuehai Qian;Jian Tang;Qinru Qiu;X. Lin
通讯作者：
Yanzhi Wang;Caiwen Ding;Zhe Li;Geng Yuan;Siyu Liao;Xiaolong Ma;Bo Yuan;Xuehai Qian;Jian Tang;Qinru Qiu;X. Lin

Implementation of a Near-Optimal Complex Root Clustering Algorithm

近最优复根聚类算法的实现