SPX: Collaborative Research: Ula! - An Integrated Deep Neural Network (DNN) Acceleration Framework with Enhanced Unsupervised Learning Capability

SPX：合作研究：乌拉！

• 批准号: 1725447
• 负责人: Yuan Xie
• 金额: $ 28万
• 依托单位: University of California-Santa Barbara
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2021-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1725447&HistoricalAwards=false
• 关键词: SPX; Collaborative; Research; Ula; Integrated

In light of very recent revolutions of unsupervised learning algorithms (e.g., generative adversarial networks and dual-learning) and the emergence of their applications, three PIs/co-PI from Duke and UCSB form a team to design Ula! - an integrated DNN acceleration framework with enhanced unsupervised learning capability. The project revolutionizes the DNN research by introducing an integrated unsupervised learning computation framework with three vertically-integrated components from the aspects of software (algorithm), hardware (computing), and application (realization). The project echoes the call from the BRAIN Initiative (2013) and the Nanotechnology-Inspired Grand Challenge for Future Computing (2015) from the White House. The research outcomes will benefit both Computational Intelligence (CI) and Computer Architecture (CA) industries at large by introducing a synergy between computing paradigm and artificial intelligence (AI). The corresponding education components  enhance existing curricula and pedagogy by introducing interdisciplinary modules on the software/hardware co-design for AI with creative teaching practices, and give special attentions to women and underrepresented minority groups.The project performs three tasks: (1) At the software level, a generalized hierarchical decision-making (GHDM) system is designed to efficiently execute the state-of-the-art unsupervised learning and reinforcement learning processes with substantially reduced computation cost; (2) At the hardware level, a novel DNN computing paradigm is designed with enhanced unsupervised learning supports, based on the novelties in near data computing, GPU architecture, and FGPA + heterogeneous platforms; (3) At the application level, the usage of Ula! is exploited in scenarios that can greatly benefit from unsupervised learning and reinforcement learning. The developed techniques are also demonstrated and evaluated on three representative computing platforms: GPU, FPGA, and emerging nanoscale computing systems, respectively.

鉴于无监督学习算法的最近革命（例如，生成对抗网络和双重学习）及其应用的出现，来自杜克和UCSB的三位PI/co-PI组成了一个团队来设计Ula！- 集成的DNN加速框架，具有增强的无监督学习能力。该项目通过引入一个集成的无监督学习计算框架，从软件（算法），硬件（计算）和应用（实现）三个方面垂直集成组件，彻底改变了DNN研究。该项目响应了来自BRAIN Initiative（2013）和白宫的纳米技术启发的未来计算大挑战（2015）的呼吁。研究成果将通过引入计算范式和人工智能（AI）之间的协同作用，使计算智能（CI）和计算机架构（CA）行业受益。相应的教育部分通过引入人工智能软硬件协同设计的跨学科模块和创造性的教学实践来加强现有的课程和教学法，并特别关注妇女和代表性不足的少数群体。该项目执行三项任务：（1）在软件层面，一个广义层次决策（GHDM）系统的设计，以有效地执行国家的，本发明涉及具有显著降低的计算成本的无监督学习和强化学习过程;（2）在硬件层面，基于近数据计算的新颖性，设计了一种具有增强的无监督学习支持的DNN计算范式，GPU架构，FGPA +异构平台;（3）在应用层面，Ula！在可以极大地受益于无监督学习和强化学习的场景中使用。所开发的技术也被证明和评估三个代表性的计算平台：GPU，FPGA，和新兴的纳米级计算系统，分别。

项目成果

Direct Training for Spiking Neural Networks: Faster, Larger, Better

• DOI: 10.1609/aaai.v33i01.33011311
• 发表时间: 2018-09
• 期刊: ArXiv
• 作者: Yujie Wu;Lei Deng;Guoqi Li;Jun Zhu;Luping Shi

HyGCN: A GCN Accelerator with Hybrid Architecture

• DOI: 10.1109/hpca47549.2020.00012
• 发表时间: 2020-01
• 期刊: 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA)
• 作者: Mingyu Yan;Lei Deng;Xing Hu;Ling Liang;Yujing Feng;Xiaochun Ye;Zhimin Zhang;Dongrui Fan

Rethinking the performance comparison between SNNS and ANNS

重新思考 SNN 和 ANN 之间的性能比较

• DOI: 10.1016/j.neunet.2019.09.005
• 发表时间: 2020-01-01
• 期刊: NEURAL NETWORKS
• 影响因子: 7.8
• 作者: Deng, Lei;Wu, Yujie;Xie, Yuan
• 通讯作者: Xie, Yuan

SmartExchange: Trading Higher-cost Memory Storage/Access for Lower-cost Computation

• DOI: 10.1109/isca45697.2020.00082
• 发表时间: 2020-05
• 期刊: 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA)
• 作者: Yang Zhao;Xiaohan Chen;Yue Wang;Chaojian Li;Haoran You;Y. Fu;Yuan Xie;Zhangyang Wang;Yingyan Lin

NNBench-X: A Benchmarking Methodology for Neural Network Accelerator Designs

NNBench-X：神经网络加速器设计的基准测试方法

• DOI: 10.1145/3417709
• 发表时间: 2020
• 期刊: ACM Transactions on Architecture and Code Optimization
• 影响因子: 1.6
• 作者: Xie, Xinfeng;Hu, Xing;Gu, Peng;Li, Shuangchen;Ji, Yu;Xie, Yuan
• 通讯作者: Xie, Yuan