SHF: Small: Enabling On-Device Bayesian Neural Network Training via An Integrated Architecture-System Approach

SHF：小型：通过集成架构系统方法实现设备上贝叶斯神经网络训练

• 批准号: 2130688
• 负责人: Xin Fu
• 金额: $ 50万
• 依托单位: University of Houston
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2130688&HistoricalAwards=false
• 关键词: SHF; Small; Enabling; Device; Bayesian

Deep-learning-based AI technologies, such as deep convolutional neural networks (DNNs), have recently achieved amazing success in numerous applications. However, DNN models can become unreliable due to the uncertainty in data, hence giving a false judgement and possibly incurring a disaster. To address this issue, Bayesian Neural Networks (BNNs), which possess a property of uncertainty estimation, have been increasingly employed in a wide range of real-world AI applications that demand reliable and robust decisions (e.g., self-driving, rescue robots, medical image diagnosis). Recently, training models in a distributed manner at mobile devices (e.g., federated learning) has become a popular and efficient training paradigm that achieves stronger data privacy, reduced data traffic and less response time compared to the cloud-centric training. Especially, federate learning on BNN models has received extensive attention. Since on-device learning is the essential step towards distributed BNN training, this project aims to enable fast and energy-efficient BNN training locally on resource-limited mobile devices. The project will have transformative impact on adopting AI technologies in many emergent domains that require swift, low-power, and most importantly, robust and reliable model training on edge and mobile devices (such as automation, medical, transportation, oil and gas, business, Internet-of-Things, and so on), helping to better explore the world and making everyday living and working more convenient and efficient. This project will also contribute to society through engaging under-represented groups, outreach to high-school students, curriculum development on machine learning, and disseminating research infrastructure for education and training.BNNs can be viewed as a probabilistic model where each model parameter (i.e., weight) is a probability distribution. There are two major BNN training approaches, which train the model via weight sampling (called Gaussian-based BNNs (GBNNs)) on the one hand and output feature map sampling (called Dropout-based Bayesian Convolutional NNs (DBCNNs)) on the other. Existing DNN accelerators are oblivious to the BNN stochastic training processes and achieving low training efficiency, and a uniform BNN accelerator is impractical due to the different sampling methods applied in GBNNs and DBCNNs. The investigators are developing a synergetic architecture-system research program that exploits and leverages the unique features of GBNNs and DBCNNs to achieve an order-of-magnitude reduction on training cost while still maintaining the model robustness, thus enabling on-device BNN training. The program comprises three objectives. (1) To enable on-device GBNN training, the investigators are dynamically eliminating/reducing the Gaussian random variables and intermediate data induced data movements, and furthermore, boosting the GBNN training speed by resorting to multi-chip-module-based DNN accelerators. (2) To enable on-device DBCNN training, the investigators are dynamically exploiting and eliminating the computation and data movement redundancy buried in the stochastic training processes. (3) The investigators are studying the efficient on-device training for mobile AI devices that involved in the distributed training for both global GBNN and DBCNN models and evaluating the overall system.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

基于深度学习的人工智能技术，如深度卷积神经网络(DNN)，最近在许多应用中取得了惊人的成功。然而，由于数据的不确定性，DNN模型可能变得不可靠，从而给出错误的判断，并可能导致灾难。为了解决这一问题，贝叶斯神经网络(BNN)具有不确定性估计的特性，已被越来越多地应用于现实世界中需要可靠和稳健决策的广泛应用中(例如，自动驾驶、救援机器人、医学图像诊断)。近年来，移动设备上的分布式训练模型(如联合学习)已经成为一种流行的高效训练范式，与以云为中心的训练相比，它实现了更强的数据隐私、更少的数据流量和更少的响应时间。尤其是基于BP神经网络模型的联邦成员学习受到了广泛的关注。由于设备上的学习是实现分布式BNN培训的基本步骤，该项目的目标是在资源有限的移动设备上实现快速和节能的BNN本地培训。该项目将对在许多新兴领域采用人工智能技术产生革命性影响，这些领域需要快速、低功耗，最重要的是在边缘和移动设备(如自动化、医疗、交通、石油和天然气、商业、物联网等)上进行稳健可靠的模型培训，帮助更好地探索世界，使日常生活和工作更加方便和高效。该项目还将通过吸引代表性不足的群体、向高中生宣传、开发关于机器学习的课程以及传播用于教育和培训的研究基础设施，为社会作出贡献。神经网络的训练方法主要有两种，一种是通过加权抽样(称为基于高斯的贝叶斯神经网络)训练模型，另一种是输出特征图抽样(称为基于丢弃的贝叶斯卷积神经网络)。现有的DNN加速器忽略了BNN的随机训练过程，训练效率低，而且由于GBNN和DBCNN采用不同的采样方法，统一的BNN加速器是不现实的。研究人员正在开发一种协同体系结构-系统研究计划，该计划利用和利用GBNN和DBCNN的独特功能，在保持模型稳健性的同时实现培训成本的数量级降低，从而实现设备上的BNN培训。该计划包括三个目标。(1)为了实现设备上的GBNN训练，研究人员正在动态地消除/减少高斯随机变量和中间数据引起的数据移动，并通过采用基于多芯片模块的DNN加速器来提高GBNN的训练速度。(2)为了实现设备上的DBCNN训练，调查人员动态地利用和消除随机训练过程中隐藏的计算和数据移动冗余。(3)调查人员正在研究移动人工智能设备的有效设备上培训，涉及全球GBNN和DBCNN模型的分布式培训，并对整个系统进行评估。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Shift-BNN: Highly-Efficient Probabilistic Bayesian Neural Network Training via Memory-Friendly Pattern Retrieving

• DOI: 10.1145/3466752.3480120
• 发表时间: 2021-10
• 期刊: MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture
• 作者: Qiyu Wan;Haojun Xia;Xingyao Zhang;Lening Wang;S. Song;Xin Fu