SHF: Small: Enabling On-Device Bayesian Neural Network Training via An Integrated Architecture-System Approach

基于深度学习的人工智能技术，如深度卷积神经网络（DNN），最近在许多应用中取得了惊人的成功。然而，由于数据的不确定性，DNN模型可能变得不可靠，从而给出错误的判断并可能引发灾难。为了解决这个问题，具有不确定性估计属性的贝叶斯神经网络（BNN）已越来越多地应用于需要可靠和鲁棒决策的广泛现实世界人工智能应用（例如，自动驾驶、救援机器人、医学影像诊断）。近来，在移动的设备（例如，联合学习）已经成为一种流行且有效的训练范例，与以云为中心的训练相比，其实现了更强的数据隐私、减少的数据流量和更少的响应时间。特别是BNN模型上的联邦成员学习得到了广泛的关注。由于设备上学习是分布式BNN训练的重要一步，因此该项目旨在在资源有限的移动的设备上本地实现快速和节能的BNN训练。该项目将对在许多新兴领域采用人工智能技术产生变革性影响，这些领域需要在边缘和移动的设备上进行快速，低功耗，最重要的是强大和可靠的模型训练（例如自动化，医疗，运输，石油和天然气，商业，物联网等），帮助更好地探索世界，使日常生活和工作更加方便和高效。该项目还将通过参与代表性不足的群体、向高中生推广、机器学习课程开发以及传播教育和培训研究基础设施来为社会做出贡献。BNN可以被视为一个概率模型，其中每个模型参数（即，权重）是概率分布。有两种主要的BNN训练方法，一方面通过权重采样（称为基于高斯的BNN（GBNN））训练模型，另一方面通过输出特征图采样（称为基于丢弃的贝叶斯卷积神经网络（DBCNN））。现有的DNN加速器忽略了BNN的随机训练过程，并且训练效率低，并且由于GBNN和DBCNN中应用的采样方法不同，因此统一的BNN加速器是不切实际的。研究人员正在开发一个协同架构系统研究计划，该计划利用并利用GBNN和DBCNN的独特功能，在保持模型鲁棒性的同时，实现训练成本的数量级降低，从而实现设备上的BNN训练。该方案包括三个目标。(1)为了实现设备上的GBNN训练，研究人员正在动态地消除/减少高斯随机变量和中间数据引起的数据移动，此外，通过采用基于多芯片模块的DNN加速器来提高GBNN训练速度。(2)为了实现设备上的DBCNN训练，研究人员正在动态地利用和消除隐藏在随机训练过程中的计算和数据移动冗余。(3)研究人员正在研究移动的AI设备的有效设备上培训，该设备涉及全球GBNN和DBCNN模型的分布式培训，并评估整个系统。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Deep-learning-based AI technologies, such as deep convolutional neural networks (DNNs), have recently achieved amazing success in numerous applications. However, DNN models can become unreliable due to the uncertainty in data, hence giving a false judgement and possibly incurring a disaster. To address this issue, Bayesian Neural Networks (BNNs), which possess a property of uncertainty estimation, have been increasingly employed in a wide range of real-world AI applications that demand reliable and robust decisions (e.g., self-driving, rescue robots, medical image diagnosis). Recently, training models in a distributed manner at mobile devices (e.g., federated learning) has become a popular and efficient training paradigm that achieves stronger data privacy, reduced data traffic and less response time compared to the cloud-centric training. Especially, federate learning on BNN models has received extensive attention. Since on-device learning is the essential step towards distributed BNN training, this project aims to enable fast and energy-efficient BNN training locally on resource-limited mobile devices. The project will have transformative impact on adopting AI technologies in many emergent domains that require swift, low-power, and most importantly, robust and reliable model training on edge and mobile devices (such as automation, medical, transportation, oil and gas, business, Internet-of-Things, and so on), helping to better explore the world and making everyday living and working more convenient and efficient. This project will also contribute to society through engaging under-represented groups, outreach to high-school students, curriculum development on machine learning, and disseminating research infrastructure for education and training.BNNs can be viewed as a probabilistic model where each model parameter (i.e., weight) is a probability distribution. There are two major BNN training approaches, which train the model via weight sampling (called Gaussian-based BNNs (GBNNs)) on the one hand and output feature map sampling (called Dropout-based Bayesian Convolutional NNs (DBCNNs)) on the other. Existing DNN accelerators are oblivious to the BNN stochastic training processes and achieving low training efficiency, and a uniform BNN accelerator is impractical due to the different sampling methods applied in GBNNs and DBCNNs. The investigators are developing a synergetic architecture-system research program that exploits and leverages the unique features of GBNNs and DBCNNs to achieve an order-of-magnitude reduction on training cost while still maintaining the model robustness, thus enabling on-device BNN training. The program comprises three objectives. (1) To enable on-device GBNN training, the investigators are dynamically eliminating/reducing the Gaussian random variables and intermediate data induced data movements, and furthermore, boosting the GBNN training speed by resorting to multi-chip-module-based DNN accelerators. (2) To enable on-device DBCNN training, the investigators are dynamically exploiting and eliminating the computation and data movement redundancy buried in the stochastic training processes. (3) The investigators are studying the efficient on-device training for mobile AI devices that involved in the distributed training for both global GBNN and DBCNN models and evaluating the overall system.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.