SHF: Small: NPU-based Architecture for Accelerating Deep Learning on Mobile Devices

SHF：小型：基于 NPU 的架构，用于加速移动设备上的深度学习

• 批准号: 2125208
• 负责人: Guohong Cao
• 金额: $ 50万
• 依托单位: Pennsylvania State Univ University Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2125208&HistoricalAwards=false
• 关键词: SHF; Small; NPU; based; Architecture

The rapid progress of deep-learning techniques has enabled many emerging artificial intelligence applications (e.g., augmented reality), and there is a tremendous demand for running these applications on mobile devices. However, deep-learning models are by nature computationally intensive, making them challenging to deploy on battery-powered mobile devices. This project systematically investigates the fundamental and challenging issues for running deep-learning applications on mobile devices by designing a mobile architecture based on Neural Processing Units (NPUs). An NPU is a microprocessor that specializes in the acceleration of deep-learning algorithms; however, it incurs accuracy loss, and it is a challenge to address this problem. This research identifies some special characteristics of running deep-learning models on NPUs and leverages such findings to design novel techniques to maximize accuracy or minimize processing time based on the application requirements. As deep learning has been successfully applied to various problems in people's daily lives, this project has great potential to benefit society by improving the performance, the energy efficiency, and the quality of running deep-learning applications on mobile devices. This project is also contributing to society through developing new curricula, disseminating research for education and training, engaging under-represented students in research, and outreaching to high-school students.The primary goal of this project is to design an NPU-based architecture for accelerating deep learning that can address the accuracy-loss problem of NPUs as well as the energy and performance limitations of current mobile architectures. The project consists of three tasks: (1) investigating model-partitioning techniques to decompose the deep-learning model into different layers running on heterogeneous processors to minimize processing time or maximize accuracy based on the application requirements; (2) designing energy-and thermal-aware architectures to address the performance limitations of the current mobile architecture, by exploring techniques to decompose the computation between heterogeneous processors to avoid overheating; (3) exploring the collaborative intelligence among edge/servers, hardware accelerators, and NPU-based architectures to optimize performance, by investigating how and where to run the computation based on the confidence level of executing deep learning models on an NPU.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度学习技术的快速发展使许多新兴的人工智能应用成为可能（例如，增强现实），并且存在在移动的设备上运行这些应用的巨大需求。然而，深度学习模型本质上是计算密集型的，这使得它们在电池供电的移动的设备上部署具有挑战性。该项目通过设计基于神经处理单元（NPU）的移动的架构，系统地研究了在移动的设备上运行深度学习应用程序的基本问题和挑战性问题。NPU是一种专门用于加速深度学习算法的微处理器;然而，它会导致精度损失，解决这个问题是一个挑战。这项研究确定了在NPU上运行深度学习模型的一些特殊特征，并利用这些发现来设计新的技术，以根据应用需求最大限度地提高准确性或最大限度地减少处理时间。由于深度学习已成功应用于人们日常生活中的各种问题，因此该项目具有巨大的潜力，可以通过提高移动的设备上运行深度学习应用程序的性能，能源效率和质量来造福社会。该项目还通过开发新课程、传播教育和培训研究成果、让代表性不足的学生参与研究以及向高中生推广等方式为社会做出贡献。该项目的主要目标是设计一种基于NPU的架构，以加速深度学习，解决NPU的准确性损失问题以及当前移动的架构的能源和性能限制。该项目包括三个任务：（1）研究模型划分技术，将深度学习模型分解为在异构处理器上运行的不同层，以根据应用需求最大限度地减少处理时间或最大限度地提高准确性;（2）设计能量和热感知架构以解决当前移动的架构的性能限制，探索异构处理器之间的计算分解技术，避免计算过热;（3）探索边缘/服务器、硬件加速器和基于NPU的架构之间的协作智能以优化性能，该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Macrotile: Toward QoE-Aware and Energy-Efficient 360-Degree Video Streaming

• DOI: 10.1109/tmc.2022.3233022
• 发表时间: 2024-02
• 期刊: IEEE Transactions on Mobile Computing
• 影响因子: 7.9
• 作者: Xianda Chen;Tianxiang Tan;Guohong Cao

Deep Learning on Mobile Devices Through Neural Processing Units and Edge Computing

• DOI: 10.1109/infocom48880.2022.9796929
• 发表时间: 2021-12
• 期刊: IEEE INFOCOM 2022 - IEEE Conference on Computer Communications
• 作者: Tianxiang Tan;G. Cao

Deep Learning on Mobile Devices With Neural Processing Units

• DOI: 10.1109/mc.2022.3215780
• 发表时间: 2023-08
• 期刊: Computer
• 影响因子: 2.2
• 作者: Tianxiang Tan;Guohong Cao

Energy-Efficient 360-Degree Video Streaming on Multicore-Based Mobile Devices

• DOI: 10.1109/infocom53939.2023.10228863
• 发表时间: 2023-05
• 期刊: IEEE INFOCOM 2023 - IEEE Conference on Computer Communications
• 作者: Xianda Chen;Guohong Cao

Deep Learning Video Analytics Through Edge Computing and Neural Processing Units on Mobile Devices

• DOI: 10.1109/tmc.2021.3105953
• 发表时间: 2021-08
• 期刊: IEEE Transactions on Mobile Computing
• 影响因子: 7.9
• 作者: Tianxiang Tan;G. Cao