FET: Small: RESONANCE: Accelerating Speech/Language Processing through Collective Training using Commodity ReRAM Chips

将机器学习技术从计算云向下移动到更接近用户的边缘计算节点，在许多需要从收集的数据集快速响应的用例中是非常可取的。一个典型的使用场景是多任务应用程序，其中云服务器保留了经过良好训练的大规模模型，这些模型根据特定的本地需求部署在边缘设备中。 示例包括多语言音频会议中的语言翻译或带有口音的语音识别。然而，在边缘设备上支持多任务应用是具有挑战性的，因为相关的高计算成本和各种各样的涉及模型。在相应的硬件设计上花费的精力很少，特别是在边缘计算设备上支持多任务语音和自然语言处理（NLP）应用程序。本研究的目的是设计一种新的计算系统，致力于这样的多任务应用程序，特别是在加速语音/NLP，通过结合算法和硬件领域的创新。该研究通过激发语音/NLP算法设计与相应计算平台之间的交互式设计理念，使大数据研究和整个行业受益。参与这项研究的本科生和研究生将接受下一代信息技术劳动力的培训。与传统的边缘计算设备主要侧重于平衡云和边缘设备之间的工作负载并优化两者之间的通信不同，该项目专注于如何有效地分解和压缩从云中的大型多任务模型中提取的特定任务子模型，以便边缘设备的部署满足特定硬件约束下的功能和性能需求。更具体地说，算法级创新实现了可分解的语音/NLP模型，始终确保资源有限的边缘设备的正确功能和性能，而硬件级创新允许这些设备有效地支持语音/NLP多任务应用，并释放基于电阻式随机存取存储器（ReRAM）的计算平台的巨大潜力。在实时操作期间，边缘设备上的模型可以放大或缩小，以适应动态硬件环境和用户需求。该研究为算法重新设计、硬件加速和集成的软件/硬件协同设计提供了一种整体方法。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Moving machine learning techniques from the computing cloud down to edge computing nodes closer to the user is highly desirable in many use cases that require quick responses from the collected data sets. A typical use scenario is multi-task applications where a cloud server retains well-trained large-scale models, which are deployed in edge devices based on specific local needs. Examples include language translation or speech recognition with accents in multi-language audio conferences. However, supporting multi-task application on edge devices is challenging due to the associated high computational cost and large variety of involved models. Very little effort has been spent on the corresponding hardware design, especially for supporting multi-task speech and natural language processing (NLP) applications on edge compute devices. This research aims to design a novel computing system dedicated to such multi-task applications, particularly on accelerating speech/NLP, by combining innovations in both algorithm and hardware domains. The study benefits big data research, and industry at large by inspiring an interactive design philosophy between the designs of speech/NLP algorithms and the corresponding computing platforms. Undergraduate and graduate students involved in this research will be trained for the next-generation information technology workforce. Different from conventional edge computing devices that mainly focuses on balancing the workloads between the cloud and the edge devices and optimizing the communication in between, this project concentrates on how to efficiently decompose and compress the task-specific sub-models extracted from a large multi-task model in the cloud so that deployment of the edge devices meet the functionality and performance needs under the specific hardware constraint. More specifically, the algorithm-level innovations enable a decomposable speech/NLP model that always assures proper function and performance in resource-limited edge devices, while the hardware-level innovations allow these devices to efficiently support speech/NLP multi-task applications and unleash the great potential of Resistive Random Access Memory (ReRAM)-based computing platforms. During the real-time operation, the model on the edge device can be scaled-up or shrunk-down to accommodate the dynamic hardware environment and user needs. The research leads to a holistic methodology across algorithm redesign, hardware acceleration, and an integrated software/hardware co-design.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.