Collaborative Research: SHF: Small: Architecture Innovations for Enabling Simultaneous Translation at the Edge

同声传译在说了几句话后就开始翻译，具有重要的现实价值，可能会破坏和造福于广泛的领域，例如部署在外国的军事人员，参加多语言会议的商人，医疗服务提供者，执法部门，客户支持服务，外交官和政治代表以及无数的国际游客。目前，准确的实时同声翻译只能通过昂贵的、经过专门训练的人工口译员或通过在服务器级图形处理单元（GPU）上运行基于机器学习的算法来实现;这两种选择对于在边缘设备中广泛和普遍部署同声翻译都是不切实际的。创新的特定领域的架构需要被设计，可以减少数量级的计算需求，同时保持翻译的准确性，从而使同步translation.This研究的挑战和机遇，在设计硬件加速器的transformer-based同声翻译。目标是利用Transformer模型的独特特性和同声翻译的独特行为来开发满足准确性、延迟、功耗和资源效率目标的跨领域解决方案。在探索的一些具体研究领域中，包括利用广泛的输入数据共享的以输入为中心的并行计算和数据重用，旨在实现线性可扩展注意力计算的计算比例架构，基于其实际效用有效降低维度的效用驱动线性变换架构，以及提供可扩展，用于同步翻译加速器的灵活和超低成本互连。除了推动计算机体系结构和自然语言处理领域的具体技术贡献外，该项目还对研究，教育和推广产生了更广泛的影响。这项研究的结果被纳入研究生课程，课程和本科生的研究经验。已计划开展各种外联活动，以扩大不同人群对该项目的教育和社会影响的包容和参与。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Simultaneous translation, which begins translating after just a few words are spoken, has significant real-world value that may disrupt and benefit a wide range of domains, such as military personnel deployed in foreign countries, businesspeople participating in multilingual meetings, medical service providers, law enforcement, customer support services, diplomats and political representatives, and countless international tourists. Currently, accurate real-time simultaneous translation is only possible by expensive, specially trained human interpreters or by running machine learning-based algorithms on server-grade graphics processing units (GPUs); both options are impractical for extensive and ubiquitous deployment of simultaneous translation in edge devices. Innovative domain-specific architecture needs to be designed that can reduce computation requirements by orders of magnitude while maintaining translation accuracy, thus enabling simultaneous translation at the edge.This research investigates the challenges and opportunities in designing hardware accelerators for transformer-based simultaneous translation. The objective is to utilize the unique characteristics of transformer models and the distinctive behaviors of simultaneous translation to develop cross-cutting solutions that meet the accuracy, latency, power, and resource efficiency goals. Among some of the specific lines of research that are explored include input-centric dataflow and data reuse that take advantage of extensive input data sharing, compute-proportional architecture that aims to achieve linearly scalable attention calculation, utility-driven linear transformation architecture that efficiently reduce dimensionality based on their actual utility, and customized routerless on-chip interconnects that provide scalable, flexible and ultra-low cost interconnects for simultaneous translation accelerators. Beyond specific technical contributions that advance the fields of computer architecture and natural language processing, this project also impacts more broadly on research, education, and outreach. Findings from this research are incorporated into graduate curricula, courses, and undergraduate research experiences. Various outreach activities have been planned to broaden inclusion and participation of diverse populations in the educational and societal impacts of this project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.