ACED Fab: Runtime Reconfigurable Array (RTRA) Technology for AI/ML

快速发展的人工智能（AI）和机器学习（ML）模型，其复杂性每3 - 4个月翻一番，超过了AI/ML底层硬件加速器的发展。目前，AI/ML算法的部署受到硬件的限制，这些硬件缺乏支持新模型的敏捷性和能效，限制了科学和工程应用的进展（例如，下一代无线系统）。需要在不牺牲能源效率的情况下进行硬件适配，以将AI/ML模型嵌入到移动的、边缘和云设备中。来自加州大学洛杉矶分校（UCLA）的研究人员沿着国立台湾大学（NTU）和台湾半导体研究院（TSRI）正在共同开发和演示概念验证运行时可重构阵列（RTRA）技术，该技术可解决现有设备的能效和灵活性问题。RTRA是一种前瞻性架构，其中敏捷的AI/ML硬件管道可以动态重新配置，以嵌入新模型或响应动态环境。这样的技术需要很好地平衡硬件和软件。简单的硬件导致复杂的软件（例如，现场可编程门阵列，或FPGA）和简单的软件导致复杂的硬件（例如，中央处理单元或CPU）。RTRA平衡硬件和软件，以实现空间和时间的灵活性。RTRA的时空随机化由于其独特的软件/硬件方法，更好地抵抗物理攻击，逆向工程，供应链和硬件木马。项目团队将使用基于系统的跨学科方法来解决RTRA的几个关键挑战：1）有效的编程范例，2）硬件资源的2D调度，3）可以实现低级别硬件潜力的高级抽象，4）多程序租赁，以及5）流水线级的重构速度（数十个时钟周期）。该项目旨在为AI/ML开发一种运行时可重构阵列技术，提供亚微秒级的程序切换决策，支持多个活动程序，多大小编译，优先处理。RTRA应该可以从高级语言（如C或Python）编程。在线硬件调度支持快速的运行时重新配置。通过在线动态多程序硬件调度、特定领域的可重配置阵列以及用于多程序租赁和灵活数据接口的面积有效互连，实现了硬件敏捷性。硬件包括用于控制的嵌入式处理器、系统存储器和用于异构系统集成的数据接口。节能处理（比FPGA好10倍，资源利用率高10倍）是对现有FPGA AI加速器的重大改进，同时为高级AI/ML模型添加时空动态。内部开发的软件工具链（从C或Python到软二进制）将用于编译和测试各种AI/ML用例。项目团队具有开发、测试和利用集成系统的独特能力。从广义上讲，该技术可以快速部署新开发的算法，加快创新应用程序的部署。此外，快速重新利用硬件的能力提供了利用“暗硅”的机会，并可能用节能的运行时可重配置结构取代当今的固定功能硬件加速器。RTRA技术旨在满足未来通信和AI/ML工作负载的需求，并提供易于访问的C或Python编程。该项目利用美国和台湾互补的学术人才来影响半导体工程和教育。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Fast-developing artificial intelligence (AI) and machine learning (ML) models, which double in complexity every 3-4 months, outpace the development of underlying hardware accelerators for AI/ML. Presently, deployment of AI/ML algorithms is limited by hardware that lacks agility and energy efficiency to support new models, limiting progress in science and engineering applications (e.g., next-generation wireless systems). Hardware adaptation without sacrificing energy efficiency is needed to embed AI/ML models into mobile, edge, and cloud devices. Researchers from the University of California, Los Angeles (UCLA) along with National Taiwan University (NTU) and Taiwan Semiconductor Research Institute (TSRI) are joining together to develop and demonstrate proof-of-concept runtime reconfigurable array (RTRA) technology that addresses the issues of energy efficiency and agility in existing devices. RTRA is a forward-looking architecture, where agile AI/ML hardware pipelines are dynamically reconfigured to embed new models or respond to dynamic environments. Such technology requires finely balanced hardware and software. Simple hardware leads to complex software (e.g., Field Programmable Gate Array, or FPGA) and simple software leads to complex hardware (e.g., Central Processing Unit, or CPU). RTRA balances hardware and software to enable spatial and temporal flexibility. Spatio-temporal randomization of RTRA is more immune against physical attacks, reverse engineering, supply chain and hardware Trojans, due to its unique software/hardware approach. The project team will use system-based cross-disciplinary approach to address several key challenges of RTRA: 1) efficient programming paradigm, 2) 2D scheduling of hardware resources, 3) high-level abstraction that can fulfill low-level hardware potential, 4) multi-program tenancy, and 5) reconfiguration speed at the pipeline level (tens of clock cycles).The project aims to develop a runtime reconfigurable array technology for AI/ML that provides program switch decisions at sub-microsecond scale, supports multiple active programs, multi-size compile, and priority handling. RTRA should be programmable from high-level languages such as C or Python. Online hardware scheduling enables rapid runtime reconfiguration. The hardware agility is enabled by online dynamic multi-program hardware scheduling, domain-specific reconfigurable array and area-efficient interconnect for multi-program tenancy and flexible data interfaces. The hardware includes embedded processor for control, system memory, and data interfaces for heterogeneous system integration. The energy efficient processing (10x better than FPGA, with another ~10x higher resource utilization) is a significant improvement over existing FPGA AI accelerators, while adding spatiotemporal dynamics for advanced AI/ML models. An internally developed software toolchain (from C or Python to soft binary) will be made available to compile and test various AI/ML use cases. The project team brings unique capability to develop, test, and utilize an integrated system. Broadly, the technology enables rapid deployment of newly developed algorithms, accelerating the deployment of innovative applications. Further, the ability to quickly repurpose hardware provides an opportunity to utilize “dark silicon” and potentially displace today’s fixed-function hardware accelerators with energy-efficient runtime reconfigurable fabric. RTRA technology is envisioned to meet the needs of future communications and AI/ML workloads, with readily accessible C or Python programming. The project leverages the complementary academic talent in the U.S. and Taiwan to impact semiconductor engineering and education.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.