FET: Medium: A Hybrid Co-processing Unit (HCU) using Phase-change Photonics in CMOS for Large-scale and Ultra-fast Machine Learning Acceleration

由人工智能(AI)和基于神经网络的机器学习(ML)的最新发展引发的新一轮技术革命正在以多种方式改变着人们的生活。然而，大规模人工神经网络的实现挑战了传统的计算模式和硬件平台，因为它要求以比摩尔定律支持的速度更快的速度获得巨大的计算能力。长期以来，光神经网络(ONN)一直被认为是一种很有前途的替代方案，它利用光相对于电的独特优势提供超快和节能的计算。然而，目前的方法缺乏现实世界人工智能应用程序所需的效率、可伸缩性和可编程性。更重要的是，这种光学处理单元只有在能够与现有的基于CMOS的高性能处理器(如CPU/GPU)无缝和高效地集成的情况下才能实用。如果没有这种可扩展性和集成，ONN的能量、速度和延迟优势将因需要将数据和模型移入/移出光处理器而受到影响。这个项目的目标是开发一个混合协处理器单元(HCU)来解决这些挑战，并展示一个适合大规模AI/ML计算的大规模、完全集成的ONN。这项研究提供的AI云系统具有可持续的能源消耗和计算能力，其社会影响将是巨大的。该计划的教育和推广活动包括光学计算和人工智能硬件设计课程开发，以及K-12科学推广计划和公开在线课程。HCU是通过新兴相变材料(PCM)与先进的硅光子工艺的整体集成实现的。这一策略使得数以千计的光子元件和数百万个晶体管可以以经济实惠和可扩展的方式在单一的cmos工艺中制造在一起。此外，建议的HCU可以与CPU/GPU进行异质共封装，以最大限度地减少数据/模型移动的能量和延迟开销。该项目旨在将光子学和电子学在AI/ML加速方面的互补优势结合起来：光子学用于使用非易失性交叉开关网络进行高速和高能效的线性运算计算，电子学用于非线性高精度激活和控制电路。HCU的能量和面积计算密度将比今天的GPU大一个数量级以上。HCU的发展也是及时的，因为光学互连已经向数据中心和超级计算机更深入地渗透到芯片间连接。除了基于云的AI/ML，拟议的HCU在延迟和功率敏感应用方面具有巨大优势，如自动驾驶车辆、机器人、太空任务和国防行动。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

A new wave of technological revolution — ignited by recent developments in artificial intelligence (AI) and machine learning (ML) based on neural networks — is transforming life in numerous ways. The implementation of large-scale artificial neural networks, however, challenges conventional computing paradigms and hardware platforms by demanding enormous computing power at a much faster pace than Moore’s law supports. Optical neural networks (ONNs) have long been proposed as a promising alternative by providing ultra-fast and energy-efficient computing utilizing the distinct advantages of light as opposed to electricity. However, current approaches lack the efficiency, scalability, and programmability that is needed for real-world AI applications. More importantly, such optical processing units will be only practical if they can be seamlessly and efficiently integrated with existing CMOS-based high-performance processors such as CPU/GPUs. Without such scalability and integration, the energy, speed, and latency benefits of ONNs would be compromised by the need for data and model movement in/out of the optical processor. The goal of this project is to develop a hybrid co-processor unit (HCU) to solve these challenges and demonstrate a large-scale, fully integrated ONN suitable for large-scale AI/ML computations. The societal impact of AI cloud systems with sustainable energy consumption and computing power that are afforded by this research will be tremendous. Education and outreach activities of this program include course development in optical computing and AI hardware design, as well as K-12 science outreach programs with publicly accessible online courses.The HCU is being realized through the monolithic integration of emerging phase-change material (PCM) with an advanced silicon photonic process. This strategy allows thousands of photonic elements and millions of transistors to be fabricated together in a single CMOS process in a cost-effective and scalable manner. Additionally, the proposed HCU can be heterogeneously co-packaged with a CPU/GPU to minimize the energy and latency overhead of data/model movement. This project aims to combine the complementary strengths of photonics and electronics for AI/ML acceleration: photonics for high-speed and energy-efficient computation of linear operations using a non-volatile cross-bar network, and electronics for nonlinear high-precision activations and control circuitry. The energy and areal computing density of the HCU will be over an order of magnitude larger than today’s GPUs. The development of HCUs is also timely as optical interconnect has already been penetrating deeper into datacenters and supercomputers toward inter-chip connections. In addition to cloud-base AI/ML, the proposed HCU has tremendous advantages for latency- and power-sensitive applications such as autonomous vehicles, robotics, space missions, and defense operations.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.