CAREER: Next-generation Optical I/O with Embedded Equalization for Disaggregated AI Computing

人工智能（AI）和机器学习（ML）模型和数据集的规模和复杂性不断增加，最近达到了1万亿个参数，这就产生了对数千个处理和存储单元的大规模并行化的迫切需求。这需要低延迟和超低功耗的多tb /s机架间/机架内光I/O以及芯片到芯片的互连。虽然最先进的芯片间互连是通过铜线实现的，但它们无法满足新兴人工智能超级计算机对速度、能效和带宽密度的严格要求。硅光子收发器通过最终将光收发器与高性能CPU/ gpu共封装，显示出解决这一挑战的巨大希望。用于下一代人工智能计算的协同封装光学器件应在低于pj /b的能效和低延迟下提供数十Tb/s的聚合数据速率。尽管最近在开发这种收发器方面做出了努力，但所提出的解决方案还不能满足未来芯片间链路的能量和延迟要求。在这项工作中，我们的目标是通过提出一种新的均衡范式和一种新的系统级架构来解决超高数据速率的这些挑战，使共封装光学器件“更智能”，而不仅仅是一个以前从未想象过的电光桥。该项目的成果可以通过将真正分解的计算架构降低到包级别来加速AI/ML计算硬件。我们还计划通过开发新课程，将我们的研究方法和现代主题融入教育课程。本课程将弥补集成电路和光子学领域之间的差距，并将提供公开的材料供其他研究所传播。我们的其他教育计划包括针对大学和K-12学生的拓展活动，以及在这个项目中吸引未被充分代表的研究生和本科生。在高数据速率下构建高能效光I/O的最关键挑战是传统上在电气领域实现的耗电均衡电路。这些均衡器可以消耗总链路能量和面积的50%以上。我们将通过彻底改变均衡技术来克服这个问题，首次将必要的均衡功能嵌入可重构光子器件中，而不是电子端。这种方法将部署在发送端和接收端，可以实现sub-pJ/b的整体链路效率。此外，当共封装光I/O在互连和交换网络中变得迫在眉睫时，我们将通过在本项目中添加更多系统级功能来展示这些芯片可以比仅仅作为电光桥更“智能”。这些新功能可以在共封装光I/O芯片中的协处理器单元上实现。这样，我们不仅可以克服使用光互连进行芯片到芯片通信的延迟开销，而且还可以利用可通过共封装光学器件解锁的新颖系统级架构。新的分解AI架构可以通过减少通信瓶颈和为多gpu系统提供直接内存访问来显着加快大规模AI/ML应用程序的训练/干扰的计算时间。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Ever-increasing size and complexity of artificial intelligence (AI) and machine learning (ML) models and datasets, recently reaching over a trillion parameters, have created a vital need for vast parallelization over thousands of processing and memory units. This requires low-latency and ultra-low power multi-Tb/s inter/intra-rack optical I/O as well as chip-to-chip interconnects. Although state-of-the-art inter-chip interconnects are realized with copper-based wirelines, they cannot meet the stringent speed, energy-efficiency, and bandwidth density requirements of emerging AI supercomputers. Silicon photonic transceivers have shown a great promise to address this challenge by ultimately co-packaging optical transceivers with high performance CPU/GPUs. Co-packaged optics for the next-generation AI computing should provide tens of Tb/s aggregate data-rates at sub-pJ/b energy-efficiency and low-latency. Despite recent efforts in developing such transceivers, proposed solutions do not yet satisfy the energy and latency requirements of future inter-chip links. In this work, we aim at solving these challenges for ultra-high data-rates by proposing a new equalization paradigm and a novel system-level architecture to make co-packaged optics “smarter” than being just an electro-optical bridge as has never been imagined before. Outcomes of this project can speed up AI/ML computing hardware by enabling a truly disaggregated computing architecture down to the package level. We also plan to integrate our research methodology and modern topics into the educational curriculum by developing a new course. This course will bridge the gap between the fields of integrated circuits and photonics and will have publicly available materials for dissemination at other institutes. Our other educational plans include outreach activities for college and K-12 students and engaging underrepresented graduate and undergraduate students in this project.The most critical challenge in building energy-efficient optical I/O at high data-rates is the power-hungry equalization circuitry conventionally implemented in the electrical domain. These equalizers can consume over 50% of the total link energy and area. We will overcome this issue by radically transforming the equalization techniques, embedding the necessary equalization functionalities in reconfigurable photonic devices rather than the electronic side for the first time. This approach will be deployed for both the transmitter and receiver sides and can achieve sub-pJ/b overall link efficiency. Moreover, while co-packaged optical I/O is becoming imminent for interconnect and switching networks, we will show that these chips can be “smarter” than being just an electro-optical bridge by adding more system-level capabilities to them in this project. These new capabilities can be implemented on a co-processor unit in co-packaged optical I/O chips. In doing so, not only we can overcome the latency overhead of using optical interconnects for chip-to-chip communications, but also, we will exploit novel system-level architectures that can be unlocked by co-packaged optics. The new disaggregated AI architecture can significantly speed up the compute time for training/interference of large-scale AI/ML applications by reducing communication bottlenecks and providing direct memory access for multi-GPU systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.