CAREER: Reliable and Accelerated Deep Neural Networks via Co-Design of Hardware and Algorithms

人工智能（AI）系统是广泛应用中不可或缺的一部分，包括安全关键领域和生命关键领域。然而，在人工智能系统部署过程中，已经观察到硬件故障和设计错误，导致系统故障和潜在的后果，如经济损失，生产力降低，甚至人员伤亡。此外，这些问题直接影响硬件安全性和系统可持续性。旨在解决这些问题的现有解决方案存在以下一个或两个限制：(1)在执行时间、功耗、硬件占用和/或数据存储资源方面的高成本；(2)覆盖范围有限，因为现有的解决方案只能解决这些问题的一个子集。该项目将通过一套全新的硬件算法协同设计方法来克服这些限制，同时最大限度地降低成本并提高覆盖范围，通过计算机体系结构、稳健系统设计和机器学习方面的跨学科知识组合来推进最先进的技术。该项目的成功完成将标志着人工智能系统的重大飞跃，使它们更加高效、可靠、值得信赖和可持续。此外，该项目将通过创造性的可视化手段和讲习班，特别是针对资源不足的高中学生，加强计算机体系结构教育。该项目还强调促进多样性和促进技术转让。该项目包含三个相互关联的重点。第一个重点是创建端到端方法，从根本上理解硬件故障和错误对高级深度学习工作负载的影响，通过硬件算法协同设计减轻这些挑战，并结合用户研究来探索针对个人用户需求量身定制的自适应架构解决方案。第二个重点是针对细粒度混合精度深度神经网络的人工智能硬件设计。这包括创建一个协同设计框架，以促进硬件和软件的协同进化，优化这些网络的加速器，同时通过反馈循环为这些加速器定制网络模型，解决对设计错误的敏感性。最后一篇文章探讨了一种生成网络参数而不是存储它们的创新方法。参数生成算法将集成到训练算法中，以优化网络精度，最小化面积、功率和存储成本，同时解决系统存储器带来的可靠性和安全威胁。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Artificial intelligence (AI) systems are integral to a broad spectrum of applications, encompassing safety-critical and life-critical domains. However, hardware failures and design bugs have been observed during AI system deployment, leading to system malfunctions and potential consequences such as financial losses, reduced productivity, and even loss of human life. Furthermore, these issues directly impact hardware security and system sustainability. Existing solutions aimed at addressing these issues suffer from one or both of the following limitations: (1) high costs in terms of execution time, power consumption, hardware footprint, and/or data storage resources; and (2) limited coverage, as existing solutions are only capable of addressing a subset of these problems. This project will overcome these limitations through a fresh set of novel hardware-algorithm co-design approaches to simultaneously minimize costs and enhance coverage, advancing the state-of-the-art through an interdisciplinary combination of knowledge in computer architecture, robust system design, and machine learning. Successful completion of this project promises to mark a significant leap forward for AI systems, enabling them to be more efficient, reliable, trustworthy, and sustainable. Additionally, the project will enhance computer architecture education through creative visualization means and workshops especially targeting students in under-resourced high schools. The project also also places emphasis on promoting diversity and facilitating technology transfer.This project encompasses three interconnected thrusts. The first thrust focuses on creating end-to-end approaches to fundamentally understand the impact of hardware failures and bugs on advanced deep learning workloads, mitigate these challenges through hardware-algorithm co-design, and incorporate a user study to explore adaptive architectural solutions tailored to individual users' needs. The second thrust targets the design of AI hardware for fine-grained mixed-precision deep neural networks. This involves creating a co-design framework to facilitate the co-evolution of hardware and software, optimizing accelerators for these networks, and simultaneously tailoring network models to these accelerators through a feedback loop, addressing susceptibility to design bugs. The last thrust explores an innovative approach for generating network parameters instead of storing them. The parameter generation algorithm will be integrated into training algorithms to optimize network accuracy and minimize area, power, and storage costs, while also addressing reliability and security threats posed by system memories.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.