CAREER: Reliable and Accelerated Deep Neural Networks via Co-Design of Hardware and Algorithms

人工智能（AI）系统是广泛应用的组成部分，包括安全关键和生命关键领域。然而，在人工智能系统部署过程中观察到硬件故障和设计缺陷，导致系统故障和潜在后果，如财务损失，生产力下降，甚至人命损失。此外，这些问题直接影响硬件安全性和系统可持续性。旨在解决这些问题的现有解决方案遭受以下限制中的一个或两个：（1）在执行时间、功耗、硬件占用空间和/或数据存储资源方面的高成本;以及（2）有限的覆盖范围，因为现有解决方案仅能够解决这些问题的子集。该项目将通过一套新的硬件算法协同设计方法来克服这些限制，同时最大限度地降低成本并提高覆盖率，通过计算机体系结构，鲁棒系统设计和机器学习知识的跨学科组合来推进最先进的技术。 该项目的成功完成有望标志着人工智能系统的重大飞跃，使其更加高效，可靠，值得信赖和可持续。此外，该项目将通过创造性的可视化手段和讲习班，特别是针对资源不足的高中学生，加强计算机体系结构教育。该项目还强调促进多样性和便利技术转让，包括三个相互关联的重点。第一个重点是创建端到端的方法，从根本上了解硬件故障和错误对高级深度学习工作负载的影响，通过硬件算法协同设计减轻这些挑战，并结合用户研究来探索针对个人用户需求的自适应架构解决方案。第二个目标是为细粒度混合精度深度神经网络设计AI硬件。这涉及到创建一个协同设计框架，以促进硬件和软件的协同进化，优化这些网络的加速器，同时通过反馈回路为这些加速器定制网络模型，解决设计缺陷的敏感性。最后一个推力探索了一种创新的方法来生成网络参数，而不是存储它们。参数生成算法将被集成到训练算法中，以优化网络精度，最大限度地减少面积、功耗和存储成本，同时解决系统存储器带来的可靠性和安全威胁。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Artificial intelligence (AI) systems are integral to a broad spectrum of applications, encompassing safety-critical and life-critical domains. However, hardware failures and design bugs have been observed during AI system deployment, leading to system malfunctions and potential consequences such as financial losses, reduced productivity, and even loss of human life. Furthermore, these issues directly impact hardware security and system sustainability. Existing solutions aimed at addressing these issues suffer from one or both of the following limitations: (1) high costs in terms of execution time, power consumption, hardware footprint, and/or data storage resources; and (2) limited coverage, as existing solutions are only capable of addressing a subset of these problems. This project will overcome these limitations through a fresh set of novel hardware-algorithm co-design approaches to simultaneously minimize costs and enhance coverage, advancing the state-of-the-art through an interdisciplinary combination of knowledge in computer architecture, robust system design, and machine learning. Successful completion of this project promises to mark a significant leap forward for AI systems, enabling them to be more efficient, reliable, trustworthy, and sustainable. Additionally, the project will enhance computer architecture education through creative visualization means and workshops especially targeting students in under-resourced high schools. The project also also places emphasis on promoting diversity and facilitating technology transfer.This project encompasses three interconnected thrusts. The first thrust focuses on creating end-to-end approaches to fundamentally understand the impact of hardware failures and bugs on advanced deep learning workloads, mitigate these challenges through hardware-algorithm co-design, and incorporate a user study to explore adaptive architectural solutions tailored to individual users' needs. The second thrust targets the design of AI hardware for fine-grained mixed-precision deep neural networks. This involves creating a co-design framework to facilitate the co-evolution of hardware and software, optimizing accelerators for these networks, and simultaneously tailoring network models to these accelerators through a feedback loop, addressing susceptibility to design bugs. The last thrust explores an innovative approach for generating network parameters instead of storing them. The parameter generation algorithm will be integrated into training algorithms to optimize network accuracy and minimize area, power, and storage costs, while also addressing reliability and security threats posed by system memories.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.