Collaborative Research: Machine Learning-assisted Modeling and Design of Approximate Computing with Generalizability and Interpretability

协作研究：机器学习辅助建模和具有通用性和可解释性的近似计算设计

• 批准号: 2202310
• 负责人: Xun Jiao
• 金额: $ 19.32万
• 依托单位: Villanova University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-08-01 至 2025-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2202310&HistoricalAwards=false
• 关键词: Collaborative; Research; Machine; Learning; assisted

By 2040, the projected energy consumed by computers will exceed the electricity the world can generate, unless radical changes are made in the way we design computers. This project aims to develop approximate computing techniques to drastically reduce the energy consumption in modern computation-intensive computing, for example, video/image processing and machine learning applications. Approximate computing is a promising technique that can trade off accuracy for energy saving and performance improvement. This project addresses one of the fundamental obstacles that has been impeding the practical usage of approximate computing: how to accurately and quickly design approximate computing systems to maximize the benefits of approximate computing without introducing too much accuracy loss or errors. The success of this project can greatly improve the practicality of approximate computing and enable its wide usage in real-world applications, such as video/image processing and machine learning. By paving the way for future approximate computing, this project can lead to considerable energy saving for future computing paradigm and carbon footprint reduction. It also advances the applications of machine learning algorithms in circuit design area, as well as identifies several fundamental machine learning questions motived by special features of the circuit design problem, such that machine learning algorithms can better benefit hardware design. This can lead to new design and testing approaches for a broad range of computing systems, from low-power embedded systems to high-performance data centers. The educational plan will integrate research activities into curriculum development and will provide students with early research training. The team is committed to broadening the participation of undergraduates and underrepresented groups in engineering research and in STEM outreach activities.Given the huge amount of energy consumed by modern computation-intensive computing such as machine learning and video/image processing applications, energy efficient computing is an urgent need. Approximate computing, by slightly trading off accuracy for better performance and/or efficiency, e.g., computation latency, area, and energy and power, has been a promising new computing paradigm. Many approximate computing approaches, such as low-precision computing, voltage scaling, inexact approximate circuits and memory, have achieved orders of speedup or energy saving. However, to safely deploy approximate computing in practice, two major challenges need to be addressed: (1) how to accurately and quickly estimate the impact of approximation on application output quality; and (2) how to accurately and quickly find the best approximation configuration to maximize the benefits of approximate computing. This proposal presents three closely-interacted research tasks to address these two challenges and to seek the wide-reaching benefits of approximate computing: (1) to develop input-aware error models of approximate circuits and input-aware simulation platform for approximate computing; (2) to develop a graph neural networks (GNNs)-based framework to quickly estimate application output quality; (3) to develop a resource-aware approximation configuration framework to optimize performance/energy while satisfying user-defined quality constraints. The goal of this project is to unveil the underlining knowledge of the intrinsic relations between output quality and input data, approximate circuits, and approximate program structures. The project will provide a practical, generalizable, and interpretable toolset that can learn to configure approximate computing once and for all.The intellectual merits of this project include both AxC design innovations as well as machine learning innovations. (1) This project will develop input-aware error models for approximate circuits considering the impact of input data, which are usually overlooked; then, it exposes the circuit-level errors to behavioral-level approximate programs by developing an input-aware simulation platform. This will form the foundation for a much-needed holistic evaluation of approximate computing. (2) This project will develop inductive GNN models and machine learning models to predict the output quality of any unseen approximate programs and approximation configurations. This will provide key generalizability and interpretability of approximate computing. In addition, the investigation of GNN in this project will uncover two new fundamental studies to GNN community, increasing GNN expressiveness power by amending graph connectivity, and utilizing graph regularity. (3) This project will design a resource-aware reinforcement learning (RL) based approach to automatically configure approximation settings for optimal performance/energy-quality tradeoff. In addition, the investigation of GNN and RL in this project will uncover a new research question, the joint optimization of the RL agent and the surrogate model. This project is a pioneering approach to the joint areas of reinforcement learning, graph neural networks, and approximate computing. It aims to establish the technological foundation for practical approximate computing.This project will bring an unprecedented transformation in our ability to understanding and designing approximate computing for practical use, by enabling a more disciplined, generalizable, and interpretable approximation. The research team will release models, tools, and infrastructures to the research and industry community. This can lead to new design and testing approaches for a broad range of computing systems, from low-power embedded systems to high-performance data centers. The educational plan will integrate research activities into curriculum development and will provide students with early exposure to research. The PIs are committed to broadening the participation of undergraduates and underrepresented groups in engineering research and in STEM outreach activities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

到2040年，除非我们在设计计算机的方式上做出根本性的改变，否则预计计算机消耗的能源将超过世界能产生的电力。该项目旨在开发近似计算技术，以大幅降低现代计算密集型计算中的能耗，例如视频/图像处理和机器学习应用。近似计算是一种很有前途的技术，它可以在精度与节能和性能改进之间进行权衡。该项目解决了阻碍近似计算实际应用的基本障碍之一：如何准确快速地设计近似计算系统，以最大限度地发挥近似计算的好处，而不会引入太多的精度损失或错误。该项目的成功可以大大提高近似计算的实用性，并使其在视频/图像处理和机器学习等现实应用中得到广泛应用。通过为未来的近似计算铺平道路，该项目可以为未来的计算范式节省大量能源，并减少碳足迹。提出了机器学习算法在电路设计领域的应用，并确定了几个由电路设计问题的特殊特征驱动的基本机器学习问题，使机器学习算法更好地有利于硬件设计。这可以为广泛的计算系统（从低功耗嵌入式系统到高性能数据中心）带来新的设计和测试方法。该教育计划将把研究活动纳入课程开发，并将为学生提供早期研究训练。该团队致力于扩大本科生和弱势群体在工程研究和STEM外展活动中的参与。鉴于现代计算密集型计算（如机器学习和视频/图像处理应用）所消耗的大量能源，节能计算是迫切需要的。近似计算，通过稍微牺牲精度来换取更好的性能和/或效率，例如，计算延迟、面积、能量和功率，已经成为一种很有前途的新计算范式。许多近似计算方法，如低精度计算、电压缩放、不精确近似电路和存储器，已经实现了数量级的加速或节能。然而，为了在实践中安全地部署近似计算，需要解决两个主要挑战：(1)如何准确快速地估计近似对应用程序输出质量的影响；(2)如何准确、快速地找到最佳近似配置，使近似计算的效益最大化。本文提出了三个密切互动的研究任务，以解决这两个挑战，并寻求近似计算的广泛影响：(1)建立近似电路的输入感知误差模型和近似计算的输入感知仿真平台；(2)开发基于图神经网络（GNNs）的框架，快速估计应用输出质量；(3)开发资源感知近似配置框架，在满足用户自定义质量约束的同时优化性能/能源。该项目的目标是揭示输出质量与输入数据、近似电路和近似程序结构之间内在关系的基础知识。该项目将提供一个实用的、可推广的和可解释的工具集，可以学习一劳永逸地配置近似计算。这个项目的智力优势包括AxC设计创新和机器学习创新。(1)本项目将开发考虑输入数据影响的近似电路的输入感知误差模型，这通常被忽视；然后，通过开发一个输入感知仿真平台，将电路级误差暴露给行为级近似程序。这将为急需的近似计算的整体评价奠定基础。(2)该项目将开发归纳GNN模型和机器学习模型，以预测任何未见过的近似程序和近似配置的输出质量。这将为近似计算提供关键的通用性和可解释性。此外，本项目对GNN的研究将为GNN社区揭示两个新的基础研究，即通过修改图的连通性来提高GNN的表达能力，以及利用图的规律性。(3)本项目将设计一种基于资源感知强化学习（RL）的方法，自动配置近似设置，以实现最佳性能/能源质量权衡。此外，本项目对GNN和RL的研究将揭示一个新的研究问题，即RL agent和代理模型的联合优化。该项目是强化学习、图神经网络和近似计算联合领域的开创性方法。旨在为实用的近似计算奠定技术基础。这个项目将为我们理解和设计实际使用的近似计算的能力带来前所未有的转变，通过实现更有纪律、可推广和可解释的近似。研究团队将向研究和行业社区发布模型、工具和基础设施。这可以为广泛的计算系统（从低功耗嵌入式系统到高性能数据中心）带来新的设计和测试方法。该教育计划将把研究活动纳入课程开发，并为学生提供早期接触研究的机会。这些项目致力于扩大本科生和弱势群体在工程研究和STEM外展活动中的参与。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

PreAxC: Error Distribution Prediction for Approximate Computing Quality Control using Graph Neural Networks

• DOI: 10.1109/isqed57927.2023.10129393
• 发表时间: 2023-04
• 期刊: 2023 24th International Symposium on Quality Electronic Design (ISQED)
• 作者: Lakshmi Sathidevi;Abhinav Sharma;Nan Wu;Xun Jiao;Cong Hao

Beyond von Neumann Era: Brain-inspired Hyperdimensional Computing to the Rescue

• DOI: 10.1145/3566097.3568354
• 发表时间: 2023-01
• 期刊: 2023 28th Asia and South Pacific Design Automation Conference (ASP-DAC)
• 作者: H. Amrouch;P. Genssler;M. Imani;Mariam Issa;Xun Jiao;Wegdan Mohammad;Gloria Sepanta;Ruixuan Wang

Robust Hyperdimensional Computing against Cyber Attacks and Hardware Errors: A Survey

针对网络攻击和硬件错误的鲁棒超维计算：一项调查

• DOI: 10.1145/3566097.3568355
• 发表时间: 2023
• 期刊: Proceedings of the 28th Asia and South Pacific Design Automation Conference
• 作者: Ma, Dongning;Zhang, Sizhe;Jiao, Xun
• 通讯作者: Jiao, Xun

Adversarial Attack on Hyperdimensional Computing-based NLP Applications

基于超维计算的 NLP 应用的对抗性攻击

• DOI: 10.23919/date56975.2023.10137289
• 发表时间: 2023
• 期刊: Automation & Test in Europe Conference & Exhibition (DATE
• 作者: Zhang, Sizhe;Wang, Zhao;Jiao, Xun
• 通讯作者: Jiao, Xun