SHF: Small: Fast Sign-Off of Machine Learning Systems: From Circuit-Level Modeling to Statistical System Validation

机器学习已经被广泛的新兴应用所采用，包括健康监测、自动驾驶、先进制造等。然而，由于机器学习算法带来的精度限制以及与其硬件实现相关的电路级非理想特性，任何机器学习系统都不能100%准确。这个项目研究了一种全新的框架，用于有效验证用纳米集成电路实现的机器学习系统。它的目标是识别和综合机器学习系统可能失败的关键转折案例。预计该项目将启动当今复杂机器学习系统设计方法的范式转变，从而对依赖机器智能的广泛工业部门产生立竿见影的影响。此外，拟议的教育活动为学术界和产业界参与者创造了大量独特的培训机会，大大改善了教育基础设施，为社会培养了高素质的研究人员和从业者。今天，验证一个具有高吞吐量、低功耗和复杂功能的机器学习系统是一项极具挑战性的任务。这个项目通过开发一个新的验证框架来应对这一巨大挑战，该框架由两个主要组件组成：(1)角例生成和(2)罕见失败率估计。为了综合大量的测试用例，提出了物理电路模型和统计生成模型，减少了物理记录难以观察的角例的实验成本。此外，基于子集划分和图嵌入，提出了一种新的公式来有效地检查可能失败的测试用例，从而估计通过随机抽样获取的罕见失败率。在这些数学工具的基础上，该项目的框架提供了一个基本的基础设施，可以促进在众多机器学习应用程序上的根本突破。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning has been adopted by a broad range of emerging applications, including health monitoring, autonomous driving, advanced manufacturing, etc. However, any machine learning system cannot be 100% accurate due to the accuracy limitation posed by machine learning algorithms and the circuit-level non-ideal features associated with its hardware implementation. This project investigates a radically new framework for efficient validation of machine learning systems implemented with nano-scale integrated circuits. It aims to identify and synthesize the critical corner cases for which a machine learning system is likely to fail. The project is expected to initialize a paradigm shift in today's design methodology for complex machine learning systems, thereby leading to an immediate impact on a broad range of industrial sectors relying on machine intelligence. In addition, the proposed education activities create a large number of unique training opportunities for both academic and industrial participants, substantially improving the education infrastructure and generate high-quality researchers and practitioners for the society. Today, validating a machine learning system with high throughout, low power and complex functionality is an extremely challenging task. This project attacks the grand challenge by developing a novel validation framework composed of two major components: (1) corner-case generation and (2) rare-failure rate estimation. Both physical circuit models and statistical generative models are proposed to synthesize a large amount of test cases, reducing the experimental cost to physically record the corner-cases that are difficult to observe. Furthermore, a novel formulation, based on subset partition and graph embedding, is developed to efficiently inspect the likely-failed test cases and consequently estimate the rare- failure rate that is expensive to capture by random sampling. Built upon these mathematical tools, the project's framework offers a fundamental infrastructure that could facilitate radical breakthroughs over numerous machine learning applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.