CAREER: Physics-inspired Machine Learning with Sparse and Asynchronous p-bits

机器学习和人工智能（AI）领域的最新进展创造了实际应用，从可以产生有意义对话的强大聊天机器人到可以产生引人注目的艺术的人工智能艺术家。然而，在舞台背后，训练这样的人工智能模型需要花费大量的精力、时间和物理资源，这使得它们成本高昂，限制了可访问性，并阻碍了民主化使用。此外，从微电子学的角度来看，这些革命性的进步来得正是时候，因为现代晶体管的尺寸已经达到了原子的尺寸，要提高它们的能量效率和性能已经变得非常困难。这个项目是关于设计一种新型的受物理启发的概率计算机，与传统的全确定性计算机形成对比。该方法是从固有噪声的磁性材料和器件开始构建概率比特（p-bits）。然后可以适当地配置连接的p位网络，以有效地解决概率机器学习中遇到的计算问题，其中包含大量难以在传统计算机中训练的强大算法。因为在这种方法中，底层构建块自然是概率性的，与传统计算机相比，它们可以更有效地用于实现概率学习算法，在传统计算机中，模仿真正的随机性需要付出高昂的面积和能源消耗。这个项目的跨学科性质将需要从设备、架构和算法等计算堆栈的几个不同层进行协同和重新思考，这样新型的节能、物理启发和概率计算机就可以被建造出来，以帮助解决社会上最大的计算挑战。这个CAREER项目的具体方法是为概率机器学习算法设计物理启发的概率计算机（p-computer）。这些p型计算机将超越现有的小型原型，将称为随机磁隧道结的磁性纳米器件与强大的基于cmos的现场可编程门阵列相结合。主要目的是展示用于概率计算的CMOS +随机MTJ架构的第一次大规模演示，其中10,000个数字p位将被100个基于随机磁隧道结的p位增加。通过随机磁隧道结自然提供的真正随机性和异步动态增强，这些异构处理器有望提供数量级的能量和性能改进，超过当前AI系统常用的优化图形和张量处理单元。这些p位计算机应用于量子和经典机器学习算法在物理启发，硬件感知和稀疏网络将导致计算优势和更好的能源效率，促进百万p位计算机的最终集成。这个项目的发现将导致独特的设备模型和算法的发展，跨学科的课程和教程。这些视频将在nanoHUB和YouTube上发布，内容涵盖统计力学、机器学习和量子计算等多种主题。通过与学术界和工业界的支持机构合作，该项目将大力促进未来“技术大师”的劳动力培训，他们在一个领域深入，但足够广泛，可以连接相关领域。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Recent advances in the fields of Machine Learning and Artificial Intelligence (AI) have created practical applications ranging from powerful chatbots that can generate meaningful conversations or AI artists that can generate striking art. Behind the stage, however, there are enormous costs in energy, time and physical resources to train such AI models, making them costly, limiting accessibility and preventing democratized use. Moreover, these revolutionary advances have come at the worst possible time from a microelectronics viewpoint, since it has become significantly hard to improve the energy efficiency and performance of modern transistors whose dimensions have reached atomic dimensions. This project is about designing a new kind of physics-inspired and probabilistic computer, contrasting conventional fully-deterministic computers. The approach is to start from inherently noisy magnetic materials and devices to build probabilistic bits (p-bits). Networks of connected p-bits can then be suitably configured to efficiently solve computational problems encountered in probabilistic machine learning containing a large family of powerful algorithms that are hard to train in conventional computers. Because the underlying building blocks are naturally probabilistic in this approach, they can be used to implement probabilistic learning algorithms far more efficiently compared to conventional computers, where mimicking true randomness comes with high costs in area and energy consumption. The interdisciplinary nature of this project will require the synergy and rethinking of several different layers of the computing stack from devices, architectures and algorithms such that new types of energy-efficient, physics-inspired and probabilistic computers can be built to help with the greatest computing challenges of society.The specific approach of this CAREER project is to design physics-inspired probabilistic computers (p-computer) tailored for probabilistic machine learning algorithms. These p-computers will go beyond existing small-scale prototypes by combining magnetic nanodevices called stochastic Magnetic Tunnel Junctions with powerful CMOS-based field programmable gate arrays. The main aim will be to demonstrate the first large-scale demonstration of a CMOS + stochastic MTJ architecture for probabilistic computing where 10,000 digital p-bits will be augmented by 100 stochastic magnetic tunnel junction-based p-bits. Augmented by the true randomness and the asynchronous dynamics naturally provided by stochastic magnetic tunnel junctions, these heterogeneous processors are expected to provide orders of magnitude energy and performance improvement over optimized Graphical and Tensor Processing Units commonly used by present-day AI systems. The application of these p-computers to quantum and classical machine learning algorithms in physics-inspired, hardware-aware and sparse networks will lead to computational advantage and better energy efficiency, facilitating the eventual integration of million p-bit computers. The findings of this project will lead to the development of unique device models and algorithms, interdisciplinary courses and tutorials. These will be disseminated on nanoHUB and YouTube covering a diverse array of topics, including statistical mechanics, machine learning and quantum computing. Through partnerships with supporting institutions in academia and industry, this project will strongly contribute to the workforce training of the “technology maestros” of the future who are deep in one field but broad enough to connect to related areas.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.