Deep physical neural networks trained with backpropagation.
Deep physical neural networks trained with backpropagation.
复制标题
DOI：
10.1038/s41586-021-04223-6
复制
发表时间：
2022-01
期刊：
影响因子：
64.8
通讯作者：
McMahon PL
中科院分区：
文献类型：
--
作者：
Wright LG;Onodera T;Stein MM;Wang T;Schachter DT;Hu Z;McMahon PL
深度学习模型已经成为科学和工程中的普遍工具。然而，他们的能源需求现在越来越限制了他们的可扩展性。深度学习加速器旨在高能效地执行深度学习，通常针对推理阶段，通常通过利用传统电子设备以外的物理衬底来实现。到目前为止，方法还不能应用反向传播算法来现场训练非传统的新型硬件。反向传播的优势使其成为大规模神经网络事实上的有效训练方法，因此这一不足构成了一大障碍。在这里，我们介绍了一种混合的原位硅胶算法，称为物理感知训练，它应用反向传播来训练可控制的物理系统。就像深度学习用由数学函数层组成的深层神经网络实现计算一样，我们的方法允许我们训练由可控物理系统层组成的深层物理神经网络，即使物理层与传统的人工神经网络层缺乏任何数学同构。为了证明我们的方法的普适性，我们训练了基于光学、力学和电子学的各种物理神经网络来实验地执行音频和图像分类任务。物理感知训练将反向传播的可扩展性与自动缓解缺陷和噪声相结合，这些缺陷和噪声可以通过在线学习算法实现。物理神经网络具有比传统电子处理器更快、更节能地执行机器学习的潜力，更广泛地说，可以赋予物理系统自动设计的物理功能，例如机器人、材料和智能传感器。一种应用反向传播的混合算法被用来训练一层层可控的物理系统，以执行像深度神经网络一样的计算，但考虑到现实世界的噪音和缺陷。
Deep-learning models have become pervasive tools in science and engineering. However, their energy requirements now increasingly limit their scalability. Deep-learning accelerators aim to perform deep learning energy-efficiently, usually targeting the inference phase and often by exploiting physical substrates beyond conventional electronics. Approaches so far have been unable to apply the backpropagation algorithm to train unconventional novel hardware in situ. The advantages of backpropagation have made it the de facto training method for large-scale neural networks, so this deficiency constitutes a major impediment. Here we introduce a hybrid in situ–in silico algorithm, called physics-aware training, that applies backpropagation to train controllable physical systems. Just as deep learning realizes computations with deep neural networks made from layers of mathematical functions, our approach allows us to train deep physical neural networks made from layers of controllable physical systems, even when the physical layers lack any mathematical isomorphism to conventional artificial neural network layers. To demonstrate the universality of our approach, we train diverse physical neural networks based on optics, mechanics and electronics to experimentally perform audio and image classification tasks. Physics-aware training combines the scalability of backpropagation with the automatic mitigation of imperfections and noise achievable with in situ algorithms. Physical neural networks have the potential to perform machine learning faster and more energy-efficiently than conventional electronic processors and, more broadly, can endow physical systems with automatically designed physical functionalities, for example, for robotics, materials and smart sensors. A hybrid algorithm that applies backpropagation is used to train layers of controllable physical systems to carry out calculations like deep neural networks, but accounting for real-world noise and imperfections.
登录
查看更多内容
影响因子：
10.4
作者：
Bueno, J.;Maktoobi, S.;Brunner, D.
通讯作者：
Brunner, D.
影响因子：
--
作者：
FRYE, RC;RIETMAN, EA;WONG, CC
通讯作者：
WONG, CC
影响因子：
16.6
作者：
Hermans, Michiel;Burm, Michael;Van Vaerenbergh, Thomas;Dambre, Joni;Bienstman, Peter
通讯作者：
Bienstman, Peter
影响因子：
3.1
作者：
Kariyappa, Sanjay;Tsai, Hsinyu;Burr, Geoffrey W.
通讯作者：
Burr, Geoffrey W.
DOI：
10.1109/tnnls.2012.2204770
发表时间：
2012-09-01
影响因子：
10.4
作者：
Adhikari, Shyam Prasad;Yang, Changju;Chua, Leon O.
通讯作者：
Chua, Leon O.