Resonant Machine Learning Based on Complex Growth Transform Dynamical Systems
Resonant Machine Learning Based on Complex Growth Transform Dynamical Systems
复制标题
DOI：
10.1109/tnnls.2020.2984267
复制
发表时间：
2019-08
影响因子：
10.4
通讯作者：
Oindrila Chatterjee;S. Chakrabartty
中科院分区：
文献类型：
--
作者：
Oindrila Chatterjee;S. Chakrabartty
传统的基于能量的学习模型将单个能量度量与底层优化过程中涉及的每个变量配置相关联。这种模型将最低能量状态与所考虑的变量的最佳配置联系起来，因此本质上是耗散的。在本文中，我们提出了一种节能学习框架，该框架利用机器学习网络和满足特勒根定理的通用电气网络之间的结构和功能相似性。与标准的基于能量的模型相比，所提出的公式将两个能量成分（即有功能量和无功能量）与网络相关联。该公式确保网络的有功功率仅在学习过程中消耗，而无功功率始终保持为零。因此，在稳定状态下，学习到的参数被存储并通过由网络节点电感和电容确定的电谐振来自我维持。基于这种方法，本文介绍了三个新颖的概念：1）学习框架，其中网络的有功功率耗散被用作学习目标函数的正则化，该学习目标函数受到零总无功功率约束； 2）基于复杂域、连续时间增长变换的动态系统，优化学习目标函数并驱动网络在稳态运行下实现电谐振； 3) 退火过程，控制有功功率耗散和收敛速度之间的权衡。作为一个代表性的例子，我们展示了如何使用所提出的框架来设计共振支持向量机（SVM），其中支持向量对应于具有自持振荡的 $LC$ 网络。我们还表明，与非谐振网络相比，该谐振网络消耗的有功功率更少。
Traditional energy-based learning models associate a single energy metric to each configuration of variables involved in the underlying optimization process. Such models associate the lowest energy state with the optimal configuration of variables under consideration and are thus inherently dissipative. In this article, we propose an energy-efficient learning framework that exploits structural and functional similarities between a machine-learning network and a general electrical network satisfying Tellegen’s theorem. In contrast to the standard energy-based models, the proposed formulation associates two energy components, namely, active and reactive energy with the network. The formulation ensures that the network’s active power is dissipated only during the process of learning, whereas the reactive power is maintained to be zero at all times. As a result, in steady state, the learned parameters are stored and self-sustained by electrical resonance determined by the network’s nodal inductances and capacitances. Based on this approach, this article introduces three novel concepts: 1) a learning framework where the network’s active-power dissipation is used as a regularization for a learning objective function that is subjected to zero total reactive-power constraint; 2) a dynamical system based on complex-domain, continuous-time growth transforms that optimizes the learning objective function and drives the network toward electrical resonance under steady-state operation; and 3) an annealing procedure that controls the tradeoff between active-power dissipation and the speed of convergence. As a representative example, we show how the proposed framework can be used for designing resonant support vector machines (SVMs), where the support vectors correspond to an $LC$ network with self-sustained oscillations. We also show that this resonant network dissipates less active power compared with its non-resonant counterpart.