Inevitable energy costs of storage capacity enhancement in an oscillatory neural network
Inevitable energy costs of storage capacity enhancement in an oscillatory neural network
复制标题
振荡神经网络中存储容量增强不可避免的能源成本
DOI：
10.1109/mwscas.2003.1562456
复制
发表时间：
2003
期刊：
影响因子：
--
通讯作者：
Vishwanathan Mohan
中科院分区：
文献类型：
--
作者：
M. Udayshankar;V. S. Chakravarthy;Vishwanathan Mohan
计算是否有不可避免的能源成本？Landaiuer（1961）提出了这个重要的问题，认为不可逆的计算过程具有不可避免的“热力学成本”，这表明计算设备消耗的能量与其“信息工作”之间存在深刻的联系。“我们以前关于神经网络模型中这种联系的可能性的研究（Amit，1989）表明，在Hopfield神经网络（HNN）中，能量耗散和性能之间存在一致的相关性。在本文中，我们在一个复杂的Hopfield神经网络（CHNN）（Chakravarthy和Ghosh，1996年; Fredkin和Tofoli，1980年）中证明了类似的结果，这是一种联想记忆，其中模式被存储为振荡。然而，当只存储单个模式时，观察到完美检索。当存储多个模式时，网络经常从一个存储的模式漫游到另一个模式，而不会停留在任何单个模式上，导致不可接受的低存储容量。我们发现，使用即使在检索过程中也能适应的权重可以大大提高存储容量。然而，这种增强的能力具有能量成本。比较电路实现的网络与固定和自适应权重，我们发现，后一种情况涉及更大的功耗。同样的结果在P的范围内得到了证实，P是网络中存储的模式数
Is there an inevitable energy cost to computation? Raising this important question, Landaiuer (1961) argued that irreversible computational processes have an inevitable "thermodynamic cost", suggesting deep link between the amount of energy spent by a computing device and its "informational work." Our previous studies (Amit, 1989) on the possibility of such a link in neural network models showed a consistent correlation between energy dissipated and performance In a Hopfield neural network (HNN). In the present paper, we demonstrate a similar result in a complex Hopfield neural network (CHNN) (Chakravarthy and Ghosh, 1996; Fredkin and Tofoli, 1980), an associative memory in which patterns are stored as oscillations. However, perfect retrieval is observed when only a single pattern is stored. When multiple patterns are stored, the network often wanders from one stored pattern to another without settling on any single pattern, resulting in unacceptably low storage capacity. We found that using weights that adapt even during retrieval dramatically enhances storage capacity. However, this enhanced capacity has an energetic cost. Comparing circuit implementations of the network with fixed and adaptive weights, we found that the latter case involves greater power dissipation. The same result is confirmed over a range of P, the number of patterns stored in the network