Transferring learning from external to internal weights in echo-state networks with sparse connectivity.
Transferring learning from external to internal weights in echo-state networks with sparse connectivity.
复制标题
DOI：
10.1371/journal.pone.0037372
复制
发表时间：
2012
期刊：
影响因子：
3.7
通讯作者：
Abbott LF
中科院分区：
文献类型：
--
作者：
Sussillo D;Abbott LF
修改递归网络中的权重以提高任务性能已被证明是困难的。其中修改被限制为网络输出上的连接的权重的回声状态网络提供了更容易的替代方案，但是以通过将来自输出的反馈包括回网络来修改网络的典型稀疏架构为代价。我们推导出使用训练回声状态网络的输出权重值来设置网络内的经常性权重的方法。这种“学习转移”的结果是一个循环网络，它在不需要原始网络中存在的输出反馈的情况下执行任务。我们还讨论了一个混合版本，其中在线学习应用于输出和经常性的权重。这两种方法都提供了训练递归网络执行复杂任务的有效方法。通过对学习迁移工作所需条件的分析，我们定义了“自感知”网络状态的概念，并将其与压缩感知进行了比较和对比。
Modifying weights within a recurrent network to improve performance on a task has proven to be difficult. Echo-state networks in which modification is restricted to the weights of connections onto network outputs provide an easier alternative, but at the expense of modifying the typically sparse architecture of the network by including feedback from the output back into the network. We derive methods for using the values of the output weights from a trained echo-state network to set recurrent weights within the network. The result of this “transfer of learning” is a recurrent network that performs the task without requiring the output feedback present in the original network. We also discuss a hybrid version in which online learning is applied to both output and recurrent weights. Both approaches provide efficient ways of training recurrent networks to perform complex tasks. Through an analysis of the conditions required to make transfer of learning work, we define the concept of a “self-sensing” network state, and we compare and contrast this with compressed sensing.
登录
查看更多内容
影响因子：
16.2
作者：
Sussillo, David;Abbott, L. F.
通讯作者：
Abbott, L. F.
影响因子：
2.5
作者：
Candès, EJ;Romberg, J;Tao, T
通讯作者：
Tao, T
影响因子：
2.9
作者：
Maass, W;Natschläger, T;Markram, H
通讯作者：
Markram, H
影响因子：
8.6
作者：
SOMPOLINSKY, H;CRISANTI, A;SOMMERS, HJ
通讯作者：
SOMMERS, HJ
影响因子：
--
作者：
BENGIO, Y;SIMARD, P;FRASCONI, P
通讯作者：
FRASCONI, P