Readouts for echo-state networks built using locally regularized orthogonal forward regression
Readouts for echo-state networks built using locally regularized orthogonal forward regression
复制标题
DOI：
10.1080/02664763.2017.1305331
复制
发表时间：
2011-10
影响因子：
1.5
通讯作者：
J. Dolinský;K. Hirose;S. Konishi
中科院分区：
文献类型：
--
作者：
J. Dolinský;K. Hirose;S. Konishi
摘要 回声状态网络（ESN）被视为一种时间扩展，它自然会产生与教师输出各种相关的回归量。我们说明，通常只有一定数量的生成的回声回归量可以有效地解释教师输出，并且我们建议通过使用局部正则化正交前向回归（LROFR）联合计算个体方差贡献和贝叶斯相关性来确定回声回归量的重要性。该信息可以有利地以多种方式用于ESN结构的分析。我们提出了使用 LROFR 构建的局部正则化线性读数。读出的维度可以比ESN模型本身更小，并且提高了ESN的鲁棒性和准确性。其主要优点是能够确定哪种类型的附加读数适合手头的任务。还提供了与 PCA 的比较。我们还提出了使用 LROFR 构建的径向基函数 (RBF) 读数，因为线性读数的灵活性有局限性，可能不足以完成复杂的任务。其出色的泛化能力使其成为前馈神经网络或相关向量机的可行替代方案。对于需要更多时间容量的情况，我们建议经过充分研究的延迟读出。
ABSTRACT Echo state network (ESN) is viewed as a temporal expansion which naturally give rise to regressors of various relevance to a teacher output. We illustrate that often only a certain amount of the generated echo-regressors effectively explain the teacher output and we propose to determine the importance of the echo-regressors by a joint calculation of the individual variance contributions and Bayesian relevance using the locally regularized orthogonal forward regression (LROFR). This information can be advantageously used in a variety of ways for an analysis of an ESN structure. We present a locally regularized linear readout built using LROFR. The readout may have a smaller dimensionality than the ESN model itself, and improves robustness and accuracy of an ESN. Its main advantage is ability to determine what type of an additional readout is suitable for a task at hand. Comparison with PCA is provided too. We also propose a radial basis function (RBF) readout built using LROFR, since flexibility of the linear readout has limitations and might be insufficient for complex tasks. Its excellent generalization abilities make it a viable alternative to feed-forward neural networks or relevance-vector-machines. For cases where more temporal capacity is required we propose well studied delay readout.