Greedy information acquisition algorithm: A new information theoretic approach to dynamic information acquisition in neural networks
Greedy information acquisition algorithm: A new information theoretic approach to dynamic information acquisition in neural networks
复制标题
贪心信息获取算法：神经网络中动态信息获取的新信息论方法
DOI：
10.1080/09540090210162065
复制
发表时间：
2002
影响因子：
5.3
通讯作者：
H. Takeuchi
中科院分区：
文献类型：
--
作者：
R. Kamimura;T. Kamimura;H. Takeuchi
本文提出了一种新的竞争性学习信息论方法。这种新方法被称为贪婪信息获取，因为网络试图在每个学习阶段吸收尽可能多的信息。在第一阶段，以最小的网络结构实现竞争，信息最大化。在第二阶段，添加新的单元，从而再次尽可能地增加信息。这个过程一直持续到信息不再可能增加为止。通过贪婪信息最大化，可以在连续的阶段中累积地发现输入模式中的不同重要特征集。我们将我们的方法应用于三个问题：偶极子问题;语言分类问题;和语音特征检测问题。实验结果表明，信息最大化可以重复应用，并逐渐发现输入模式中的不同特征。我们还比较了我们的方法与传统的竞争学习和多变量分析。实验结果表明，该方法能更好地检测出输入模式中的显著特征。
In this paper, we proopose a new information theoretic approach to competitive learning. The new approach is called greedy information acquisition , because networks try to absorb as much information as possible in every stage of learning. In the first phase, with minimum network architecture for realizing competition, information is maximized. In the second phase, a new unit is added, and thereby information is again increased as much as possible. This proceess continues until no more increase in information is possible. Through greedy information maximization, different sets of important features in input patterns can be cumulatively discovered in successive stages. We applied our approach to three problems: a dipole problem; a language classification problem; and a phonological feature detection problem. Experimental results confirmed that information maximization can be repeatedly applied and that different features in input patterns are gradually discovered. We also compared our method with conventional competitive learning and multivariate analysis. The experimental results confirmed that our new method can detect salient features in input patterns more clearly than the other methods.