Selective Binding, Generalization/Abstraction, and Internal Model Creation by means of Network Structure Pruning

对于人工神经网络来说，由于经济、简化、泛化能力等原因，剪枝是很重要的，目前已有很多算法。修剪也发生在真实的大脑中，被认为是冗余减少原理的一种形式，可能是许多高阶大脑功能的基础。从这个角度出发，我们考虑了选择绑定、整合、抽象和内部模型归纳等相互关联的课题。特别地，我们已经处理了下面的子项目，使用一个修剪算法称为CSDF，由该首席研究员早些时候设计。(1)类比学习/推理类比在心理学、认识论、教育学、科学史和认知科学等各个领域都有研究。人工智能方法也存在，但这种基于规则的尝试往往是硬力的，并且会受到组合爆炸和其他问题的影响。基于CSDF，我们开发了一种新的方法，称为abcap （abstract…More action-Based Connectionist Analogy Processor）。通过修剪学习的结果是，AB-CAP自动创建一个内部抽象模型，并在具体实体和抽象实体之间引入适当的绑定。例如，内部模型充当新的相关数据集的吸引器，以允许AB-CAP能够处理多个类比。这些特性已经通过一些例子进行了演示。(2)盲源分离（BSS）盲源分离是一种新的信息技术创新，它可以从传感器观测到的混合信号中提取未知信号。本研究开发的方法与现有的基于信息/概率论的方法有本质的不同。它使用一个自编码器神经网络，以传感器信号作为输入向量，最小化与输入-输出恒等映射相关的误差。将CSDF应用于解码器部分。在CSDF剪枝中幸存下来的隐藏非线性单元将作为盲信号提取器。此外，解码器矩阵重构了外部混合矩阵，使解码器部分可以解释为整个外部情况的内部模型。仿真结果表明，该方法具有较强的适应性和鲁棒性。(3)在SOM中的应用近年来，本课题组将SOM的范式从向量空间推广到函数空间。这叫做mnSOM。我们一直在考虑将CSDF修剪纳入与mnSOM相关的竞争学习中。这一部分还有待取得成功的结果。少

For artificial neural networks, pruning is important for reasons of economy, simplification, generalization ability and so on, and a number of algorithms are available today. Pruning occurs in real brains as well, and is thought to be one form of the Principle of Redundancy Reduction that presumably underlies many higher-order brain functions. From this standpoint, we have considered such inter-related subjects as selected binding, integration, abstraction and internal model induction. In particular, we have dealt with the following sub-projects by using a pruning algorithm called CSDF devised earlier by this principal investigator.(1)Analogical Learning/InferenceAnalogy has been studied in various areas including psychology, epistemology, pedagogy, science history, and cognitive science. AI approaches also exist but such rule-based attempts tend to be bruit-force ones and suffer from combinatorial explosion and other problems. Based on CSDF, we have developed a new method called Abstr … More action-Based Connectionist Analogy Processor (AB-CAP). As a result of the learning with the pruning, AB-CAP automatically creates an internal abstraction model as well as induces proper bindings between concrete and abstract entities. For instance, the internal model acts as an attractor of new relevant dataset, to allow AB-CAP to be able to deal with multiple analogies. These features have been demonstrated by a number of examples.(2)Blind Source Separation(BSS)BSS is a new IT innovation by which to extract otherwise unknown signals from their mixtures observed by sensors. The method developed in this study is fundamentally different from existing ones that are all based on information/probability theories. It uses an auto-encoder neural net which minimizes the error associated with the input-output identity mapping with the sensor signals as the input vector. CSDF is applied to the decoder part. The hidden nonlinear units that have survived the CSDF pruning will be the blind signal extractors. Furthermore, the decoder matrix reconstructs the external mixing matrix, so that the decoder part can be interpreted as an internal model of the whole external situation. The method has high adaptability and robustness, as has been shown by many simulation examples including audio and visual real-world data.(3)Application to SOMRecently, this research group has generalized the paradigm of SOM from the vector space to function space. This is called mnSOM. We have been considering incorporation of the CSDF pruning into the competitive learning associated with mnSOM. Successful results remain to be made for this part. Less