3 Learning distributed representations of concepts
3 Learning distributed representations of concepts
复制标题
DOI：
--
复制
发表时间：
2010
影响因子：
--
通讯作者：
Geoffrey E. Hinton
中科院分区：
文献类型：
--
作者：
Geoffrey E. Hinton
对于如何在神经网络中表示概念信息，有许多不同的建议。这些理论的范围从极端的局部主义理论，其中每个概念由单个神经单元表示（Barlow 1972）到极端的分布式理论，其中一个概念对应于大部分皮层的活动模式。这两个极端是两种不同语义理论的自然实现。在结构主义方法中，概念是由它们与其他概念的关系来定义的，而不是由某种内在本质来定义的。这种方法在神经网络中的自然表达是使每个概念成为没有内部结构的单个单元，并使用单元之间的连接来编码概念之间的关系。在成分方法中，每个概念只是一组特征，因此神经网络可以通过为每个特征分配一个单元并设置单元之间连接的强度来实现一组概念，以便每个概念对应于分布在整个网络上的稳定活动模式（Hopfield 1982; Kohonen 1977; Willshaw，Buneman和Longuet-Higgins 1969）。然后，网络可以执行概念完成（即从其特征的足够子集中检索整个概念）。成分理论的问题在于，它们对概念如何用于结构化推理几乎没有什么可说的。它们主要关注概念之间的相似性或成对关联。它们没有提供一种明显的方式来表示由在结构中扮演不同角色的许多概念组成的铰接结构。
There have been many different proposals for how conceptual information may be represented in neural networks. These range from extreme localist theories in which each concept is represented by a single neural unit (Barlow 1972) to extreme distributed theories in which a concept corresponds to a pattern of activity over a large part of the cortex. These two extremes are the natural implementations of two different theories of semantics. In the structuralist approach, concepts are defined by their relationships to other concepts rather than by some internal essence. The natural expression of this approach in a neural net is to make each concept be a single unit with no internal structure and to use the connections between units to encode the relationships between concepts. In the componential approach each concept is simply a set offeatures and so a neural net can be made to implement a set of concepts by assigning a unit to each feature and setting the strengths of the connections between units so that each concept corresponds to a stable pattern of activity distributed over the whole network (Hopfield 1982; Kohonen 1977; Willshaw, Buneman, and Longuet-Higgins 1969). The network can then perform concept completion (i.e. retrieve the whole concept from a sufficient subset of its features). The problem with componential theories is that they have little to say about how concepts are used for structured reasoning. They are primarily concerned with the similarities between concepts or with pairwise associations. They provide no obvious way of representing articulated structures composed of a number of concepts playing different roles within the structure.