Resonator Networks, 1: An Efficient Solution for Factoring High-Dimensional, Distributed Representations of Data Structures
Resonator Networks, 1: An Efficient Solution for Factoring High-Dimensional, Distributed Representations of Data Structures
复制标题
DOI：
10.1162/neco_a_01331
复制
发表时间：
2020-12-01
影响因子：
2.9
通讯作者：
Sommer, Friedrich T.
中科院分区：
文献类型：
--
作者：
Frady, E. Paxon;Kent, Spencer J.;Sommer, Friedrich T.
使用分布式神经表示对数据结构进行编码和操作的能力可以通过支持基于规则的符号推理来定性地增强传统神经网络的能力，基于规则的符号推理是认知的核心属性。在这里，我们展示了如何在向量符号体系结构(VSA)的框架内实现这一点(Platform，1991；Gayler，1998；Kanerva，1996)，其中数据结构通过将高维向量与一起形成分布式表示空间上的代数的操作相结合来编码。特别是，对于在解码VSA数据结构的元素时出现的硬组合搜索问题，我们提出了一种有效的解决方案：多个码向量的乘积的因式分解。我们提出的算法，称为谐振器网络，是一种新型的递归神经网络，它交织了VSA乘法运算和模式完成。我们通过两个例子--树形数据结构的解析和视觉场景的解析--展示了因式分解问题是如何产生的，以及谐振器网络是如何解决它的。更广泛地说，谐振器网络为将VSA应用于现实世界领域中的无数人工智能问题提供了可能性。本期的配套文章(Kent，Frady，Sommer和Olshausen，2020)对谐振器网络的性能进行了严格的分析和评估，表明它的性能优于其他方法。
The ability to encode and manipulate data structures with distributed neural representations could qualitatively enhance the capabilities of traditional neural networks by supporting rule-based symbolic reasoning, a central property of cognition. Here we show how this may be accomplished within the framework of Vector Symbolic Architectures (VSAs) (Plate, 1991; Gayler, 1998; Kanerva, 1996), whereby data structures are encoded by combining high-dimensional vectors with operations that together form an algebra on the space of distributed representations. In particular, we propose an efficient solution to a hard combinatorial search problem that arises when decoding elements of a VSA data structure: the factorization of products of multiple codevectors. Our proposed algorithm, called a resonator network, is a new type of recurrent neural network that interleaves VSA multiplication operations and pattern completion. We show in two examples-parsing of a tree-like data structure and parsing of a visual scene-how the factorization problem arises and how the resonator network can solve it. More broadly, resonator networks open the possibility of applying VSAs to myriad artificial intelligence problems in real-world domains. The companion article in this issue (Kent, Frady, Sommer, & Olshausen, 2020) presents a rigorous analysis and evaluation of the performance of resonator networks, showing it outperforms alternative approaches.