Associative Memory Using Dictionary Learning and Expander Decoding
Associative Memory Using Dictionary Learning and Expander Decoding
复制标题
使用字典学习和扩展器解码的联想记忆
DOI：
10.1609/aaai.v31i1.10515
复制
发表时间：
2016
期刊：
影响因子：
--
通讯作者：
A. Rawat
中科院分区：
文献类型：
--
作者：
A. Mazumdar;A. Rawat
关联内存是一种内容可寻址内存框架，它通过神经网络存储消息向量（或数据集）的集合，同时使神经上可行的机制能够从数据集中的噪声版本中恢复任何消息。设计联想记忆需要解决两个主要任务：1)学习阶段：给定一个数据集，以图形模型（或神经网络）的形式学习数据集的简明表示；2)召回阶段：给定数据集的消息向量的噪声版本，通过学习阶段学习的网络上的神经可行算法输出正确的消息向量。本文研究了一类神经联想记忆的设计问题，该问题学习了大型数据集的网络表示，以确保在回忆阶段对大量对抗性错误进行纠正。具体来说，本文设计的联想存储器可以在O(n)个节点的网络上存储包含exp(n) n个长度的消息向量的数据集，并且可以容忍Ω（n / polylog）对抗性错误。本文通过将学习阶段和召回阶段分别映射到使用方形字典的字典学习任务和扩展代码中的迭代纠错任务来实现这种内存设计。
An associative memory is a framework of content-addressable memory that stores a collection of message vectors (or a dataset) over a neural network while enabling a neurally feasible mechanism to recover any message in the dataset from its noisy version. Designing an associative memory requires addressing two main tasks: 1) learning phase: given a dataset, learn a concise representation of the dataset in the form of a graphical model (or a neural network), 2) recall phase: given a noisy version of a message vector from the dataset, output the correct message vector via a neurally feasible algorithm over the network learnt during the learning phase. This paper studies the problem of designing a class of neural associative memories which learns a network representation for a large dataset that ensures correction against a large number of adversarial errors during the recall phase. Specifically, the associative memories designed in this paper can store dataset containing exp(n) n-length message vectors over a network with O(n) nodes and can tolerate Ω(n / polylog) adversarial errors. This paper carries out this memory design by mapping the learning phase and recall phase to the tasks of dictionary learning with a square dictionary and iterative error correction in an expander code, respectively.