EDNC: Evolving Differentiable Neural Computers
EDNC: Evolving Differentiable Neural Computers
复制标题
EDNC：不断发展的可微神经计算机
DOI：
10.1016/j.neucom.2020.06.018
复制
发表时间：
2020
期刊：
影响因子：
6
通讯作者：
Faramarz Safi
中科院分区：
文献类型：
--
作者：
M. Rasekh;Faramarz Safi
深度学习领域最近的一项研究是通过将人工神经网络（ANN）耦合到外部存储资源来扩展它们。神经图灵机（NTM）和可微神经计算机（DNC）是这一领域的两个对应。研究活动分为两类：与内存交互或选择控制器组件。第一种方法使用尝试和错误的方式来提供控制器的结构，这将是一个问题，当没有关于特定任务的控制器的结构的先验知识。第二种方法包括选择控制器的结构和重量优化的方法。神经进化福尔斯属于第二类，它自动地并且在没有先验知识的情况下获得适当的网络结构。本研究提出了进化微分神经计算机（EDNC），它使用了一种新的神经进化算法，该算法引入了两种类型的嵌套面向对象的编码，称为自适应层神经进化（ALNE）和基于矩阵的称为M_ALNE。这些神经进化算法使用以下子算法：初始群体自适应性（SAIP）、进化结构自适应性（SAES）、层简化（LR）、层突变（LM）和突变自适应性（SAM）。进化过程从SAIP开始，然后通过SAES算法，使用LR和LM算法来提供各种结构，以探索各种神经结构。最后，SAM算法将所选择的结构引导到目标结构。在这项研究中，实验将EDNC应用于众所周知的任务，包括Facebook bAbI，复制，图形（最短路径）和8-puzzle。实验表明，该方法在自动编码和没有先验知识，产生一个合适的控制器结构在最短的时间。然而，它表明，与基线方法相比，这两种编码方法至少减少了73%的进化时间。
One recent study in the field of deep learning is extending Artificial Neural Networks (ANNs) by coupling them to external memory resources. Neural Turing Machine (NTM) and Differentiable Neural Computer (DNC) are two counterparts in this field. Research activities fall into two categories of either interacting with memory or choosing controller components. The first approach uses a try and error fashion to provide the controller’s structure that would be a problem when there is no prior knowledge on the controller’s structure for particular tasks. The second approach includes methods for choosing the controller’s structure and weight optimization. NeuroEvolution falls in the second category that automatically and without prior knowledge obtains an appropriate network structure. This research presents Evolutionary Differentiable Neural Computer (EDNC), which uses a novel NeuroEvolution algorithm that is introduced in two types of nested object-oriented encoding called Adaptive Layer NeuroEvolution (ALNE) and Matrix-based one called M_ALNE. These NeuroEvolution algorithms use the following sub-algorithms: Self-Adaptivity in the Initial Population (SAIP), Self-Adaptivity in Evolutionary Structures (SAES), Layer Recombination (LR), Layer Mutation (LM), and Self-Adaptivity in Mutation (SAM). The evolution process starts with SAIP, and then it takes short and long steps to explore a variety of neural structures by SAES algorithm that uses both LR and LM algorithms to provide a variety of structures. Finally, the SAM algorithm guides the selected structures to a target structure. In this research, experiments applied EDNC on well-known tasks, including Facebook bAbI, copy, graph (shortest path), and 8-puzzle. The experiments show that the proposed method in both encodings automatically and without prior knowledge, produces an appropriate controller structure in the shortest time. It though shows that both proposed encodings reduce the evolution time by at least 73% in comparison with the baseline methods.