A Primer for Neural Arithmetic Logic Modules
A Primer for Neural Arithmetic Logic Modules
复制标题
DOI：
--
复制
发表时间：
2021-01
期刊：
影响因子：
--
通讯作者：
Bhumika Mistry;K. Farrahi;Jonathon S. Hare
中科院分区：
文献类型：
--
作者：
Bhumika Mistry;K. Farrahi;Jonathon S. Hare
神经算术逻辑模块已经成为一个越来越受关注的领域，尽管仍然是一个利基领域。这些模块是神经网络，旨在实现学习算术和/或逻辑运算的系统概括，如$\{+，-，\times，\div，\leq，\textrm{AND}\}$，同时也是可解释的。本文首先讨论了这一领域的进展现状，解释了关键工作，从神经算术逻辑单元（NALU）开始。针对NALU的不足，对近年来模块设计选择的原因进行了深入的分析。模块之间的交叉比较是在实验设置和发现，在那里我们突出了一个基本的实验，导致无法直接比较跨文件的不一致。为了减轻现有的不一致，我们创建了一个基准比较所有现有的算术NALM。最后，我们对NALU的现有应用和需要进一步探索的研究方向进行了新的讨论。
Neural Arithmetic Logic Modules have become a growing area of interest, though remain a niche field. These modules are neural networks which aim to achieve systematic generalisation in learning arithmetic and/or logic operations such as $\{+, -, \times, \div, \leq, \textrm{AND}\}$ while also being interpretable. This paper is the first in discussing the current state of progress of this field, explaining key works, starting with the Neural Arithmetic Logic Unit (NALU). Focusing on the shortcomings of the NALU, we provide an in-depth analysis to reason about design choices of recent modules. A cross-comparison between modules is made on experiment setups and findings, where we highlight inconsistencies in a fundamental experiment causing the inability to directly compare across papers. To alleviate the existing inconsistencies, we create a benchmark which compares all existing arithmetic NALMs. We finish by providing a novel discussion of existing applications for NALU and research directions requiring further exploration.