Robust and Scalable Differentiable Neural Computer for Question Answering
Robust and Scalable Differentiable Neural Computer for Question Answering
复制标题
用于问答的稳健且可扩展的可微神经计算机
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
A. Waibel
中科院分区：
文献类型：
--
作者：
Jörg K.H. Franke;J. Niehues;A. Waibel
深度学习模型通常不容易适应新任务，需要针对特定任务进行调整。可微神经计算机（DNC）是一种记忆增强神经网络，被设计为通用问题求解器，可用于广泛的任务。但在现实中，很难将这种模式应用于新的任务。我们分析了DNC，并确定可能的改进问题回答的应用程序。这激发了更强大和可扩展的DNC（rsDNC）。客观前提是保持该模型的一般特征不变，同时使其应用更可靠，并加快其所需的训练时间。rsDNC的特点是更强大的训练，苗条的存储单元和双向架构。我们不仅在bAbI任务上实现了新的最先进的性能，而且还最小化了不同初始化之间的性能差异。此外，我们证明了rsDNC对新任务的简化适用性，在没有自适应的CNN RC任务上具有可通过的结果。
Deep learning models are often not easily adaptable to new tasks and require task-specific adjustments. The differentiable neural computer (DNC), a memory-augmented neural network, is designed as a general problem solver which can be used in a wide range of tasks. But in reality, it is hard to apply this model to new tasks. We analyze the DNC and identify possible improvements within the application of question answering. This motivates a more robust and scalable DNC (rsDNC). The objective precondition is to keep the general character of this model intact while making its application more reliable and speeding up its required training time. The rsDNC is distinguished by a more robust training, a slim memory unit and a bidirectional architecture. We not only achieve new state-of-the-art performance on the bAbI task, but also minimize the performance variance between different initializations. Furthermore, we demonstrate the simplified applicability of the rsDNC to new tasks with passable results on the CNN RC task without adaptions.