Compositional Data and Task Augmentation for Instruction Following
Compositional Data and Task Augmentation for Instruction Following
复制标题
用于遵循指令的成分数据和任务增强
DOI：
--
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
D. Roth
中科院分区：
文献类型：
--
作者：
Soham Dan;Xinran Han;D. Roth
在物理基础域中执行自然语言指令需要一个模型，该模型既理解诸如left_of和above之类的空间概念，又理解用于识别地标和与地标相关的发音指令的组合语言。本文研究了块世界域中的指令理解。给定一个初始的块和一个自然语言指令，系统通过操作选定的块来执行指令。高度组合的指令由原子组件组成，理解这些组件是执行指令的必要步骤。我们表明，虽然端到端训练（仅由正确的块位置监督）无法解决此任务的挑战，并且在涉及单个原子组件的指令上表现不佳，但无知识辅助信号可以通过为指令的组件提供监督来显着提高性能。具体来说，我们生成的信号旨在帮助模型逐渐理解组成指令的组成部分，以及帮助它更好地理解空间概念的信号，并显示它们对两个数据集和两个最先进（SOTA）模型的整体任务的贝内，特别是当训练数据有限时-这在此类任务中很常见。
Executing natural language instructions in a physically grounded domain requires a model that understands both spatial concepts such as left_of and above , and the compositional language used to identify landmarks and articu-late instructions relative to them. In this paper, we study instruction understanding in the blocks world domain. Given an initial arrange-ment of blocks and a natural language instruction, the system executes the instruction by manipulating selected blocks. The highly compositional instructions are composed of atomic components and understanding these components is a necessary step to executing the instruction. We show that while end-to-end training (supervised only by the correct block location) fails to address the challenges of this task and performs poorly on instructions involving a single atomic component, knowledge-free auxiliary signals can be used to significantly improve performance by providing supervision for the instruction’s components. Specifi-cally, we generate signals that aim at helping the model gradually understand components of the compositional instructions, as well as those that help it better understand spatial concepts, and show their benefit to the overall task for two datasets and two state-of-the-art (SOTA) models, especially when the training data is limited—which is usual in such tasks.