Sequence-level Mixed Sample Data Augmentation
Sequence-level Mixed Sample Data Augmentation
复制标题
DOI：
10.18653/v1/2020.emnlp-main.447
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Demi Guo;Yoon Kim;Alexander M. Rush
中科院分区：
文献类型：
--
作者：
Demi Guo;Yoon Kim;Alexander M. Rush
尽管在经验上取得了成功，但神经网络仍然难以捕捉自然语言的组成方面。这项工作提出了一种简单的数据增强方法，以鼓励序列到序列问题的神经模型中的组合行为。我们的方法SeqMix通过将训练集中的输入/输出序列进行软组合来创建新的合成示例。我们将这种方法连接到现有的技术，如SwitchOut和字辍学，并表明这些技术都是一个单一的目标的近似变体。SeqMix在强大的Transformer基线上对五个不同的翻译数据集持续产生约1.0 BLEU的改进。对于需要强大的组合泛化的任务，如SCAN和语义解析，SeqMix还提供了进一步的改进。
Despite their empirical success, neural networks still have difficulty capturing compositional aspects of natural language. This work proposes a simple data augmentation approach to encourage compositional behavior in neural models for sequence-to-sequence problems. Our approach, SeqMix, creates new synthetic examples by softly combining input/output sequences from the training set. We connect this approach to existing techniques such as SwitchOut and word dropout, and show that these techniques are all approximating variants of a single objective. SeqMix consistently yields approximately 1.0 BLEU improvement on five different translation datasets over strong Transformer baselines. On tasks that require strong compositional generalization such as SCAN and semantic parsing, SeqMix also offers further improvements.