MUTANT: A Training Paradigm for Out-of-Distribution Generalization in Visual Question Answering
MUTANT: A Training Paradigm for Out-of-Distribution Generalization in Visual Question Answering
复制标题
DOI：
10.18653/v1/2020.emnlp-main.63
复制
发表时间：
2020-09
期刊：
影响因子：
--
通讯作者：
Tejas Gokhale;Pratyay Banerjee;Chitta Baral;Yezhou Yang
中科院分区：
文献类型：
--
作者：
Tejas Gokhale;Pratyay Banerjee;Chitta Baral;Yezhou Yang
虽然在视觉问答排行榜上取得了进展，但模型通常利用i.i.d.下数据集中的虚假相关性和先验。设置.因此，评价的分布（OOD）测试样本已成为一个代理泛化。在本文中，我们提出了MUTANT，这是一种训练范式，它将模型暴露给感知相似但语义不同的输入突变，以提高OOD泛化能力，例如VQA-CP挑战。在这种范式下，模型利用一致性约束的训练目标来理解输入（问题-图像对）中的语义变化对输出（答案）的影响。与VQA-CP上的现有方法不同，MUTANT不依赖于关于训练和测试答案分布的性质的知识。MUTANT在VQA-CP上建立了一个新的最先进的准确性，提高了10.57\%。我们的工作开辟了使用语义输入突变的OOD泛化问题回答的途径。
While progress has been made on the visual question answering leaderboards, models often utilize spurious correlations and priors in datasets under the i.i.d. setting. As such, evaluation on out-of-distribution (OOD) test samples has emerged as a proxy for generalization. In this paper, we present MUTANT, a training paradigm that exposes the model to perceptually similar, yet semantically distinct mutations of the input, to improve OOD generalization, such as the VQA-CP challenge. Under this paradigm, models utilize a consistency-constrained training objective to understand the effect of semantic changes in input (question-image pair) on the output (answer). Unlike existing methods on VQA-CP, MUTANT does not rely on the knowledge about the nature of train and test answer distributions. MUTANT establishes a new state-of-the-art accuracy on VQA-CP with a $10.57\%$ improvement. Our work opens up avenues for the use of semantic input mutations for OOD generalization in question answering.