Unified Questioner Transformer for Descriptive Question Generation in Goal-Oriented Visual Dialogue
Unified Questioner Transformer for Descriptive Question Generation in Goal-Oriented Visual Dialogue
复制标题
DOI：
10.1109/iccv48922.2021.00191
复制
发表时间：
2021-06
期刊：
影响因子：
--
通讯作者：
Shoya Matsumori;Kosuke Shingyouchi;Yukikoko Abe;Yosuke Fukuchi;K. Sugiura;M. Imai
中科院分区：
文献类型：
--
作者：
Shoya Matsumori;Kosuke Shingyouchi;Yukikoko Abe;Yosuke Fukuchi;K. Sugiura;M. Imai
构建一个可以对真实的世界提出问题的交互式人工智能是视觉和语言问题的最大挑战之一。特别是，面向目标的视觉对话，其中的代理的目的是寻求信息，通过提问在一个轮换对话，最近已经获得了学术界的关注。虽然现有的几个模型基于GuessWhat？！[10]，怀疑者通常会问简单的基于类别的问题或绝对空间的问题。对于对象共享属性的复杂场景，或者需要描述性问题来区分对象的情况，这可能会有问题。在本文中，我们提出了一种新的提问者架构，称为统一的提问者Transformer（UniQer），描述性的问题生成与引用表达式。此外，我们建立了一个目标导向的视觉对话任务，称为CLEVR问。它综合了复杂的场景，需要怀疑者提出描述性的问题。我们用CLEVR Ask数据集的两个变体训练我们的模型。定量和定性评估的结果表明，UniQer优于基线。
Building an interactive artificial intelligence that can ask questions about the real world is one of the biggest challenges for vision and language problems. In particular, goal-oriented visual dialogue, where the aim of the agent is to seek information by asking questions during a turn-taking dialogue, has been gaining scholarly attention recently. While several existing models based on the GuessWhat?! dataset [10] have been proposed, the Questioner typically asks simple category-based questions or absolute spatial questions. This might be problematic for complex scenes where the objects share attributes, or in cases where descriptive questions are required to distinguish objects. In this paper, we propose a novel Questioner architecture, called Unified Questioner Transformer (UniQer), for descriptive question generation with referring expressions. In addition, we build a goal-oriented visual dialogue task called CLEVR Ask. It synthesizes complex scenes that require the Questioner to generate descriptive questions. We train our model with two variants of CLEVR Ask datasets. The results of the quantitative and qualitative evaluations show that UniQer outperforms the baseline.