Continual Learning in Multimodal Interactive Settings

为了让人类和人工代理人在情景环境中有效地沟通，代理人需要能够推理对话期间所说的话和他们所感知的东西。尽管视觉对话领域取得了进展，但目前基于深度神经网络的模型不能持续学习新概念，并遭受灾难性遗忘。我的研究旨在将持续学习和视觉对话联系起来，以开发能够在不忘记先前知识的情况下逐步学习新概念和技能的代理。

In order for humans and artificial agents to communicate effectively in situated settings, agents need to be able to reason about what has been said during the dialogue and what they perceive. Despite the advances within the field of Visual Dialogue, current models based on deep neural networks are not able to continually learn new concepts and suffer from catastrophic forgetting. My research aims to bridge Continual Learning and Visual Dialogue in order to develop agents that are able to incrementally learn new concepts and skills without forgetting previous knowledge.