Evaluating Theory of Mind in Question Answering
Evaluating Theory of Mind in Question Answering
复制标题
评估问答中的心理理论
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
T. Griffiths
中科院分区：
文献类型：
--
作者：
Aida Nematzadeh;Kaylee Burns;Erin Grant;A. Gopnik;T. Griffiths
我们提出了一个新的数据集，用于评估问答模型的能力，原因有关的信念。我们的任务受到心理理论实验的启发，这些实验考察儿童是否能够推理他人的信念，特别是当这些信念与现实不同时。我们评估了一些最近的神经模型与记忆增强。我们发现，所有的失败在我们的任务，这需要保持跟踪不一致的状态的世界;此外，模型的准确性显着下降时，随机句子被引入到测试任务。
We propose a new dataset for evaluating question answering models with respect to their capacity to reason about beliefs. Our tasks are inspired by theory-of-mind experiments that examine whether children are able to reason about the beliefs of others, in particular when those beliefs differ from reality. We evaluate a number of recent neural models with memory augmentation. We find that all fail on our tasks, which require keeping track of inconsistent states of the world; moreover, the models’ accuracy decreases notably when random sentences are introduced to the tasks at test.