EAGER: Language Learning through Machine Theory of Mind

EAGER：通过机器心理理论学习语言

• 批准号: 2141751
• 负责人: Yonatan Bisk
• 金额: $ 30万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2141751&HistoricalAwards=false
• 关键词: EAGER; Language; Learning; through; Machine

As natural language systems become ubiquitous (e.g. phone trees, chatbots, and smart homes) they must learn to adapt to users by modeling them each as individuals with different abilities, knowledge, and tastes. Theory of mind is the human ability to reason about the hidden mental states of others, but is a complex phenomenon that does not emerge in children until late in their development compared to other more basic communicative skills. The questions of interest to this EAGER project are: (1) what makes this skill hard for children to learn, (2) what can computers learn from how children are taught, and (3) in what ways can machine learning models provide insight into human development. This project sits at this intersection of machine learning, developmental psychology, and pedagogy. This project includes formal models of information sharing and teaching grounded in shared referential games. Agents and children are tasked with asking an instructor to efficiently distinguish similar objects -- a task which requires understanding common ground and identifying distinguishing features. While the learner will often make ambiguous statements, the teacher will provide corrections and instruction to guide the learning process. This formulation allows for variation along several dimensions of relevance to successful communication: working memory, visual and lexical complexity, and specificity of instruction. Experiments with children will provide benchmarks against which computational agents can be compared, and experiments with agents will allow us to decompose the contribution of each of these factors to the difficulty of developing a theory of mind.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着自然语言系统变得无处不在（例如电话树、聊天机器人和智能家居），它们必须学会通过将用户建模为具有不同能力、知识和品味的个体来适应用户。心智理论是人类推理他人隐藏的心理状态的能力，但与其他更基本的沟通技能相比，它是一种复杂的现象，直到发展后期才会在儿童中出现。 这个 EAGER 项目感兴趣的问题是：(1) 是什么让儿童难以学习这项技能；(2) 计算机可以从儿童的教学方式中学到什么；(3) 机器学习模型可以通过哪些方式提供对人类发展的洞察。 该项目位于机器学习、发展心理学和教育学的交叉点。该项目包括基于共享参考游戏的信息共享和教学的正式模型。 特工和孩子们的任务是要求教练有效地区分相似的物体——这项任务需要理解共同点并识别区别特征。虽然学习者经常会做出模棱两可的陈述，但老师会提供纠正和指导来指导学习过程。这种表述允许在与成功沟通相关的几个维度上进行变化：工作记忆、视觉和词汇复杂性以及教学的特异性。对儿童的实验将提供比较计算代理的基准，而对代理的实验将使我们能够分解每个因素对发展心智理论的难度的贡献。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Symmetric Machine Theory of Mind

• 发表时间: 2022
• 作者: Melanie Sclar;Graham Neubig;Yonatan Bisk

Computational Language Acquisition with Theory of Mind

计算语言习得与心理理论

• 发表时间: 2023
• 期刊: International Conference on Learning Representations
• 作者: Liu, Andy;Zhu, Hao;Liu, Emmy;Bisk, Yonatan;Neubig, Graham
• 通讯作者: Neubig, Graham

Simulated Language Learning from Communicative Goals and Linguistic Input

从交际目标和语言输入模拟语言学习

• 发表时间: 2022
• 期刊: Proceedings of the Annual Meeting of the Cognitive Science Society
• 作者: Zhu, Hao;Bisk, Yonatan;Neubig, Graham
• 通讯作者: Neubig, Graham

Don’t Copy the Teacher: Data and Model Challenges in Embodied Dialogue

不要模仿老师：具身对话中的数据和模型挑战

• DOI: 10.18653/v1/2022.emnlp-main.635
• 发表时间: 2022
• 期刊: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
• 作者: Min, So Yeon;Zhu, Hao;Salakhutdinov, Ruslan;Bisk, Yonatan
• 通讯作者: Bisk, Yonatan

EXCALIBUR: Encouraging and Evaluating Embodied Exploration

• DOI: 10.1109/cvpr52729.2023.01434
• 发表时间: 2023-06
• 期刊: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Hao Zhu;Raghav Kapoor;So Yeon Min;Winson Han;Jiatai Li;Kaiwen Geng;Graham Neubig;Yonatan Bisk;Aniruddha Kembhavi;Luca Weihs