CAREER: Cognitively-Informed Memory Models for Language-Capable Robots

能够通过口语与人交流的机器人将推动人类工作的未来，并帮助社会中最脆弱的成员，包括儿童和老年人、残疾人、自闭症或精神疾病患者，以及遭受孤立、欺凌或创伤的人。机器人在与普通人交谈时需要完成的关键任务之一是指表情生成，这是创建描述的过程，比如“走廊尽头的办公室”。当机器人生成这样的描述时，它们需要以一种准确（描述不应该是错误的）、自然（描述不应该听起来尴尬）、可理解（听众应该能够快速、毫不费力地解释描述）和高效（机器人应该能够生成描述，而不必暂停和思考太长时间）的方式进行。为了理解机器人如何以满足这些属性的方式生成描述，我们可以从理解人类是如何做到的开始。我们擅长生成引用表达式的一个原因可能是因为我们的工作记忆或短期记忆，我们用它来保持少量及时和重要的信息，以便我们可以快速而毫不费力地访问。这个项目的关键思想是赋予机器人同样类型的工作记忆能力，以及同样的思考人类工作记忆的方式，这样他们就能够利用及时和重要的信息来更好地生成参考表达式。通过采用这种认知启发的方法，这项工作将推动包括人工智能、机器人和心理学在内的多个领域的最新技术。此外，该项目的教育方面旨在开发有助于培养在这些领域交叉领域工作的下一代学生的材料。为了确保产生最广泛的影响，这些努力将与个人信息计划部门有关扩大参与计算的活动结合起来，使这些活动能够覆盖目前代表性不足的群体。从技术角度来看，本研究的关键目标是展示工作记忆模型如何适当地缓存与任务相关的关于目标相关物体的信念，从而使机器人能够更好地执行参考表达式生成。为此，这项工作将评估两个关键假设：工作记忆的认知启发模型将使机器人能够以一种更准确、更自然、计算效率更高、听者处理的认知效率更高的方式生成参考表达；并且可以利用目标相关性来确保在这些模型中保留大多数与任务相关的信息。通过解决这些假设，研究将发展：(1)基于当前人类工作记忆心理学理论的机器人认知架构中引用表达生成的第一个算法；(2)对机器人如何在人工工作记忆模型中智能管理和分配资源有了根本性的新认识；(3)从机器人和认知建模的角度了解哪些记忆模型将产生最佳性能；(4)对如何在集成认知架构中自动评估实体及其属性的目标相关性有了根本性的新认识；(5)了解目标关联如何在工作记忆机器人模型中用于分配认知资源；(6)从机器人和认知建模的角度理解哪种目标驱动的资源分配策略将产生最佳性能；(7)免费提供的人机对话数据集，以及一个免费提供的实验框架，允许其他研究人员收集更多的此类对话，这两者都将通过开放科学框架永久存档。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Robots that can communicate with people through spoken language stand to advance the future of human work and to assist the most vulnerable members of society, including children and older adults, people with disabilities, autism, or mental illness, and people experiencing isolation, bullying, or trauma. One of the key tasks that robots will need to do when talking with everyday people is referring expression generation, which is the process of creating descriptions like "the office at the end of the hallway." When robots generate such descriptions, they need to do so in a way that is accurate (the description shouldn't be wrong), natural (the description shouldn't sound awkward), understandable (the listener should be able to interpret the description quickly and effortlessly), and efficient (the robot should be able to generate the description without having to pause and think for too long). To understand how robots might generate descriptions in a way that satisfies these properties, we can start by trying to understand how people do so. One reason we are good at generating referring expressions may be because of our working, or short term, memory, which we use to keep a small amount of timely and important information available in a way that we can quickly and effortlessly access. The key idea of this project is to give robots the same type of working memory capabilities, and the same ways of thinking about what might be in peoples' working memories, so they will be able to use that timely and important information to do a better job at generating referring expressions. By taking this cognitively inspired approach, this work will advance the state of the art of multiple fields, including AI, robotics, and psychology. In addition, the educational aspect of this project aims to develop materials that will help train the next generation of students working at the intersection of these fields. To ensure the broadest possible impact, these efforts will be integrated with the PI's department's activities relating to Broadening Participation in Computing so that they reach currently underrepresented groups. From a technical perspective, the key goal of this research is to show how models of working memory that appropriately cache task-relevant beliefs about goal-relevant objects will enable robots to better perform referring expression generation. To this end, the work will assess two key hypotheses: that cognitively inspired models of working memory will enable robots to generate referring expressions in a way that is more accurate, natural, computationally efficient to generate, and cognitively efficient for the listener to process; and that goal relevance can be leveraged to ensure that the most task-relevant information is retained within those models. By addressing these hypotheses, the research will develop: (1) the first algorithms for referring expression generation in robot cognitive architectures that are informed by current psychological theories of human working memory; (2) a fundamental new understanding of how robots can intelligently manage and allocate resources within artificial working memory models, (3) an understanding of which memory models will produce optimal performance from both robotics and cognitive modeling perspectives; (4) fundamental new understanding of how the goal relevance of entities and their properties can be automatically assessed within integrated cognitive architectures; (5) understanding of how goal relevance can be used to allocate cognitive resources within robotic models of working memory; (6) understanding of which goal-driven resource allocation strategies will produce optimal performance from both robotics and cognitive modeling perspectives; and (7) freely-available datasets of human-robot dialogues, and a freely-available experimental framework to allow other researchers to collect additional such dialogues, both of which will be permanently archived via the Open Science Framework.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.