CAREER: Dialogue Engagement for Educational Robots

这个教师早期职业发展项目的活动植根于一个社会教育机器人，它与学生在STEM学习中合作。这个机器人扮演同伴学习者的角色，要求学生教他们如何处理数学和计算机科学问题。解释一个人推理的过程对加深理解有教学上的好处。教育机器人有可能接触到广泛的学生群体，但其好处取决于能否促进学生对学习活动的强烈参与。在现有的人机协作活动研究中，很多注意力都集中在执行自然语言命令或回答独立问题的机器人上；很少有人关注那些能维持更长时间对话互动的机器人。该项目开发了一个框架，用于提高人机会话交互中的用户参与度——与机器人的社会联系和对任务的参与。提高人机对话参与度将使科学进步从专注于原子交互的机器人发展到能够长期维持交互的社交智能机器人。该项目整合了大量的计算机科学教育活动，以扩大在计算机领域代表性不足的学生的研究机会，通过教授会说话的机器人的研讨会来拓宽计算的视角，并扩大研究者为培训同伴导师创建包容性学习环境而建立的课程的影响。该项目旨在通过描述感知输入和机器人动作的多种模式如何有效地结合在物理位置的交互中，提高教育机器人维持学生参与STEM学习活动的能力。组合模式的一个重大挑战是定义适当的时间段，以捕获有意义的单位，并以一种允许它们相互补充的方式对齐。该项目涉及社会教育机器人参与的三个相互关联的方面。首先，它探索了使用语音、视觉和词汇输入来估计参与度的机器学习方法，重点是对齐感知数据的时间动态。其次，该项目开发了基于感知参与的自适应响应生成策略，利用了人类和人类-机器人同伴学习对话的语料库分析。第三，该项目采用强化学习方法，通过学习整合语言和非语言机器人动作（如头部运动、语音反向通道和凝视手势）的对话策略来促进参与。综上所述，这些结果为教育机器人与更高的社会智能互动并保持长期参与提供了基础，并最终可以转移到与其他教育领域以及非教育领域的机器人进行社会对话。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The activities in this Faculty Early Career Development project are rooted in a social educational robot that collaborates with students in STEM learning. The robot acts as a peer learner who asks students to teach them how to approach mathematics and computer science problems. The process of explaining one’s reasoning has the pedagogical benefit of deepening understanding. Educational robots have the potential to reach a broad population of students, but the benefits depend on facilitating strong engagement in the learning activity. In existing research on human-robot collaborative activities, much attention has been directed towards robots that execute natural language commands or respond to stand-alone questions; less attention has been given to robots that sustain longer conversational interactions. This project develops a framework for improving user engagement---social connection to the robot and involvement in the task---in conversational human-robot interaction. Improving human-robot conversational engagement will enable scientific progress from robots that focus on atomic interactions to socially intelligent robots that sustain interaction over time. This project integrates substantial computer science education activities to expand research opportunities for students who are underrepresented in computing, to broaden perspectives of computing by teaching a seminar on talking robots, and to extend the impact of a curriculum established by the investigator for training peer mentors in creating inclusive learning environments.This project aims to improve educational robots’ ability to sustain student engagement in STEM learning activities by characterizing how multiple modalities of perceptual inputs and of robot actions can be effectively combined in physically-situated interaction. A significant challenge in combining modalities is defining appropriate temporal segments that capture meaningful units and align in a way that allows them to complement one another. The project addresses three interconnected aspects of engagement for social educational robots. First, it explores machine learning methods for estimating engagement using speech, vision, and lexical inputs, with a focus on the temporal dynamics of aligning the perceptual data. Second, the project develops strategies for adaptive response generation based on perceived engagement, drawing on corpus analyses of human-human and human-robot peer-learning dialogues. Third, the project takes a reinforcement learning approach to fostering engagement by learning dialogue policies that integrate verbal and non-verbal robot actions such as head movements, spoken backchannels, and gaze gestures. Taken together, the results provide a foundation for enabling educational robots to interact with greater social intelligence and maintain long-term engagement, and ultimately could transfer to social conversations with robots in other fields of education as well as non-educational domains.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.