NRI: INT: Designing Effective Dialogue, Gaze, and Gesture Behaviors in a Social Robot that Supports Collaborative Learning in Middle School Mathematics

当两个学生与教学代理一起工作时，他们往往会在与代理和彼此交谈时学到更多，解释他们的推理并建立在彼此的想法上。目前尚不清楚的是，使用物理机器人而不是虚拟代理是否以及如何改善学生互动的方式，并最终改善他们的学习。该奖项研究了机器人的非语言行为如何补充它对学生所说的话，以促进他们的思考和发展他们的理解。使用机器人凝视学生，鼓励他们说话或做出数学手势，帮助学生澄清自己的想法，可能会利用机器人带来的独特能力进行交互，并最终增强机器人与学生对话的效果。该奖项旨在研究如何将机器人的两个沟通渠道-手势和凝视-的战略设计与对话相结合，以增强中学生在数学方面的协作互动。最终，该项目的成功将有助于更广泛地了解机器人如何在未来的学习环境中有效地整合，以及增加对协作机器人如何促进协作的理解。参加研究的中学生将通过接触新技术和研究， 跨学科的研究生和本科生将在人工智能、人机交互、学习科学和认知心理学的交叉领域接受培训。该奖项汇集了两个理论框架，即ICAP认知参与理论和交互式沟通对齐模型（IAM），以做出两项贡献：1）如何使用数据，使机器人自动学习有效的社会行为，以及2）什么是经过验证测试的符合特定框架的理想机器人行为？ICAP假设，互动活动，其中两个学生都建设性地促进合作是最好的学习，而IAM假设，作为合作者追求成功的沟通，他们将夹带（或模仿）对方的对话选择和手势。该奖项将使用强化学习来确定哪些特定的凝视对话行为（例如，在特定类型的对话期间哪个人类合作者正被注视）促进两个人类合作者之间的平衡交互，以及何时以及如何引入数学相关的术语和手势。这个过程旨在创建机器人对话，手势和凝视模式，这些模式对先验知识和动机的个体差异敏感和敏感。该项目的第一年将用于开发与机器人的协作交互。在项目的第2年和第3年，自适应凝视，手势和对话行为将在非自适应控制条件下进行测试。最后，在第四年，该项目将最佳自适应策略与虚拟代理中实施的类似策略进行比较，探索机器人的具体性质，结合这些战略性设计的行为，该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响进行评估，被认为值得支持审查标准。

When two students work together with a pedagogical agent, they tend to learn more as they talk to the agent and each other, explaining their reasoning and building on each other's ideas. What is not clear is whether and how the use of a physical robot rather than a virtual agent might improve the ways in which students interact and ultimately their learning. This award investigates how a robot's nonverbal behaviors might complement what it says to the students in order to prompt their thinking and develop their understanding. Using the robot to gaze at a student to encourage them to speak or make a mathematical gesture that helps the students clarify their own thinking might leverage the unique capabilities the robot brings to the interaction and ultimately enhance the effects of the robot's conversation with the students. This award investigates how the strategic design of two channels of communication of the robot – gesture and gaze – can be combined with dialogue to enhance middle school students' collaborative interactions within math. Ultimately, success in this project will contribute to broader understanding of how robots can be integrated effectively in learning environments in the future, as well as increase understanding of how co-robots can facilitate collaboration. Middle school students who participate in the studies will experience positive impact through the exposure to novel technologies and research, and transdisciplinary graduate and undergraduate students will be trained in the intersection of artificial intelligence, human-computer interaction, learning sciences, and cognitive psychology.This award brings together two theoretical frameworks, the ICAP theory of cognitive engagement and the Interactive Alignment Model of communication (IAM), to make two contributions: 1) How can data can be used such that the robot automatically learns effective social behaviors, and 2) What are empirically-tested desired robot behaviors that align to a particular framework? ICAP postulates that interactive activities where both students contribute constructively to the collaboration are best for learning, while IAM postulates that as collaborators pursue a successful communication, they will entrain to (or mimic) each other's dialogue choices and gestures. The award will use reinforcement learning to determine which specific gaze-dialogue behaviors (e.g., which human collaborator is being looked at during a particular type of dialogue) promote balanced interaction amongst the two human collaborators and when and how to introduce mathematically relevant terminology and gestures. This process is intended to create robot dialogue, gesture, and gaze patterns that are sensitive and responsive to individual differences in prior knowledge and motivation. Year 1 of the project will be spent developing the collaborative interaction with the robot. In Years 2 and 3 of the project, the adaptive gaze, gesture, and dialogue behaviors will be tested against non-adaptive control conditions. Finally, in Year 4, the project compares the optimal adaptive policies derived to a similar policy implemented within a virtual agent, exploring whether the embodied nature of the robot, combined with these strategically designed behaviors, offers significant advantages over a parallel agent.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.