NRI: Real Time Observation, Inference and Intervention of Co-Robot Systems Towards Individually Customized Performance Feedback Based on Students' Affective States

这个NSF国家机器人计划项目将调查合作机器人系统的观察、推理和干预循环的潜力，以增强学生的情感状态，并提高他们在工程实验室任务中的表现。协作机器人是与人类并肩工作的机器人，帮助他们并适应他们的需求。学生和合作机器人之间的双向知识交流创造了一种互惠关系，在这种关系中，双方为了共同的目标向对方学习。情感状态，如挫折和投入，对学生在日常学习任务中的表现起着重要作用。过度紧张或分心的学生可能会犯一些本来很容易避免的错误。一个能够识别学生情感状态的协作机器人系统可以进行干预，以防止这些错误。这个项目的结果可能会为基于技能的教学提供一个模板，这些主题远远超出了工程学的范畴。目前，这样的学习需要学生和教师之间进行广泛的互动，教师始终提供密集的反馈。在许多情况下，教师和学生之间的个性不匹配或其他问题可能会导致挫折、学习困难，并最终辍学。此外，一对一学习受到可伸缩性挑战的限制，因为增加学生人数，而没有按比例增加训练有素的教师，可能会导致分配给每个学生的教师时间的质量和数量减少。协作机器人学习系统将能够通过提供实时和可扩展的反馈系统来缓解这些挑战，这些反馈系统适应学生的个人需求，并有助于将每个学生所需的人工教练时间降至最低。这项研究将使用协作机器人的集成视觉、听觉和深度感知系统来获取学生的面部、听觉和身体手势数据。该系统将基于机器学习对面部和肢体语言数据进行分类，对学生的情感状态进行统计推断。视觉反馈将被用来向学生提供旨在增强他们的情感状态和改善他们在实验室任务中的表现的干预(视觉指导和评论)。该项目将评估合作机器人在反复学习和测试的过程中改善学生情感状态和提高学生在实验室任务中的表现的能力的影响。本项目将有助于更好地理解学生在潜在压力较大的实验室活动中如何互动和发挥作用。本文提出的协作机器人系统将有助于发现学生的情绪和任务绩效之间存在的关联。合作机器人将主动适应学生学习复杂工程任务的方式以及伴随学习而来的情感状态。联合机器人系统预测针对每个学生和情况的特定干预策略的有效性，将导致个性化的学生反馈，为学生和教师提供服务，随着时间的推移提高学生的表现。这项提议将合作机器人的影响推进到教育研究和实践中，并扩展了如何以数字形式简洁地表示人类行为的复杂性的知识。

This NSF National Robotics Initiative project will investigate the potential of a cycle of observation, inference and intervention by co-robot systems to enhance students' affective states and improve their performance on engineering laboratory tasks. Co-robots are robots that work side-by-side with humans, assisting them and adapting to their needs. The two-way exchange of knowledge between students and co-robots creates a reciprocal relationship, in which each party learns from the other in service of a common goal. Affective states, such as frustration and engagement, play a major role in students' performance on everyday learning tasks. A student who is overly stressed or distracted may commit errors that would be otherwise easy to avoid. A co-robot system that is cognizant of students' affective states can intervene to prevent these errors. The results of this project may provide a template for skill-based instruction on topics well beyond engineering. Currently, such learning requires extensive interactions between a student and an instructor, with the instructor providing intensive feedback at all times. In many cases, personality mismatches or other issues between instructor and student can lead to frustration, learning difficulties, and eventual dropout. Furthermore, one-on-one learning is limited by scalability challenges, as an increase in the number of students, without a proportional increase in trained instructors, can result in decreases in quality and quantity of instructor time allocated to each student. Co-robot learning systems will be able to mitigate these challenges by providing both real time and scalable feedback systems that adapt to the individual needs of students and help to minimize the amount of human instructor time required by each student. This research will acquire facial, auditory, and body gesture data from students using the integrated visual, audio and depth sensory system of the co-robot. The system will make statistical inferences of students' affective states, based on machine learning classification of facial and body language data. Visual feedback will be used to present students with interventions (visual instructions and commentary) intended to enhance their affective state and improve their performance on laboratory tasks. The project will assess the impact of co-robots' ability to improve students' affective states and enhance students' performance on laboratory tasks over repeated iterations of learning and testing. This project will lead to a better understanding of how students interact and function during potentially stressful laboratory activities. The co-robot systems proposed in this work will help discover the correlations that exists between students' affect and task performance. Co-robots will actively adapt to the manner in which students learn complex engineering tasks and the affective states that accompany that learning. Co-robot systems that predict the effectiveness of specific intervention strategies for each student and situation will lead to individually-tailored student feedback that serves both students and instructors towards enhancing student performance over time. This proposal advances the impact of co-robots into educational research and practice and extends knowledge of how to succinctly represent the complexities of human behavior in digital form.