NRI: Real Time Observation, Inference and Intervention of Co-Robot Systems Towards Individually Customized Performance Feedback Based on Students' Affective States

这个NSF国家机器人计划项目将调查合作机器人系统的观察，推理和干预周期的潜力，以增强学生的情感状态，提高他们在工程实验室任务中的表现。合作机器人是与人类并肩工作的机器人，帮助他们并适应他们的需求。学生和合作机器人之间的双向知识交流创造了一种互惠关系，在这种关系中，每一方都在为共同的目标服务中相互学习。情感状态，如挫折和参与，在学生的日常学习任务的表现中发挥着重要作用。过度紧张或分心的学生可能会犯一些本来很容易避免的错误。认知学生情感状态的协作机器人系统可以进行干预以防止这些错误。这个项目的结果可能会提供一个模板，以技能为基础的教学主题远远超出工程。目前，这种学习需要学生和教师之间的广泛互动，教师随时提供密集的反馈。在许多情况下，教师和学生之间的性格不匹配或其他问题可能导致挫折感，学习困难，最终辍学。此外，一对一学习受到可扩展性挑战的限制，因为学生人数的增加，而训练有素的教师人数没有按比例增加，可能导致分配给每个学生的教师时间的质量和数量下降。协作机器人学习系统将能够通过提供真实的时间和可扩展的反馈系统来缓解这些挑战，这些反馈系统适应学生的个人需求，并有助于最大限度地减少每个学生所需的人类教师时间。本研究将使用合作机器人的整合视觉、听觉和深度感觉系统，从学生那里获取面部、听觉和身体姿势数据。 该系统将根据面部和肢体语言数据的机器学习分类，对学生的情感状态进行统计推断。 视觉反馈将用于向学生提供干预措施（视觉指导和评论），旨在增强他们的情感状态，提高他们在实验室任务中的表现。 该项目将评估合作机器人改善学生情感状态的能力的影响，并通过反复的学习和测试来提高学生在实验室任务中的表现。 这个项目将导致学生如何互动和功能在潜在的压力实验室活动的更好的理解。在这项工作中提出的合作机器人系统将有助于发现学生的情感和任务绩效之间存在的相关性。合作机器人将积极适应学生学习复杂工程任务的方式以及伴随学习的情感状态。预测每个学生和情况的特定干预策略的有效性的合作机器人系统将导致个性化的学生反馈，为学生和教师提供服务，随着时间的推移提高学生的表现。该提案将合作机器人的影响推进到教育研究和实践中，并扩展了如何以数字形式简洁地表示人类行为复杂性的知识。

This NSF National Robotics Initiative project will investigate the potential of a cycle of observation, inference and intervention by co-robot systems to enhance students' affective states and improve their performance on engineering laboratory tasks. Co-robots are robots that work side-by-side with humans, assisting them and adapting to their needs. The two-way exchange of knowledge between students and co-robots creates a reciprocal relationship, in which each party learns from the other in service of a common goal. Affective states, such as frustration and engagement, play a major role in students' performance on everyday learning tasks. A student who is overly stressed or distracted may commit errors that would be otherwise easy to avoid. A co-robot system that is cognizant of students' affective states can intervene to prevent these errors. The results of this project may provide a template for skill-based instruction on topics well beyond engineering. Currently, such learning requires extensive interactions between a student and an instructor, with the instructor providing intensive feedback at all times. In many cases, personality mismatches or other issues between instructor and student can lead to frustration, learning difficulties, and eventual dropout. Furthermore, one-on-one learning is limited by scalability challenges, as an increase in the number of students, without a proportional increase in trained instructors, can result in decreases in quality and quantity of instructor time allocated to each student. Co-robot learning systems will be able to mitigate these challenges by providing both real time and scalable feedback systems that adapt to the individual needs of students and help to minimize the amount of human instructor time required by each student. This research will acquire facial, auditory, and body gesture data from students using the integrated visual, audio and depth sensory system of the co-robot. The system will make statistical inferences of students' affective states, based on machine learning classification of facial and body language data. Visual feedback will be used to present students with interventions (visual instructions and commentary) intended to enhance their affective state and improve their performance on laboratory tasks. The project will assess the impact of co-robots' ability to improve students' affective states and enhance students' performance on laboratory tasks over repeated iterations of learning and testing. This project will lead to a better understanding of how students interact and function during potentially stressful laboratory activities. The co-robot systems proposed in this work will help discover the correlations that exists between students' affect and task performance. Co-robots will actively adapt to the manner in which students learn complex engineering tasks and the affective states that accompany that learning. Co-robot systems that predict the effectiveness of specific intervention strategies for each student and situation will lead to individually-tailored student feedback that serves both students and instructors towards enhancing student performance over time. This proposal advances the impact of co-robots into educational research and practice and extends knowledge of how to succinctly represent the complexities of human behavior in digital form.