NRI: Real Time Observation, Inference and Intervention of Co-Robot Systems Towards Individually Customized Performance Feedback Based on Students' Affective States

NRI：协作机器人系统的实时观察、推理和干预，以实现基于学生情感状态的个性化定制表现反馈

• 批准号: 1527148
• 负责人: Conrad Tucker
• 金额: $ 34.26万
• 依托单位: Pennsylvania State Univ University Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2018-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1527148&HistoricalAwards=false
• 关键词: NRI; Real; Time; Observation; Inference

This NSF National Robotics Initiative project will investigate the potential of a cycle of observation, inference and intervention by co-robot systems to enhance students' affective states and improve their performance on engineering laboratory tasks. Co-robots are robots that work side-by-side with humans, assisting them and adapting to their needs. The two-way exchange of knowledge between students and co-robots creates a reciprocal relationship, in which each party learns from the other in service of a common goal. Affective states, such as frustration and engagement, play a major role in students' performance on everyday learning tasks. A student who is overly stressed or distracted may commit errors that would be otherwise easy to avoid. A co-robot system that is cognizant of students' affective states can intervene to prevent these errors. The results of this project may provide a template for skill-based instruction on topics well beyond engineering. Currently, such learning requires extensive interactions between a student and an instructor, with the instructor providing intensive feedback at all times. In many cases, personality mismatches or other issues between instructor and student can lead to frustration, learning difficulties, and eventual dropout. Furthermore, one-on-one learning is limited by scalability challenges, as an increase in the number of students, without a proportional increase in trained instructors, can result in decreases in quality and quantity of instructor time allocated to each student. Co-robot learning systems will be able to mitigate these challenges by providing both real time and scalable feedback systems that adapt to the individual needs of students and help to minimize the amount of human instructor time required by each student. This research will acquire facial, auditory, and body gesture data from students using the integrated visual, audio and depth sensory system of the co-robot. The system will make statistical inferences of students' affective states, based on machine learning classification of facial and body language data. Visual feedback will be used to present students with interventions (visual instructions and commentary) intended to enhance their affective state and improve their performance on laboratory tasks. The project will assess the impact of co-robots' ability to improve students' affective states and enhance students' performance on laboratory tasks over repeated iterations of learning and testing. This project will lead to a better understanding of how students interact and function during potentially stressful laboratory activities. The co-robot systems proposed in this work will help discover the correlations that exists between students' affect and task performance. Co-robots will actively adapt to the manner in which students learn complex engineering tasks and the affective states that accompany that learning. Co-robot systems that predict the effectiveness of specific intervention strategies for each student and situation will lead to individually-tailored student feedback that serves both students and instructors towards enhancing student performance over time. This proposal advances the impact of co-robots into educational research and practice and extends knowledge of how to succinctly represent the complexities of human behavior in digital form.

这个NSF国家机器人计划项目将研究协作机器人系统的观察、推理和干预循环的潜力，以增强学生的情感状态，提高他们在工程实验室任务中的表现。协作机器人是指与人类并肩工作、协助人类并适应人类需求的机器人。学生和协作机器人之间的双向知识交流创造了一种互惠关系，在这种关系中，每一方都为了共同的目标向对方学习。情感状态，如沮丧和投入，在学生的日常学习任务中发挥着重要作用。压力过大或注意力不集中的学生可能会犯一些本来很容易避免的错误。一个能够识别学生情感状态的协作机器人系统可以干预以防止这些错误。这个项目的结果可能会提供一个模板，以技能为基础的指导主题远远超出工程。目前，这种学习需要学生和教师之间广泛的互动，教师随时提供密集的反馈。在许多情况下，教师和学生之间的性格不匹配或其他问题可能导致挫折，学习困难，最终辍学。此外，一对一的学习受到可扩展性挑战的限制，因为学生数量的增加，而没有训练有素的教师的比例增加，可能导致分配给每个学生的教师时间的质量和数量的下降。协作机器人学习系统将能够通过提供实时和可扩展的反馈系统来缓解这些挑战，这些反馈系统可以适应学生的个性化需求，并有助于最大限度地减少每个学生所需的人工指导时间。这项研究将利用协作机器人的集成视觉、音频和深度感知系统，从学生身上获取面部、听觉和身体手势数据。该系统将基于面部和肢体语言数据的机器学习分类，对学生的情感状态进行统计推断。视觉反馈将用于向学生展示干预措施（视觉指导和评论），旨在增强他们的情感状态，提高他们在实验室任务中的表现。该项目将评估协作机器人改善学生情感状态的能力，并通过反复的学习和测试提高学生在实验室任务中的表现。这个项目将有助于更好地了解学生在潜在压力的实验室活动中如何互动和发挥作用。在这项工作中提出的协作机器人系统将有助于发现学生的情感和任务绩效之间存在的相关性。协作机器人将积极适应学生学习复杂工程任务的方式以及伴随学习的情感状态。协作机器人系统预测每个学生和情况的具体干预策略的有效性，将导致个性化的学生反馈，为学生和教师提供服务，以提高学生的表现。该提案推进了协作机器人对教育研究和实践的影响，并扩展了如何以数字形式简洁地表示人类行为复杂性的知识。