A Scalable and Accessible System for Automated Coaching of Human Motion

这项拟议的工作将开发一个自动教授手语、舞蹈和人-机器人协作等身体动作的系统。这种动作技能支持教育和广泛的可及性，可以通过创造性的表达和身体健康来增强幸福感，并使新的工作形式成为可能。教授动作通常需要人类教练的密集努力：教练演示动作，评估学习者的表现，并为练习提供反馈和量身定做的计划，以提高流利性。拟议的工作旨在更好地了解如何开发系统，自行设计和执行指导过程的这些组成部分。这样的系统将使那些不能不间断地接触到专门的人类教练的人更容易学习基于运动的技能。该项目的更广泛影响还包括培养心理学、认知科学、教育学和计算机科学交叉领域的下一代科学家。该工作的目标是将自动教学代理的能力扩展到身体运动领域，并将反映学习者认知和情感状态的信号纳入系统。使用预先记录的专家动作例子，以及学习者感知到的动作和姿势，该系统将识别教师和学习者之间的定性和定量差异。低水平的错误将被跟踪，以建立一个不断演变的认知模型，衡量学习者对该过程的舒适度，以及对技能和技能组合的掌握程度。该项目将开发使用该模型的计算工具，以确定哪种反馈可能最有效地提高学习者的表现。该项目还将开发和评估提供这种反馈的硬件和软件平台，并提供音频、视觉或触觉形式的提示。为了使该系统尽可能容易访问，该项目将评估低成本和无处不在的传感方法，包括用于传感的网络摄像头，以及从互联网视频中解析的专家演示。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The proposed work will develop a system for teaching physical motions such as sign language, dance, and human-robot collaboration automatically. Such motion skills support education and broad accessibility, can enhance well-being through creative expression and physical fitness, and enable new forms of work. Teaching motion often makes use of the intensive efforts of a human coach: the coach demonstrates a motion, evaluates the learner’s performance, and provides feedback and a tailored plan for practice to improve fluency. The proposed work aims to better understand how to develop systems that devise and carry out these components of the coaching process on their own. Such systems would make learning motion-based skills more accessible for those who do not have uninterrupted access to a dedicated human coach. The broader impact of the project also includes the training of the next generation of scientists at the intersection of psychology, cognitive science, education, and computer science.The goal of the work is to expand the capabilities of automatic teaching agents into the domain of physical motion, and to incorporate signals reflecting cognitive and emotional states of the learner into the system. Using pre-recorded motion examples from experts, and sensed actions and poses of the learner, this system will identify qualitative and quantitative differences between the teacher and learner. Low-level errors will be tracked to build an evolving cognitive model that measures the learner’s level of comfort with the process, as well as mastery of skills and combinations of skills. The project will develop computational tools that use this model to determine which feedback may be most effective to improve the learner’s performance. The project will also develop and evaluate hardware and software platforms that provide this feedback, with cues that may be presented in audio, visual, or tactile forms. To make the system as accessible as possible, the project will evaluate low-cost and ubiquitous approaches to sensing, including web cameras for sensing, and expert demonstrations parsed from internet videos.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.