A Scalable and Accessible System for Automated Coaching of Human Motion

拟议的工作将开发一个系统，用于自动教授身体动作，如手语，舞蹈和人机协作。这种运动技能支持教育和广泛的可获得性，可以通过创造性表达和身体健康来提高福祉，并实现新的工作形式。教学动作通常利用人类教练的密集努力：教练演示动作，评估学习者的表现，并提供反馈和定制的练习计划以提高流畅性。拟议的工作旨在更好地了解如何开发系统，设计和执行这些组件的辅导过程中自己。这样的系统将使那些无法不间断地获得专门的人类教练的人更容易学习基于动作的技能。该项目的更广泛影响还包括培养心理学、认知科学、教育学和计算机科学交叉领域的下一代科学家。该工作的目标是将自动教学代理的能力扩展到物理运动领域，并将反映学习者认知和情感状态的信号纳入系统。 使用来自专家的预先记录的动作示例，以及感测到的学习者的动作和姿势，该系统将识别教师和学习者之间的定性和定量差异。低水平的错误将被跟踪，以建立一个不断发展的认知模型，衡量学习者的舒适度与过程，以及掌握的技能和技能的组合。该项目将开发计算工具，使用这个模型来确定哪些反馈可能是最有效的，以提高学习者的表现。该项目还将开发和评估提供这种反馈的硬件和软件平台，并以音频，视觉或触觉形式提供提示。为了使该系统尽可能容易使用，该项目将评估低成本和无处不在的传感方法，包括用于传感的网络摄像头，以及从互联网视频中解析的专家演示。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The proposed work will develop a system for teaching physical motions such as sign language, dance, and human-robot collaboration automatically. Such motion skills support education and broad accessibility, can enhance well-being through creative expression and physical fitness, and enable new forms of work. Teaching motion often makes use of the intensive efforts of a human coach: the coach demonstrates a motion, evaluates the learner’s performance, and provides feedback and a tailored plan for practice to improve fluency. The proposed work aims to better understand how to develop systems that devise and carry out these components of the coaching process on their own. Such systems would make learning motion-based skills more accessible for those who do not have uninterrupted access to a dedicated human coach. The broader impact of the project also includes the training of the next generation of scientists at the intersection of psychology, cognitive science, education, and computer science.The goal of the work is to expand the capabilities of automatic teaching agents into the domain of physical motion, and to incorporate signals reflecting cognitive and emotional states of the learner into the system. Using pre-recorded motion examples from experts, and sensed actions and poses of the learner, this system will identify qualitative and quantitative differences between the teacher and learner. Low-level errors will be tracked to build an evolving cognitive model that measures the learner’s level of comfort with the process, as well as mastery of skills and combinations of skills. The project will develop computational tools that use this model to determine which feedback may be most effective to improve the learner’s performance. The project will also develop and evaluate hardware and software platforms that provide this feedback, with cues that may be presented in audio, visual, or tactile forms. To make the system as accessible as possible, the project will evaluate low-cost and ubiquitous approaches to sensing, including web cameras for sensing, and expert demonstrations parsed from internet videos.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.