NRI: FND: Robust Learning of Sequential Motion from Human Demonstrations to Enable Robot-Guided Exercise Training

治疗性锻炼对于健康生活和从受伤、手术、疾病或虚弱中有效恢复至关重要。物理或职业治疗师通常负责指导治疗练习。目前这些服务的供需不匹配。预计到2025年，全国将短缺26,600名物理治疗师，到2030年将短缺50,000名职业治疗师。以技术为基础的家庭项目正迅速成为解决熟练劳动力短缺的一种方式。技术辅助的锻炼计划促进高度结构化的练习，并提供实时反馈，被认为可以改善健康，但尚未被设想出来。这个项目通过设计智能机器人来弥补这一差距，这些机器人可以在治疗性运动训练中扮演治疗师的角色。这个想法是，临床医生将通过演示教机器人任何有组织的练习，然后机器人将扮演教练的角色，教用户并提供性能的定量评估。要让机器人做到这一点，我们需要一种智能算法，让治疗师可以教机器人任何新的练习，而不需要对机器人进行编程，并使机器人能够从治疗师的演示中学习。该项目将开发一种新颖的从演示中学习（LfD）框架来实现运动训练机器人。为健身教练机器人设计LfD框架的核心技术挑战是：i)从非专业用户的演示中稳健地学习人类运动序列，同时适应个人之间和个人内部的变化；ii)提供一个定量指标，以一种有上下文意义的方式解释用户轨迹与演示序列的偏差。解决这些挑战需要我们当前学习运动轨迹（低级策略学习）的方式发生重大变化，并对学习顺序任务的轨迹之间的关系进行建模（高级策略学习）。因此，本研究将仅基于运动的运动学和动力学变量来设计LfD的整个管道。LfD框架的核心是用于学习任务轨迹的相空间模型（PSM）。PSM利用动态系统理论分析运动变量来分割任务轨迹，并建立一个对时空变化具有鲁棒性的参数表示。图形模型使用PSM生成的紧凑参数集来学习演示任务背后的高级策略，同时利用人类四肢的典型解剖约束。使用相同的参数集来设计定量度量来评估学习结果。该项目将在一系列用户研究中评估由该LfD框架驱动的协作机器人运动教练的保真度，以教授上肢运动。ABB YuMi机器人将被用作测试平台。演示数据将从学生治疗师佩戴在手、前臂、上臂和躯干上的惯性测量单元（imu）收集。机器人将向年长的成年参与者（OA）演示学习过的练习，然后他们将模仿机器人进行练习。在培训阶段，oa也将佩戴imu，以便他们的表现可以根据治疗师的原始演示进行评估。运动传输的保真度将通过高速3D运动捕捉视频系统进行测试，从治疗师到机器人，再到患者，这是运动学分析的黄金标准。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Therapeutic exercises are crucial for healthy living and effective recovery from injury, surgery, disease, or frailty. Physical or occupational therapists are typically responsible for directing therapeutic exercises. There is currently a mismatch between supply and demand for these services. There is a predicted shortage of 26,600 physical therapists nationally by year 2025 and 50,000 occupational therapists by year 2030. Technology-based home programs are rapidly emerging as a way to combat this skilled labor shortage. Technology assisted exercise programs that promote highly structured practice and provide real-time feedback are believed to improve well-being but have yet to be conceived. This project bridges that gap through designing intelligent robots that can take the role of a therapist during therapeutic exercise training. The idea is that a clinician will teach a robot any structured exercise through demonstrations and the robot will then take the role of a coach to teach users and provide quantitative evaluation of performance. For a robot to do that, we need an intelligent algorithm that will allow the therapists to teach a robot any new exercise without actually programming the robot and enable the robot to learn from therapists' demonstrations. This project will develop a novel Learning from demonstration (LfD) framework to realize exercise trainer robots. The core technical challenges of designing a LfD framework for a exercise trainer robot are i) robustly learning sequence of human movements from lay users' demonstrations while accommodating inter- and intra-personal variations and ii) offers a quantitative metric to explain the deviation of a user's trajectory from the demonstrated sequence in a contextually meaningful way. Solving these challenges requires major changes in the way we currently learn motion trajectories (low-level policy learning) and model the relations among trajectories for learning sequential tasks (high-level policy learning). Accordingly, this research will design the entire pipeline of LfD based only on the kinematic and kinetic variables of motion. The core of this LfD framework is a phase space model (PSM) for learning task trajectories. The PSM leverages dynamic system theories to analyze motion variables to segment a task trajectory and build a parametric representation that is robust against spatio-temporal variations. The compact parameter set that PSM generates are used by a graphical model to learn the high-level policy underlying the demonstrated task while leveraging the typical anatomical constraints of human limbs. The same parameter set is used to design a quantitative metric to evaluate the learning outcome. The project will evaluate the fidelity of a co-robot exercise trainer powered by this LfD framework to teach upper extremity exercises in a series of user studies. An ABB YuMi robot will be used as the test platform. The demonstration data will be collected from inertial measurement units (IMUs) worn by student-therapists on the hand, forearm, upper arm and torso. The robot will demonstrate learned exercises to older adult participants (OA), who then will perform the exercises by mirroring the robot. During the training phase, the OAs will also be wearing IMUs so that their performance can be assessed with respect to the original demonstration from the therapist. The fidelity of movement transmission will be tested from the therapist, to the robot, to the patient with a high-speed, 3D motion capture video system which is the gold standard for kinematic analysis.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.