NRI: FND: Robust Learning of Sequential Motion from Human Demonstrations to Enable Robot-Guided Exercise Training

治疗性锻炼对于健康的生活和有效地从受伤、手术、疾病或虚弱中恢复至关重要。物理或职业治疗师通常负责指导治疗练习。目前，这些服务的供需之间存在错配。据预测，到2025年，全国将缺少2.66万名物理治疗师，到2030年，全国将缺少5万名职业治疗师。以技术为基础的家庭项目正在迅速崛起，作为解决熟练劳动力短缺的一种方式。促进高度结构化练习并提供实时反馈的技术辅助锻炼计划被认为可以改善幸福感，但尚未被构思出来。这个项目通过设计智能机器人来弥合这一差距，这些机器人可以在治疗运动训练中扮演治疗师的角色。其想法是，临床医生将通过演示教授机器人任何结构化的练习，然后机器人将扮演教练的角色，教授用户并提供性能的量化评估。要让机器人做到这一点，我们需要一种智能算法，允许治疗师在不对机器人进行实际编程的情况下教机器人任何新的练习，并使机器人能够从治疗师的演示中学习。该项目将开发一种新的从演示中学习(LFD)框架来实现运动训练器机器人。为运动教练机器人设计LFD框架的核心技术挑战是：i)从非专业用户的演示中稳健地学习人类动作序列，同时适应人与人之间和人内的变化；ii)提供一种量化度量来解释用户轨迹与演示序列之间的偏差。解决这些挑战需要对我们目前学习运动轨迹的方式进行重大改变(低级策略学习)，并为学习连续任务的轨迹之间的关系建模(高级策略学习)。因此，本研究将仅根据运动的运动学和运动学变量来设计LFD的整个管道。LFD框架的核心是用于学习任务轨迹的相空间模型(PSM)。PSM利用动态系统理论来分析运动变量，以分割任务轨迹，并构建对时空变化具有鲁棒性的参数表示。图形模型使用PSM生成的紧凑参数集来学习作为所演示任务的基础的高级策略，同时利用人体肢体的典型解剖约束。使用相同的参数集来设计量化度量来评估学习结果。该项目将评估由该LFD框架驱动的协作机器人运动训练器在一系列用户研究中教授上肢运动的保真度。ABB YuMi机器人将作为测试平台。演示数据将从学生治疗师佩戴在手、前臂、上臂和躯干上的惯性测量单元(IMU)收集。机器人将向年长的成年人参与者(OA)演示学习过的练习，然后他们将通过镜像机器人来执行练习。在培训阶段，美洲国家组织还将佩戴IMUS，以便根据治疗师的原始演示来评估他们的表现。运动传输的保真度将通过高速3D运动捕捉视频系统测试，从治疗师到机器人，再到患者，这是运动学分析的黄金标准。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Therapeutic exercises are crucial for healthy living and effective recovery from injury, surgery, disease, or frailty. Physical or occupational therapists are typically responsible for directing therapeutic exercises. There is currently a mismatch between supply and demand for these services. There is a predicted shortage of 26,600 physical therapists nationally by year 2025 and 50,000 occupational therapists by year 2030. Technology-based home programs are rapidly emerging as a way to combat this skilled labor shortage. Technology assisted exercise programs that promote highly structured practice and provide real-time feedback are believed to improve well-being but have yet to be conceived. This project bridges that gap through designing intelligent robots that can take the role of a therapist during therapeutic exercise training. The idea is that a clinician will teach a robot any structured exercise through demonstrations and the robot will then take the role of a coach to teach users and provide quantitative evaluation of performance. For a robot to do that, we need an intelligent algorithm that will allow the therapists to teach a robot any new exercise without actually programming the robot and enable the robot to learn from therapists' demonstrations. This project will develop a novel Learning from demonstration (LfD) framework to realize exercise trainer robots. The core technical challenges of designing a LfD framework for a exercise trainer robot are i) robustly learning sequence of human movements from lay users' demonstrations while accommodating inter- and intra-personal variations and ii) offers a quantitative metric to explain the deviation of a user's trajectory from the demonstrated sequence in a contextually meaningful way. Solving these challenges requires major changes in the way we currently learn motion trajectories (low-level policy learning) and model the relations among trajectories for learning sequential tasks (high-level policy learning). Accordingly, this research will design the entire pipeline of LfD based only on the kinematic and kinetic variables of motion. The core of this LfD framework is a phase space model (PSM) for learning task trajectories. The PSM leverages dynamic system theories to analyze motion variables to segment a task trajectory and build a parametric representation that is robust against spatio-temporal variations. The compact parameter set that PSM generates are used by a graphical model to learn the high-level policy underlying the demonstrated task while leveraging the typical anatomical constraints of human limbs. The same parameter set is used to design a quantitative metric to evaluate the learning outcome. The project will evaluate the fidelity of a co-robot exercise trainer powered by this LfD framework to teach upper extremity exercises in a series of user studies. An ABB YuMi robot will be used as the test platform. The demonstration data will be collected from inertial measurement units (IMUs) worn by student-therapists on the hand, forearm, upper arm and torso. The robot will demonstrate learned exercises to older adult participants (OA), who then will perform the exercises by mirroring the robot. During the training phase, the OAs will also be wearing IMUs so that their performance can be assessed with respect to the original demonstration from the therapist. The fidelity of movement transmission will be tested from the therapist, to the robot, to the patient with a high-speed, 3D motion capture video system which is the gold standard for kinematic analysis.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.