SCH: INT: Collaborative Research: Replicating Clinic Physical Therapy at Home: Touch, Depth, and Epidermal Electronics in an Interactive Avatar System

物理治疗往往受到缺乏接触治疗师的机会和缺乏坚持家庭治疗方案的阻碍。本研究开发了一种家庭使用的物理治疗辅助系统，重点是中风康复。当一个人锻炼时，廉价的相机观察颜色和深度，不显眼的纹身传感器监测详细的肌肉活动。导出3D运动轨迹，并与专家治疗师完成的练习进行比较。患者在屏幕上观看虚拟形象，其中箭头和颜色编码指导患者正确移动。除了运动跟踪、皮肤传感器和辅助系统等先进领域外，该项目还具有广泛影响的潜力，因为它通过与健康相关的工程课程和项目吸引妇女和代表性不足的少数民族参与工程，并且因为家庭物理治疗援助可以特别帮助农村和服务不足的人口。本项目利用生物电子学、计算机视觉、计算机游戏、高维机器学习、人为因素等，开发一套家庭物理治疗辅助系统。在家庭锻炼期间，患者的运动和生理状况由Kinect彩色/深度摄像头和可转移到带有临时纹身的皮肤上的无线表皮电子设备监测。该项目涉及优化电极设计和表皮电子的无线信号，以监测肌肉招募的时空方面，手部和身体姿势估计和跟踪算法，对快速运动和遮挡具有强大的功能，以及用于多模态传感器融合的机器学习和虚拟角色渲染算法的开发和专家训练的最优控制引导逻辑，用于云和本地使用。该系统旨在提供实时反馈，使家庭治疗与专家治疗师的办公室就诊一样有效，从而减少完全康复所需的时间和金钱。

Physical therapy is often hampered by lack of access to therapists, and lack of adherence to home therapy regimens. This research develops a physical therapy assistance system for home use, with emphasis on stroke rehabilitation. As a person exercises, inexpensive cameras observe color and depth, and unobtrusive tattoo sensors monitor detailed muscle activity. The 3D movement trajectory is derived and compared against the exercise done with an expert therapist. The patient watches a screen avatar where arrows and color coding guide the patient to move correctly. In addition to advancing fields such as movement tracking, skin sensors, and assistive systems, the project has the potential for broad impact by attracting women and under-represented minorities to engineering through health-related engineering coursework and projects, and because home physical therapy assistance can especially help rural and under-served populations. This project uses bio-electronics, computer vision, computer gaming, high-dimensional machine learning, and human factors to develop a home physical therapy assistance system. During home exercises, patient kinematics and physiology are monitored with a Kinect color/depth camera and wireless epidermal electronics transferable to the skin with a temporary tattoo. The project involves optimization of electrode design and wireless signaling for epidermal electronics to monitor spatiotemporal aspects of muscle recruitment, hand and body pose estimation and tracking algorithms that are robust to rapid motion and occlusions, and development of machine learning and avatar rendering algorithms for multi-modal sensor fusion and expert-trained optimal control guidance logic, for both cloud and local usage. The system aims to provide real-time feedback to make home sessions as effective as office visits with an expert therapist, reducing the time and money required for full recovery.