RUI: Improving motor learning and rehabilitation via experimental bidirectional dynamic human-virtual reality interaction system

这项拨款支持的研究产生了人类和虚拟现实系统如何相互作用的新知识，以帮助学习身体康复的运动技能。 当人类戴上虚拟现实耳机时，他们被传送到一个沉浸式的数字世界，可以是任何东西或任何地方。虚拟现实系统的最新进展使这些数字世界能够密切模仿现实世界-为训练康复技能开辟了许多可能性。然而，为了充分实现这一潜力，需要更好地理解沉浸式虚拟环境的组成部分，这些组成部分可以转化为现实世界的运动技能发展。相应地，该奖项支持与虚拟现实环境中学习相关的人类运动和体验的基础研究，以最适合他们的学习需求。目前研究的结果将有利于美国经济和社会，因为许多学校和工作场所正在使用虚拟现实系统为人们提供复杂运动技能的可扩展培训。为了正确研究虚拟世界中运动学习的复杂性，该项目涉及多个学科的研究人员，包括机械工程，运动机能学，心理学和视觉科学。此外，该项目的多学科方法有助于扩大代表性不足群体的参与，沿着帮助弥合工程和社会科学研究领域之间的差距。 该项目中的多学科研究团队将开发两个动态的人-虚拟现实交互系统-一个涉及在虚拟世界中行走，同时避开障碍物，另一个用脚控制器控制假肢和手。每个系统都包括一个虚拟现实头戴式耳机，该头戴式耳机具有根据用户的技能水平进行调整的战略性设计的自适应环境，以及其他子系统，用于记录重要的用户数据，例如步态，眼睛注视行为和脚趾敲击事件（同时控制假肢）。开发的系统依赖于各种传感器，如眼动仪，3D运动捕捉系统，脚控制器鞋垫，可以完全捕捉用户的生物力学和他们在执行指定任务的策略。分析收集的数据将有助于更清楚地了解在运动学习或康复任务中人类与虚拟现实系统之间的双向关系-人类提高他们的技能，VR系统适应用户需求。该研究将通过展示虚拟现实通过设计的系统改善学习和康复的潜力，填补人类-虚拟现实交互研究的知识空白。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

This grant supports research that generates new knowledge of how humans and virtual reality systems interact to help learn motor skills for physical rehabilitation. When humans put on a virtual reality headset, they are transported into an immersive digital world that can be anything or anywhere. Recent advancements of virtual reality systems make it so that these digital worlds closely mimic the real-world -- opening up many possibilities to train rehabilitative skills. However, in order for this potential to be fully realized, a better understanding of the components of immersive virtual environments that translate to real-world motor skill development is needed. Correspondingly, this award supports the fundamental research of human movements and experiences linked to learning in virtual reality environments that best fit their learning needs. Findings from the current study will benefit the U.S. economy and society as many schools and workplaces are using virtual reality systems to provide people with scalable training of complex motor skills. To properly examine the complexities of motor learning in virtual worlds, this project involves researchers from several disciplines including mechanical engineering, kinesiology, psychology, and vision science. Moreover, the multi-disciplinary approach of this project helps broaden participation of underrepresented groups, along with aids to bridge gaps between engineering and social science research fields. The multidisciplinary research team in this project will develop two dynamic human-virtual reality interaction systems—one involving walking through a virtual world while avoiding obstacles, and the other controlling a prosthetic arm and hand with a foot controller. Each system includes a virtual reality headset with strategically designed adaptive environments that scale to the skill level of the user, and other subsystems to record vital user data such as gait, eye gaze behaviors, and toe tapping events (while controlling the prosthesis). The developed systems rely on various sensors, such as eye trackers, 3D motion capture systems, foot controller insoles, that can fully capture the biomechanics of the users and their strategies in performing the assigned tasks. Analyzing the collected data will lead to clearer understanding of the bidirectional relationships between humans and the virtual reality systems during motor learning or rehabilitation tasks—humans enhancing their skills, and the VR system adapting to the user needs. This research will fill a knowledge gap in the human-virtual reality interaction research by demonstrating the potential of virtual reality to improve learning and rehabilitation through the designed systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.