II-EN: Acquisition of Sensors and Displays for Research on Motion Synthesis and Rehabilitation

实时动作捕捉和交互式3D计算机生成环境正在迅速崛起，作为一个集成和强大的人机交互环境，有能力在许多领域进行革命性的应用。传感器和显示技术的最新进展现在可以在低成本的消费者导向和高端工业导向的以人为中心的计算机系统中看到。该提案旨在通过加强和扩大加州大学默塞德分校现有的研究基础设施来支持和加强新的研究和应用领域的研究和教育活动，特别是通过收购(a)高端数据手套，(b)无遮挡运动捕捉服，以及(c)集成Kinect传感器的便携式交互式显示器来增强现有的可视化和运动捕捉设施。拟议的设备增强将允许在以下主题上开发新的研究项目：(a)具有协调的手臂运动的手势合成模型，(b)用于上肢运动康复评估和远程治疗交付的远程虚拟协作会议，(c)针对给定运动协议的手部运动康复的自动进度监测，以及(d)在狭窄空间中用于训练应用的类人全身运动规划。这些项目将以互动式培训和治疗应用为目标，这些应用将受益于位于虚拟环境中的新型人机界面。来自计算机科学和认知科学的智力梅里坦跨学科团队将探索新的以人为中心的计算研究，重点是在物理训练和治疗期间并行使用的通信渠道。提议的便携式和无遮挡的高端运动传感设备将允许开发类似人类的运动模型，用于手势合成以及全身任务导向运动。这些模型将在沉浸式远程交互过程中发现新的交互范式，这将涉及自主合成的类人动作和将被捕获并流回系统的人类动作。将开发一个手势分析工具来提供算法基础设施，以自动分析和提取手势的组成部分。基础设施将允许在实际实践中对康复和治疗进行评估。科学和实践研究项目将提高虚拟人在各种教育，培训和治疗应用中的有效性，并将有助于使普通用户更容易获得这种3D计算机生成的环境。更广泛的影响基础设施将支持与运动康复相关的研究项目，这有可能使许多需要物理治疗的美国人受益，比如背部、颈部和肩部的疾病。这项研究将促进对使用虚拟人进行广泛的实际培训和治疗应用的原则方法的理解。仪器和研究将为研究生和本科生提供独特的学习机会。研究成果将在认知科学、计算机科学和康复治疗出版物中广泛传播。基础设施和研究将向当地高中和社区大学的学生展示，以激发他们对科学和工程的兴趣。

Real-time motion-capture and interactive 3D computer-generated environments are rapidly emerging as an integrated and powerful human-computer interaction context with the ability to revolutionize applications in many areas. Recent advances in sensor and display technologies can now be seen in both low-cost consumer-oriented and in high-end industrial-oriented human-centered computer systems. This proposal aims to support and enhance research and educational activities in new research and application areas by strengthening and expanding the available research infrastructure at the University of California - Merced, specifically by enhancing an existing visualization and motion capture facility with the acquisition of (a) a high-end data glove, (b) an occlusion-free motion capture suit, and (c) portable interactive displays integrated with Kinect sensors. The proposed equipment enhancement will allow the development of new research projects on the following topics: (a) gesture synthesis models with coordinated hand-arm motions, (b) remote virtual collaborative sessions for upper body motion rehabilitation assessment and remote therapy delivery, (c) automated progress monitoring of hand motor rehabilitation in respect to given motion protocols, and (d) human-like full-body motion planning in tight spaces for training applications. The projects will target interactive training and therapy applications that can benefit from new human-computer interfaces that are situated in virtual environments.Intellectual MeritAn interdisciplinary team from computer science and cognitive science will explore novel human-centered computing research with a focus on the channels of communication that are used in parallel during physical training and therapy. The proposed portable and occlusion-free high-end motion sensing equipment will allow the development of human-like motion models for synthesis of gestures as well as full-body task-oriented motions. These models will enable, during immersive remote interactions, the discovery of new interaction paradigms, which will involve autonomously synthesized human-like motions and human motions that will be captured and streamed back to the system. A gesture analysis tool will be developed to provide algorithmic infrastructure to automatically analyze and extract the components of the gestures. The infrastructure will permit rehabilitation and therapy to be evaluated in actual practice. The scientific and practical research projects will advance the effectiveness of virtual humans in diverse educational, training, and therapeutic applications, and will help make such 3D computer-generated environments more accessible to the common user.Broader ImpactThe infrastructure will support research projects that relate to motion rehabilitation, which has the potential to benefit the many Americans who are in need of physical therapy such as for ailments in the back, neck, and shoulders. The research will advance an understanding of a principled approach for using virtual humans for a broad range of practical training and therapy applications. The instrumentation and research will provide unique learning opportunities for graduate and undergraduate students. The research outcomes will be disseminated broadly in cognitive science, computer science, and rehabilitation therapy publications. The infrastructure and research will be demonstrated to local high school and community college students to motivate interest in science and engineering.