MRI: Development of the Collaborative-Research Augmented Immersive Virtual Environment Laboratory (CRAIVE-Lab)

提案#：12-29391PI(S)：Braasch，Jonas；Chang，Ben；Cutler，Barbra M.；Goebel，Johannes；Radke，RichardInstitution：伦斯勒理工学院标题：磁共振成像/开发：协作研究增强沉浸式虚拟环境实验室(CRAIVE-Lab)项目建议：该项目开发一个专门的虚拟现实(VR)系统，引入了一个沉浸式虚拟环境的新概念，该环境针对协作小组活动进行了优化。该系统以同等的优先级处理听觉和视觉感觉，目前只有头盔显示系统支持这一点。与后者不同，该系统还支持参与者之间的直接通信，而不需要电信设备。S触觉地板显示器可用于重建地板振动(例如，模拟音乐会舞台)，其精度目前仅为较小的显示器(如汽车模拟器)所知。该工作解决了对专门的虚拟现实(VR)系统的需求，以研究和启用通信驱动的任务，使用户组沉浸在位于同一物理空间的高保真多模式环境中。虽然目前的多模式VR系统已经实现了高度的真实感，但它们要么专注于单个或非常小的用户组的沉浸，要么专注于在电影院式的环境中向更大的用户组呈现素材。在这两种情况下，这些系统都提供了均匀的视觉和声学领域。对于群组交流任务，提供个性化视听视角或每个用户的异质场可以更好地从VR系统中获取相关信息-S显示，同时增加交互任务的体验感和感知真实感。该项目解决了需要克服的技术障碍，以建立一个大型(18mx12mx4.3米)、多用户、多视角、多模型的显示器。对于视觉领域，将使用多种汇聚点渲染技术在7投影仪显示器上(重新)创建场景。在声学领域，将设计一个192个扬声器通道的系统，用于波场合成(WFS)，并支持高阶双声(HoA)声投影来呈现不均匀的声场。由16个平台元件组成的触觉显示器将用于模拟地板振动，并为其他振动物体(例如手持设备)提供基础设施。智能位置跟踪系统估计当前用户位置和头部方向，以及其他对象的定位数据。对于跟踪系统，将使用混合视觉/声音传感器系统来模拟人类依赖不同模式提取稳健信息的活动。广泛影响：随着研究人类感知的新工具的出现，该系统将使研究能够探索新的多模式、多用户数据显示，战略性地利用人类整合跨模型感觉信息的能力。它还将成为研究人类与类人智能系统之间互动的平台，这些系统可以模拟不同复杂程度的环境。该仪器支持包括为残疾人开发新接口的研究。CRAIVE系统将是三个全校中心的综合设施，即认知、交流和文化中心(CCC)、CCNI超级计算机和实验媒体和表演艺术中心(EMPC)，充分发挥它们在该领域合作研究的潜力，并作为从事其中一个已启用研究领域的学生的培训平台。

Proposal #: 12-29391PI(s): Braasch, Jonas; Chang, Ben; Cutler, Barbra M.; Goebel, Johannes; Radke, RichardInstitution: Rensselaer Polytechnic Institute Title: MRI/Dev.: Collaborative-Research Augmented Immersive Virtual Environment Laboratory (CRAIVE-Lab)Project Proposed:This project, developing a specialized virtual-reality (VR) system, introduces a new concept for an immersive virtual environment which is optimized for collaborative group activities. The system addresses the auditory and visual senses with equal priority, which is currently only championed by head-mounted display systems. Unlike the latter, this system also enables direct communication between participants without the need for telecommunication devices. The system?s haptic floor display can be used to recreate floor vibrations (e.g., simulate concert stages) with an accuracy that is currently only known for smaller displays (e.g., car simulators).The work addresses the need for specialized virtual reality (VR) system for the study and enablement of communication-driven tasks with groups of users immersed in a high-fidelity multi-modal environment located in the same physical space. While current multi-modal VR systems have achieved a high degree of realism, they either focus on the immersion of a single or very small group of users or on presenting material to a larger group of users in a cinema-type environment. In both cases, the systems provide homogeneous visual and acoustic fields. For group communication tasks, inhomogeneous fields that provide personalized visual and acoustic perspectives or each user, could provide better access to relevant information from the VR system?s display and at the same time increase the experiential degree of presence and perceived realism for interactive tasks. The project addresses the technical hurdles that need to be surmounted to establish a large-scale (18mx12mx4.3m), muti-user, muti-perspective, muti-model display. For visual domain, multiple point-of-convergence rendering techniques will be used to (re-)create scenes on a 7-projector display.. For the acoustic domain, a 192-loudspeaker-channel system will be designed for Wave-Field Synthesis (WFS) with the support of Higher-Order-Ambisonic (HoA) sound projection to render inhomogeneous acoustic fields. A haptic display, consisting of 16 platforms elements, will be used to simulate floor vibrations and also to provide infrastructure for other vibrating objects (e.g., handheld devices). An intelligent position-tracking system estimates current user positions and head orientations as well as positioning data for other objects. For the tracking system, a hybrid visual/acoustic sensor system will be used to emulate the human activity of extracting robust information by relying on different modalities.Broader Impacts: With new tools to study human perception, the system will enable research to explore new multi-modal, multi-user data displays that strategically utilize human ability to integrate cross-model sensory information. It will also serve as a platform to study interactions between humans and humanoid intelligent systems that can simulate environments at different degrees of complexity. The instrumentation supports research that will include the development of new interfaces for people with disabilities. The CRAIVE system will be an integrative facility for three school-wide centers, the center for Cognition, Communication, and Culture (CCC), the CCNI supercomputer, and the Experimental Media and Performing Arts Center (EMPC) to their full potential for collaborative research in this area, and to serve as a training platform for students who will engage in one of the enabled research areas.