MRI: Development of the Collaborative-Research Augmented Immersive Virtual Environment Laboratory (CRAIVE-Lab)

提案#:12-29391PI(s): Braasch, Jonas；张,本;芭芭拉·m·卡特勒；Goebel约翰内斯;机构：伦斯勒理工学院职称：MRI/Dev。项目建议：该项目开发一种专门的虚拟现实（VR）系统，引入了一种新的沉浸式虚拟环境概念，该虚拟环境针对协作小组活动进行了优化。该系统以同等的优先级处理听觉和视觉感官，目前只有头戴式显示系统支持这一点。与后者不同的是，该系统还可以在不需要电信设备的情况下实现参与者之间的直接通信。这个系统吗?触觉地板显示器可用于再现地板振动（例如，模拟音乐会舞台），其精度目前仅适用于较小的显示器（例如，汽车模拟器）。这项工作解决了对专门的虚拟现实（VR）系统的需求，用于研究和实现通信驱动的任务，用户群沉浸在位于同一物理空间的高保真多模态环境中。虽然目前的多模态VR系统已经实现了高度的真实感，但它们要么专注于单个或非常小的用户群体的沉浸感，要么专注于在电影类型的环境中向更大的用户群体呈现材料。在这两种情况下，系统提供均匀的视觉和声场。对于群体交流任务，提供个性化视觉和声学视角或每个用户的非同质场是否可以更好地从VR系统获取相关信息？S显示，同时增加了交互式任务的体验度和感知真实感。该项目解决了建立大型（18mx12mx4.3m）、多用户、多视角、多模型显示所需要克服的技术障碍。对于视觉域，将使用多个收敛点渲染技术（重新）在7个投影仪显示器上创建场景。在声学领域，将设计一个192扬声器通道系统，用于波场合成（WFS），支持高阶双声（HoA）声音投影，以呈现非均匀声场。由16个平台元素组成的触觉显示器将用于模拟地板振动，并为其他振动物体（例如手持设备）提供基础设施。智能位置跟踪系统估计当前用户的位置和头部方向以及其他物体的定位数据。对于跟踪系统，将使用混合视觉/声学传感器系统来模拟人类活动，通过依赖不同的模态提取鲁棒信息。更广泛的影响：通过研究人类感知的新工具，该系统将使研究能够探索新的多模态、多用户数据显示，战略性地利用人类整合跨模型感官信息的能力。它还将作为一个平台，研究人类和类人智能系统之间的相互作用，这些系统可以模拟不同复杂程度的环境。该仪器支持的研究将包括为残疾人开发新的接口。该系统将成为三个全校范围内的中心，认知、交流和文化中心（CCC）、CCNI超级计算机和实验媒体和表演艺术中心（EMPC）的综合设施，以充分发挥其在该领域合作研究的潜力，并为将从事某一研究领域的学生提供培训平台。

Proposal #: 12-29391PI(s): Braasch, Jonas; Chang, Ben; Cutler, Barbra M.; Goebel, Johannes; Radke, RichardInstitution: Rensselaer Polytechnic Institute Title: MRI/Dev.: Collaborative-Research Augmented Immersive Virtual Environment Laboratory (CRAIVE-Lab)Project Proposed:This project, developing a specialized virtual-reality (VR) system, introduces a new concept for an immersive virtual environment which is optimized for collaborative group activities. The system addresses the auditory and visual senses with equal priority, which is currently only championed by head-mounted display systems. Unlike the latter, this system also enables direct communication between participants without the need for telecommunication devices. The system?s haptic floor display can be used to recreate floor vibrations (e.g., simulate concert stages) with an accuracy that is currently only known for smaller displays (e.g., car simulators).The work addresses the need for specialized virtual reality (VR) system for the study and enablement of communication-driven tasks with groups of users immersed in a high-fidelity multi-modal environment located in the same physical space. While current multi-modal VR systems have achieved a high degree of realism, they either focus on the immersion of a single or very small group of users or on presenting material to a larger group of users in a cinema-type environment. In both cases, the systems provide homogeneous visual and acoustic fields. For group communication tasks, inhomogeneous fields that provide personalized visual and acoustic perspectives or each user, could provide better access to relevant information from the VR system?s display and at the same time increase the experiential degree of presence and perceived realism for interactive tasks. The project addresses the technical hurdles that need to be surmounted to establish a large-scale (18mx12mx4.3m), muti-user, muti-perspective, muti-model display. For visual domain, multiple point-of-convergence rendering techniques will be used to (re-)create scenes on a 7-projector display.. For the acoustic domain, a 192-loudspeaker-channel system will be designed for Wave-Field Synthesis (WFS) with the support of Higher-Order-Ambisonic (HoA) sound projection to render inhomogeneous acoustic fields. A haptic display, consisting of 16 platforms elements, will be used to simulate floor vibrations and also to provide infrastructure for other vibrating objects (e.g., handheld devices). An intelligent position-tracking system estimates current user positions and head orientations as well as positioning data for other objects. For the tracking system, a hybrid visual/acoustic sensor system will be used to emulate the human activity of extracting robust information by relying on different modalities.Broader Impacts: With new tools to study human perception, the system will enable research to explore new multi-modal, multi-user data displays that strategically utilize human ability to integrate cross-model sensory information. It will also serve as a platform to study interactions between humans and humanoid intelligent systems that can simulate environments at different degrees of complexity. The instrumentation supports research that will include the development of new interfaces for people with disabilities. The CRAIVE system will be an integrative facility for three school-wide centers, the center for Cognition, Communication, and Culture (CCC), the CCNI supercomputer, and the Experimental Media and Performing Arts Center (EMPC) to their full potential for collaborative research in this area, and to serve as a training platform for students who will engage in one of the enabled research areas.