MRI: Development of the Collaborative-Research Augmented Immersive Virtual Environment Laboratory (CRAIVE-Lab)

MRI：协作研究增强沉浸式虚拟环境实验室 (CRAIVE-Lab) 的开发

• 批准号: 1229391
• 负责人: Jonas Braasch
• 金额: $ 30万
• 依托单位: Rensselaer Polytechnic Institute
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-10-01 至 2016-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1229391&HistoricalAwards=false
• 关键词: MRI; Development; Collaborative; Research; Augmented

Proposal #: 12-29391PI(s): Braasch, Jonas; Chang, Ben; Cutler, Barbra M.; Goebel, Johannes; Radke, RichardInstitution: Rensselaer Polytechnic Institute Title: MRI/Dev.: Collaborative-Research Augmented Immersive Virtual Environment Laboratory (CRAIVE-Lab)Project Proposed:This project, developing a specialized virtual-reality (VR) system, introduces a new concept for an immersive virtual environment which is optimized for collaborative group activities. The system addresses the auditory and visual senses with equal priority, which is currently only championed by head-mounted display systems. Unlike the latter, this system also enables direct communication between participants without the need for telecommunication devices. The system?s haptic floor display can be used to recreate floor vibrations (e.g., simulate concert stages) with an accuracy that is currently only known for smaller displays (e.g., car simulators).The work addresses the need for specialized virtual reality (VR) system for the study and enablement of communication-driven tasks with groups of users immersed in a high-fidelity multi-modal environment located in the same physical space. While current multi-modal VR systems have achieved a high degree of realism, they either focus on the immersion of a single or very small group of users or on presenting material to a larger group of users in a cinema-type environment. In both cases, the systems provide homogeneous visual and acoustic fields. For group communication tasks, inhomogeneous fields that provide personalized visual and acoustic perspectives or each user, could provide better access to relevant information from the VR system?s display and at the same time increase the experiential degree of presence and perceived realism for interactive tasks. The project addresses the technical hurdles that need to be surmounted to establish a large-scale (18mx12mx4.3m), muti-user, muti-perspective, muti-model display. For visual domain, multiple point-of-convergence rendering techniques will be used to (re-)create scenes on a 7-projector display.. For the acoustic domain, a 192-loudspeaker-channel system will be designed for Wave-Field Synthesis (WFS) with the support of Higher-Order-Ambisonic (HoA) sound projection to render inhomogeneous acoustic fields. A haptic display, consisting of 16 platforms elements, will be used to simulate floor vibrations and also to provide infrastructure for other vibrating objects (e.g., handheld devices). An intelligent position-tracking system estimates current user positions and head orientations as well as positioning data for other objects. For the tracking system, a hybrid visual/acoustic sensor system will be used to emulate the human activity of extracting robust information by relying on different modalities.Broader Impacts: With new tools to study human perception, the system will enable research to explore new multi-modal, multi-user data displays that strategically utilize human ability to integrate cross-model sensory information. It will also serve as a platform to study interactions between humans and humanoid intelligent systems that can simulate environments at different degrees of complexity. The instrumentation supports research that will include the development of new interfaces for people with disabilities. The CRAIVE system will be an integrative facility for three school-wide centers, the center for Cognition, Communication, and Culture (CCC), the CCNI supercomputer, and the Experimental Media and Performing Arts Center (EMPC) to their full potential for collaborative research in this area, and to serve as a training platform for students who will engage in one of the enabled research areas.

提案编号：12- 29391 PI（s）：Braasch，Jonas; Chang，Ben; Cutler，Barbra M.; Goebel，Johannes; Radke，RichardInstitution：Rensselaer Polytechnic Institute标题：MRI/Dev.：协作研究增强沉浸式虚拟环境实验室（CRAIVE-Lab）项目已提出：该项目开发一个专门的虚拟现实（VR）系统，引入了沉浸式虚拟环境的新概念，该环境针对协作小组活动进行了优化。该系统以同等的优先级解决听觉和视觉问题，目前只有头戴式显示系统支持这一点。与后者不同的是，该系统还使参与者之间能够直接通信，而无需电信设备。系统？触觉地板显示器可用于再现地板振动（例如，模拟音乐会舞台），其精度目前仅为较小的显示器所知（例如，这项工作解决了对专用虚拟现实（VR）系统的需求，该系统用于研究和实现通信驱动的任务，其中用户组沉浸在位于同一物理空间中的高保真多模态环境中。虽然目前的多模态VR系统已经实现了高度的真实感，但它们要么专注于单个或非常小的用户群体的沉浸感，要么专注于在电影院类型的环境中向更大的用户群体呈现材料。在这两种情况下，系统提供均匀的视觉和声场。对于群组通信任务，提供个性化视觉和声学视角或每个用户的非均匀场可以更好地访问VR系统的相关信息？的显示，并在同一时间增加体验程度的存在和感知的现实主义的互动任务。该项目解决了需要克服的技术障碍，以建立一个大规模（18mx12mx4.3m），多用户，多视角，多模型显示。对于视觉领域，将使用多个会聚点渲染技术在7投影仪显示器上（重新）创建场景。对于声学领域，将设计192个扬声器通道系统用于波场合成（WFS），并支持高阶高保真度立体声（HoA）声音投影，以呈现不均匀的声场。由16个平台元件组成的触觉显示器将用于模拟地板振动，并为其他振动物体提供基础设施（例如，手持设备）。智能位置跟踪系统估计当前用户位置和头部取向以及其他对象的定位数据。对于跟踪系统，混合视觉/声学传感器系统将被用来模拟人类的活动，通过依赖于不同的模态提取鲁棒的信息。更广泛的影响：随着新的工具来研究人类的感知，该系统将使研究探索新的多模态，多用户的数据显示，战略性地利用人类的能力，整合跨模型的感官信息。它还将作为一个平台，研究人类与人形智能系统之间的交互，这些系统可以模拟不同复杂程度的环境。该仪器支持的研究将包括为残疾人开发新的界面。CRAIVE系统将成为三个全校中心的综合设施，认知，交流和文化中心（CCC），CCNI超级计算机和实验媒体和表演艺术中心（EMPC），以充分发挥其在这一领域的合作研究潜力，并作为学生的培训平台，他们将从事其中一个启用的研究领域。