Transcended Reality: Modeling, Fabrication, Interaction, Navigation and Perception in Immersive (AR/VR) Environments

随着消费级AR/VR设备和实时计算机图形的出现，增强现实和虚拟现实（AR/VR）正在经历前所未有的复苏，以提供虚拟和真实难以区分地融合在一起的沉浸式体验。这个提议雄心勃勃，旨在进一步推动AR/VR的发展，成为TR或超越现实之一，在AR中，我们的真实物理环境不仅被添加到AR中，而且被删减、复制、抽象和转换，或者更普遍地超越。从早期的科幻小说开始，VR思想家的梦想就不仅仅是重现我们的物理现实，而是要超越它，超越它。***我们将通过耦合一套允许3D形状和运动共存于数字和物理领域的设备来实现这一目标：将AR/VR沉浸式显示器（如Oculus Rift， HTC/Valve Vive， Meta-AR或微软HoloLens）和实时3D形状和运动采集设备（如微软Kinect或VICON运动捕捉）与机器人执行器，激光切割机和3D打印相结合。***具体来说，拟研究将集中在沉浸式环境中的三个领域：对象的建模形式和功能、角色交互和导航；所有这些都是基于沉浸式环境的感知能力。***建模和制作：我们期望沉浸式环境中的3D模型超越形状或形式，吸收物理对象的功能，以便我们可以积极地与它们互动。因此，在本提案中，我们将重点关注3D形式和功能的同时建模。因此，使用我们研究的数字设计将立即可用，既可以作为身临其境的数字模型，也可以作为3D打印的实物，或者更有趣的是，作为两者的组合。***互动：随着我们环境中越来越多的设备具有“互联网意识”，TR将成为一个人类通过“物联网”与物理环境进行交流的空间。在身临其境的环境中，人与机器之间的互动交流将是这个提议的另一个线索。该提案将进一步探索建模和动画技术，产生情感上引人入胜的化身，在恐怖谷的两边。***导航和感知：随着新的AR/VR显示和人类对这些设备的暴露程度的增加，人们对使用立体和全息投影感知形状和运动的能力和局限性的理解不断发展。评估各种沉浸式交互和导航技术的感知影响，是贯穿我们提出的研究的一个重要线索

Augmented and Virtual Reality (AR/VR) are experiencing an unprecedented resurgence, with the onset of consumer-grade AR/VR devices and real-time computer graphics, to deliver an immersive experience where the virtual and real blend indistinguishably together. This proposal ambitiously aims to take the promise of AR/VR further, to one of TR or Transcended Reality, where our real physical environment is not only added to as in AR but subtracted from, replicated, abstracted and transformed, or more generally transcended. The dream of VR thinkers since early science fiction writings, has not been to merely recreate our physical reality, but to go beyond, indeed to transcend it.***We will accomplish this by coupling a suite of devices that allow 3D shape and motion to co-exist in the digital and physical realm: combining AR/VR immersive displays (like the Oculus Rift, HTC/Valve Vive, Meta-AR or Microsoft HoloLens) and real-time 3D shape and motion acquisition devices (like the Microsoft Kinect or VICON motion capture), with robotic actuators, laser cutters and 3D printing.***Specifically, the proposed research will focus on three areas within immersive environments: modeling form and function of objects, character interaction and navigation; all grounded by the perceptual capabilities of the immersive setting. ***Modeling and Fabrication: We expect 3D models in immersive environments to go beyond shape or form, to imbibe the functionality of physical objects, so that we may actively interact with them. In this proposal, we will thus focus on the simultaneous modeling of both 3D form and function. Digital designs using our research would thus instantly usable, both as digital models in an immersive setting, as 3D printed physical artifacts, or more interestingly as combinations of both.***Interaction: As more devices in our environment become "internet aware", TR will be a space where humans communicate with their physical environments via the "internet of things". Interactive communication between man and machine in an immersive setting will be another thread of this proposal.This proposal will further explore modeling and animation techniques that produce emotionally engaging avatars, on either side of the uncanny valley.***Navigation and Perception: Understanding the affordances and limitations of shape and motion perceived using stereoscopic and holographic projections continues to evolve with new AR/VR displays, and increased human exposure to these devices. Evaluating the perceptual impact of various immersive interaction and navigation techniques, is an important thread that will pervade our proposed research.**