HCC: Small: Super Reality -- Immersive Visual Interfaces with No Line-of-Sight Restriction

HCC：小：超现实——无视线限制的沉浸式视觉界面

• 批准号: 2219842
• 负责人: Voicu Popescu
• 金额: $ 25万
• 依托单位: Purdue University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2219842&HistoricalAwards=false
• 关键词: HCC; Small; Super; Reality; Immersive

The project develops Super Reality, a new class of human-computer visual interfaces designed for efficient and effective 3D dataset exploration. Super Reality (SR) interfaces build upon conventional virtual reality interfaces, preserving their benefits of immersion, intuitiveness, and depth cues, while radically increasing the information bandwidth of the images shown to the user. SR interfaces let the user see more of the dataset from the current location by removing the line-of-sight restriction of conventional virtual reality interfaces. The user sees not only the regions of the dataset that are visible from the current viewpoint, but also regions that are hidden behind occluders. The user can preview an occluded dataset region from the current position, without having to navigate around occluders, thereby avoiding the wasted navigation when the occluded region proves to be of no interest. The user can compare distant dataset regions directly, in the same image, without having to rely on the memory of dataset regions seen earlier. With a SR interface, the user explores the dataset in parallel, from multiple viewpoints, but without the burden of a complex visualization interface based on a matrix of conventional images. Through the leap-forward increase of 3D dataset exploration efficiency and effectiveness, this project will lead to a foundational level innovation influencing the domains where data visualization is used. This SR interface has the potential to minimize user disorientation and cybersickness, which are triggered by mismatched user motion and user visual perception in the traditional virtual reality interfaces. The project will be carried out by involving a diverse group of undergraduate students in research.The goal of the project is the design and development of Super Reality (SR) interfaces that will remove the line-of-sight restriction through a generalization of the computational camera that projects the 3D dataset onto the 2D image plane. The linear rays of conventional cameras are replaced with curved rays that are routed around occluders to connect the user to dataset regions of interest. Super Reality interfaces essentially dispatch a team of virtual subordinate users that each image the dataset from their own viewpoint and present the user with an integration of the data captured from the multiple viewpoints. These integrated images are constructed to meet multiple design requirements: (a) image continuity, by bounding the difference between trajectories of neighboring rays; (b) image non-redundancy, by avoiding ray intersections; (c) adaptive image resolution, by routing more rays towards dataset regions of higher complexity or importance; (d) in-place disocclusion, by projecting the disoccluded region where the user would see it in the absence of the occluder; (e) minimal image distortion, by limiting the curvature of the non-linear rays; fast rendering, by designing the generalized cameras with a fast projection operation. The SR interfaces will be designed to render the part of these integrated images close to the user with the conventional, first-person view. This way, the part of the frame that corresponds to nearby geometry responds to the user’s view changes as the user expects, anchoring the user to avoid disorientation and cybersickness. Also, SR interface limits the number of degrees of freedom the user needs to manipulate in order to deploy and retract the desired disocclusion effects. The Super Reality interfaces are iteratively refined and validated in controlled user studies based on objective and subjective metrics that quantify the degree to which the design requirements are met, as well as the efficiency and effectiveness of 3D dataset exploration.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目开发了超级现实，这是一种新的人机视觉界面，旨在高效和有效地探索3D数据集。超现实(SR)界面建立在传统虚拟现实界面的基础上，保留了沉浸感、直观性和深度提示的优势，同时从根本上增加了显示给用户的图像的信息带宽。SR界面通过消除传统虚拟现实界面的视线限制，使用户能够从当前位置看到更多数据集。用户不仅可以看到从当前视点可见的数据集区域，还可以看到隐藏在遮挡物后面的区域。用户可以从当前位置预览被遮挡的数据集区域，而不必绕过遮挡器导航，从而避免在被遮挡的区域被证明不感兴趣时浪费导航。用户可以在同一图像中直接比较遥远的数据集区域，而不必依赖于先前看到的数据集区域的记忆。通过SR界面，用户可以从多个视角并行浏览数据集，而无需承担基于传统图像矩阵的复杂可视化界面的负担。通过3D数据集勘探效率和效果的跨越式提高，该项目将导致一项基础性的创新，影响数据可视化应用领域。这种SR界面有可能最大限度地减少传统虚拟现实界面中由不匹配的用户运动和用户视觉感知引发的用户迷失方向和晕屏。该项目将通过让不同的本科生参与研究来实施。该项目的目标是设计和开发超现实(SR)界面，通过将3D数据集投影到2D图像平面上的计算相机的泛化来消除视线限制。传统相机的线性射线被曲线射线取代，曲线射线绕过遮挡器将用户连接到感兴趣的数据集区域。超现实界面本质上是派遣一组虚拟下属用户，每个用户从自己的视点对数据集进行成像，并向用户呈现从多个视点捕获的数据的集成。这些综合图像的构建是为了满足多种设计要求：(A)通过限制相邻光线的轨迹之间的差异来实现图像的连续性；(B)通过避免光线相交来实现图像的非冗余性；(C)通过将更多的光线投射到更复杂或更重要的数据集区域来实现自适应图像分辨率；(D)就地消除遮挡，方法是将错开的区域投影到用户在没有遮挡器的情况下可以看到的区域；(E)通过限制非线性光线的曲率来实现最小的图像失真；(D)通过设计具有快速投影操作的通用相机来实现快速渲染。SR界面将被设计为使用传统的第一人称视图将这些集成图像的部分呈现给用户。这样，框中与附近几何体对应的部分会像用户预期的那样响应用户的视图变化，从而锚定用户，以避免迷失方向和晕车。此外，SR界面限制了用户需要操纵的自由度数量，以便部署和收回所需的非咬合效果。超现实界面在受控用户研究中反复改进和验证，基于客观和主观指标，量化设计要求的满足程度，以及3D数据集探索的效率和有效性。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

AR Interfaces for Disocclusion—A Comparative Study

• DOI: 10.1109/vr55154.2023.00068
• 发表时间: 2023-03
• 期刊: 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR)
• 作者: Shuqi Liao;Yuqi Zhou;V. Popescu