Characterizing Human Spatial Thinking When Designing 3D Objects.

随着当前的虚拟现实(VR)和增强现实(AR)技术变得负担得起和可获得，它们的使用已经扩展到研究实验室之外的教室、设计工作室和人们的家中。一个流行的领域是使用VR/AR进行创造性任务，如3D草图绘制。三维草图正在改变设计活动，因为它是一种为设计师提供传统二维草图的表现力、直接性和可编辑性的媒介。此外，它还支持传统3D设计工具(如原型制作)提供的以身体为中心的空间感知、存在和多角度。设计师已经在他们的工作流程中使用了VR/AR草图系统的成功案例，比如TiltBrush和GravitySketch。除了直接在3D中绘制草图的能力外，这些系统还集成了CAD软件的工具，如在某些情况下的自适应辅助，例如通过自动检测形状中心或反映用户动作，使设计过程更加简单。但由于缺乏人们在设计3D对象时如何在空间上思考的知识，新用户界面(UI)的潜力目前受到阻碍。空间思维给用户带来了巨大的认知负荷，这限制了他们在设计对象时的有效性。我的研究项目旨在通过识别人类在空间思考时使用的动作和元素，为3D设计创造新的自适应和智能的用户界面。我把空间思维作为创意设计中的关键元素，它会给用户带来认知负荷，特别是那些空间能力较低的用户。在这里，我提出了三个项目，每个项目都考虑了虚拟环境中的不同元素，这将为新的用户界面提供优化用户流量的信息。项目1将创建VR用户界面，通过了解用户视角对设计过程的影响，自动建议相机的位置和方向。项目2是关于设计AR用户界面，使用触觉和音频反馈将现实世界的对象延伸到空中。为此，我们将研究参照系(例如，墙、桌子等)的影响。在设计过程中。最后，项目3是关于制造AR和VR用户界面，它们提供非侵入性的向导，实时增强用户的空间手势。我们将研究有助于人类空间认知的手势动作。这些新的智能用户界面将允许用户更多地专注于创造性，而不是理解虚拟空间，这将通过减轻用户的认知负担来帮助普及3D设计工具的使用。我对空间思维的研究延伸到了教育、空间认知和数据分析等领域。这些活动依赖于人们对复杂环境的理解或在精神上可视化他们行为的结果，这取决于人们的空间能力和空间思维。这些领域的研究人员可以利用这些知识来设计智能用户界面。

As current virtual reality (VR) and augmented reality (AR) technologies become affordable and accessible, their use has extended outside research labs into classrooms, design studios, and people's homes. One area that has become prevalent is using VR/AR for creative tasks, such as 3D sketching. 3D sketching is transforming design activities as it is a medium that provides designers with both the expressiveness, immediacy, and editability of traditional 2D sketching. Additionally, it supports the body-centered spatial awareness, presence, and multiple perspectives afforded by traditional 3D design tools, such as prototyping. There are already success stories of VR/AR sketching systems that designers use in their workflow, like Tiltbrush and GravitySketch. Besides the ability to directly sketch in 3D, these systems also incorporate tools from CAD software, like adaptive assistance in some scenarios, e.g., by automatically detecting the center of a shape or mirroring the user actions, making the design process even simpler. But the potential of new user interfaces (UIs) is currently held back by a lack of knowledge about how people think spatially when designing 3D objects. Spatial thinking creates a significant cognitive load for the user, which limits their effectiveness when designing an object. My research program aims to create new adaptive and intelligent UIs for 3D design by identifying the actions and elements humans use when thinking spatially. I focus on spatial thinking as the key element in creative design that leads to cognitive load on the users, especially to those with lower spatial abilities. Here I propose three projects, each of which considers different elements inside the virtual environment, which will inform new UIs that optimize the user's flow. Project 1 will create VR UIs that automatically suggest a camera position and orientation by understanding the effect of a user's point of view on the design process. Project 2 is about designing AR UIs that use haptic and audio feedback to extend real-world objects to mid-air. For this, we will study the effect of reference frames (e.g., walls, tables, etc.) on the design process. Finally, Project 3 is about making AR and VR UIs that provide non-intrusive guides that enhance the user's spatial gestures in real-time. We will study the gestural actions that aid the spatial cognition of a person. These new intelligent UIs will allow users to focus more on being creative and less on understanding the virtual space, which will help democratize the use of 3D design tools by diminishing the cognitive load of the users. My research on spatial thinking extends to areas like education, spatial cognition, and data analysis. These are activities that rely on people making sense of complex environments or mentally visualizing the results of their actions, which depend on people's spatial abilities and spatial thinking. Researchers in these areas can use this knowledge to design intelligent UIs.