Virtual and Augmented Reality for Knowledge Work

近年来，已投入数亿美元开发面向消费者的虚拟现实(VR)和增强现实(AR)耳机或“头盔显示器”(HMD)。VR耳机给人一种被虚拟世界包围的错觉，而现实世界通常是完全隐藏的。在流行的游戏平台STeam上，已经有1500多款VR游戏可供选择。AR耳机，如微软的HoloLens，允许用户仍然看到物理世界，但这些耳机可以显示似乎位于物理世界中的虚拟内容，例如浮动的虚拟窗口或文件，用户可以四处走动，从不同的角度查看。一些耳机，如HoloLens，包含一个嵌入式的独立计算机，多个传感器，当用户移动他们的头时，通过“内向外”跟踪来计算它们的位置，而不需要任何外部设备。AR耳机不如VR耳机成熟，但在未来几年，它们的分辨率、视场(FOV)、延迟、重量和价格将继续提高。最终，用户可能能够一次佩戴几个小时，就像戴眼镜一样。 如果发生这种情况，用户将受益于在他们周围的任何地方显示信息的无限能力，包括在他们的工作环境中或在旅途中。这类AR耳机可以补充智能手机、笔记本电脑和台式电脑等物理设备。 该提案询问用户，尤其是知识工作者，如何才能最好地与未来的头戴式耳机互动：他们将如何打开、定位以及在多个虚拟浮动窗口和文档之间移动？如何使用不同的输入模式(鼠标、手势、头部和身体的运动、眼睛的凝视方向)来更快、更有效地与大量信息交互？该提案概述了用户界面原型制作并通过实验进行评估的计划。这项工作还将培训一支研究生团队，掌握为新兴技术开发用户界面的技术。

In recent years, hundreds of millions of dollars have been invested to develop consumer-ready Virtual Reality (VR) and Augmented Reality (AR) headsets or "head-mounted displays" (HMDs). VR headsets create the illusion that a user is surrounded by virtual world, with the physical world normally completely hidden. There are already more than 1500 games available for VR on the popular gaming platform Steam. AR headsets, such as Microsoft's HoloLens, allow a user to still see the physical world, but these headsets can display virtual content that appears to be situated in the physical world, such as floating virtual windows or documents that the user can walk around and look at from different angles. Some headsets, like HoloLens, contain an embedded, independent computer, multiple sensors, and calculate their position as the user moves their head with "inside-out" tracking, without the need for any external devices. AR headsets are less mature than VR headsets, but in coming years, their resolution, field-of-view (FOV), latency, weight, and price will continue to improve. Eventually, users may be able to wear them for hours at a time, like a pair of eyeglasses. If this happens, users will benefit from an unlimited ability to display information anywhere around them, including in their work environments or while on the go. Such AR headsets could complement physical devices like smartphones, laptops, and desktop computers. This proposal asks how users, especially knowledge workers, can best interact with such headsets of the future: How might they open, position, and move between multiple virtual floating windows and documents? How can different input modalities (mouse, hand gestures, motions of the head and body, eye gaze direction) be used to interact more quickly and effectively with large collections of information? The proposal outlines a plan for prototyping user interfaces and evaluating these with experiments. The work would also train a team of graduate students in techniques for developing user interfaces for an emerging class of technologies.