Virtual and Augmented Reality for Knowledge Work

近年来，已经投入了数亿美元用于开发面向消费者的虚拟现实（VR）和增强现实（AR）头盔或“头戴式显示器”（hmd）。虚拟现实头戴设备会让用户产生一种被虚拟世界包围的错觉，而现实世界通常是完全隐藏的。在流行的游戏平台Steam上，已经有超过1500款VR游戏。增强现实头盔，如微软的HoloLens，允许用户仍然看到物理世界，但这些耳机可以显示虚拟内容，似乎位于物理世界，如浮动的虚拟窗口或文件，用户可以走动，从不同的角度看。一些头显，如HoloLens，包含一个嵌入式的独立计算机，多个传感器，并在用户移动头部时通过“由内到外”跟踪计算自己的位置，而不需要任何外部设备。AR头显不如VR头显成熟，但在未来几年，它们的分辨率、视场（FOV）、延迟、重量和价格将继续改善。最终，用户可以像戴眼镜一样一次戴上几个小时。如果实现了这一点，用户将受益于无限制地在他们周围的任何地方显示信息，包括在他们的工作环境中或在旅途中。这种增强现实耳机可以补充智能手机、笔记本电脑和台式电脑等物理设备。该提案询问用户，特别是知识工作者，如何才能最好地与未来的这种头戴式耳机交互：他们如何打开、定位和在多个虚拟浮动窗口和文档之间移动？如何使用不同的输入方式（鼠标、手势、头部和身体的运动、眼睛注视的方向）来更快、更有效地与大量信息进行交互？该提案概述了用户界面原型设计的计划，并通过实验对其进行评估。这项工作还将培训一组研究生，学习为新兴技术开发用户界面的技术。

In recent years, hundreds of millions of dollars have been invested to develop consumer-ready Virtual Reality (VR) and Augmented Reality (AR) headsets or "head-mounted displays" (HMDs). VR headsets create the illusion that a user is surrounded by virtual world, with the physical world normally completely hidden. There are already more than 1500 games available for VR on the popular gaming platform Steam. AR headsets, such as Microsoft's HoloLens, allow a user to still see the physical world, but these headsets can display virtual content that appears to be situated in the physical world, such as floating virtual windows or documents that the user can walk around and look at from different angles. Some headsets, like HoloLens, contain an embedded, independent computer, multiple sensors, and calculate their position as the user moves their head with "inside-out" tracking, without the need for any external devices. AR headsets are less mature than VR headsets, but in coming years, their resolution, field-of-view (FOV), latency, weight, and price will continue to improve. Eventually, users may be able to wear them for hours at a time, like a pair of eyeglasses. If this happens, users will benefit from an unlimited ability to display information anywhere around them, including in their work environments or while on the go. Such AR headsets could complement physical devices like smartphones, laptops, and desktop computers. This proposal asks how users, especially knowledge workers, can best interact with such headsets of the future: How might they open, position, and move between multiple virtual floating windows and documents? How can different input modalities (mouse, hand gestures, motions of the head and body, eye gaze direction) be used to interact more quickly and effectively with large collections of information? The proposal outlines a plan for prototyping user interfaces and evaluating these with experiments. The work would also train a team of graduate students in techniques for developing user interfaces for an emerging class of technologies.