Virtual and Augmented Reality for Knowledge Work

近年来，已投资数亿美元开发消费者就绪的虚拟现实（VR）和增强现实（AR）头戴式设备或“头戴式显示器”（HMD）。 VR头显创造了一种错觉，用户被虚拟世界所包围，而物理世界通常完全隐藏。 在流行的游戏平台Steam上已经有超过1500款VR游戏。 AR头戴设备，如微软的HoloLens，允许用户仍然看到物理世界，但这些头戴设备可以显示似乎位于物理世界中的虚拟内容，例如用户可以四处走动并从不同角度查看的浮动虚拟窗口或文档。 一些耳机，例如HoloLens，包含嵌入式独立计算机和多个传感器，并在用户移动头部时通过“由内而外”跟踪计算其位置，而无需任何外部设备。 AR头戴设备的成熟度低于VR头戴设备，但在未来几年，它们的分辨率、视场（FOV）、延迟、重量和价格将继续改善。 最终，用户可以像戴眼镜一样，一次戴上几个小时。 如果发生这种情况，用户将受益于在他们周围任何地方显示信息的无限能力，包括在他们的工作环境中或在旅途中。 这样的AR头显可以补充智能手机、笔记本电脑和台式电脑等物理设备。 该提案询问用户，特别是知识工作者，如何最好地与未来的此类耳机进行交互：他们如何打开，定位以及在多个虚拟浮动窗口和文档之间移动？ 如何使用不同的输入方式（鼠标、手势、头部和身体的运动、眼睛注视方向）来更快速有效地与大量信息进行交互？ 该提案概述了一个计划，原型用户界面和评估这些实验。 这项工作还将培训一组研究生，掌握为新兴技术开发用户界面的技术。

In recent years, hundreds of millions of dollars have been invested to develop consumer-ready Virtual Reality (VR) and Augmented Reality (AR) headsets or "head-mounted displays" (HMDs). VR headsets create the illusion that a user is surrounded by virtual world, with the physical world normally completely hidden. There are already more than 1500 games available for VR on the popular gaming platform Steam. AR headsets, such as Microsoft's HoloLens, allow a user to still see the physical world, but these headsets can display virtual content that appears to be situated in the physical world, such as floating virtual windows or documents that the user can walk around and look at from different angles. Some headsets, like HoloLens, contain an embedded, independent computer, multiple sensors, and calculate their position as the user moves their head with "inside-out" tracking, without the need for any external devices. AR headsets are less mature than VR headsets, but in coming years, their resolution, field-of-view (FOV), latency, weight, and price will continue to improve. Eventually, users may be able to wear them for hours at a time, like a pair of eyeglasses. If this happens, users will benefit from an unlimited ability to display information anywhere around them, including in their work environments or while on the go. Such AR headsets could complement physical devices like smartphones, laptops, and desktop computers. This proposal asks how users, especially knowledge workers, can best interact with such headsets of the future: How might they open, position, and move between multiple virtual floating windows and documents? How can different input modalities (mouse, hand gestures, motions of the head and body, eye gaze direction) be used to interact more quickly and effectively with large collections of information? The proposal outlines a plan for prototyping user interfaces and evaluating these with experiments. The work would also train a team of graduate students in techniques for developing user interfaces for an emerging class of technologies.