Virtual and Augmented Reality for Knowledge Work

近年来，已经投入了数亿美元用于开发面向消费者的虚拟现实（VR）和增强现实（AR）头盔或“头戴式显示器”（hmd）。虚拟现实头戴设备会让用户产生一种被虚拟世界包围的错觉，而现实世界通常是完全隐藏的。在流行的游戏平台Steam上，已经有超过1500款VR游戏。增强现实头盔，如微软的HoloLens，允许用户仍然看到物理世界，但这些耳机可以显示虚拟内容，似乎位于物理世界，如浮动的虚拟窗口或文件，用户可以走动，从不同的角度看。一些头显，如HoloLens，包含一个嵌入式的独立计算机，多个传感器，并在用户移动头部时通过“由内到外”跟踪计算自己的位置，而不需要任何外部设备。AR头显不如VR头显成熟，但在未来几年，它们的分辨率、视场（FOV）、延迟、重量和价格将继续改善。最终，用户可以像戴眼镜一样一次戴上几个小时。

In recent years, hundreds of millions of dollars have been invested to develop consumer-ready Virtual Reality (VR) and Augmented Reality (AR) headsets or "head-mounted displays" (HMDs). VR headsets create the illusion that a user is surrounded by virtual world, with the physical world normally completely hidden. There are already more than 1500 games available for VR on the popular gaming platform Steam. AR headsets, such as Microsoft's HoloLens, allow a user to still see the physical world, but these headsets can display virtual content that appears to be situated in the physical world, such as floating virtual windows or documents that the user can walk around and look at from different angles. Some headsets, like HoloLens, contain an embedded, independent computer, multiple sensors, and calculate their position as the user moves their head with "inside-out" tracking, without the need for any external devices. AR headsets are less mature than VR headsets, but in coming years, their resolution, field-of-view (FOV), latency, weight, and price will continue to improve. Eventually, users may be able to wear them for hours at a time, like a pair of eyeglasses. If this happens, users will benefit from an unlimited ability to display information anywhere around them, including in their work environments or while on the go. Such AR headsets could complement physical devices like smartphones, laptops, and desktop computers. This proposal asks how users, especially knowledge workers, can best interact with such headsets of the future: How might they open, position, and move between multiple virtual floating windows and documents? How can different input modalities (mouse, hand gestures, motions of the head and body, eye gaze direction) be used to interact more quickly and effectively with large collections of information? The proposal outlines a plan for prototyping user interfaces and evaluating these with experiments. The work would also train a team of graduate students in techniques for developing user interfaces for an emerging class of technologies.