Virtual and Augmented Reality for Knowledge Work

点击翻译按钮获取中文摘要

In recent years, hundreds of millions of dollars have been invested to develop consumer-ready Virtual Reality (VR) and Augmented Reality (AR) headsets or "head-mounted displays" (HMDs). VR headsets create the illusion that a user is surrounded by virtual world, with the physical world normally completely hidden. There are already more than 1500 games available for VR on the popular gaming platform Steam. AR headsets, such as Microsoft's HoloLens, allow a user to still see the physical world, but these headsets can display virtual content that appears to be situated in the physical world, such as floating virtual windows or documents that the user can walk around and look at from different angles. Some headsets, like HoloLens, contain an embedded, independent computer, multiple sensors, and calculate their position as the user moves their head with "inside-out" tracking, without the need for any external devices. AR headsets are less mature than VR headsets, but in coming years, their resolution, field-of-view (FOV), latency, weight, and price will continue to improve. Eventually, users may be able to wear them for hours at a time, like a pair of eyeglasses. If this happens, users will benefit from an unlimited ability to display information anywhere around them, including in their work environments or while on the go. Such AR headsets could complement physical devices like smartphones, laptops, and desktop computers. This proposal asks how users, especially knowledge workers, can best interact with such headsets of the future: How might they open, position, and move between multiple virtual floating windows and documents? How can different input modalities (mouse, hand gestures, motions of the head and body, eye gaze direction) be used to interact more quickly and effectively with large collections of information? The proposal outlines a plan for prototyping user interfaces and evaluating these with experiments. The work would also train a team of graduate students in techniques for developing user interfaces for an emerging class of technologies.