Operating System Support for Augmented Reality Applications
Operating System Support for Augmented Reality Applications
复制标题
增强现实应用程序的操作系统支持
DOI：
--
复制
发表时间：
2013
期刊：
影响因子：
--
通讯作者：
Helen J. Wang
中科院分区：
文献类型：
--
作者：
Loris D'antoni;Alan M. Dunn;S. Jana;Tadayoshi Kohno;B. Livshits;D. Molnar;Alexander Moshchuk;E. Ofek;Franziska Roesner;Scott Saponas;Margus Veanes;Helen J. Wang
增强现实（AR）采用自然用户输入（NUI），例如手势、语音和眼睛凝视，并在用户看到的现实之上产生数字视觉覆盖。今天，存在多种出货的AR应用程序，最值得注意的是Microsoft Kinect和智能手机应用程序，如Layar，Wikitude和Junaio。尽管有这些活动，但很少有人关注操作系统对AR应用程序的支持。相反，今天的每个AR应用程序都在OpenCV或Microsoft Kinect SDK等用户级库的帮助下进行自己的感知和渲染。
在本文中，我们将探讨操作系统应该如何发展以支持AR应用程序。由于AR应用程序使用全新的输入和输出，因此支持AR应用程序的操作系统需要重新考虑暴露给应用程序的输入和显示抽象。与鼠标和键盘不同，它们为用户输入形成了明确的、单独的通道，NUI需要对现实世界环境进行连续的感知，而现实世界环境中通常会有敏感数据与用户输入混合在一起。因此，操作系统输入抽象必须确保用户隐私不被侵犯，并且操作系统必须提供细粒度的权限系统以访问识别的对象，如用户的面部和骨架。此外，由于AR应用程序的视觉输出混合了现实世界和虚拟对象，传统GUI中的合成窗口抽象不再可行，操作系统必须重新考虑显示抽象及其管理。我们讨论解决这些问题和其他问题的研究方向，并建立一个操作系统，让多个应用程序共享一个（增强）现实。
Augmented reality (AR) takes natural user input (NUI), such as gestures, voice, and eye gaze, and produces digital visual overlays on top of reality seen by a user. Today, multiple shipping AR applications exist, most notably titles for the Microsoft Kinect and smartphone applications such as Layar, Wikitude, and Junaio. Despite this activity, little attention has been paid to operating system support for AR applications. Instead, each AR application today does its own sensing and rendering, with the help of user-level libraries like OpenCV or the Microsoft Kinect SDK.
In this paper, we explore how operating systems should evolve to support AR applications. Because AR applications work with fundamentally new inputs and outputs, an OS that supports AR applications needs to re-think the input and display abstractions exposed to applications. Unlike mouse and keyboard, which form explicit, separate channels for user input, NUI requires continuous sensing of the real-world environment, which often has sensitive data mixed with user input. Hence, the OS input abstractions must ensure that user privacy is not violated, and the OS must provide a fine-grained permission system for access to recognized objects like a user's face and skeleton. In addition, because visual outputs of AR applications mix real-world and virtual objects, the synthetic window abstraction in traditional GUIs is no longer viable, and OSes must rethink the display abstractions and their management. We discuss research directions for solving these and other issues and building an OS that let multiple applications share one (augmented) reality.