Mixed-Reality Interfaces for Remote Collaboration

传统的视频聊天在距离分离的协作者之间提供音频和视频连接。但是传统的视频聊天工具不足以满足关注对象和环境的远程协作任务。这种远程协作任务将越来越常见，包括辅助任务（例如机器维修）、指导任务（例如远程巡视）和检查任务（例如设计评论）。传统的视频聊天在给予合作者以下方面做得很差：（i）关于远程合作者正在做什么的意识信息，诸如他们正在看哪里，或者他们正在做出的手势作为语音的一部分（例如，在说“看这个”的同时指向），以及（ii）对远程空间的理解。这些接口至关重要，因为远程对象和位置已成为移动的设备支持的协作的主要焦点。**我们将探讨如何设计混合现实（MR）界面来支持远程协作任务。这样的接口通常做两件事：首先，它们使用增强现实（例如，投影的视觉、头戴式显示器或透视显示器）在真实的世界中可视地锚虚拟内容，其次，它们使用传感器来捕获环境的各方面（例如，环境的深度相机扫描/视频捕获）或协作者与环境的交互（例如，他们的姿势或他们正在看什么）。我们将产生理论和原型，描述和说明如何设计MR接口，捕获协作者的意识信息（如凝视，视图和手势信息），并允许协作者独立探索数字远程环境。该项目由三个子项目组成：*（1）软件基础设施。我们将设计和实施新的基础设施，使我们能够快速原型化新型MR体验。(2)捕捉和可视化意识信息。我们的MR原型将专注于捕捉协作者与环境的交互（例如，他们在看哪里，他们在指向什么等），向远程协作者呈现并可视化此内容。** (3)允许独立探索。我们将探索数字化捕获和重建远程环境的方法，以允许合作者独立地查看和探索这些空间。我们的研究将带来关于如何设计和构建MR接口的新知识，这些接口允许相距遥远的合作者有效地共同工作。我们将识别和解决在这个设计领域出现的挑战，包括用户界面设计，易用性和流动性，以及技术基础设施。我们的工作将通知研究界设计有效的远程协作MR接口。它将告知加拿大公司，因为他们继续开发远程和远程工作方案的技术。

Conventional video chat provides audio and video connection between distance-separated collaborators. But conventional video chat tools are inadequate for remote collaboration tasks that focus on objects and environments. Such remote collaboration tasks will increasingly become commonplace, including assistance tasks (e.g. machine repair), guidance tasks (e.g. remote tour), and inspection tasks (e.g. design critique). Conventional video chat does a poor job of giving collaborators: (i) awareness information about what a remote collaborator is doing, such as where they are looking, or the hand gestures they are making as part of speech (e.g. pointing while saying “Look at this one”), and (ii) an understanding of the remote space. These interfaces are critical, because remote objects and locations have become be the main focus of collaboration enabled by mobile devices.******We will explore how to design Mixed Reality (MR) interfaces to support remote collaboration tasks. Such interfaces typically do two things: first, they visually anchor virtual content in the real world using augmented reality (e.g. projected visuals, head-mounted displays, or see-through displays), and second, they use sensors to capture aspects of environment (e.g. a depth camera scan/video capture of an environment) or collaborators' interactions with the environment (e.g. their gestures or what they are looking at). We will produce theories and prototypes that describe and illustrate how to design MR interfaces that capture awareness information about collaborators (such as gaze, view and gesture information), and allow collaborators to independently explore a digital remote environment.******This program is comprised of three sub-projects:******(1) Software infrastructure. We will design and implement new infrastructure that allows us to rapidly prototype new kinds of MR experiences.******(2) Capturing and visualizing awareness information. Our MR prototypes will focus on capturing collaborators' interactions with the environment (e.g. where are they looking, what are they pointing at, etc.), representing and visualizing this to remote collaborators.******(3) Enabling independent exploration. We will explore ways of digitally capturing and reconstructing remote environments to allow collaborators to independently view and explore these spaces.******Our research will result in new knowledge about how to design and build MR interfaces that allow distance-separated collaborators to work together effectively. We will identify and address challenges that arise in this design space including user interface design, ease of use and fluidity, and technical infrastructure. Our work will inform the research community on designing effective MR interfaces for remote collaboration. It will inform Canadian companies as they continue to develop technologies for remote and telework scenarios.