Mixed Reality Groupware to Support Physical Collaborations

支持物理协作的混合现实群件物理协作是非常常见的活动，在物理协作中，人们一起工作来移动、操作、构建和修复物理对象。无论是构建数字电路还是修理发动机，玩游戏还是进行练习，人们都在一起工作，分享想法并接受指导。混合现实系统非常适合支持物理协作，因为它们允许虚拟对象和其他有用的信息与物理任务一起显示，这使得通过技术提供和接收指导更容易、更自然。问题是，我们不知道如何设计基于混合现实的群件（MR-Groupware），以便在协作者因时间或距离而分开时最好地支持物理任务中的协作。例如，使用当前的视频会议技术指导朋友如何修理漏水的水龙头是缓慢而繁琐的：远程助手很难描述和演示要做的动作，被引导的人也很难看到和理解助手想让他们做什么。当我们与他人分离时，困难产生的主要原因有两个。首先，当我们与某人面对面工作时，我们缺乏对所有丰富信息的意识（例如，其他人在做什么，他们在看什么，他们把工具放在哪里，等等）。当使用当前的通信技术时，很难收集到这些意识问题。其次，我们缺乏与某人在一起的具体体现，这使得面对面的合作变得容易。例如，当一个合作者在数千公里之外试图修理水龙头时，很难向某人演示如何使用扳手。该研究计划将通过开发新的交互和技术，解决通过MR-Groupware与分离的合作者一起工作的两个关键挑战（意识和具体化）。我的长期研究目标是让与他人在物理任务上进行远程或分时合作，就像让他们站在你身边一样自然。该研究项目将通过一个高度协作的计划，利用我们与领先的混合现实公司和学术合作伙伴已经建立的战略关系，朝着这一目标取得实质性进展。为了使我们的工作产生最大的影响，我们将生产新的开源软件工具包，使MR-Groupware迅速成熟。MR-Groupware是加拿大的一项战略技术，因为它已经成功地应用于广泛的战略行业（例如，航空航天、国防、培训/教育、视频游戏和卫生）。加拿大的混合现实产业正在迅速发展，通过我的研究项目培养出来的HQP将会在这个新的领域做出重要的贡献。

Mixed Reality Groupware to Support Physical Collaborations Physical collaborations, where people work together to move, manipulate, build, and repair physical objects, are extremely common activities. Whether building a digital circuit or repairing an engine, playing a game or following an exercise, people work together to share ideas and receive guidance. Mixed reality systems are well-suited to support physical collaborations, since they allow virtual objects and other useful information to be displayed alongside physical tasks, which makes providing and receiving guidance through technology easier and more natural. The problem is that we do not know how to design mixed reality-based groupware (MR-Groupware) to best support collaboration in physical tasks when collaborators are separated by time or distance. For example, providing guidance to a friend on how to repair a leaky faucet using current video conferencing technology is slow and cumbersome: its difficult for the remote helper to describe and demonstrate the movements to be made, and its difficult for the person being guided to see and understand what the helper wants them to do. When we are separated from others there are two primary reasons that difficulties arise. First, we lack an awareness of all of the rich information we are able to take in when we are working with someone face-to-face (e.g., what is the other person doing, what are they looking at, where did they put the tools, etc). These awareness questions are tough to gather when using current communication technology. Second, we lack the embodiments that come from being physically collocated with someone, and that make face-to-face collaborations easy. For example, it is tough to demonstrate to someone how to use a wrench when the tool is thousands of kilometers away with a collaborator who is trying to fix the faucet. This research program will address the two key challenges (awareness and embodiment) for working with a separated collaborator through MR-Groupware by developing new interactions and technologies. My long-term research objective is to allow remote or time-separated collaboration with others on physical tasks to be as natural as having them stand beside you. This research program will make substantial progress towards this objective through a highly collaborative plan, which leverages our already established strategic relationships with leading mixed reality companies and academic partners. To maximize the impact of our work, we will produce new open-source software toolkits that allow MR-Groupware to be rapidly matured. MR-Groupware is a strategic technology for Canada, since it has already been successfully applied to a wide range of strategic industries (e.g., aeropsace, defence, training/education, video games, and health). The mixed reality industry in Canada is rapidly growing, and the HQP trained through my research program will be well positioned to make important contributions to this new field.