Mixed Reality Groupware to Support Physical Collaborations

支持物理协作的混合现实群件 物理协作是非常常见的活动，在这种协作中，人们一起移动、操纵、构建和修复物理对象。无论是建造数字电路还是修理发动机，玩游戏还是做运动，人们都会一起工作，分享想法，接受指导。混合现实系统非常适合支持物理协作，因为它们允许虚拟对象和其他有用信息与物理任务一起显示，这使得通过技术提供和接收指导变得更容易、更自然。 问题是，当协作者因时间或距离分开时，我们不知道如何设计基于混合现实的群件(MR-Groupware)来最好地支持物理任务中的协作。例如，使用当前的视频会议技术向朋友提供如何修复漏水的水龙头的指导既慢又麻烦：远程助手很难描述和演示要做的动作，被引导的人很难看到和理解助手想要他们做什么。当我们与他人分开时，出现困难的主要原因有两个。首先，当我们与某人面对面工作时，我们缺乏对我们能够吸收的所有丰富信息的意识(例如，其他人在做什么，他们在看什么，他们把工具放在哪里，等等)。在使用当前的通信技术时，这些意识问题很难收集到。其次，我们缺乏与某人身体搭配的具体表现，这使得面对面的合作变得容易。例如，当工具在数千公里外与试图修理水龙头的合作者一起时，很难向某人演示如何使用扳手。这项研究计划将通过开发新的交互和技术，解决通过MR群件与独立的合作者合作所面临的两个关键挑战(意识和实施)。 我的长期研究目标是让在物理任务上与他人进行远程或不同时间的协作，就像让他们站在你身边一样自然。这一研究计划将通过高度合作的计划，利用我们与领先的混合现实公司和学术合作伙伴已经建立的战略关系，在实现这一目标方面取得实质性进展。为了最大限度地发挥我们工作的影响，我们将生产新的开源软件工具包，使MR-Groupware迅速成熟。MR-Groupware是加拿大的一项战略技术，因为它已经成功地应用于广泛的战略行业(例如，航空航天、国防、培训/教育、视频游戏和医疗)。加拿大的混合现实行业正在迅速发展，通过我的研究计划培训的HQP将处于有利地位，为这一新领域做出重要贡献。

Mixed Reality Groupware to Support Physical Collaborations Physical collaborations, where people work together to move, manipulate, build, and repair physical objects, are extremely common activities. Whether building a digital circuit or repairing an engine, playing a game or following an exercise, people work together to share ideas and receive guidance. Mixed reality systems are well-suited to support physical collaborations, since they allow virtual objects and other useful information to be displayed alongside physical tasks, which makes providing and receiving guidance through technology easier and more natural. The problem is that we do not know how to design mixed reality-based groupware (MR-Groupware) to best support collaboration in physical tasks when collaborators are separated by time or distance. For example, providing guidance to a friend on how to repair a leaky faucet using current video conferencing technology is slow and cumbersome: its difficult for the remote helper to describe and demonstrate the movements to be made, and its difficult for the person being guided to see and understand what the helper wants them to do. When we are separated from others there are two primary reasons that difficulties arise. First, we lack an awareness of all of the rich information we are able to take in when we are working with someone face-to-face (e.g., what is the other person doing, what are they looking at, where did they put the tools, etc). These awareness questions are tough to gather when using current communication technology. Second, we lack the embodiments that come from being physically collocated with someone, and that make face-to-face collaborations easy. For example, it is tough to demonstrate to someone how to use a wrench when the tool is thousands of kilometers away with a collaborator who is trying to fix the faucet. This research program will address the two key challenges (awareness and embodiment) for working with a separated collaborator through MR-Groupware by developing new interactions and technologies. My long-term research objective is to allow remote or time-separated collaboration with others on physical tasks to be as natural as having them stand beside you. This research program will make substantial progress towards this objective through a highly collaborative plan, which leverages our already established strategic relationships with leading mixed reality companies and academic partners. To maximize the impact of our work, we will produce new open-source software toolkits that allow MR-Groupware to be rapidly matured. MR-Groupware is a strategic technology for Canada, since it has already been successfully applied to a wide range of strategic industries (e.g., aeropsace, defence, training/education, video games, and health). The mixed reality industry in Canada is rapidly growing, and the HQP trained through my research program will be well positioned to make important contributions to this new field.