Spatio-temporal reasoning in mixed reality with a hybrid approach to support scalability and development of virtuality-reality continnum software

最近的技术进步导致了一类新的计算设备的引入，这种设备将计算机生成的信息与现实-虚拟连续体中的“物理”世界结合在一起，例如混合现实(MR)。MR依靠上下文信息将虚拟内容集成到物理环境中，并识别用户的行为以促进直观和自然的交互。在这个项目中，我们致力于MR中的第一个挑战，即通过识别3D世界中的对象并在用户在现实-虚拟连续体中交互时跟踪它们来改进时空推理。作为第二个挑战，我们将致力于支持若干用户与若干本地或远程物理空间之间的知识转让和在MR中构建环境的共同表示。该项目的预期结果将包括用于MR的算法和模型，这些算法和模型将在工业4.0和医疗保健的两个基于MR的情景感知决策支持应用原型中实施，以及在实验室和与工业4.0和医疗保健的潜在用户进行现场测试期间收集的实验结果。该项目的行业合作伙伴是智能多云生态系统的领先者VMware Canada。预计本项目考虑的混合现实用例将在下一代移动网络--即6G--中发挥关键作用，这是VMWare分布式边缘(DE)的关键研究领域之一。因此，VMware最感兴趣的是在该项目中进行研究，以增强混合现实体验、其可扩展性及其在共同确定的基础设施中的演示(在行业4.0和医疗保健中)。

Recent advances in technology have led to the introduction of a new class of computing devices that combine computer-generated information with the "physical" world in a reality-virtuality continuum, such as Mixed Reality (MR). MR relies on contextual information to integrate virtual content within the physical environment and recognize users' actions to foster intuitive and natural interactions. In this project, we work on a first challenge in MR which resides in improving spatiotemporal reasoning by recognizing objects in a 3D world and tracking them while the users interact in a reality-virtuality continuum. As a second challenge, we will work in supporting the knowledge transfers and the construction of a common representation of an environment in MR between several users and several local or remote physical spaces. The expected results of the project will consist in algorithms and models for MR that will be implemented in two MR based context-aware decision support application prototypes for the Industry 4.0 and healthcare, as well as experimental results collected during in labs and on the field tests with potential users in Industry 4.0 and healthcare. The industry partner in this project is VMware Canada, a leader in intelligent multi-cloud ecosystems. It is expected that the use case of Mixed Reality, which is considered in this project, will play a pivotal role in the next generation - namely 6G - of mobile networks, which is one of the key research areas for the VMWare Distributed Edge (DE). The research conducted in this project to enhance Mixed Reality experiences, its scability and its demonstration (in Industry 4.0 and in healthcare) in a jointly identified infrastructure is therefore of utmost interest for VMware.