Enhancing Human Abilities and Social Interaction in Context-Aware Pervasive Extended Reality

在情境感知的普遍扩展现实中增强人类能力和社会互动

• 批准号: RGPIN-2022-03294
• 负责人: Kim, Kangsoo
• 金额: $ 2.11万
• 依托单位: University of Calgary
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=752364
• 关键词: Enhancing; Human; Abilities; Social; Interaction

Notwithstanding the recent technological advances and increasing public interests in the field of eXtended Reality (XR), the realization of XR is still largely based on egocentric and sporadic information collected by ad-hoc open-loop XR systems. This restricts a comprehensive understanding of the user's context and needs in the dynamic and large-scale XR environment. The influence of virtual entities over the users is limited due to the inherent lack (or absence) of physicality and interactivity in the real world, and the implications on individuals and society are largely under-explored despite the timely importance. This program tackles these issues and aims to realize more influential XR experience by conducting convergence research that combines XR with complementary cutting-edge technologies, e.g., Artificial Intelligence and Ubiquitous Computing with Internet of Things (IoT). The complementary concepts and technologies manifested in the form of intelligent and pervasive embodied virtual entities in XR would be transformative. While pursuing more interactive XR systems, better understanding of human experience in XR, and practically effective use cases of XR in real life, two short-term objectives are established for this program: (1) develop a context-aware pervasive XR framework, and (2) investigate the effects of multimodal (embodied) XR experience in enhancing human abilities and experience. The context-aware pervasive XR framework will be designed and developed integrating novel sensing and actuating platforms, e.g., smart IoT devices and robotic units, in a concept of digital twin paradigm, while employing advanced machine-learning techniques to recognize the user's context in the environment. Various interactive XR prototypes will be developed involving embodied virtual entities, and rigorous user studies will be conducted to comprehensively investigate the effects on human abilities and experience, adopting theories and scientific knowledge in human factors sciences. The research is novel in the technical aspect that employs a large volume of multimodal data and advanced machine-learning methods to develop a comprehensive large-scale smart XR environment, and also in the scientific aspect that aims to reveal the impacts of XR in different dimensions of human abilities and experience. While training a diverse group of highly qualified personnel in an inclusive and equitable research and education environment, the research outcomes with a technological development of context-aware XR framework and formative knowledge on the effects of XR experience will guide researchers and practitioners to design more communicative and collaborative XR systems. The research has significant impacts on improving existing interaction and learning mechanisms in different social settings, e.g., education, collaboration, and medical/industrial simulation and training, which will directly benefit the future of XR industry and research in Canada.

尽管最近技术取得了进步，公众对延展实境（XR）领域的兴趣也越来越大，但XR的实现仍然在很大程度上基于临时开环XR系统收集的以自我为中心的零星信息。这限制了在动态和大规模XR环境中对用户上下文和需求的全面理解。由于真实的世界中固有的物理性和交互性的缺乏（或不存在），虚拟实体对用户的影响是有限的，尽管及时的重要性，但对个人和社会的影响在很大程度上还没有得到充分的探讨。该计划旨在解决这些问题，并通过将XR与互补的尖端技术相结合的融合研究，实现更有影响力的XR体验，例如，人工智能和物联网（IoT）的普适计算。XR中以智能和普遍体现的虚拟实体形式表现出来的互补概念和技术将是变革性的。在追求更具交互性的XR系统，更好地理解XR中的人类体验以及XR在真实的生活中的实际有效用例的同时，该计划建立了两个短期目标：（1）开发上下文感知的普适XR框架，以及（2）调查多模态（具体化）XR体验在增强人类能力和体验方面的效果。上下文感知的普适XR框架将被设计和开发，集成新颖的传感和致动平台，例如，智能物联网设备和机器人单元，在数字孪生范式的概念，同时采用先进的机器学习技术来识别用户在环境中的上下文。将开发各种涉及具体虚拟实体的交互式XR原型，并将进行严格的用户研究，以全面调查对人类能力和经验的影响，采用人因科学的理论和科学知识。该研究在技术方面是新颖的，采用大量的多模态数据和先进的机器学习方法来开发一个全面的大规模智能XR环境，在科学方面也是新颖的，旨在揭示XR对人类能力和经验的不同维度的影响。在包容和公平的研究和教育环境中培养多样化的高素质人才的同时，研究成果与上下文感知XR框架的技术发展和XR经验影响的形成知识将指导研究人员和从业人员设计更具沟通性和协作性的XR系统。该研究对改善不同社会环境中现有的互动和学习机制具有重大影响，例如，教育，合作，医疗/工业模拟和培训，这将直接有利于加拿大XR行业和研究的未来。