Enhancing Human Abilities and Social Interaction in Context-Aware Pervasive Extended Reality

点击翻译按钮获取中文摘要

Notwithstanding the recent technological advances and increasing public interests in the field of eXtended Reality (XR), the realization of XR is still largely based on egocentric and sporadic information collected by ad-hoc open-loop XR systems. This restricts a comprehensive understanding of the user's context and needs in the dynamic and large-scale XR environment. The influence of virtual entities over the users is limited due to the inherent lack (or absence) of physicality and interactivity in the real world, and the implications on individuals and society are largely under-explored despite the timely importance. This program tackles these issues and aims to realize more influential XR experience by conducting convergence research that combines XR with complementary cutting-edge technologies, e.g., Artificial Intelligence and Ubiquitous Computing with Internet of Things (IoT). The complementary concepts and technologies manifested in the form of intelligent and pervasive embodied virtual entities in XR would be transformative. While pursuing more interactive XR systems, better understanding of human experience in XR, and practically effective use cases of XR in real life, two short-term objectives are established for this program: (1) develop a context-aware pervasive XR framework, and (2) investigate the effects of multimodal (embodied) XR experience in enhancing human abilities and experience. The context-aware pervasive XR framework will be designed and developed integrating novel sensing and actuating platforms, e.g., smart IoT devices and robotic units, in a concept of digital twin paradigm, while employing advanced machine-learning techniques to recognize the user's context in the environment. Various interactive XR prototypes will be developed involving embodied virtual entities, and rigorous user studies will be conducted to comprehensively investigate the effects on human abilities and experience, adopting theories and scientific knowledge in human factors sciences. The research is novel in the technical aspect that employs a large volume of multimodal data and advanced machine-learning methods to develop a comprehensive large-scale smart XR environment, and also in the scientific aspect that aims to reveal the impacts of XR in different dimensions of human abilities and experience. While training a diverse group of highly qualified personnel in an inclusive and equitable research and education environment, the research outcomes with a technological development of context-aware XR framework and formative knowledge on the effects of XR experience will guide researchers and practitioners to design more communicative and collaborative XR systems. The research has significant impacts on improving existing interaction and learning mechanisms in different social settings, e.g., education, collaboration, and medical/industrial simulation and training, which will directly benefit the future of XR industry and research in Canada.