Intelligent scene understanding for collaborative mobile augmented reality

点击翻译按钮获取中文摘要

Augmented Reality (AR) is currently one of the most anticipated emerging technology, where the combination of virtual objects with the real world provide the end-user with a natural and interactive environment. AR has found its application in many fields, including digital media, entertainment, security, healthcare, etc.****Current AR technology limits the possibilities of AR through only having a rudimentary understanding of the scene surrounding a user. Partnering with AWE Company Ltd., we will develop new algorithms that will enable semantic understanding of the scene for intelligent AR, such as understanding what objects are around the user. This level of understanding will help AWE in delivering more robust AR experiences. We are especially interested in enabling collaborative AR experiences, where multiple users can enjoy and interact with the same AR experience at the same time. Specifically, we will : 1)Develop an automated method to semantically understand the user's environment. The proposed method will search the user's data (2D image and 3D point cloud) using a library of pre-trained objects, and compute locations and sizes of the identified objects. Segmenting the data and localizing detected objects will take advantage of state-of-the-art deep learning techniques to account for changes in object and environment appearance. 2. Achieve state-of-the-art in Visual-Based Localization (VBL) by using the semantic information obtained from the first contribution, and represent all the users with a common coordinate system to achieve collaborative AR. ****Together, the proposed methods will result in a robust framework for creating intelligent and collaborative mobile AR applications. The proposed research will help to position Canada as a leader in multimedia technologies, while the resultant technology transfer to Canadian industry will strengthen Canada's global competitiveness and create positive impacts to Canadian economy and society.********