Intelligent scene understanding for collaborative mobile augmented reality

协作移动增强现实的智能场景理解

• 批准号: 530666-2018
• 负责人: Khan, Naimul
• 金额: $ 4.37万
• 依托单位: Ryerson University
• 依托单位国家: 加拿大
• 项目类别: Collaborative Research and Development Grants
• 财政年份: 2018
• 资助国家: 加拿大
• 起止时间: 2018-01-01 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=655422
• 关键词: Intelligent; scene; understanding; collaborative; mobile

Augmented Reality (AR) is currently one of the most anticipated emerging technology, where the combination of virtual objects with the real world provide the end-user with a natural and interactive environment. AR has found its application in many fields, including digital media, entertainment, security, healthcare, etc.****Current AR technology limits the possibilities of AR through only having a rudimentary understanding of the scene surrounding a user. Partnering with AWE Company Ltd., we will develop new algorithms that will enable semantic understanding of the scene for intelligent AR, such as understanding what objects are around the user. This level of understanding will help AWE in delivering more robust AR experiences. We are especially interested in enabling collaborative AR experiences, where multiple users can enjoy and interact with the same AR experience at the same time. Specifically, we will : 1)Develop an automated method to semantically understand the user's environment. The proposed method will search the user's data (2D image and 3D point cloud) using a library of pre-trained objects, and compute locations and sizes of the identified objects. Segmenting the data and localizing detected objects will take advantage of state-of-the-art deep learning techniques to account for changes in object and environment appearance. 2. Achieve state-of-the-art in Visual-Based Localization (VBL) by using the semantic information obtained from the first contribution, and represent all the users with a common coordinate system to achieve collaborative AR. ****Together, the proposed methods will result in a robust framework for creating intelligent and collaborative mobile AR applications. The proposed research will help to position Canada as a leader in multimedia technologies, while the resultant technology transfer to Canadian industry will strengthen Canada's global competitiveness and create positive impacts to Canadian economy and society.********

增强现实（AR）是当前最令人期待的新兴技术之一，其中虚拟对象与真实的世界的组合为最终用户提供自然的和交互的环境。AR已经在许多领域得到应用，包括数字媒体、娱乐、安全、医疗保健等。当前的AR技术通过仅对用户周围的场景有基本的了解来限制AR的可能性。与AWE Company Ltd合作，我们将开发新的算法，使智能AR能够对场景进行语义理解，例如理解用户周围的物体。这种理解水平将有助于AWE提供更强大的AR体验。我们对实现协作式AR体验特别感兴趣，多个用户可以同时享受相同的AR体验并与之互动。具体来说，我们将：1）开发一种自动化的方法来从语义上理解用户的环境。所提出的方法将使用预先训练的对象库搜索用户的数据（2D图像和3D点云），并计算所识别对象的位置和大小。分割数据和定位检测到的对象将利用最先进的深度学习技术来考虑对象和环境外观的变化。2.通过使用从第一个贡献中获得的语义信息来实现基于视觉的定位（VBL）的最新技术，并使用公共坐标系来表示所有用户以实现协作AR。* 总之，提出的方法将产生一个强大的框架，用于创建智能和协作的移动的AR应用程序。拟议的研究将有助于将加拿大定位为多媒体技术的领导者，而由此产生的技术转让给加拿大工业将加强加拿大的全球竞争力，并对加拿大经济和社会产生积极影响。