Research and development of a cloud-based context-aware API for semantic scene understanding

基于云的上下文感知API的语义场景理解研究与开发

• 批准号: 558247-2020
• 负责人: Khan, Naimul
• 金额: $ 2.17万
• 依托单位: Ryerson University
• 依托单位国家: 加拿大
• 项目类别: Alliance Grants
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=706640
• 关键词: Research; development; cloud; based; context

Augmented Reality (AR) has become a promising tool for enhancing the quality of experience in digital media products. However, the technical tools available for AR lacks in providing the rich understanding required to design interactive digital experiences quickly and efficiently. In partnership with AWE Company Ltd, this research project will attempt at developing a context-aware Application Programming Interface (API) that can provide a semantic understanding of a user's surrounding by intelligently distributing the camera feed data from a user's device on to three separate sub-modules for semantic understanding: building facade segmentation, floor plan creation, and object localization. Through understanding the user's context (i.e. the requirement of the understanding of the aforementioned semantics), the proposed API can reduce the computational and data overhead required to run the neural network-based systems on the cloud. The proposed project will employ Distributed Deep Neural Network (DDNN) design principles to carefully distribute the computational task between the clients and the cloud, while building the semantic understanding stack on top of available technology tools such as ARCore/ARKit, so that AWE's software developers can easily integrate the proposed solution into their product development pipeline. The proposed API will result in a robust framework for quick distribution of digital twins projects that AWE plans to roll out for many of its clients within the next two years. The proposed research will help to position Canada as a leader in multimedia technologies, while the resultant technology transfer to Canadian industry will strengthen Canada's global competitiveness and create positive impacts to Canadian economy and society.

增强现实技术（AR）已成为提高数字媒体产品体验质量的一种有前途的工具。然而，可用于增强现实的技术工具在提供快速有效地设计交互式数字体验所需的丰富理解方面缺乏。与AWE公司合作，该研究项目将尝试开发一个上下文感知的应用程序编程接口（API），该接口可以通过智能地将来自用户设备的摄像头馈线数据分发到三个独立的子模块上进行语义理解，从而提供对用户周围环境的语义理解：建筑立面分割、平面图创建和对象定位。通过理解用户的上下文（即理解上述语义的要求），提议的API可以减少在云上运行基于神经网络的系统所需的计算和数据开销。拟议的项目将采用分布式深度神经网络（dddnn）设计原则，在客户端和云之间仔细分配计算任务，同时在可用的技术工具（如ARCore/ARKit）上构建语义理解堆栈，以便AWE的软件开发人员可以轻松地将拟议的解决方案集成到他们的产品开发管道中。