Promoting 3D Environment Perception Ability of Autonomous Systems

提升自主系统3D环境感知能力

• 批准号: RGPIN-2021-04244
• 负责人: Wang, Guanghui
• 金额: $ 2.11万
• 依托单位: Ryerson University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=749942
• 关键词: Promoting; 3D; Environment; Perception; Ability

Environment perception and modeling from image sequences is a challenging task of intelligent vehicles in dynamic environments. As a critical component of visual perception, how to efficiently model and represent the world has been an active research area during the past decades. Although considerable progress has been made both in theory and in practice, the problem of environment perception remains far from being solved, owing primarily to three critical challenges: (i) how to recover the 3D structure of dynamic natural scenes coupled with rigid, nonrigid, articulated, and moving objects; (ii) how to efficiently and robustly represent the working environment in practical navigation scenarios; and (iii) how to increase the generalization ability of a machine learning model. These issues are very complex and have thus seldom been well-addressed in the literature. The main objective of this project is to solve these challenges and promote the 3D perception ability of intelligent systems by developing an efficient and robust representation of dynamic natural environments. Specifically, we will investigate three critical research problems in this project. The first one is on scene depth estimation and semantic segmentation. We propose a unified learning model for depth estimation and semantic segmentation using multi-task learning. By integrating the geometric constraint and the temporal information of the sequence, the proposed model can recover the depth and semantic information simultaneously with high accuracy. The second problem is semantic object detection. We propose a new learning model for object detection with the ability to make reasoning and deduction from training examples by mimicking human brains. A new learning paradigm is developed to increase the model generalizability via active and reinforcement learning. For the third problem, we propose a novel unified framework to represent dynamic environments by exploring all information from the scene to form a hybrid representation. The study envisions a novel perspective for environment perception by integrating all information from the scene. In particular, the project emphasizes new algorithmic aspects for problems of depth estimation, semantic segmentation, object detection, and 3D mapping. The proposed technique makes the environment mapping more efficient and effective for autonomous systems to navigate in unknown dynamic environments. This research is not only academically significant but also urgently needed by automotive and robotic industries. This research is a fundamental step towards the investigator's long-term research endeavor: building a highly intelligent system through visual cues. We will also integrate the proposed framework and research results into our education and outreach activities to enhance existing course curriculums and attract more students, especially those from underrepresented minorities, to work in computer science and engineering.

基于图像序列的环境感知和建模是动态环境下智能车辆的一项具有挑战性的任务。作为视觉感知的一个重要组成部分，如何有效地对世界进行建模和表示一直是近几十年来的研究热点。尽管在理论和实践上都取得了很大的进展，但环境感知问题仍然远未得到解决，主要是由于三个关键挑战：（i）如何恢复与刚性，非刚性，铰接和移动物体相耦合的动态自然场景的3D结构;（ii）如何有效和鲁棒地表示实际导航场景中的工作环境;以及（iii）如何提高机器学习模型的泛化能力。这些问题非常复杂，因此在文献中很少得到很好的解决。该项目的主要目标是解决这些挑战，并通过开发动态自然环境的高效和鲁棒表示来提高智能系统的3D感知能力。具体来说，我们将调查三个关键的研究问题，在这个项目中。第一个是场景深度估计和语义分割。我们提出了一个统一的学习模型，使用多任务学习的深度估计和语义分割。该模型综合了序列的几何约束和时间信息，能够同时恢复序列的深度和语义信息，具有较高的精度。第二个问题是语义对象检测。我们提出了一种新的对象检测学习模型，能够通过模仿人脑来对训练示例进行推理和推导。通过主动学习和强化学习，提出了一种新的学习范式来提高模型的泛化能力。对于第三个问题，我们提出了一种新的统一框架来表示动态环境，通过探索来自场景的所有信息来形成混合表示。 该研究通过整合来自场景的所有信息，为环境感知提供了一个新的视角。特别是，该项目强调深度估计，语义分割，对象检测和3D映射问题的新算法方面。所提出的技术使环境映射更高效和有效的自治系统在未知的动态环境中导航。这项研究不仅具有学术意义，而且也是汽车和机器人行业迫切需要的。这项研究是研究人员长期研究奋进的基础一步：通过视觉线索构建高度智能的系统。我们亦会把建议的架构和研究成果融入教育和外展活动，以加强现有的课程，吸引更多学生，特别是来自少数族裔的学生，投身计算机科学和工程学。