室外复杂视觉条件下的机器人感知与目标识别方法

本项目将研究以视觉为主且具有复杂室外环境自适应能力的机器人环境感知和目标识别理论、方法及系统。结合机器人视觉嵌入性、移动性和实时性的应用需求，我们创新性地从大气物理和光学成像机理出发，融合多模异构信号（图像、偏振、雷达），重点解决机器人感知中的深度估计、光照及天气变化、动态背景等挑战性问题，在此基础上提出2D图像/3D点云跨模态、弱监督、自适应实时语义分割、目标识别和跟踪算法。本项目研发的新型机器人视觉系统可以主动感知所处的环境，自动判定光照与气象条件并进行相应的图像阴影、反光和雨雪雾去除，将复杂光照和恶劣天气统一解决并屏蔽在算法底层，为后续的机器人场景语义感知、目标识别等算法提供优质数据。从目前的研究现状看，这是一个新的研究思路和系统性的解决方案。为突破现有机器人视觉系统在复杂室外条件下过分依赖数学优化求解以及缺乏物理环境适应能力的瓶颈问题，提供了一种十分有效的理论方法和技术路线。

This project will study the vision-based robot perception and target recognition theories, methods, and system with complex outdoor environment adaptability. Combining the application requirements of robot vision embedding, mobility and real-time, we start from the mechanism of physical optical imaging and adopt a combination of software and hardware to study and solve the challenging problems of robot perception, which are depth estimation, illumination and weather change, dynamic background and so on, from the perspective of fusion of multi-mode heterogeneous signals (image, polarization, radar). On this basis, we propose a 2D image / 3D point cloud cross-modality, weak supervision, adaptive real-time semantic segmentation and target recognition algorithm. The new robot vision system developed by this project can actively perceive the environment, automatically judge the illumination and meteorological conditions, and select the corresponding processing algorithm to shield the scene 3D geometric perception, complex illumination and bad weather at the bottom of the algorithm, which lays the foundation for the subsequent robot scene semantic perception and target recognition algorithm development. From the current research situation, this is a new research idea and systematic solution. It provides a very effective theoretical method and technical route to break through the bottleneck of existing methods which rely too much on mathematical analytic solution, strong ambiguity, poor real-time performance and lack of physical explanation.