Active Robotic Vision: Dynamic Sensor Systems for Informed Robotic Perception

主动机器人视觉：用于知情机器人感知的动态传感器系统

• 批准号: RGPIN-2019-05939
• 负责人: Waslander, Steven
• 金额: $ 4.01万
• 依托单位: University of Toronto
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=690501
• 关键词: Active; Robotic; Vision; Dynamic; Sensor

***Increasingly, robotic systems employ large numbers of sensors to perceive their environment in all directions. These static arrays with fixed viewpoints are expensive, require large computing power, and have limited ability to adapt to changing task requirements. In this program, I will develop active perception methods for dynamic sensor systems, which use actuated sensors to gain viewpoint flexibility and reduce computing and cost requirements for robots.******Dynamic sensor systems add two complexities to typical robotic perception. First, careful calibration is required to fuse actuated and static sensor data. This includes the kinematic parameters of actuators and the relative pose estimation between each sensor and the robot base frame, which vary during operation. Second, methods for the active selection of sensor viewpoints are needed to predict the importance of future measurements to the perception task at hand. I will focus on simultaneous localization and mapping throughout this program, as a canonical example of an active perception task, but the methods developed can generalize to other domains such as object tracking, grasping, 3D reconstruction and inspection.******The program includes three main objectives geared toward unlocking the potential of dynamic sensor systems. I will develop novel calibration methods based on factor graph optimization that simultaneously resolve robot motion through an unknown environment and identify temporal, kinematic and extrinsic calibration for static and dynamic sensors. These methods will unify the calibration process for dynamic sensor systems and remove the current reliance on specially design targets. I will then develop active perception methods based on information gain prediction that inform a greedy viewpoint selection strategy. These predictions will be based on the existing map and localization uncertainty, and will evaluate the tradeoff between refining the current map state and exploring new areas to reach the desired goal location. Finally, I will refine information gain prediction through hybrid recurrent neural network training and apply reinforcement learning to produce viewpoints directly in an end-to-end approach. As information gain prediction is challenging to model precisely, training deep networks over varied environments, paths, and sensor viewpoints will lead to improved prediction performance.******This work will enable robotic systems with actuated sensors and manipulators to better achieve their objectives, and will lead to a deeper understanding of the connections between information gain, viewpoint selection and localization and mapping robustness. Drones, ground vehicles and mobile manipulators will then be able to operate in more complex spaces safely, perform detailed inspections of aging infrastructure, produce accurate 3D models of objects for grasping and manipulation, and even operate in close proximity to humans through improved situational awareness. *****

* 机器人系统越来越多地使用大量传感器来感知各个方向的环境。这些具有固定视点的静态阵列是昂贵的，需要大量的计算能力，并且适应不断变化的任务要求的能力有限。在这个项目中，我将开发动态传感器系统的主动感知方法，该方法使用驱动传感器来获得视点灵活性，并降低机器人的计算和成本要求。动态传感器系统为典型的机器人感知增加了两个复杂性。首先，需要仔细校准以融合致动和静态传感器数据。这包括致动器的运动学参数和每个传感器与机器人基架之间的相对姿态估计，其在操作期间变化。其次，需要主动选择传感器视点的方法来预测未来测量对当前感知任务的重要性。在整个项目中，我将专注于同时定位和映射，作为主动感知任务的典型示例，但开发的方法可以推广到其他领域，如物体跟踪，抓取，3D重建和检查。该计划包括三个主要目标，旨在释放动态传感器系统的潜力。我将开发基于因子图优化的新型校准方法，同时解决机器人运动通过一个未知的环境，并确定静态和动态传感器的时间，运动学和外部校准。这些方法将统一动态传感器系统的校准过程，并消除目前对专门设计的目标的依赖。然后，我将开发基于信息增益预测的主动感知方法，以告知贪婪的视点选择策略。这些预测将基于现有地图和定位不确定性，并将评估优化当前地图状态和探索新区域以达到所需目标位置之间的权衡。最后，我将通过混合递归神经网络训练改进信息增益预测，并应用强化学习以端到端的方式直接生成视点。由于信息增益预测很难精确建模，因此在不同的环境、路径和传感器视角下训练深度网络将提高预测性能。这项工作将使机器人系统与驱动传感器和机械手，以更好地实现其目标，并将导致更深入地了解信息增益，视点选择和定位和映射鲁棒性之间的联系。届时，无人机、地面车辆和移动的操纵器将能够在更复杂的空间中安全地运行，对老化的基础设施进行详细检查，生成用于抓取和操纵的物体的精确3D模型，甚至通过改进的态势感知在人类附近运行。*****