Active Robotic Vision: Dynamic Sensor Systems for Informed Robotic Perception

越来越多的机器人系统使用大量的传感器从各个方向感知环境。这些具有固定视点的静态数组价格昂贵，需要很大的计算能力，并且适应不断变化的任务需求的能力有限。在这个项目中，我将开发动态传感器系统的主动感知方法，它使用驱动传感器来获得视点灵活性，并减少机器人的计算和成本要求。******动态传感器系统为典型的机器人感知增加了两个复杂性。首先，需要仔细校准以融合驱动和静态传感器数据。这包括执行器的运动学参数以及每个传感器与机器人基架之间的相对姿态估计，这些参数在运行过程中会发生变化。其次，需要主动选择传感器视点的方法来预测未来测量对手头感知任务的重要性。作为主动感知任务的典型例子，我将在整个程序中重点关注同步定位和映射，但所开发的方法可以推广到其他领域，如物体跟踪，抓取，3D重建和检查。******该计划包括三个主要目标，旨在释放动态传感器系统的潜力。我将开发基于因子图优化的新型校准方法，同时解决机器人在未知环境中的运动，并识别静态和动态传感器的时间，运动学和外部校准。这些方法将统一动态传感器系统的校准过程，消除目前对特殊设计目标的依赖。然后，我将开发基于信息增益预测的主动感知方法，为贪婪的视点选择策略提供信息。这些预测将基于现有地图和定位的不确定性，并将评估优化当前地图状态和探索新区域以达到预期目标位置之间的权衡。最后，我将通过混合递归神经网络训练来改进信息增益预测，并应用强化学习以端到端方式直接生成视点。由于信息增益预测很难精确建模，因此在不同的环境、路径和传感器视点上训练深度网络将提高预测性能。******这项工作将使具有驱动传感器和操纵器的机器人系统能够更好地实现其目标，并将对信息获取、视点选择、定位和映射鲁棒性之间的联系有更深入的理解。然后，无人机、地面车辆和移动机械手将能够在更复杂的空间中安全操作，对老化的基础设施进行详细检查，生成精确的物体3D模型以进行抓取和操作，甚至通过提高态势感知能力在接近人类的情况下操作。* * * * *

***Increasingly, robotic systems employ large numbers of sensors to perceive their environment in all directions. These static arrays with fixed viewpoints are expensive, require large computing power, and have limited ability to adapt to changing task requirements. In this program, I will develop active perception methods for dynamic sensor systems, which use actuated sensors to gain viewpoint flexibility and reduce computing and cost requirements for robots.******Dynamic sensor systems add two complexities to typical robotic perception. First, careful calibration is required to fuse actuated and static sensor data. This includes the kinematic parameters of actuators and the relative pose estimation between each sensor and the robot base frame, which vary during operation. Second, methods for the active selection of sensor viewpoints are needed to predict the importance of future measurements to the perception task at hand. I will focus on simultaneous localization and mapping throughout this program, as a canonical example of an active perception task, but the methods developed can generalize to other domains such as object tracking, grasping, 3D reconstruction and inspection.******The program includes three main objectives geared toward unlocking the potential of dynamic sensor systems. I will develop novel calibration methods based on factor graph optimization that simultaneously resolve robot motion through an unknown environment and identify temporal, kinematic and extrinsic calibration for static and dynamic sensors. These methods will unify the calibration process for dynamic sensor systems and remove the current reliance on specially design targets. I will then develop active perception methods based on information gain prediction that inform a greedy viewpoint selection strategy. These predictions will be based on the existing map and localization uncertainty, and will evaluate the tradeoff between refining the current map state and exploring new areas to reach the desired goal location. Finally, I will refine information gain prediction through hybrid recurrent neural network training and apply reinforcement learning to produce viewpoints directly in an end-to-end approach. As information gain prediction is challenging to model precisely, training deep networks over varied environments, paths, and sensor viewpoints will lead to improved prediction performance.******This work will enable robotic systems with actuated sensors and manipulators to better achieve their objectives, and will lead to a deeper understanding of the connections between information gain, viewpoint selection and localization and mapping robustness. Drones, ground vehicles and mobile manipulators will then be able to operate in more complex spaces safely, perform detailed inspections of aging infrastructure, produce accurate 3D models of objects for grasping and manipulation, and even operate in close proximity to humans through improved situational awareness. *****