Active Robotic Vision: Dynamic Sensor Systems for Informed Robotic Perception

点击翻译按钮获取中文摘要

Increasingly, robotic systems employ large numbers of sensors to perceive their environment in all directions. These static arrays with fixed viewpoints are expensive, require large computing power, and have limited ability to adapt to changing task requirements. In this program, I will develop active perception methods for dynamic sensor systems, which use actuated sensors to gain viewpoint flexibility and reduce computing and cost requirements for robots. Dynamic sensor systems add two complexities to typical robotic perception. First, careful calibration is required to fuse actuated and static sensor data. This includes the kinematic parameters of actuators and the relative pose estimation between each sensor and the robot base frame, which vary during operation. Second, methods for the active selection of sensor viewpoints are needed to predict the importance of future measurements to the perception task at hand. I will focus on simultaneous localization and mapping throughout this program, as a canonical example of an active perception task, but the methods developed can generalize to other domains such as object tracking, grasping, 3D reconstruction and inspection. The program includes three main objectives geared toward unlocking the potential of dynamic sensor systems. I will develop novel calibration methods based on factor graph optimization that simultaneously resolve robot motion through an unknown environment and identify temporal, kinematic and extrinsic calibration for static and dynamic sensors. These methods will unify the calibration process for dynamic sensor systems and remove the current reliance on specially design targets. I will then develop active perception methods based on information gain prediction that inform a greedy viewpoint selection strategy. These predictions will be based on the existing map and localization uncertainty, and will evaluate the tradeoff between refining the current map state and exploring new areas to reach the desired goal location. Finally, I will refine information gain prediction through hybrid recurrent neural network training and apply reinforcement learning to produce viewpoints directly in an end-to-end approach. As information gain prediction is challenging to model precisely, training deep networks over varied environments, paths, and sensor viewpoints will lead to improved prediction performance. This work will enable robotic systems with actuated sensors and manipulators to better achieve their objectives, and will lead to a deeper understanding of the connections between information gain, viewpoint selection and localization and mapping robustness. Drones, ground vehicles and mobile manipulators will then be able to operate in more complex spaces safely, perform detailed inspections of aging infrastructure, produce accurate 3D models of objects for grasping and manipulation, and even operate in close proximity to humans through improved situational awareness.