Active Robotic Vision: Dynamic Sensor Systems for Informed Robotic Perception

*机器人系统越来越多地使用大量传感器来全方位感知它们的环境。这些具有固定视点的静态数组价格昂贵，需要强大的计算能力，并且适应不断变化的任务要求的能力有限。在这个项目中，我将开发动态传感器系统的主动感知方法，它使用致动传感器来获得视角灵活性，并降低对机器人的计算和成本要求。动态传感器系统为典型的机器人感知增加了两个复杂性。首先，需要仔细校准以融合致动和静态传感器数据。这包括执行器的运动学参数以及每个传感器和机器人基架之间的相对位姿估计，这些参数在操作过程中会发生变化。其次，需要主动选择传感器视点的方法来预测未来测量对手头感知任务的重要性。作为主动感知任务的典型例子，我将在整个项目中专注于同时定位和测绘，但所开发的方法可以推广到其他领域，如物体跟踪、抓取、3D重建和检测。*该项目包括三个主要目标，旨在释放动态传感器系统的潜力。我将开发基于因子图优化的新校准方法，同时解决机器人在未知环境中的运动，并识别静态和动态传感器的时间、运动学和外部校准。这些方法将统一动态传感器系统的校准过程，消除目前对特殊设计目标的依赖。然后，我将开发基于信息增益预测的主动感知方法，以告知贪婪的观点选择策略。这些预测将基于现有地图和定位的不确定性，并将评估完善当前地图状态和探索新区域以达到预期目标位置之间的权衡。最后，我将通过混合递归神经网络训练来改进信息增益预测，并应用强化学习以端到端的方法直接产生观点。由于信息增益预测很难精确建模，在不同的环境、路径和传感器视角上训练深度网络将提高预测性能。*这项工作将使具有致动传感器和机械手的机器人系统更好地实现其目标，并将导致对信息增益、视点选择和定位和映射鲁棒性之间的关系的更深入的理解。无人机、地面车辆和移动机械手将能够安全地在更复杂的空间中操作，对老化的基础设施进行详细检查，产生准确的物体3D模型用于抓取和操纵，甚至通过改进的情境感知在靠近人类的地方操作。*****

***Increasingly, robotic systems employ large numbers of sensors to perceive their environment in all directions. These static arrays with fixed viewpoints are expensive, require large computing power, and have limited ability to adapt to changing task requirements. In this program, I will develop active perception methods for dynamic sensor systems, which use actuated sensors to gain viewpoint flexibility and reduce computing and cost requirements for robots.******Dynamic sensor systems add two complexities to typical robotic perception. First, careful calibration is required to fuse actuated and static sensor data. This includes the kinematic parameters of actuators and the relative pose estimation between each sensor and the robot base frame, which vary during operation. Second, methods for the active selection of sensor viewpoints are needed to predict the importance of future measurements to the perception task at hand. I will focus on simultaneous localization and mapping throughout this program, as a canonical example of an active perception task, but the methods developed can generalize to other domains such as object tracking, grasping, 3D reconstruction and inspection.******The program includes three main objectives geared toward unlocking the potential of dynamic sensor systems. I will develop novel calibration methods based on factor graph optimization that simultaneously resolve robot motion through an unknown environment and identify temporal, kinematic and extrinsic calibration for static and dynamic sensors. These methods will unify the calibration process for dynamic sensor systems and remove the current reliance on specially design targets. I will then develop active perception methods based on information gain prediction that inform a greedy viewpoint selection strategy. These predictions will be based on the existing map and localization uncertainty, and will evaluate the tradeoff between refining the current map state and exploring new areas to reach the desired goal location. Finally, I will refine information gain prediction through hybrid recurrent neural network training and apply reinforcement learning to produce viewpoints directly in an end-to-end approach. As information gain prediction is challenging to model precisely, training deep networks over varied environments, paths, and sensor viewpoints will lead to improved prediction performance.******This work will enable robotic systems with actuated sensors and manipulators to better achieve their objectives, and will lead to a deeper understanding of the connections between information gain, viewpoint selection and localization and mapping robustness. Drones, ground vehicles and mobile manipulators will then be able to operate in more complex spaces safely, perform detailed inspections of aging infrastructure, produce accurate 3D models of objects for grasping and manipulation, and even operate in close proximity to humans through improved situational awareness. *****