CAREER: Belief Space Planning and Learning for Uncertainty-Immersed Underwater Robots

地球上的海洋和河流对我们的生活非常重要，详细了解它们非常重要，例如它们的水流如何随时间变化。 虽然很难从第一原理开发详细的电流模型，但可以构建良好的近似。 这些粗糙的模型，然后可以使用的自主水下机器人计划和执行路径。 通过在穿越环境期间收集数据，可以学习更详细的模型，在仿真中进行测试，然后用于使机器人的行动更加可靠。 该项目还将为WaterBotics的课程做出贡献，这是一个K-12计划，通过与水下机器人的实践学习，使数千名学生了解工程原理。研究成果将被整合到该项目的新教育模块中。该项目使用已成功应用于学习机器人技能的强化学习来学习广阔环境的动态。具体而言，强化学习将适用于处理由于物理干扰和多路径返回而导致概率分布在整个环境中急剧变化的声学传感器。这些现象的性质最初并不知道高精度，但将学习机器人巡逻。具体而言，以粗略的初始模型为起点，将通过规划、模拟和物理重复的组合来学习在这种现象下的机动和巡逻的最佳策略。基于模型的信念空间运动规划将用于引导和加速最优策略的情景学习。这项工作将采用一种新的信念空间规划度量，降低了规划的计算复杂性，无论是在粗，全球规模和精细，局部规模。

The Earth's oceans and rivers are very important to our lives, and it is important to understand them in detail, such as how their current flows change over time. While it is difficult to develop detailed models of current flow from first principles, good approximations can be constructed. These coarse models can then be used by an autonomous underwater robot to plan and execute paths. By collecting data during its traversals of the environment, more detailed models can be learned, tested in simulation, and then used to make the robot's actions more reliable. This project will also contribute to the curriculum of WaterBotics, a K-12 program that exposes thousands of students to engineering principles via hands-on learning with underwater robots. The results of the proposed research will be integrated into new educational modules for this program.This project uses reinforcement learning, which has been successfully applied in learning robotic skills, to learn the dynamics of an expansive environment. Specifically, reinforcement learning will be adapted to deal with acoustic sensors whose probability distributions, due to physical disturbances and multi-path returns, vary sharply throughout the environment. The properties of these phenomena are not initially known with high accuracy, but will be learned as the robot patrols. Specifically, with a coarse initial model as a starting point, optimal policies for maneuvering and patrolling under such phenomena will be learned through a combination of planning, simulation, and physical repetition. Model-based belief space motion planning will be used to bootstrap and accelerate the episodic learning of optimal policies. The work will employ a novel belief-space planning metric that reduces the computational complexity of planning both at coarse, global scales and at fine, local scales.