CAREER: Belief Space Planning and Learning for Uncertainty-Immersed Underwater Robots

地球上的海洋和河流对我们的生活非常重要，重要的是要详细了解它们，比如它们的当前流量如何随着时间的推移而变化。虽然很难根据第一原理建立详细的电流流动模型，但可以建立良好的近似值。然后，自主水下机器人可以使用这些粗略模型来规划和执行路径。通过在机器人遍历环境期间收集数据，可以学习更详细的模型，在模拟中进行测试，然后使用这些模型来使机器人的行动更可靠。该项目还将对水机器人的课程做出贡献，这是一个K-12项目，通过与水下机器人的实践学习，让数千名学生接触工程原理。这项研究的结果将被整合到这个项目的新教育模块中。这个项目使用强化学习，它已经成功地应用于学习机器人技能，以学习扩展环境的动态。具体地说，强化学习将适用于处理声传感器，其概率分布由于物理干扰和多径返回而在整个环境中变化很大。这些现象的性质最初并不是很准确地知道，但会随着机器人的巡逻而学习。具体地说，以一个粗略的初始模型为起点，通过规划、模拟和物理重复相结合的方法，学习在这种现象下的最优机动和巡逻策略。基于模型的信念空间运动规划将用于引导和加速最优策略的情景学习。这项工作将采用一种新的信念空间规划指标，该指标可以降低粗、全局和细、局部规模的规划的计算复杂性。

The Earth's oceans and rivers are very important to our lives, and it is important to understand them in detail, such as how their current flows change over time. While it is difficult to develop detailed models of current flow from first principles, good approximations can be constructed. These coarse models can then be used by an autonomous underwater robot to plan and execute paths. By collecting data during its traversals of the environment, more detailed models can be learned, tested in simulation, and then used to make the robot's actions more reliable. This project will also contribute to the curriculum of WaterBotics, a K-12 program that exposes thousands of students to engineering principles via hands-on learning with underwater robots. The results of the proposed research will be integrated into new educational modules for this program.This project uses reinforcement learning, which has been successfully applied in learning robotic skills, to learn the dynamics of an expansive environment. Specifically, reinforcement learning will be adapted to deal with acoustic sensors whose probability distributions, due to physical disturbances and multi-path returns, vary sharply throughout the environment. The properties of these phenomena are not initially known with high accuracy, but will be learned as the robot patrols. Specifically, with a coarse initial model as a starting point, optimal policies for maneuvering and patrolling under such phenomena will be learned through a combination of planning, simulation, and physical repetition. Model-based belief space motion planning will be used to bootstrap and accelerate the episodic learning of optimal policies. The work will employ a novel belief-space planning metric that reduces the computational complexity of planning both at coarse, global scales and at fine, local scales.