S&AS: FND: Uncertainty-Aware Safe Deep Reinforcement Learning

机器人设计者不能总是预见到现实世界中的所有可能性；因此，机器人将需要使用能够学习适应其环境中意外变化的算法。然而，如果机器人要在现实世界中学习和适应，它们必须以在学习中继续保持安全的方式进行适应。这个项目开发了一些方法，确保机器人在学习和适应环境中意想不到的变化时继续保持安全。通过使机器人认识到自己的不确定性，这些方法可以帮助机器人更安全地操作。此外，这些方法将使机器人能够适应环境中不可预见的变化。这些方法将应用于自动驾驶，使机器人能够在摩擦程度不同的表面或在原本可能不安全的崎岖地形上安全操作。这些方法将使自动驾驶车辆在恶劣天气条件下的城市驾驶更加安全，也可以在越野环境中进行搜索和救援行动，巡逻车辆检测动物偷猎者，以及其他应用。本研究开发了一套感知不确定性的安全机器人学习方法。这些方法将使机器人能够估计其行动效果的不确定性；基于估计的不确定性，机器人将确定如何在安全和谨慎操作的同时提高其性能。此外，这些方法将使用估计的不确定性来确定机器人应该如何调整其参数以有效地响应环境变化。该项目中的方法将在复杂的策略类上操作，例如那些由深度强化学习训练的神经网络表示的策略类。这种方法将使机器人能够实现安全和自适应的长期自主。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Robot designers cannot always anticipate all real-world eventualities; hence robots will need to use algorithms that can learn to adapt to unexpected changes in their environment. However, if robots are to learn and adapt in the real world, they must adapt in ways that continue to maintain safety while learning. This project develops methods that ensure that robots continue to be safe even as they learn and adapt to unexpected changes in their environment. By enabling robots to be cognizant of their uncertainty, these methods can help robots operate more safely. Further, the methods will enable robots to be adaptive to unforeseen changes in their environment. These methods will be applied to autonomous driving, to enable robots to operate safely on surfaces with different levels of friction or on uneven terrain that might otherwise be unsafe to operate on. Such methods will enable safer autonomous vehicles for city driving in poor weather conditions, as well as for operating in off-road settings for search and rescue operations, patrol vehicles to detect animal poachers, and for other applications.This research develops a set of methods for uncertainty-aware safe robot learning. These methods will enable a robot to estimate the uncertainty of the effect of its actions; based on the estimated uncertainty, the robot will determine how to improve its performance while operating safely and cautiously. Furthermore, these methods will use the estimated uncertainty to determine how the robot should adapt its parameters to efficiently respond to environmental changes. The methods in this project will operate on complex policy classes such as those represented by neural networks trained with deep reinforcement learning. Such an approach will enable robots to achieve safe and adaptive long-term autonomy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.