Cross-Layer Uncertainty-Aware Reinforcement Learning for Safe Autonomous Driving

自动驾驶有着巨大的市场，在学术界和工业界都受到了极大的关注。为了处理复杂的场景，自动驾驶车辆(AVs)将使用强化学习(RL)来设计功能层的高级规划器，但在从模拟到真实的传输过程中总是会遇到安全问题。其中一个主要挑战是，目前的功能层设计实践没有充分考虑体系结构层(如软件层和硬件层)的不确定性。本项目将通过全面研究RL和架构层不确定性之间的相互作用来解决这一开放挑战。具体地说，我们将在仿真平台上构建虚拟AD场景，并基于真实世界数据(WP1)对架构层不确定性进行形式化建模。通过跨层不确定性感知RL(WP2)的设计，讨论了不确定性对RL的影响。相反，我们还将通过计算给定RL对其具有健壮性的最大软件/硬件不确定性模式的Pareto前沿(WP3)来研究RL相对于跨层不确定性的健壮性。将进行广泛的分析，包括验证(WP2，WP3)、仿真(WP2，WP3)和真实世界实验(WP4)。该项目的成功将极大地提高RL在AD中的实用性，并对其他机器人应用产生更广泛的影响。

Autonomous driving (AD) has a huge market and IS receiving enormous attention in both academia and industry. To deal with complex scenarios, autonomous vehicles (AVs) will use reinforcement learning (RL) to design high-level planners in the functional layer but always suffer from safety issues during sim-to-real transfer. One of the main challenges is that the current practice of functional-layer design does not sufficiently consider the uncertainty in the architecture layer, e.g., the software layer and hardware layer. This open challenge will be tackled in this project by a comprehensive study of the interaction between RL and architecture-layer uncertainty. Specifically, we will build virtual AD scenarios on the simulation platform with formal modeling of architecture-layer uncertainty based on real-world data (WP1). The impact of uncertainties on RL will be discussed via the design of cross-layer uncertainty-aware RL (WP2). Inversely, we will also study the robustness of an RL with respect to cross-layer uncertainty by computing the Pareto front of the largest software/hardware uncertainty patterns that a given RL is robust to (WP3). Extensive analysis including verification (WP2, WP3), simulation (WP2, WP3), and real-world experiments (WP4) will be carried out. The success of this project will greatly improve the practicability of RL in AD with a broader impact on other robotics applications.