Learning Interpretable, High-Performing Policies for Autonomous Driving
Learning Interpretable, High-Performing Policies for Autonomous Driving
复制标题
DOI：
10.15607/rss.2022.xviii.068
复制
发表时间：
2022-02
期刊：
影响因子：
--
通讯作者：
Rohan R. Paleja;Yaru Niu;Andrew Silva;Chace Ritchie;Sugju Choi;M. Gombolay
中科院分区：
文献类型：
--
作者：
Rohan R. Paleja;Yaru Niu;Andrew Silva;Chace Ritchie;Sugju Choi;M. Gombolay
强化学习（RL）中基于直觉的方法在自动驾驶汽车的学习策略方面取得了巨大的成功。虽然这些方法的性能保证了现实世界的采用，但这些政策缺乏可解释性，限制了自动驾驶（AD）安全关键和法律监管领域的可部署性。AD需要可解释和可验证的控制策略，以保持高性能。我们提出了可解释的连续控制树（ICCT），这是一种基于树的模型，可以通过现代的，基于梯度的RL方法进行优化，以产生高性能的，可解释的策略。我们的方法的关键是一个程序，允许直接优化的稀疏决策树表示。我们根据六个领域的基线验证了ICCT，表明ICCT能够学习可解释的政策表示，在AD场景中，这些政策表示与基线持平或优于基线高达33%，同时与深度学习基线相比，政策参数的数量减少了300 - 600倍。此外，我们通过一个14辆车的物理机器人演示证明了我们的ICCT的可解释性和实用性。
Gradient-based approaches in reinforcement learning (RL) have achieved tremendous success in learning policies for autonomous vehicles. While the performance of these approaches warrants real-world adoption, these policies lack interpretability, limiting deployability in the safety-critical and legally-regulated domain of autonomous driving (AD). AD requires interpretable and verifiable control policies that maintain high performance. We propose Interpretable Continuous Control Trees (ICCTs), a tree-based model that can be optimized via modern, gradient-based, RL approaches to produce high-performing, interpretable policies. The key to our approach is a procedure for allowing direct optimization in a sparse decision-tree-like representation. We validate ICCTs against baselines across six domains, showing that ICCTs are capable of learning interpretable policy representations that parity or outperform baselines by up to 33% in AD scenarios while achieving a 300x-600x reduction in the number of policy parameters against deep learning baselines. Furthermore, we demonstrate the interpretability and utility of our ICCTs through a 14-car physical robot demonstration.