An Inverse Reinforcement Learning Approach for Customizing Automated Lane Change Systems
An Inverse Reinforcement Learning Approach for Customizing Automated Lane Change Systems
复制标题
DOI：
10.1109/tvt.2022.3179332
复制
发表时间：
2022
影响因子：
6.8
通讯作者：
Jundi Liu;L. Boyle;A. Banerjee
中科院分区：
文献类型：
--
作者：
Jundi Liu;L. Boyle;A. Banerjee
车辆自动化旨在加强道路安全，改善驾驶体验。然而，标准系统并没有考虑到用户和驾驶条件的变化。基于用户偏好定制车辆自动化旨在提高用户体验和技术采用率。本研究引入一个系统范例，从自然驾驶数据出发，识别自定义自动变道系统的驾驶行为和风格。首先利用最小先验专家知识的多元泛函主成分分析（MFPCA）提取驾驶行为。对提取的驾驶行为进行聚类，识别驾驶风格。然后使用逆强化学习（IRL）算法从已识别驾驶风格的分组演示中训练自动变道系统，以捕获具有相似驾驶风格的一组驾驶员的偏好。将所提出的定制自动变道系统的性能与(1)在所有样本行程中训练的非定制系统，(2)基于专家编码奖励函数的定制系统，以及(3)使用生成对抗模仿学习（GAIL）算法训练的定制系统进行比较。结果表明，我们的方法在变道动作的预测精度方面优于所有其他系统。此外，我们的方法还可以深入了解不同驾驶风格的典型行为，从而实现自动变道系统的定制。
Vehicle automation seeks to enhance road safety and improve the driving experience. However, a standard system does not account for variations in users and driving conditions. Customizing vehicle automation based on users' preferences aims to improve the user experience and adoption of the technologies. This study introduces a systematic paradigm that starts with naturalistic driving data to identify the driving behaviors and styles for a customized automated lane change system. The driving behaviors are first extracted using Multivariate Functional Principal Component Analysis (MFPCA) with minimum prior expert knowledge. The driving styles are identified by clustering the extracted driving behaviors. An Inverse Reinforcement Learning (IRL) algorithm is then used to train the automated lane change system from grouped demonstrations of the identified driving styles to capture the preferences of a group of drivers with a similar driving style. The performance of the proposed customized automated lane change system is compared to (1) a non-customized system trained on all the sample trips, (2) customized systems built on expert-coded reward functions, and (3) customized systems trained using a Generative Adversarial Imitation Learning (GAIL) algorithm. The results show that our method outperforms all the other systems with respect to the prediction accuracy of the lane change actions. Additionally, our method gains insights on the representative behaviors of different driving styles to enable customization of automated lane change systems.