Multi-Agent Reinforcement Learning for Order-dispatching via Order-Vehicle Distribution Matching
Multi-Agent Reinforcement Learning for Order-dispatching via Order-Vehicle Distribution Matching
复制标题
DOI：
10.1145/3357384.3357799
复制
发表时间：
2019-10
期刊：
影响因子：
--
通讯作者：
Ming Zhou;Jiarui Jin;Weinan Zhang;Zhiwei Qin;Yan Jiao;Chenxi Wang;Guobin Wu;Yong Yu;
中科院分区：
文献类型：
--
作者：
Ming Zhou;Jiarui Jin;Weinan Zhang;Zhiwei Qin;Yan Jiao;Chenxi Wang;Guobin Wu;Yong Yu;
提高订单到车辆的调度效率是网约车系统的研究热点。现有的订单调度解决方案大多是集中控制，这需要考虑所有可能的匹配之间的可用订单和车辆。对于大规模的拼车平台来说，每秒钟都有成千上万的车辆和订单需要匹配，这是非常高的计算成本。针对大规模订单调度问题，提出了一种基于多智能体强化学习的分散执行订单调度方法.不同于以往的合作多智能体强化学习算法，在我们的方法中，所有的代理独立工作的指导下，从评估的联合政策，因为有没有必要的通信或明确的合作代理之间。此外，我们在每个时间步使用KL发散优化来加快学习过程并平衡车辆（供应）和订单（需求）。在解释性环境和真实世界模拟器上的实验表明，所提出的方法在各种交通环境中的累积驾驶员收入（ADI）和订单响应率（ORR）方面优于基线。此外，在滴滴出行在线平台的支持下，我们设计了一个混合系统来部署我们的模型。
Improving the efficiency of dispatching orders to vehicles is a research hotspot in online ride-hailing systems. Most of the existing solutions for order-dispatching are centralized controlling, which require to consider all possible matches between available orders and vehicles. For large-scale ride-sharing platforms, there are thousands of vehicles and orders to be matched at every second which is of very high computational cost. In this paper, we propose a decentralized execution order-dispatching method based on multi-agent reinforcement learning to address the large-scale order-dispatching problem. Different from the previous cooperative multi-agent reinforcement learning algorithms, in our method, all agents work independently with the guidance from an evaluation of the joint policy since there is no need for communication or explicit cooperation between agents. Furthermore, we use KL-divergence optimization at each time step to speed up the learning process and to balance the vehicles (supply) and orders (demand). Experiments on both the explanatory environment and real-world simulator show that the proposed method outperforms the baselines in terms of accumulated driver income (ADI) and Order Response Rate (ORR) in various traffic environments. Besides, with the support of the online platform of Didi Chuxing, we designed a hybrid system to deploy our model.