Coordinated Learning for Lane Changing
Based on Coordination Graph
and Reinforcement Learning
Hao Zuo
(&)
, Huaiwei Si, Nan Ding, Xin Wang, and Guozhen Tan
Dalian University of Technology, Dalian, China
1507154320@qq.com
Abstract. The rapid development of autonomous driving has aroused widespread interest in academia and industry. Due to vehicular mobility, it is not
feasible to adopt the existing coordinated learning approach based on static
topology directly. To solve this issue, we propose a coordinated learning
approach based on dynamic graph and reinforcement learning to enable distributed learning of cooperative lane changing. The dynamic graph model is
constructed by evaluating the driving risk between each pair of vehicles. The
lane change decision making of the vehicles is guided by the global optimal
action based on the dynamic coordinated graph. Experiments verify that the
proposed approach can achieve the accuracy and safety of lane change decision
making, and with the increase in the number of vehicles, the approach has good
scalability.
Keywords: Coordination graph Á Reinforcement learning Á Lane change Á
Autonomous driving
1 Introduction
Automated driving is one of the most important artificial intelligence applications,
which attracts a lot of interest from technology companies and research institutions [2].
Baidu Apollo Project, Google Self-Drive Car and DARPA Urban Challenge et al. show
great potential for autonomous driving. In the field of autonomous driving, lanechanging processes have received great attention [1]. Lane change decision is considered to be essential driving behaviors that affect both traffic safety [16] and efficiency [9]. However, due to the variability of road traffic conditions, even with
extensive expert experience, manual design rules to deal with all possible lane change
situations may lead to overly simple and sub-optimal decision.
The above problem can be solved using the reinforcement learning (RL) method.
A large number of studies [5, 15] have applied RL to autonomous driving. However, all
of these studies only focus on learning the driving decisions for a single vehicle, which
contradicts the fact that autonomous driving is a typical multi-agent system
(MAS) [17]. Some scholars [4, 13] have studied how to design control strategies
through vehicles communication to coordinate multiple vehicles. However, all of these
© Springer Nature Singapore Pte Ltd. 2020
Q. Liang et al. (Eds.): Artificial Intelligence in China, LNEE 572, pp. 599–607, 2020.
https://doi.org/10.1007/978-981-15-0187-6_71
Based on Coordination Graph
and Reinforcement Learning
Hao Zuo
(&)
, Huaiwei Si, Nan Ding, Xin Wang, and Guozhen Tan
Dalian University of Technology, Dalian, China
1507154320@qq.com
Abstract. The rapid development of autonomous driving has aroused widespread interest in academia and industry. Due to vehicular mobility, it is not
feasible to adopt the existing coordinated learning approach based on static
topology directly. To solve this issue, we propose a coordinated learning
approach based on dynamic graph and reinforcement learning to enable distributed learning of cooperative lane changing. The dynamic graph model is
constructed by evaluating the driving risk between each pair of vehicles. The
lane change decision making of the vehicles is guided by the global optimal
action based on the dynamic coordinated graph. Experiments verify that the
proposed approach can achieve the accuracy and safety of lane change decision
making, and with the increase in the number of vehicles, the approach has good
scalability.
Keywords: Coordination graph Á Reinforcement learning Á Lane change Á
Autonomous driving
1 Introduction
Automated driving is one of the most important artificial intelligence applications,
which attracts a lot of interest from technology companies and research institutions [2].
Baidu Apollo Project, Google Self-Drive Car and DARPA Urban Challenge et al. show
great potential for autonomous driving. In the field of autonomous driving, lanechanging processes have received great attention [1]. Lane change decision is considered to be essential driving behaviors that affect both traffic safety [16] and efficiency [9]. However, due to the variability of road traffic conditions, even with
extensive expert experience, manual design rules to deal with all possible lane change
situations may lead to overly simple and sub-optimal decision.
The above problem can be solved using the reinforcement learning (RL) method.
A large number of studies [5, 15] have applied RL to autonomous driving. However, all
of these studies only focus on learning the driving decisions for a single vehicle, which
contradicts the fact that autonomous driving is a typical multi-agent system
(MAS) [17]. Some scholars [4, 13] have studied how to design control strategies
through vehicles communication to coordinate multiple vehicles. However, all of these
© Springer Nature Singapore Pte Ltd. 2020
Q. Liang et al. (Eds.): Artificial Intelligence in China, LNEE 572, pp. 599–607, 2020.
https://doi.org/10.1007/978-981-15-0187-6_71