In order to verify the scalability of the approaches, we apply the above approach to
different vehicle densities environments. The parameter settings are the same as before.
We gave the results of five vehicles, eight vehicles and 11 vehicles in turn. Figure 3
shows that as the number of vehicles increases, DR-DCG approach always obtains
higher average rewards, indicating that it has better scalability and can be extended to
more complex situations. In summary, through the analysis of vehicle motion characteristics and experiments of different vehicle densities, it shows that our approach can
effectively solve the latent conflict risk between vehicles by coordination lane changing
and further verifies the effectiveness of our approach.
5 Conclusions
Autonomous driving is a typical multi-agent domain, but the application of RL to
multiple vehicles coordination problem has been a challenging task due to the changing
topology of vehicles. This paper proposes a MARL approach based on the combination
of DCG and RL. This approach contributes to the safety and coordination lane change
decision making of autonomous driving which are increasingly concerned by people.
Our work has the following specific points: (1) Propose an approach to construct DCG
for vehicles with latent conflicts based on driving risk evaluation, (2) Propose a distributed MARL approach for multi-vehicle coordination lane change decision making
0
500 1000 1500 2000 2500 3000 3500 4000 4500 5000
0
50
100
150
200
250
300
Individual Learning
Expert
DR-DCG
MOBIL
0
500 1000 1500 2000 2500 3000 3500 4000 4500 5000
0
50
100
150
200
250
Individual Learning
Expert
DR-DCG
MOBIL
0
500 1000 1500 2000 2500 3000 3500 4000 4500 5000
0
20
40
60
80
100
120
140
160
Individual Learning
Expert
DR-DCG
MOBIL
(a) 5 vehicles
(b) 8 vehicles
(c) 11 vehicles
average
reward
average reward
average
reward
Fig. 3. Average reward of different vehicle densities with regard to learning episode
Coordinated Learning for Lane Changing Based on Coordination …
605
different vehicle densities environments. The parameter settings are the same as before.
We gave the results of five vehicles, eight vehicles and 11 vehicles in turn. Figure 3
shows that as the number of vehicles increases, DR-DCG approach always obtains
higher average rewards, indicating that it has better scalability and can be extended to
more complex situations. In summary, through the analysis of vehicle motion characteristics and experiments of different vehicle densities, it shows that our approach can
effectively solve the latent conflict risk between vehicles by coordination lane changing
and further verifies the effectiveness of our approach.
5 Conclusions
Autonomous driving is a typical multi-agent domain, but the application of RL to
multiple vehicles coordination problem has been a challenging task due to the changing
topology of vehicles. This paper proposes a MARL approach based on the combination
of DCG and RL. This approach contributes to the safety and coordination lane change
decision making of autonomous driving which are increasingly concerned by people.
Our work has the following specific points: (1) Propose an approach to construct DCG
for vehicles with latent conflicts based on driving risk evaluation, (2) Propose a distributed MARL approach for multi-vehicle coordination lane change decision making
0
500 1000 1500 2000 2500 3000 3500 4000 4500 5000
0
50
100
150
200
250
300
Individual Learning
Expert
DR-DCG
MOBIL
0
500 1000 1500 2000 2500 3000 3500 4000 4500 5000
0
50
100
150
200
250
Individual Learning
Expert
DR-DCG
MOBIL
0
500 1000 1500 2000 2500 3000 3500 4000 4500 5000
0
20
40
60
80
100
120
140
160
Individual Learning
Expert
DR-DCG
MOBIL
(a) 5 vehicles
(b) 8 vehicles
(c) 11 vehicles
average
reward
average reward
average
reward
Fig. 3. Average reward of different vehicle densities with regard to learning episode
Coordinated Learning for Lane Changing Based on Coordination …
605