[10]. DR-DCG learning denotes the coordinated learning approach proposed by us, and
LCC denotes the local cooperative control model. We simulate the learning process at
two-lane highway.
We averaged more than 10 runs as the final result. Since individual learning of
optimal behavior may not guarantee the best group performance, the comparison with
individual learning approach shows the effectiveness of the DR-DCG approach in
decision making of cooperative lane changing. In the proposed approach, based on the
independent MDP, the two-vehicle local cooperative control model is transferred to the
edge of the DCG we constructed after pretraining and calculated all the payoff contribution between the pairs of vehicle on the edges. Figure 2a shows the learning of the
average reward over five vehicles. It can be seen from the figure that the local cooperative control model can obtain better results than individual learning, indicating that it
can accurately calculate the coordination payoff contribution between the pairs of
vehicles that have latent conflicts relations. More obvious is that the DR-DCG
approach we proposed can achieve a higher average reward than the other approaches,
which shows the benefits of coordination between vehicles. Figure 2b–d plots the
microscopic mobility of different approaches include the minimum forward distance,
safe travel time and number of vehicles on the driving lane.
2.5
3
3.5
4
4.5
5
Individual Learning
Expert
DR-DCG
MOBIL
LCC
0
50
100
150
200
250
300
350
400
Individual Learning
Expert
DR-DCG
MOBIL
LCC
0
10
20
30
40
50
60
70
80
Individual Learning
Expert
DR-DCG
MOBIL
LCC
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
0
20
40
60
80
100
120
140
160
Individual Learning
Expert
DR-DCG
MOBIL
LCC
(a) average reward
(b) minimum forward distance
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
(c) safe travel time
(d) the number of vehicles on
the driving lane
Fig. 2. Average reward and microscopic mobility of different approaches
604
H. Zuo et al.
LCC denotes the local cooperative control model. We simulate the learning process at
two-lane highway.
We averaged more than 10 runs as the final result. Since individual learning of
optimal behavior may not guarantee the best group performance, the comparison with
individual learning approach shows the effectiveness of the DR-DCG approach in
decision making of cooperative lane changing. In the proposed approach, based on the
independent MDP, the two-vehicle local cooperative control model is transferred to the
edge of the DCG we constructed after pretraining and calculated all the payoff contribution between the pairs of vehicle on the edges. Figure 2a shows the learning of the
average reward over five vehicles. It can be seen from the figure that the local cooperative control model can obtain better results than individual learning, indicating that it
can accurately calculate the coordination payoff contribution between the pairs of
vehicles that have latent conflicts relations. More obvious is that the DR-DCG
approach we proposed can achieve a higher average reward than the other approaches,
which shows the benefits of coordination between vehicles. Figure 2b–d plots the
microscopic mobility of different approaches include the minimum forward distance,
safe travel time and number of vehicles on the driving lane.
2.5
3
3.5
4
4.5
5
Individual Learning
Expert
DR-DCG
MOBIL
LCC
0
50
100
150
200
250
300
350
400
Individual Learning
Expert
DR-DCG
MOBIL
LCC
0
10
20
30
40
50
60
70
80
Individual Learning
Expert
DR-DCG
MOBIL
LCC
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
0
20
40
60
80
100
120
140
160
Individual Learning
Expert
DR-DCG
MOBIL
LCC
(a) average reward
(b) minimum forward distance
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
(c) safe travel time
(d) the number of vehicles on
the driving lane
Fig. 2. Average reward and microscopic mobility of different approaches
604
H. Zuo et al.