Through experimental tests (Fig. 2), the load balancing scheduling operation time is
23.562 s, the Spark scheduling operation time is 30.17 s, and the actual optimization
percentage is 22%. It shows that during the execution of the task, the load scheduling
avoids the node 3 with high load, but the native Spark scheduling cannot do it, and the
node 3 drags the execution progress of the native Spark.
Through experimental tests (Fig. 3), the load balancing scheduling operation time is
21.637 s, and the Spark scheduling operation time is 23.054 s. In test 3, although the
nodes are all empty, the load balancing is faster than the running time of the native
Spark schedule.
Fig. 2. Experimental test 2
Fig. 3. Experimental test 3
136
Y. Liang et al.