Online optimization algorithms 203
of the RCDS, RSimplex, N-M simplex, and GP methods for the Rosenbrock-4
function, using the cases ranking the 30th out of the 100 runs for each noise
level as examples.
While the Rosenbrock function is challenging for the RCDS method in
terms of the convergence efficiency, real-life online optimization applications
typically do not have such a behavior in the parameter space. For many problems, the design operation condition corresponds to an extremum of the objective function, around which the performance behaves as a quadratic function
of the control parameters. The RCDS method would be an ideal algorithm
to bring the machine toward the design performance in such cases, especially
when the starting point is close to the optimum.
7.3.5 Testing of multi-objective optimization with stochastic algorithms
Deterministic optimization algorithms usually converge to the nearby local
minimum. If in an application the objective function has local minima that
could intercept the convergence path of the deterministic algorithms toward
the global minimum, stochastic algorithms can be used to look for the global
minimum. Stochastic algorithms may not be as efficient, but typically have
better ability to overcome the attraction of local minima.
The NSGA-II genetic algorithm and the particle swarm optimization
(PSO) algorithm are two popular stochastic algorithms that are used for accelerator designs. The MG-GPO algorithm is a new stochastic optimization
method. These algorithms naturally apply to multi-objective problems. We
tested their performances under noise with a multi-objective problem, using
the two objective functions, f 1 (x) in Eq. (7.31) and the Rosenbrock-4 function.
The parameter range is the same as used in the previous tests. For all three
methods, the population of solutions is set to N = 50 and the algorithms are
run for 40 generations in the tests. The initial population is randomly chosen
from the entire parameter space with a uniform distribution.
In the NSGA-II setup, 90% of the new solutions are generated through
crossover and the 10% by mutation. The control parameters for random number generation in Eq. (7.19) and Eq. (7.21) are set to µ c = µ m = 20.
For the PSO setup, the weight factors for velocity calculation in Eq. (7.23)
are set to w = 0.4 and c 1 = c 2 = 1. The components of all initial velocities
are drawn from the uniform distribution between 0 and 0.1.
The MG-GPO setup assumes θ = 0.4 for the correlation length parameter.
The GP-LCB acquisition function uses κ = 0.5. There are 40N trial solutions,
half generated with crossover and the other half with mutation, before the
selection by the GP model is applied.
Three levels of noise are applied to the function evaluations as was done
in the single-objective tests. About 2000 solutions are evaluated for each algorithm, out of which the best 100 solutions are selected through non-dominated
sorting. The results are shown in Figure 7.15 for NSGA-II and PSO. The final
fronts of MG-GPO (not shown) are similar to that of PSO. At the low noise
of the RCDS, RSimplex, N-M simplex, and GP methods for the Rosenbrock-4
function, using the cases ranking the 30th out of the 100 runs for each noise
level as examples.
While the Rosenbrock function is challenging for the RCDS method in
terms of the convergence efficiency, real-life online optimization applications
typically do not have such a behavior in the parameter space. For many problems, the design operation condition corresponds to an extremum of the objective function, around which the performance behaves as a quadratic function
of the control parameters. The RCDS method would be an ideal algorithm
to bring the machine toward the design performance in such cases, especially
when the starting point is close to the optimum.
7.3.5 Testing of multi-objective optimization with stochastic algorithms
Deterministic optimization algorithms usually converge to the nearby local
minimum. If in an application the objective function has local minima that
could intercept the convergence path of the deterministic algorithms toward
the global minimum, stochastic algorithms can be used to look for the global
minimum. Stochastic algorithms may not be as efficient, but typically have
better ability to overcome the attraction of local minima.
The NSGA-II genetic algorithm and the particle swarm optimization
(PSO) algorithm are two popular stochastic algorithms that are used for accelerator designs. The MG-GPO algorithm is a new stochastic optimization
method. These algorithms naturally apply to multi-objective problems. We
tested their performances under noise with a multi-objective problem, using
the two objective functions, f 1 (x) in Eq. (7.31) and the Rosenbrock-4 function.
The parameter range is the same as used in the previous tests. For all three
methods, the population of solutions is set to N = 50 and the algorithms are
run for 40 generations in the tests. The initial population is randomly chosen
from the entire parameter space with a uniform distribution.
In the NSGA-II setup, 90% of the new solutions are generated through
crossover and the 10% by mutation. The control parameters for random number generation in Eq. (7.19) and Eq. (7.21) are set to µ c = µ m = 20.
For the PSO setup, the weight factors for velocity calculation in Eq. (7.23)
are set to w = 0.4 and c 1 = c 2 = 1. The components of all initial velocities
are drawn from the uniform distribution between 0 and 0.1.
The MG-GPO setup assumes θ = 0.4 for the correlation length parameter.
The GP-LCB acquisition function uses κ = 0.5. There are 40N trial solutions,
half generated with crossover and the other half with mutation, before the
selection by the GP model is applied.
Three levels of noise are applied to the function evaluations as was done
in the single-objective tests. About 2000 solutions are evaluated for each algorithm, out of which the best 100 solutions are selected through non-dominated
sorting. The results are shown in Figure 7.15 for NSGA-II and PSO. The final
fronts of MG-GPO (not shown) are similar to that of PSO. At the low noise