10 Sequential Parameter Optimization for Mixed-Discrete Problems
341
Fig. 10.1 Example of three
different sampling methods in
creating an initial design of
size 4 in a two-dimensional
space: Sobol sequence (red
triangles), Latin hypercube
(black crosses) and full
factorial design (blue circles)
10.4.1.3 Factorial Designs
In the field of design of experiments, a set of statistically well-profound designs
have emerged, which are commonly applied to analyse and optimize industrial
problems. Common designs of this field are full factorial, fractional factorial, BoxBehnken and central composite [36]. All these designs are designated to fit linear
models for the response surface methodology, commonly with second-order and
quadratic effects. For example, in a full factorial design for two variables with two
levels for each design variable, a set of 2 2 evenly spaced points is determined. In
case of continuous variables, they are determined by [r min , r max ]. With this design,
we are able to analyse main and second-order effects. For quadratic effects, centre
points need to be added. A full factorial design has the disadvantage of requiring an
exponentially increasing number of experiments with rising number of variables. To
prohibit an infeasible number of experiments, usually optimized fractional factorial
or other screening designs are utilized.
From Fig. 10.1, one can see that, contrary to LHS and the Sobol sequence, a full
factorial sampling exhibits a particular grid structure that eases distinguishing the
effects of all design variables on the objective function.
10.4.2 Modeling
10.4.2.1 Modeling in Mixed-Integer Space
Once the first dataset has been created and observed, it is used to train a surrogate
model that aims to replicate the behaviour of the objective function (see Algo-
341
Fig. 10.1 Example of three
different sampling methods in
creating an initial design of
size 4 in a two-dimensional
space: Sobol sequence (red
triangles), Latin hypercube
(black crosses) and full
factorial design (blue circles)
10.4.1.3 Factorial Designs
In the field of design of experiments, a set of statistically well-profound designs
have emerged, which are commonly applied to analyse and optimize industrial
problems. Common designs of this field are full factorial, fractional factorial, BoxBehnken and central composite [36]. All these designs are designated to fit linear
models for the response surface methodology, commonly with second-order and
quadratic effects. For example, in a full factorial design for two variables with two
levels for each design variable, a set of 2 2 evenly spaced points is determined. In
case of continuous variables, they are determined by [r min , r max ]. With this design,
we are able to analyse main and second-order effects. For quadratic effects, centre
points need to be added. A full factorial design has the disadvantage of requiring an
exponentially increasing number of experiments with rising number of variables. To
prohibit an infeasible number of experiments, usually optimized fractional factorial
or other screening designs are utilized.
From Fig. 10.1, one can see that, contrary to LHS and the Sobol sequence, a full
factorial sampling exhibits a particular grid structure that eases distinguishing the
effects of all design variables on the objective function.
10.4.2 Modeling
10.4.2.1 Modeling in Mixed-Integer Space
Once the first dataset has been created and observed, it is used to train a surrogate
model that aims to replicate the behaviour of the objective function (see Algo-