336
L. Gentile et al.
would help escaping misleading local minima. On the contrary, at the end of
the process, exploitation would gain more importance, assuming that the most
promising area has already been identified. For these reasons, algorithms able to
auto-tune (i.e. perform on-the-fly parameter control) all along the optimization
process, such as [31, 40], can be a promising choice. More details on auto-tune
can be found in Chap. 11. The combination of these features strongly increases the
problem difficulty. Optimization algorithms need to be designed to solve specific
problems, presenting different combinations of these features. We will focus on two
essential problems in this chapter, namely, the dimensionality and the uncertainty.
10.3.2 High Dimensionality
The dimensionality of the search space is defined by the number of design variables.
Referring to the notation in Eq. (10.1), we define the dimensionality as n = n r +
n z + n d . It is intuitive that a large number of variables pose a demanding challenge
that affects many algorithm’s aspects. Dealing with this particular problem requires
a great modeling capability, a huge amount of acquired data and, consequently, a
large budget of objective function evaluations. Every modeling technique requires a
sufficiently large dataset such that an accurate model can be trained.
Furthermore, high dimensionality leads to severe practical issues in the development of surrogate models as well. Kriging, besides linear regression, is one of
the most popular techniques in SMBO, see [8]. For example, depending on the
employed distance measures [1], it is widely recognized that Kriging may perform
poorly for problems with more than approximately 20 variables [19].
A spectrum of countermeasures to these issues comes from different fields of
engineering and data analytics. Most commonly, methods attempt to use some
screening or mapping approach. The former attempts to remove insignificant
variables, while the latter attempts to map the original search space to a lowdimensional subspace.
10.3.2.1 Screening
In the effort of reducing the problem complexity and dimensionality, screening
identifies and retains important input variables and interaction terms. Screening is
often implemented via sampling and the analysis of sampling results [46].
Sensitivity analysis studies how the variability of a function’s output responds to
changes of its inputs. It includes local and global sensitivity analyses. The local
sensitivity indicates the variability of the output with respect to input variable
changes at a given point; in other words it evaluates the numeric partial derivatives.
The global sensitivity, contrarily, explains the global variability of the output over
the entire design space, which provides an overall view of the impact of input
L. Gentile et al.
would help escaping misleading local minima. On the contrary, at the end of
the process, exploitation would gain more importance, assuming that the most
promising area has already been identified. For these reasons, algorithms able to
auto-tune (i.e. perform on-the-fly parameter control) all along the optimization
process, such as [31, 40], can be a promising choice. More details on auto-tune
can be found in Chap. 11. The combination of these features strongly increases the
problem difficulty. Optimization algorithms need to be designed to solve specific
problems, presenting different combinations of these features. We will focus on two
essential problems in this chapter, namely, the dimensionality and the uncertainty.
10.3.2 High Dimensionality
The dimensionality of the search space is defined by the number of design variables.
Referring to the notation in Eq. (10.1), we define the dimensionality as n = n r +
n z + n d . It is intuitive that a large number of variables pose a demanding challenge
that affects many algorithm’s aspects. Dealing with this particular problem requires
a great modeling capability, a huge amount of acquired data and, consequently, a
large budget of objective function evaluations. Every modeling technique requires a
sufficiently large dataset such that an accurate model can be trained.
Furthermore, high dimensionality leads to severe practical issues in the development of surrogate models as well. Kriging, besides linear regression, is one of
the most popular techniques in SMBO, see [8]. For example, depending on the
employed distance measures [1], it is widely recognized that Kriging may perform
poorly for problems with more than approximately 20 variables [19].
A spectrum of countermeasures to these issues comes from different fields of
engineering and data analytics. Most commonly, methods attempt to use some
screening or mapping approach. The former attempts to remove insignificant
variables, while the latter attempts to map the original search space to a lowdimensional subspace.
10.3.2.1 Screening
In the effort of reducing the problem complexity and dimensionality, screening
identifies and retains important input variables and interaction terms. Screening is
often implemented via sampling and the analysis of sampling results [46].
Sensitivity analysis studies how the variability of a function’s output responds to
changes of its inputs. It includes local and global sensitivity analyses. The local
sensitivity indicates the variability of the output with respect to input variable
changes at a given point; in other words it evaluates the numeric partial derivatives.
The global sensitivity, contrarily, explains the global variability of the output over
the entire design space, which provides an overall view of the impact of input