338
L. Gentile et al.
component analysis (PCA). This method is especially used for problems with only
continuous variables [13].
Contrarily, other methods are based on nonlinear mapping and projection. Space
mapping (SM) intends to map the design space of a “coarse” and low-dimensional
model to a fine, “expensive” and higher-dimensional one [5]. A good survey of
related approaches is given in [46].
Another class of dimensionality reduction approaches is based on unsupervised
learning. Two promising examples are autoencoders and self-organizing maps.
Autoencoders are neural networks that aim to reconstruct their own inputs. As
such, an encoder network maps from the high-dimensional space to the coded
space. Then, a decoder network maps back to the high-dimensional space, with
as little loss of information as possible. By constraining the coded space to have
a smaller dimension than the input space, the autoencoder is forced to learn the
most salient features of the input data [42]. Self-organizing maps are a particularly
interesting class of unsupervised systems that are based on competitive learning.
The output neurons compete among themselves to be activated. Hence, only one
is activated at one particular time. This competitive system forces the neurons to
organize themselves. Commonly, self-organizing maps target to map from the highdimensional space to one- or two-dimensional space [29].
10.3.3 Uncertainty
A large variety of optimization problems in scheduling, finance, transportation and
engineering design requires that decisions are made in the presence of uncertainty.
Uncertainties are present in all real-world application problems, e.g. due to inaccuracies in the manufacturing process, uncertain operating conditions or system
component failures. However, different forms of uncertainties can be distinguished;
a good overview can be found in [26]. In the following a description of the
most common forms of uncertainty in real-world application problems, noise and
robustness, will be given.
An optimization problem is considered subjected to noise if the objective function is perturbed. This can be due to several factors such as sensor measurements
errors or heuristic simulations. Mathematically, noise is often assumed normally
distributed with zero mean and variance [26].
In other cases, perturbation can afflict the design variables. Therefore, it is often
required that the optimal solution should still work satisfactorily when the design
variables change slightly, e.g. due to manufacturing tolerances [26].
In these cases, successfully performing global optimization means facing a
variety of challenging issues. Resources have to be allocated to perform an
uncertainty analysis in order to direct the research to stable and robust optima.
The correct balance between exploitation, exploration and uncertainty quantification has to be addressed. Furthermore, adopting surrogate model-based optimization
in noisy functions causes an additional problem. The use of derivative-based
L. Gentile et al.
component analysis (PCA). This method is especially used for problems with only
continuous variables [13].
Contrarily, other methods are based on nonlinear mapping and projection. Space
mapping (SM) intends to map the design space of a “coarse” and low-dimensional
model to a fine, “expensive” and higher-dimensional one [5]. A good survey of
related approaches is given in [46].
Another class of dimensionality reduction approaches is based on unsupervised
learning. Two promising examples are autoencoders and self-organizing maps.
Autoencoders are neural networks that aim to reconstruct their own inputs. As
such, an encoder network maps from the high-dimensional space to the coded
space. Then, a decoder network maps back to the high-dimensional space, with
as little loss of information as possible. By constraining the coded space to have
a smaller dimension than the input space, the autoencoder is forced to learn the
most salient features of the input data [42]. Self-organizing maps are a particularly
interesting class of unsupervised systems that are based on competitive learning.
The output neurons compete among themselves to be activated. Hence, only one
is activated at one particular time. This competitive system forces the neurons to
organize themselves. Commonly, self-organizing maps target to map from the highdimensional space to one- or two-dimensional space [29].
10.3.3 Uncertainty
A large variety of optimization problems in scheduling, finance, transportation and
engineering design requires that decisions are made in the presence of uncertainty.
Uncertainties are present in all real-world application problems, e.g. due to inaccuracies in the manufacturing process, uncertain operating conditions or system
component failures. However, different forms of uncertainties can be distinguished;
a good overview can be found in [26]. In the following a description of the
most common forms of uncertainty in real-world application problems, noise and
robustness, will be given.
An optimization problem is considered subjected to noise if the objective function is perturbed. This can be due to several factors such as sensor measurements
errors or heuristic simulations. Mathematically, noise is often assumed normally
distributed with zero mean and variance [26].
In other cases, perturbation can afflict the design variables. Therefore, it is often
required that the optimal solution should still work satisfactorily when the design
variables change slightly, e.g. due to manufacturing tolerances [26].
In these cases, successfully performing global optimization means facing a
variety of challenging issues. Resources have to be allocated to perform an
uncertainty analysis in order to direct the research to stable and robust optima.
The correct balance between exploitation, exploration and uncertainty quantification has to be addressed. Furthermore, adopting surrogate model-based optimization
in noisy functions causes an additional problem. The use of derivative-based