10 Sequential Parameter Optimization for Mixed-Discrete Problems
343
of this strategy can be found in [25], where an optimization process based on random
forest models has been employed for a high-dimensional algorithm tuning problem.
On the one hand, inherently discrete models are easy to use, since they require
no additional work to adapt them to discrete problems. On the other hand, in a
mixed-variable case, a tree-based model would not be well-suited to represent the
continuous parameters in the mixture. Also, tree-based models may not provide the
useful features that models like Kriging has. For example, uncertainty estimates can
be derived from random forests, but unlike Kriging, these estimates do not go to
zero at observed sample locations.
10.4.2.4 Similarity-Based Models
Similarity-based modeling is a promising strategy that is gaining more and more
interest. Here, suitable measures of similarity are used to model discrete data. With
respect to their interpretation and use, the measures are referred to as similarity
measures, dissimilarity measures, distance measures, correlation measures or kernels. Although this approach is potentially very powerful, it requires the definition
of proper measures. This may be problematic if these measures have to fulfil
further requirements, e.g. definiteness, and if the problem involves different types
of variables.
Fonseca et al. [18] defined similarity-based models that keep a memory of
solutions and estimate the performance of new samples by comparing them to
that memory. Three models from this class are of particular interest: radial basis
function networks (RBFN), support vector machine (SVM) and Kriging. Various
model-based variants applied to different optimization problems can be found in
the literature, e.g. [4, 16, 21, 22, 25, 30, 37, 58]. Several of these works involve the
development of appropriate similarity measures for discrete or mixed search spaces.
As these previous developments indicate, similarity-based models like Kriging
are very promising approaches towards handling mixed and discrete variables.
Hence, we focus on Kriging in the following.
10.4.2.5 Handling Factor Variables in Kriging Model
Kriging is a similarity-based model and assumes that the data follows a multivariate Gaussian distribution, where errors are spatially correlated. A detailed and
comprehensible description of Kriging is given by Forrester et al. [19].
We consider a simplified variant of the optimization problem defined in
Eq. (10.1):
min
x∈R n
f ( x)
343
of this strategy can be found in [25], where an optimization process based on random
forest models has been employed for a high-dimensional algorithm tuning problem.
On the one hand, inherently discrete models are easy to use, since they require
no additional work to adapt them to discrete problems. On the other hand, in a
mixed-variable case, a tree-based model would not be well-suited to represent the
continuous parameters in the mixture. Also, tree-based models may not provide the
useful features that models like Kriging has. For example, uncertainty estimates can
be derived from random forests, but unlike Kriging, these estimates do not go to
zero at observed sample locations.
10.4.2.4 Similarity-Based Models
Similarity-based modeling is a promising strategy that is gaining more and more
interest. Here, suitable measures of similarity are used to model discrete data. With
respect to their interpretation and use, the measures are referred to as similarity
measures, dissimilarity measures, distance measures, correlation measures or kernels. Although this approach is potentially very powerful, it requires the definition
of proper measures. This may be problematic if these measures have to fulfil
further requirements, e.g. definiteness, and if the problem involves different types
of variables.
Fonseca et al. [18] defined similarity-based models that keep a memory of
solutions and estimate the performance of new samples by comparing them to
that memory. Three models from this class are of particular interest: radial basis
function networks (RBFN), support vector machine (SVM) and Kriging. Various
model-based variants applied to different optimization problems can be found in
the literature, e.g. [4, 16, 21, 22, 25, 30, 37, 58]. Several of these works involve the
development of appropriate similarity measures for discrete or mixed search spaces.
As these previous developments indicate, similarity-based models like Kriging
are very promising approaches towards handling mixed and discrete variables.
Hence, we focus on Kriging in the following.
10.4.2.5 Handling Factor Variables in Kriging Model
Kriging is a similarity-based model and assumes that the data follows a multivariate Gaussian distribution, where errors are spatially correlated. A detailed and
comprehensible description of Kriging is given by Forrester et al. [19].
We consider a simplified variant of the optimization problem defined in
Eq. (10.1):
min
x∈R n
f ( x)