342
L. Gentile et al.
rithm 1). In SPOT, a surrogate model is used to determine promising candidate
solutions. To that end, it aims to learn the relation between problem variables and
the corresponding function’s response.
Compared to their frequent use for real-valued problems, surrogate model-driven
approaches are less often used in mixed optimization [27]. According to BartzBeielstein and Zaefferer [8], few expensive, real-world optimization problems of
this type have been brought to the science community’s attention, e.g. in the engineering domain [3, 50, 51, 54], bioinformatics [41] or data science [47]. One reason
for the scarce use of discrete surrogate model-based optimization is the availability
of suitable methods. Bartz-Beielstein and Zaefferer [8] identified six strategies for
surrogate modeling in mixed-discrete search spaces: the naive approach, customized
models, inherently discrete models, feature extraction, mapping and similaritybased models. These strategies explain how modeling techniques can be used in
the general cases. The six strategies are not mutually exclusive. Some methods may
belong to several categories or combine different strategies. Here, we focus on three
of the six strategies: the naive approach, inherently discrete models and similaritybased models since these are more commonly used.
10.4.2.2 The Naive Approach
The naive approach to discrete modeling is to ignore the discrete nature of the search
space. Standard continuous methods are applied to solve the optimization problem.
An application of this approach can be found in [7]. There, the authors faced an
expensive parameter tuning problem and employed Kriging models. Especially
if the discrete variables are of an ordinal nature, the naive approach may be
successful. Indeed, this strategy could even be adopted to deal with categorical
variables: Addressing an arbitrary order would create a one-to-one correlation
between categorical variables and ordinal values. Several potential drawbacks can
arise if this strategy is employed for problems that are too complex:
• Large areas of redundancy in the model’s input space
• Creation of infeasible solutions
• Degeneration of performance due to bias caused by a misinterpretation of
variables
The naive approach is attractive, due to its ease of use and the ability to stick to
continuous variable handling methods. Practitioners have to carefully evaluate if
this option suits the characteristics of the problem under study.
10.4.2.3 Inherently Discrete Models
There are models that are discrete in their design and hence need no further
adaptation to discrete variables. For example, tree-based models, like regression
trees or random forests, are inherently discrete models. A representative application
L. Gentile et al.
rithm 1). In SPOT, a surrogate model is used to determine promising candidate
solutions. To that end, it aims to learn the relation between problem variables and
the corresponding function’s response.
Compared to their frequent use for real-valued problems, surrogate model-driven
approaches are less often used in mixed optimization [27]. According to BartzBeielstein and Zaefferer [8], few expensive, real-world optimization problems of
this type have been brought to the science community’s attention, e.g. in the engineering domain [3, 50, 51, 54], bioinformatics [41] or data science [47]. One reason
for the scarce use of discrete surrogate model-based optimization is the availability
of suitable methods. Bartz-Beielstein and Zaefferer [8] identified six strategies for
surrogate modeling in mixed-discrete search spaces: the naive approach, customized
models, inherently discrete models, feature extraction, mapping and similaritybased models. These strategies explain how modeling techniques can be used in
the general cases. The six strategies are not mutually exclusive. Some methods may
belong to several categories or combine different strategies. Here, we focus on three
of the six strategies: the naive approach, inherently discrete models and similaritybased models since these are more commonly used.
10.4.2.2 The Naive Approach
The naive approach to discrete modeling is to ignore the discrete nature of the search
space. Standard continuous methods are applied to solve the optimization problem.
An application of this approach can be found in [7]. There, the authors faced an
expensive parameter tuning problem and employed Kriging models. Especially
if the discrete variables are of an ordinal nature, the naive approach may be
successful. Indeed, this strategy could even be adopted to deal with categorical
variables: Addressing an arbitrary order would create a one-to-one correlation
between categorical variables and ordinal values. Several potential drawbacks can
arise if this strategy is employed for problems that are too complex:
• Large areas of redundancy in the model’s input space
• Creation of infeasible solutions
• Degeneration of performance due to bias caused by a misinterpretation of
variables
The naive approach is attractive, due to its ease of use and the ability to stick to
continuous variable handling methods. Practitioners have to carefully evaluate if
this option suits the characteristics of the problem under study.
10.4.2.3 Inherently Discrete Models
There are models that are discrete in their design and hence need no further
adaptation to discrete variables. For example, tree-based models, like regression
trees or random forests, are inherently discrete models. A representative application