11 Parameter Control in Evolutionary Optimisation
375
By performing a proper parameter tuning, we get values for control parameters
that are good for the optimisation problems chosen in the tuning process. To reduce
the computational burden of tuning, simpler, synthetic problems are often used that
might not reflect the characteristics of real-world problems for which the tuned
algorithm will be used. It was also shown that good control parameter values for one
problem can behave poorly on similarly looking problems [8]. This is connected to
the fact that the optimisation algorithm performance is usually quite sensitive with
respect to control parameter values. A small change in control parameter value can
result in great changes in algorithm overall performance.
A potential advantage of the static approach is that the process of parameter tuning is pre-described and theoretically well founded since tuning is an optimisation
problem. From a design point of view, this is not the case for parameter control.
Searching for an optimal control strategy can be considered an optimisation problem
in infinite dimensional search space. Therefore, how to develop suitable update
rules for parameter control is theoretically still an open question. Nevertheless,
optimisation parameter control is a very active area of research that brought great
performance improvements to several optimisation algorithms [11, 13].
Parameter control can also introduce additional parameters on its own. Such
parameters determine how parameter control behaves. Therefore, the use of parameter control does not necessarily bring parameterless optimisation algorithms.
However, parameter control rules should be designed in a way that the additional
parameters (so called meta-parameters) of the parameter control are much less
sensitive compared to original parameters of the optimisation algorithm [4, 54].
An advantage of the non-static parameter approach is the gain of flexibility and
the possibility to adjust the parameter values to the current state of the search
process. Since characteristics of the fitness landscape change during optimisation, it
is much more efficient to change parameters during optimisation and use appropriate
control parameter values in each stage of the optimisation. Another advantage
of parameter control is that it can reduce the need for parameter tuning. If
adaptive or self-adaptive parameter control is used, at the first stage of optimisation,
the algorithm discovers the proper parameters itself, without tuning. This can
slightly prolong the optimisation, but it can eliminate the need for time-consuming
parameter tuning.
The choice of whether to use parameter tuning, or some combination of tuning,
and control (see Fig. 11.9), depends on the complexity of the problems and in what
frequency these particular optimisation problems are solved. If the problem is very
hard and needs to be solved only once, parameter tuning represents an enormous
overhead. If a hard problem needs to be solved over and over again, parameter
tuning can represent a negligible overhead and can reduce the optimisation time
considerably. In some cases it might be useful to first search for fine-tuned initial
values and then to use parameter control. This way the tuning gives the advantage
of tailoring to a specific problem class and the control offers the benefits of
dynamically varying parameter values [8].
375
By performing a proper parameter tuning, we get values for control parameters
that are good for the optimisation problems chosen in the tuning process. To reduce
the computational burden of tuning, simpler, synthetic problems are often used that
might not reflect the characteristics of real-world problems for which the tuned
algorithm will be used. It was also shown that good control parameter values for one
problem can behave poorly on similarly looking problems [8]. This is connected to
the fact that the optimisation algorithm performance is usually quite sensitive with
respect to control parameter values. A small change in control parameter value can
result in great changes in algorithm overall performance.
A potential advantage of the static approach is that the process of parameter tuning is pre-described and theoretically well founded since tuning is an optimisation
problem. From a design point of view, this is not the case for parameter control.
Searching for an optimal control strategy can be considered an optimisation problem
in infinite dimensional search space. Therefore, how to develop suitable update
rules for parameter control is theoretically still an open question. Nevertheless,
optimisation parameter control is a very active area of research that brought great
performance improvements to several optimisation algorithms [11, 13].
Parameter control can also introduce additional parameters on its own. Such
parameters determine how parameter control behaves. Therefore, the use of parameter control does not necessarily bring parameterless optimisation algorithms.
However, parameter control rules should be designed in a way that the additional
parameters (so called meta-parameters) of the parameter control are much less
sensitive compared to original parameters of the optimisation algorithm [4, 54].
An advantage of the non-static parameter approach is the gain of flexibility and
the possibility to adjust the parameter values to the current state of the search
process. Since characteristics of the fitness landscape change during optimisation, it
is much more efficient to change parameters during optimisation and use appropriate
control parameter values in each stage of the optimisation. Another advantage
of parameter control is that it can reduce the need for parameter tuning. If
adaptive or self-adaptive parameter control is used, at the first stage of optimisation,
the algorithm discovers the proper parameters itself, without tuning. This can
slightly prolong the optimisation, but it can eliminate the need for time-consuming
parameter tuning.
The choice of whether to use parameter tuning, or some combination of tuning,
and control (see Fig. 11.9), depends on the complexity of the problems and in what
frequency these particular optimisation problems are solved. If the problem is very
hard and needs to be solved only once, parameter tuning represents an enormous
overhead. If a hard problem needs to be solved over and over again, parameter
tuning can represent a negligible overhead and can reduce the optimisation time
considerably. In some cases it might be useful to first search for fine-tuned initial
values and then to use parameter control. This way the tuning gives the advantage
of tailoring to a specific problem class and the control offers the benefits of
dynamically varying parameter values [8].