372
M. Antoniou et al.
considers the average improvement in the properties of all the solutions in the
population, the extreme which refers to outliers and learned quality attribution
using other learning-based techniques, such as machine learning.
• Parameter update: The final step of adaptive parameter control is the update
mechanism. The update of the parameters is a settlement between control
parameter values with high quality and exploring new values. The categories
are the following:
– Quality proportionate: Each control parameter value is assigned with a
probability, which defines how frequently it is used in future iterations.
– Quality proportionate with minimum probability: The same as above, but
each control parameter value has a minimum probability. That way, a value
that is not performing well is not lost completely, as it can be successful in
future iterations.
– Greedy: The best control parameter value is selected to be used in the next
iteration.
– Deterministic: A rule depending on the current value is used to update the
control parameter values of the next iteration.
Therefore, designing a good adaptive parameter control strategy requires some
consideration. The difficulty of designing a good parameter control strategy is
strongly connected to the problem of landscape analysis [45]. We can see parameter
control as performing some sort of landscape analysis during optimisation, and on
top of that, the features of the landscape are mapped to control parameter values that
work well in such landscapes. One approach to circumvent these design difficulties
is to let a machine learning model guide the parameter control [46]. By this method
only the feedback collection step needs to be designed, whilst other steps are left
to the machine learning model. Using a sufficient amount of data, the model could
learn good rules for parameter control and in essence design remaining steps of
parameter control based on which rules worked well during training of the model.
Reinforcement learning is especially suitable to find such rules for optimisation
parameter control. This type of machine learning was found to be extremely
successful to construct agents that are capable of playing various games [47]. In
this regard, optimisation parameter control can be viewed as an agent playing a
game. In the setting of parameter control being a game, a player performs actions
(propose new values of optimisation parameters) that influence the behaviour of the
optimisation process. At the end, the score of the player’s actions is equivalent to
the fitness found by the optimisation process. In this way, Zhang and Lu devised
an evolutionary algorithm that uses a mutation operator proposed by a machine
learning model trained using reinforcement learning [48]. However, this type of
parameter control construction is still in its infancy and is currently an active area
of research.
M. Antoniou et al.
considers the average improvement in the properties of all the solutions in the
population, the extreme which refers to outliers and learned quality attribution
using other learning-based techniques, such as machine learning.
• Parameter update: The final step of adaptive parameter control is the update
mechanism. The update of the parameters is a settlement between control
parameter values with high quality and exploring new values. The categories
are the following:
– Quality proportionate: Each control parameter value is assigned with a
probability, which defines how frequently it is used in future iterations.
– Quality proportionate with minimum probability: The same as above, but
each control parameter value has a minimum probability. That way, a value
that is not performing well is not lost completely, as it can be successful in
future iterations.
– Greedy: The best control parameter value is selected to be used in the next
iteration.
– Deterministic: A rule depending on the current value is used to update the
control parameter values of the next iteration.
Therefore, designing a good adaptive parameter control strategy requires some
consideration. The difficulty of designing a good parameter control strategy is
strongly connected to the problem of landscape analysis [45]. We can see parameter
control as performing some sort of landscape analysis during optimisation, and on
top of that, the features of the landscape are mapped to control parameter values that
work well in such landscapes. One approach to circumvent these design difficulties
is to let a machine learning model guide the parameter control [46]. By this method
only the feedback collection step needs to be designed, whilst other steps are left
to the machine learning model. Using a sufficient amount of data, the model could
learn good rules for parameter control and in essence design remaining steps of
parameter control based on which rules worked well during training of the model.
Reinforcement learning is especially suitable to find such rules for optimisation
parameter control. This type of machine learning was found to be extremely
successful to construct agents that are capable of playing various games [47]. In
this regard, optimisation parameter control can be viewed as an agent playing a
game. In the setting of parameter control being a game, a player performs actions
(propose new values of optimisation parameters) that influence the behaviour of the
optimisation process. At the end, the score of the player’s actions is equivalent to
the fitness found by the optimisation process. In this way, Zhang and Lu devised
an evolutionary algorithm that uses a mutation operator proposed by a machine
learning model trained using reinforcement learning [48]. However, this type of
parameter control construction is still in its infancy and is currently an active area
of research.