7 Introduction to Optimisation
227
for optimisation will give the same average performance when averaged overall
possible functions, which means that the universally best method does not exist for
all optimisation problems. This theorem proves the importance of applying problemspecific information when deciding upon an appropriate algorithm to achieve better
than average results.
7.1.2 Local vs Global Optimisation
There are two categories of optimal solutions that can be found as a result of an
optimisation process: local solutions and global solutions. Mathematically, a local
solution is a solution for which an optimal solution x ∗
local is better than all other
values of x in its neighbourhood. A global solution describes an optimal solution
x ∗
global which is better than all other values of x across the whole search space. As
a result, all global minima are also local minima. This distinction highlights the
importance of a correct problem formulation. For a linear, convex problem, a local
solution is a global solution. In the case of a complex, non-convex problem, a local
solution is not necessarily a global one. In this case the choice of the initial guess,
from which the optimisation algorithm performs the search, can be crucial for the
performance of the algorithm itself because of the possibility of converging into one
of the local optima close to the initial guess rather than the global one. Hence global
optimisation algorithms are designed with particular strategies that are aiming at
avoiding being trapped in local optima.
7.1.3 Single- vs Multi-Objective
The objective functions drive the optimisation algorithm to find an optimum value,
depending on whether the result has to be minimised or maximised. In singleobjective optimisation, the main goal is to find the ‘optimal’ solution for only one
objective function.
For a problem with more than one objective, there is rarely one solution that is
the optimal solution for each of the objective functions. In this case, a set of optimal
solutions is found. Finding the optimum solution for multiple objective functions
can be difficult and computationally expensive. One method of simplification is
to reduce the number of objective functions. Multiple objective functions can be
lumped into one objective functions through a weighted sum approach, where the
function outputs are scaled then multiplied a constant representing its importance
relative to the other objectives. It should be noted that, although conceptually
easy, the weighted sum approach only finds solution on the convex regions of
the Pareto front and are difficult to implement when the objective functions have
different orders of magnitude. Another method is the -constraint one, which
considers all objectives except one, as constraints in the optimisation process.
227
for optimisation will give the same average performance when averaged overall
possible functions, which means that the universally best method does not exist for
all optimisation problems. This theorem proves the importance of applying problemspecific information when deciding upon an appropriate algorithm to achieve better
than average results.
7.1.2 Local vs Global Optimisation
There are two categories of optimal solutions that can be found as a result of an
optimisation process: local solutions and global solutions. Mathematically, a local
solution is a solution for which an optimal solution x ∗
local is better than all other
values of x in its neighbourhood. A global solution describes an optimal solution
x ∗
global which is better than all other values of x across the whole search space. As
a result, all global minima are also local minima. This distinction highlights the
importance of a correct problem formulation. For a linear, convex problem, a local
solution is a global solution. In the case of a complex, non-convex problem, a local
solution is not necessarily a global one. In this case the choice of the initial guess,
from which the optimisation algorithm performs the search, can be crucial for the
performance of the algorithm itself because of the possibility of converging into one
of the local optima close to the initial guess rather than the global one. Hence global
optimisation algorithms are designed with particular strategies that are aiming at
avoiding being trapped in local optima.
7.1.3 Single- vs Multi-Objective
The objective functions drive the optimisation algorithm to find an optimum value,
depending on whether the result has to be minimised or maximised. In singleobjective optimisation, the main goal is to find the ‘optimal’ solution for only one
objective function.
For a problem with more than one objective, there is rarely one solution that is
the optimal solution for each of the objective functions. In this case, a set of optimal
solutions is found. Finding the optimum solution for multiple objective functions
can be difficult and computationally expensive. One method of simplification is
to reduce the number of objective functions. Multiple objective functions can be
lumped into one objective functions through a weighted sum approach, where the
function outputs are scaled then multiplied a constant representing its importance
relative to the other objectives. It should be noted that, although conceptually
easy, the weighted sum approach only finds solution on the convex regions of
the Pareto front and are difficult to implement when the objective functions have
different orders of magnitude. Another method is the -constraint one, which
considers all objectives except one, as constraints in the optimisation process.