DATA ASSIMILATION IN BIOGEOCHEMICAL MODELLING
539
strained problem. Accordingly, the minimum of L(p, λ, x) can “simply”
be found by setting all its partial derivatives to zero. It turns out that
the derivatives with respect to the components of the Lagrangian multipliers return the model dynamics, whereas the derivatives with respect
to the components of the state vector will, after repeated application
of the chain rule, return what is called the “adjoint model”. The
joint model can be understood as a model that runs the dynamics of the
original “forward” model backward in time while being forced by the
model-data misfits. A single backward run of the adjoint model, which
uses the final state of a run of the forward model as initial condition,
returns the full gradient of the cost function, ∇ p J(p), at the position of
the actual parameter vector p . A gradient descent algorithm will then
be needed to find a new parameter vector to start the next iteration of
forward model run and adjoint model run.
The main advantage of the adjoint technique is its very efficient computation of a complete gradient in N -dimensional parameter space: Only
a single forward and a single backward model integration are needed,
whereas the other methods mentioned above require order N model
tegrations to compute an approximate version of the gradient. There
is, of course, nothing like a free lunch: construction of the backward,
or so-called adjoint model is a major effort. (Semi-)automatic compilers
exist than can help turning the computer code of a forward model into
its adjoint counterpart (e.g., described by Marotzke et al., [1999]).
Applications of the adjoint method to parameter estimation for marine
ecosystem models have shown some success [Spitz et al., 1998, 2001;
Friedrichs, 2002; Gunson et al., 1999]. However, Schartau et al. [2001]
reported that the likely existence of local minima in the cost function
may require restarting the gradient search from a large variety of initial
estimates of the parameter vector. This is a problem any gradientdescent minimisation method will have. So far, however, it has not
been conclusively demonstrated that local minima of the cost function
do indeed exist. They are very difficult to identify in N -dimensional
parameter space. Visualisations of two-dimensional sections through
the N -dimensional cost function topography often show local minima
[Athias et al., 2000; Vallino, 2000; Schartau et al., 2001], but it is by no
means clear whether these are also minima in the other N − 2 directions.
Stochastic minimisation methods.
In order to avoid the expensive and often cumbersome computation of the cost function’s gradient
and also to cope with possibly existing local minima without having to
fully scan the complete parameter space in a “brute force” mode, search
algorithms have been developed that contain stochastic elements. These
d
a
in
,
539
strained problem. Accordingly, the minimum of L(p, λ, x) can “simply”
be found by setting all its partial derivatives to zero. It turns out that
the derivatives with respect to the components of the Lagrangian multipliers return the model dynamics, whereas the derivatives with respect
to the components of the state vector will, after repeated application
of the chain rule, return what is called the “adjoint model”. The
joint model can be understood as a model that runs the dynamics of the
original “forward” model backward in time while being forced by the
model-data misfits. A single backward run of the adjoint model, which
uses the final state of a run of the forward model as initial condition,
returns the full gradient of the cost function, ∇ p J(p), at the position of
the actual parameter vector p . A gradient descent algorithm will then
be needed to find a new parameter vector to start the next iteration of
forward model run and adjoint model run.
The main advantage of the adjoint technique is its very efficient computation of a complete gradient in N -dimensional parameter space: Only
a single forward and a single backward model integration are needed,
whereas the other methods mentioned above require order N model
tegrations to compute an approximate version of the gradient. There
is, of course, nothing like a free lunch: construction of the backward,
or so-called adjoint model is a major effort. (Semi-)automatic compilers
exist than can help turning the computer code of a forward model into
its adjoint counterpart (e.g., described by Marotzke et al., [1999]).
Applications of the adjoint method to parameter estimation for marine
ecosystem models have shown some success [Spitz et al., 1998, 2001;
Friedrichs, 2002; Gunson et al., 1999]. However, Schartau et al. [2001]
reported that the likely existence of local minima in the cost function
may require restarting the gradient search from a large variety of initial
estimates of the parameter vector. This is a problem any gradientdescent minimisation method will have. So far, however, it has not
been conclusively demonstrated that local minima of the cost function
do indeed exist. They are very difficult to identify in N -dimensional
parameter space. Visualisations of two-dimensional sections through
the N -dimensional cost function topography often show local minima
[Athias et al., 2000; Vallino, 2000; Schartau et al., 2001], but it is by no
means clear whether these are also minima in the other N − 2 directions.
Stochastic minimisation methods.
In order to avoid the expensive and often cumbersome computation of the cost function’s gradient
and also to cope with possibly existing local minima without having to
fully scan the complete parameter space in a “brute force” mode, search
algorithms have been developed that contain stochastic elements. These
d
a
in
,