(change in magnification or sample shrinkage for example). The problem arising is
that images are easily aligned 2 by 2 without taking into account global projection
geometry but then some deformations in images are due to change of tilt angle. It is
then necessary to add some prior information such as tilt axis position and tilt angle
of images to prevent some wrong correction in projection geometry. When information about tilt axis position and tilt angle is not known a priori, the possibility of
deformation is unusable.
Once the type of transformation is fixed, the multiscale approach also allows
different optimization techniques to find the best transformation. It can be brute
force testing, where an exhaustive but limited set of possibilities is taken at each
scale. It helps to stay around the local minima found at coarsest scale. Better
optimization scheme involves the use of gradient descent algorithms, often powell
or convex. The interests of these approaches were well debated in the medical
imaging field [21–24] to register data in 2D and 3D from computer axial
tomography (CAT), positron emission tomography (PET), magnetic resonance
imaging (MRI) or ultrasound.
To estimate the better result of alignment, there is need of a metric. Many
algorithms use mean (or sum) of square difference or correlation (see part 7.1 for
more concise description), but other metrics exists such as mutual information.
Mutual information was designed to allow the registration of images with very
different contrasts [25, 26]. Formally, the mutual information of two discrete random variables X and Y can be defined as:
I X; Y
ð
Þ ¼
X
x;y
p x; y
ð Þlog
p x; y
ð Þ
p x
ð Þp y
ð Þ
Where p x; y
ð Þ is the joint probability distribution function of X and Y, and p x
ð Þ
and p y
ð Þ are the marginal probability distribution functions of X and Y respectively.
In the case of 3D chemical mapping by EFTEM, mutual information was designed
to align energy filtered images together between them or with zero-loss or plasmon
images which represents the ultra-structural information. So, mutual information is
needed to obtain better alignment between chemical and structural images (two
modalities). Even if some studies demonstrated higher robustness and lesser sensitivity to noise [23], the higher computation cost over cross-correlation makes the
use of mutual information approach questionable to align images coming from a
single modality (ultra-structural tilt series from standard tomographic approach for
example).
Multiscale registration allows to have sometimes a better alignment than
cross-correlation methods but it does not compensate the fact that it is not sufficient
as it will still align image two by two and so some propagation of errors will still
occurs even if smaller. There is also need to take into account the fact that the
information inside the image is a projection from a 3D sample and so the
improvement is not competitive enough compared to feature-based registration.
Therefore, multiscale registration is suitable for prealignment but it will frequently
require further refinement based on landmarks.
196
A. Verguet et al.
that images are easily aligned 2 by 2 without taking into account global projection
geometry but then some deformations in images are due to change of tilt angle. It is
then necessary to add some prior information such as tilt axis position and tilt angle
of images to prevent some wrong correction in projection geometry. When information about tilt axis position and tilt angle is not known a priori, the possibility of
deformation is unusable.
Once the type of transformation is fixed, the multiscale approach also allows
different optimization techniques to find the best transformation. It can be brute
force testing, where an exhaustive but limited set of possibilities is taken at each
scale. It helps to stay around the local minima found at coarsest scale. Better
optimization scheme involves the use of gradient descent algorithms, often powell
or convex. The interests of these approaches were well debated in the medical
imaging field [21–24] to register data in 2D and 3D from computer axial
tomography (CAT), positron emission tomography (PET), magnetic resonance
imaging (MRI) or ultrasound.
To estimate the better result of alignment, there is need of a metric. Many
algorithms use mean (or sum) of square difference or correlation (see part 7.1 for
more concise description), but other metrics exists such as mutual information.
Mutual information was designed to allow the registration of images with very
different contrasts [25, 26]. Formally, the mutual information of two discrete random variables X and Y can be defined as:
I X; Y
ð
Þ ¼
X
x;y
p x; y
ð Þlog
p x; y
ð Þ
p x
ð Þp y
ð Þ
Where p x; y
ð Þ is the joint probability distribution function of X and Y, and p x
ð Þ
and p y
ð Þ are the marginal probability distribution functions of X and Y respectively.
In the case of 3D chemical mapping by EFTEM, mutual information was designed
to align energy filtered images together between them or with zero-loss or plasmon
images which represents the ultra-structural information. So, mutual information is
needed to obtain better alignment between chemical and structural images (two
modalities). Even if some studies demonstrated higher robustness and lesser sensitivity to noise [23], the higher computation cost over cross-correlation makes the
use of mutual information approach questionable to align images coming from a
single modality (ultra-structural tilt series from standard tomographic approach for
example).
Multiscale registration allows to have sometimes a better alignment than
cross-correlation methods but it does not compensate the fact that it is not sufficient
as it will still align image two by two and so some propagation of errors will still
occurs even if smaller. There is also need to take into account the fact that the
information inside the image is a projection from a 3D sample and so the
improvement is not competitive enough compared to feature-based registration.
Therefore, multiscale registration is suitable for prealignment but it will frequently
require further refinement based on landmarks.
196
A. Verguet et al.