204
8: Stefan A. Robila, Pramod K. Varshney
band variance. Therefore, if PCA is applied on this dataset, the component
containing information on the target will have low variance and it is highly likely
that this component may be discarded thereby missing important information
in the dataset. Hence, the ICA-FE algorithm may not work accurately for the
datasets containing small targets. In this case, we would like to use all the PCA
derived components to produce only those independent components that will
correspond to the classes present in the image. Therefore, a modification in
the design of the ICA algorithm is desired.
Consider the minimum mutual information (MMI) framework for the solution ofICA (see Sect. 5.3.2):
u= Wx,
(8.1)
where x is an n-dimensional random vector, u is an m-dimensional random
vector and W is an m x n matrix. Intuitively, when the number of observations
is larger than the number of sources (n > m), the data observed can be
represented as a linear combination of a smaller set of components (the data is
undercompletely represented) (Shriki et al. 2000). Thus, for m < n the problem
is called undercomplete MMl (undercomplete leA).
We derive a gradient-based algorithm to solve the undercomplete MMI
problem. Unfortunately, we cannot directly apply the information maximization algorithm (refer to Sect. 4.3.2), since we had assumed m = n. In that case,
W was a full rank matrix, and the probability density function (pdf) of u was
directly expressed in terms of the pdf of x as:
p(x)
p(u) = Idet WI .
(8.2)
In the case of m < n, expressing the pdf of the random vector u using the
conditional pdfs, we have,
p(u) = f p(ulx)p(x)dx.
(8.3)
Assume now that the output random vector u is perturbed by additive independent Gaussian noise and consider the limit when the variance of the noise
goes to zero. In this case, we can use the multivariate Gaussian distribution to
express the conditional pdf as (Shriki et al. 2000):
1
_--L Ilu-wxf
p(ulx) = lim
In e 20 2
•
a
2 -+0 ( J2na 2 )
(8.4)
When the realization of the random vector u is in the image of Wx there
exists a vector Xo such that:
u = Wxo.
(8.5)
8: Stefan A. Robila, Pramod K. Varshney
band variance. Therefore, if PCA is applied on this dataset, the component
containing information on the target will have low variance and it is highly likely
that this component may be discarded thereby missing important information
in the dataset. Hence, the ICA-FE algorithm may not work accurately for the
datasets containing small targets. In this case, we would like to use all the PCA
derived components to produce only those independent components that will
correspond to the classes present in the image. Therefore, a modification in
the design of the ICA algorithm is desired.
Consider the minimum mutual information (MMI) framework for the solution ofICA (see Sect. 5.3.2):
u= Wx,
(8.1)
where x is an n-dimensional random vector, u is an m-dimensional random
vector and W is an m x n matrix. Intuitively, when the number of observations
is larger than the number of sources (n > m), the data observed can be
represented as a linear combination of a smaller set of components (the data is
undercompletely represented) (Shriki et al. 2000). Thus, for m < n the problem
is called undercomplete MMl (undercomplete leA).
We derive a gradient-based algorithm to solve the undercomplete MMI
problem. Unfortunately, we cannot directly apply the information maximization algorithm (refer to Sect. 4.3.2), since we had assumed m = n. In that case,
W was a full rank matrix, and the probability density function (pdf) of u was
directly expressed in terms of the pdf of x as:
p(x)
p(u) = Idet WI .
(8.2)
In the case of m < n, expressing the pdf of the random vector u using the
conditional pdfs, we have,
p(u) = f p(ulx)p(x)dx.
(8.3)
Assume now that the output random vector u is perturbed by additive independent Gaussian noise and consider the limit when the variance of the noise
goes to zero. In this case, we can use the multivariate Gaussian distribution to
express the conditional pdf as (Shriki et al. 2000):
1
_--L Ilu-wxf
p(ulx) = lim
In e 20 2
•
a
2 -+0 ( J2na 2 )
(8.4)
When the realization of the random vector u is in the image of Wx there
exists a vector Xo such that:
u = Wxo.
(8.5)