Independent Component Analysis
117
In this case, the role of the cost function c(·) in (4.4) is played by the
mutual information I. In order to compute the gradient, expand the mutual
information as (Papoulis 1991):
m
i=!
m
= E {log (p( u) ) } - L E {log (p ( Ui) )} .
(4.21)
i=1
Since u is a function of x when both have the same number of components
(i. e. m = n), we can express p(u) using p(x) and the Jacobian matrix J(x) of u
with respect to x:
p(x)
p(u) = detJ(x) ,
where 'del' represents the determinant and
(4.22)
(4.23)
Using (4.22) and (4.23), express the first term of the mutual information
given in (4.21) using x and W:
n
I(uJ, ... ,Un) = E {log (p(x))} -log(detW)- LE{log(p(Ui))}. (4.24)
i=1
The gradient of 1(·) with respect to W can be expressed as:
n
aE{log(p(x))} alog(detW) a~E{log(P(Ui))}
aw
aw
aw
=_ alog (det W) _ ~aE{log(p(ui))} .
(4.25)
aw
~
aw
1=1
Since the first term E {log (p(x))} does not involve W, we will analyze the
two remaining terms separately. The first term becomes (Lee 1998):
_a _log_(,--de_t..c....(W_)...:....) =
a det W = 1 (ad· (W)) l' = (W-l)1'
aw
detW aw
det(W)
J
,
(4.26)
where adj represents the adjoint of the matrix.