110
4: Stefan A. Robila
the problem of recovering unknown signals that were mixed in an unknown
manner (Lee 1998). Our goal is to distinguish the conversation made by each
person present in the room. This problem can formally be defined as follows.
Consider an m-dimensional random vector s, and a matrix A of size n x m.
The problem is to recover this pair (s,A) from the available n-dimensional
observation vector x defined as:
x=As.
(4.1)
Generally, this problem is called Blind Source Separation (BSS). Each component of s corresponds to a source (thus there are m sources). The term blind
indicates that little or no information on the mixing matrix A, or the source
signals is available (Hyvarinen et al. 2001). The number of possible solutions
for this problem can be infinite, i. e., for a given x there is an infinite number
of pairs (s,A) that satisfy (4.1).
Consider now the situation when the components of the source s are assumed
to be statistically independent meaning thereby that the probability density
function of s, p(s) can be expressed as:
rn
p(s) = np(Si) .
(4.2)
i=l
In this case, the problem is called the independent component problem, and
a solution to this problem is called an Independent Component Analysis (ICA)
solution (Common 1994). In addition to the independence assumption, in order to provide a unique solution, ICA has the following restrictions (Hyvarinen
et al. 2001):
1. All the components of s have non-Gaussian distribution
2. The number of sources is smaller or equal to the number of observations
(m.:::: n)
3. Only low noise is permitted
The above conditions are required to ensure the existence and the uniqueness of the ICA solution. The second restriction states that there should be
enough observations available in order to be able to recover the sources. It
is similar to the condition in linear algebra, where the number of equations
needs to be at least equal to the number of variables. If this condition is not
satisfied, the solution obtained for ICA is not unique (Lee 1998). For simplicity,
in most applications, it is assumed that m = n and that the mixing matrix A
is invertible. In this case, once the matrix A is computed, its inverse can be
used to retrieve the independent components. Thus, in the rest of this chapter,
the matrix A will be assumed to be square. Extension of ICA to non-square
matrix cases has been proposed in the literature (Common 1994; Girolami
2000; Chang et al. 2002).
The restriction regarding non-Gaussian components also contributes to the
uniqueness of the solution. If non-Gaussian components are involved, the ICA
4: Stefan A. Robila
the problem of recovering unknown signals that were mixed in an unknown
manner (Lee 1998). Our goal is to distinguish the conversation made by each
person present in the room. This problem can formally be defined as follows.
Consider an m-dimensional random vector s, and a matrix A of size n x m.
The problem is to recover this pair (s,A) from the available n-dimensional
observation vector x defined as:
x=As.
(4.1)
Generally, this problem is called Blind Source Separation (BSS). Each component of s corresponds to a source (thus there are m sources). The term blind
indicates that little or no information on the mixing matrix A, or the source
signals is available (Hyvarinen et al. 2001). The number of possible solutions
for this problem can be infinite, i. e., for a given x there is an infinite number
of pairs (s,A) that satisfy (4.1).
Consider now the situation when the components of the source s are assumed
to be statistically independent meaning thereby that the probability density
function of s, p(s) can be expressed as:
rn
p(s) = np(Si) .
(4.2)
i=l
In this case, the problem is called the independent component problem, and
a solution to this problem is called an Independent Component Analysis (ICA)
solution (Common 1994). In addition to the independence assumption, in order to provide a unique solution, ICA has the following restrictions (Hyvarinen
et al. 2001):
1. All the components of s have non-Gaussian distribution
2. The number of sources is smaller or equal to the number of observations
(m.:::: n)
3. Only low noise is permitted
The above conditions are required to ensure the existence and the uniqueness of the ICA solution. The second restriction states that there should be
enough observations available in order to be able to recover the sources. It
is similar to the condition in linear algebra, where the number of equations
needs to be at least equal to the number of variables. If this condition is not
satisfied, the solution obtained for ICA is not unique (Lee 1998). For simplicity,
in most applications, it is assumed that m = n and that the mixing matrix A
is invertible. In this case, once the matrix A is computed, its inverse can be
used to retrieve the independent components. Thus, in the rest of this chapter,
the matrix A will be assumed to be square. Extension of ICA to non-square
matrix cases has been proposed in the literature (Common 1994; Girolami
2000; Chang et al. 2002).
The restriction regarding non-Gaussian components also contributes to the
uniqueness of the solution. If non-Gaussian components are involved, the ICA