Appendix A: On the Independence of Certain Random Variables
263
x
2
1 +
n
i=2
x
2
i =
y 1 −
n
i=2
y i
2
+
n
i=2
(y i + y 1 )
2
= y
2
1 − 2y 1
n
i=2
y i +
n
i=2
y i
2
+
n
i=2
(y
2
i + 2y 1 y i + y
2
1 )
= ny
2
1 +
n
i=2
y i
2
+
n
i=2
y
2
i ,
(A.8)
which shows that the exponent is a sum containing one term depending on y 1 only
and the remaining terms are assembled in the form that characterizes the variance S
2
in (A.1). The distribution function therefore factors and that shows that y 1 = ¯
x and
the rest of the variables are statistically independent.
Following the definition of χ
2
p in (7.36), we introduced the test statistics f in (7.37)
and used that χ
2
p − χ
2
q in the numerator and χ
2
q in the denominator are statistically
independent. Here χ
2
q is given by (7.36) with the N × p-matrix A replaced by the
N × q-matrix B, which is given by A with q − p additional columns corresponding
to the additional fit parameters. The statistical independence made it possible to write
the distribution function n,m ( f ) as the product of two χ
2 -distributions in (7.39). In
order to motivate why the two distributions are independent, we introduce the random
variables s = y − Ax and t = y − Bw, whose squares add up to the respective χ
2 -
distributions. Here we assume all σ i to be unity to simplify the notation. Note that
the dimension of s, t and y is N .
Let us explicitely calculate the random variables s and investigate their relation
to the “measurements” y. We find
s = y − Ax =
1 − A
A
t A
−1 A
t
y = P A y ,
(A.9)
where we determine the fit parameters x =
A
t A
−1 A
t y with the help of the pseudo
inverse from (7.4). It is easy to show that the matrix P A is a projection operator with
the properties P
2
A = P A and P
t
A = P A . Being a projector implies that its eigenvalues
λ A are either zero or one, where the number of ones determines the number of degrees
of freedom of the χ
2 -distribution χ
2
p =
N
j=1 s
2
j , because fitting with A introduces
p constraints among the N normally distributed s
2
j , such that only N − p of them
are independent.
Before addressing this point further, let us consider the matrix B with the q − p
additional columns. The projector derived from B is P B =
1 − B
B
t B
−1 B
t
.
Without proof, we state that P A and P B commute P A P B − P B P A = 0, which implies
that both projectors can be diagonalized simultaneously by an orthogonal matrix O,
which transforms s to r = Os and t to r
= Ot. Since O is orthogonal, normally distributed independent random variables are mapped onto normally distributed independent random variables. Before expressing χ
2
p and χ
2
q in the new variables, we
263
x
2
1 +
n
i=2
x
2
i =
y 1 −
n
i=2
y i
2
+
n
i=2
(y i + y 1 )
2
= y
2
1 − 2y 1
n
i=2
y i +
n
i=2
y i
2
+
n
i=2
(y
2
i + 2y 1 y i + y
2
1 )
= ny
2
1 +
n
i=2
y i
2
+
n
i=2
y
2
i ,
(A.8)
which shows that the exponent is a sum containing one term depending on y 1 only
and the remaining terms are assembled in the form that characterizes the variance S
2
in (A.1). The distribution function therefore factors and that shows that y 1 = ¯
x and
the rest of the variables are statistically independent.
Following the definition of χ
2
p in (7.36), we introduced the test statistics f in (7.37)
and used that χ
2
p − χ
2
q in the numerator and χ
2
q in the denominator are statistically
independent. Here χ
2
q is given by (7.36) with the N × p-matrix A replaced by the
N × q-matrix B, which is given by A with q − p additional columns corresponding
to the additional fit parameters. The statistical independence made it possible to write
the distribution function n,m ( f ) as the product of two χ
2 -distributions in (7.39). In
order to motivate why the two distributions are independent, we introduce the random
variables s = y − Ax and t = y − Bw, whose squares add up to the respective χ
2 -
distributions. Here we assume all σ i to be unity to simplify the notation. Note that
the dimension of s, t and y is N .
Let us explicitely calculate the random variables s and investigate their relation
to the “measurements” y. We find
s = y − Ax =
1 − A
A
t A
−1 A
t
y = P A y ,
(A.9)
where we determine the fit parameters x =
A
t A
−1 A
t y with the help of the pseudo
inverse from (7.4). It is easy to show that the matrix P A is a projection operator with
the properties P
2
A = P A and P
t
A = P A . Being a projector implies that its eigenvalues
λ A are either zero or one, where the number of ones determines the number of degrees
of freedom of the χ
2 -distribution χ
2
p =
N
j=1 s
2
j , because fitting with A introduces
p constraints among the N normally distributed s
2
j , such that only N − p of them
are independent.
Before addressing this point further, let us consider the matrix B with the q − p
additional columns. The projector derived from B is P B =
1 − B
B
t B
−1 B
t
.
Without proof, we state that P A and P B commute P A P B − P B P A = 0, which implies
that both projectors can be diagonalized simultaneously by an orthogonal matrix O,
which transforms s to r = Os and t to r
= Ot. Since O is orthogonal, normally distributed independent random variables are mapped onto normally distributed independent random variables. Before expressing χ
2
p and χ
2
q in the new variables, we