102
T. Basu et al.
y i ∼ N(x
T
i β, σ
2 ).
However, this assumption is too restrictive for many real data situations.
One can use generalized linear models to relax the assumption of normality. We
introduce a function g, which acts as a link function such that
g(E(y i |x i )) = x
T
i β;
(3.43)
here, y i can possess any exponential family distribution, such as Poisson, Binomial,
or Gamma. Note that if y i ∈ {0, 1} then
μ i ≡ E(y i |x i ) = P (y i = 1|x i );
(3.44)
hence we can (for our purposes) define
Definition 3.2 (Classification) Classification is the process of carrying out a
regression problem with 0/1-valued response and allocating observations to one
of the two classes according to the decision rule μ i ≥ 0.5.
3.5.1 Logistic Regression
In logistic regression we start with the logistic model,
log
μ i
1 − μ i
= x
T
i β
(3.45)
with “logit” link function g(μ i ) = log
μ i
1−μ i
. An alternative formulation of
Eq. (3.45) is
P (y i = 1|x i ) = h(x
T
i β)
(3.46)
where the logistic function
h(t) =
exp(t)
1 + exp(t)
(3.47)
maps the range (−∞, ∞)–[−1, 1]. The parameters in the logistic model are
estimated through an iteratively weighted least squares technique known as “Fisher
Scoring,” for details of which we refer to [9].
Example 3.2 (Sonar Dataset) Gorman and Sejnowski used this dataset in their
study of the classification of sonar signals using a neural network [12]. The
objective of the study was to discriminate between sonar signals bounced off a
metal cylinder and a cylindrical rock. Each observation is a set of 60 numbers