135
Clustering and Classification
the decision is always made based on the previously known values of P(ω 1 ) and P(ω 2 ).
Loosely speaking, such decisions are like predicting that a summer day is sunny based
on the yearly statistics that identifies a fixed probability for summer days being sunny.
Such a decision ignores any observation for some particular year. Even though this
type of decision making based only on a priori probability can have a relatively high
success rate in certain applications, it is blind to observations made from the system.
In many practical applications, however, the information contained in P(ω i ) probability measure is too limited to make a good decision. This is why we often use another
variable to improve our decision about the class of the new samples. In the aforementioned example, assuming that the image to be processed is in color, the color of each
pixel can be a suitable variable in helping us make a better classification decision. For
each pixel, we have three color elements, r (the intensity of the red component), g (the
intensity of the green component), and b (the intensity of the green component). Based on
the class of the pixel (tumor or nontumor), the color elements will be in a specific range.
The color information then leads us to define a new probability called “class conditional probability.” P(r, g, b|ω 1 ) is the class conditional probability of class 1, and
P(r, g, b|ω 2 ) is the class conditional probability of class 2. For simplicity, we name the
vector (r, g, b) as X. This reduces the notation for the class conditional probability to
P(X|ω i ). The probability measure P(X|ω i ) quantifies the probability that the color of a
pixel belonging to class ω i be in the range of X. However, what we normally need to do
for classification is rather the opposite of what we get from the class conditional probability, i.e., we are to determine the class of a selected pixel based on its color vector X.
For instance, we know the color elements of a pixel and we intend to decide whether the
pixel is a tumor or a nontumor pixel. This means that we need to determine the probabilities P(ω i |X) as opposed to the conditional probabilities P(X|ω i ). The probability
P(ω i |X) quantifies the likelihood that a given pixel belongs to class ω i , knowing that the
color vector of the selected pixel is X. We call this probability “a posteriori probability.”
While it is often difficult to compute a posteriori probability directly, one can compute
a posteriori probability using a priori and class conditional probabilities as follows:
P(w |X P X ) = ( w i P
i
) (
P X| ) ( w i )
(7.4)
Therefore,
P X|w P w i )
(
i ) (
P(w i |X) =
(7.5)
P X
( )
Practically, for any given pixel, we need to compute P(ω i |X) for each of the classes i
and then vote for the class whose a posteriori probability is the largest. Since P(X)
is the same for all classes, we can disregard this probability in the decision-making
process and compare P(X|ω i )P(ω i ) terms with each other. For example, for the simple
tumor example, the decision-making process becomes choosing ω 1 if
(
1 ) ( 1 P X 2 w 2
P X|w P w ) > ( |w ) (
P )
(7.6)
and ω 2 otherwise.
Clustering and Classification
the decision is always made based on the previously known values of P(ω 1 ) and P(ω 2 ).
Loosely speaking, such decisions are like predicting that a summer day is sunny based
on the yearly statistics that identifies a fixed probability for summer days being sunny.
Such a decision ignores any observation for some particular year. Even though this
type of decision making based only on a priori probability can have a relatively high
success rate in certain applications, it is blind to observations made from the system.
In many practical applications, however, the information contained in P(ω i ) probability measure is too limited to make a good decision. This is why we often use another
variable to improve our decision about the class of the new samples. In the aforementioned example, assuming that the image to be processed is in color, the color of each
pixel can be a suitable variable in helping us make a better classification decision. For
each pixel, we have three color elements, r (the intensity of the red component), g (the
intensity of the green component), and b (the intensity of the green component). Based on
the class of the pixel (tumor or nontumor), the color elements will be in a specific range.
The color information then leads us to define a new probability called “class conditional probability.” P(r, g, b|ω 1 ) is the class conditional probability of class 1, and
P(r, g, b|ω 2 ) is the class conditional probability of class 2. For simplicity, we name the
vector (r, g, b) as X. This reduces the notation for the class conditional probability to
P(X|ω i ). The probability measure P(X|ω i ) quantifies the probability that the color of a
pixel belonging to class ω i be in the range of X. However, what we normally need to do
for classification is rather the opposite of what we get from the class conditional probability, i.e., we are to determine the class of a selected pixel based on its color vector X.
For instance, we know the color elements of a pixel and we intend to decide whether the
pixel is a tumor or a nontumor pixel. This means that we need to determine the probabilities P(ω i |X) as opposed to the conditional probabilities P(X|ω i ). The probability
P(ω i |X) quantifies the likelihood that a given pixel belongs to class ω i , knowing that the
color vector of the selected pixel is X. We call this probability “a posteriori probability.”
While it is often difficult to compute a posteriori probability directly, one can compute
a posteriori probability using a priori and class conditional probabilities as follows:
P(w |X P X ) = ( w i P
i
) (
P X| ) ( w i )
(7.4)
Therefore,
P X|w P w i )
(
i ) (
P(w i |X) =
(7.5)
P X
( )
Practically, for any given pixel, we need to compute P(ω i |X) for each of the classes i
and then vote for the class whose a posteriori probability is the largest. Since P(X)
is the same for all classes, we can disregard this probability in the decision-making
process and compare P(X|ω i )P(ω i ) terms with each other. For example, for the simple
tumor example, the decision-making process becomes choosing ω 1 if
(
1 ) ( 1 P X 2 w 2
P X|w P w ) > ( |w ) (
P )
(7.6)
and ω 2 otherwise.