278
F. Firouzi et al.
T
P
F
P
F
N
T
N
Predicted
Actual
Positive Negative
Positive
Negative
Accuracy= (TP+TN)/(TP+FP+FN+TN)
T
P
F
P
F
N
T
N
Predicted
Actual
Positive Negative
Positive
Negative
Precision= TP/(TP+FP)
T
P
F
P
F
N
T
N
Predicted
Actual
Positive Negative
Positive
Negative
Specificity = TN/(TN+FP)
Fig. 5.29 Accuracy, precision, and specificity performance metrics
Accuracy is a good performance metric when the target classes are balanced.
However, it should be avoided when the number of samples in each class is very
different (i.e., imbalanced dataset). The reason is that in imbalanced datasets, the
probability of instances belonging to a minority class is significantly low compared
to a majority class. Therefore, the classifier tends to classify new observations
mostly as the majority class. In our example, imagine that there are only five cases
of cancer out of every 100 cases. In this situation, if the system predicts all 100
cases as noncancerous, the accuracy of the model is 95%, but apparently, the model
is terrible at predicting cancer.
Precision (Positive Predictive Value)
Precision represents the proportion of true, relevant predictions (i.e., the percentage
of your model results, which are relevant, or the ratio between the relevant instances
and the total retrieved instances). Precision is formally defined as the ratio between
the number of true positives and the number of true positives plus the number of
false positives. In our example, the precision is defined as how many of the people
detected as cancerous have cancer. In other words, precision indicates how much
the model is precise. For example, if we predict just one cancerous patient, and the
patient has cancer, the precision is 100%.
Precision =
TP
TP + FP
Recall (Sensitivity)
Recall or sensitivity expresses the ability of the model to detect all the relevant cases
(all the points of interest) within a dataset. Formally, recall is the number of true
positives divided by the number of true positives plus the number of false negatives.
In our cancer detection example, recall demonstrates how many of the cancerous
patients are predicted as having cancer. If we mark every patient as cancerous, the
recall is 100%.
Recall =
TP
TP + FN
F. Firouzi et al.
T
P
F
P
F
N
T
N
Predicted
Actual
Positive Negative
Positive
Negative
Accuracy= (TP+TN)/(TP+FP+FN+TN)
T
P
F
P
F
N
T
N
Predicted
Actual
Positive Negative
Positive
Negative
Precision= TP/(TP+FP)
T
P
F
P
F
N
T
N
Predicted
Actual
Positive Negative
Positive
Negative
Specificity = TN/(TN+FP)
Fig. 5.29 Accuracy, precision, and specificity performance metrics
Accuracy is a good performance metric when the target classes are balanced.
However, it should be avoided when the number of samples in each class is very
different (i.e., imbalanced dataset). The reason is that in imbalanced datasets, the
probability of instances belonging to a minority class is significantly low compared
to a majority class. Therefore, the classifier tends to classify new observations
mostly as the majority class. In our example, imagine that there are only five cases
of cancer out of every 100 cases. In this situation, if the system predicts all 100
cases as noncancerous, the accuracy of the model is 95%, but apparently, the model
is terrible at predicting cancer.
Precision (Positive Predictive Value)
Precision represents the proportion of true, relevant predictions (i.e., the percentage
of your model results, which are relevant, or the ratio between the relevant instances
and the total retrieved instances). Precision is formally defined as the ratio between
the number of true positives and the number of true positives plus the number of
false positives. In our example, the precision is defined as how many of the people
detected as cancerous have cancer. In other words, precision indicates how much
the model is precise. For example, if we predict just one cancerous patient, and the
patient has cancer, the precision is 100%.
Precision =
TP
TP + FP
Recall (Sensitivity)
Recall or sensitivity expresses the ability of the model to detect all the relevant cases
(all the points of interest) within a dataset. Formally, recall is the number of true
positives divided by the number of true positives plus the number of false negatives.
In our cancer detection example, recall demonstrates how many of the cancerous
patients are predicted as having cancer. If we mark every patient as cancerous, the
recall is 100%.
Recall =
TP
TP + FN