Confusion Matrix: No more confusion

Confusion Matrix and other Classification Metrics

5 min readJun 9, 2021

It is pretty tough to avoid the confusing trap of the Confusion Matrix; but however tough a concept might seem, if you try to relate it to a real-life example, there are high chances of remembering it for a long period. Confusion Matrix is no exception.

I will divide this post into 2 major sessions :

Confusion Matrix
Accuracy, Precision, Recall & Specificity

And at the end, I will relate these concepts to a Fun Analogy.

Confusion Matrix

Confusion Matrix is basically a technique to summarize the performance of our Machine Learning model. With the help of Confusion Matrix, we can determine some important predictive analyses such as Accuracy, Precision, Recall (Sensitivity), Specificity, etc.

It is referred to as a Confusion Matrix as it makes it easier for us to understand, visualize and check whether our ML model is confusing while predicting the output classes or not.

Suppose our model is trained to predict binary classes: Yes or No; and once the model is trained and ready to make predictions, a Confusion Matrix for the model predictions can be created as follows :

(By the way, Confusion Matrix can be created for Train data, as well as for Test data. We just need the Actual Values and the Predicted Values)

True Positives (TP) : [ +ve = +ve ]

Classes which were actually +ve and predicted +ve as well.

2. True Negatives (TN) : [ -ve = -ve ]

Classes which were actually -ve and predicted -ve as well.

3. False Positives (FP) : [ -ve = +ve ]

Classes which were actually -ve but Falsely predicted as +ve.

4. False Negatives (FN) : [ +ve = -ve ]

Classes which were actually +ve but Falsely predicted as -ve.

The True Values (TP & TN) are the correct predictions made by our model for both +ve and -ve classes.

The False Values (FP & FN) are the incorrect predictions made by our model for both +ve and -ve classes. Hence there are errors associated with these predictions.

These errors are :

1. Type 1 Error : [ False Positive Rate (FPR) ]

The error caused because our model Falsely predicted +ve output for the values which were actually -ve.

2. Type 2 Error : [ False Negative Rate (FNR) ]

The error caused because our model Falsely predicted -ve output for the values which were actually +ve.

Accuracy, Precision, Recall & Specificity

Consider we have an imbalanced dataset for binary classification with 30–70 distribution for +ve class & -ve class respectively as follows :

Once the model is trained on this dataset and ready to make predictions, we can determine important Classification Metrics using the Confusion Matrix.

By having a Question-Answer approach, we can remember these metrics quite easily.

ACCURACY

Question :

Out of all the predictions made for both +ve as well as -ve classes, how many of them did we actually predict correctly?

Accuracy is the ratio of correctly predicted values (True Values) and the total predicted values (True Values + False Values)

PRECISION

Question :

Out of all the predictions we made for the +ve class, how many of them did we truly predict as +ve?

Precision is the ratio of correctly predicted +ve values and total predictions made for the +ve class

RECALL & SPECIFICITY

RECALL

Question :

Out of all the values belonging to the +ve class, how many of them did we predict correctly as +ve?

Recall is the ratio of correctly predicted +ve values and total actually +ve values

SPECIFICITY

Question :

Out of all the values belonging to the -ve class, how many of them did we predict correctly as -ve?

Specificity is the ratio of correctly predicted -ve values and total actually -ve values

Recall & Specificity are exactly opposite to each other

FUN ANALOGY

Suppose there are sweets made in your house on some special occasion and you see one bucket full of Gulab Jamuns (+ve class) and another bucket full of Rasgullas (-ve class). When everyone is busy with their work and nobody is watching you, it's your time to taste these delicacies. You secretly go near these buckets containing sweets and :

If you grab Gulab Jamuns (+ve class) with your left hand, and Rasgullas (-ve class) with your right hand as much as you can, then

Now, suppose that all the Gulab Jamuns and Rasgullas are mixed together and kept in a single bucket, and if you try to grab them with both of your hands then,

Now, when Gulab Jamuns and Rasgullas are still mixed together and kept in the same bucket, and someone like me who likes Gulab Jamuns more than Rasgullas, trying to grab as much Gulab Jamuns as I can, then