top of page

CIFAR-10 image classifier using Pytorch

  • Vinay Anant
  • Nov 2, 2021
  • 4 min read

Overview

CIFAR-10 is a dataset which consists of 60,000 32x32 colour images in 10 classes. They were collected by Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton.




We are using the same code provided by pytorch.org at this link.


Step1: Importing packages, dataset and then normalizing it


We are importing essential packages to load CIFAR-10 dataset using torchvision.



Now we are loading CIFAR-10 dataset into trainset and testset. This dataset is downloading from this link.



Let's load some of the images to see how it looks.



Step2: Defining a Convolution Neural Network, but what is it?


A Convolutional Neural Network is an algorithm in deep Learning that takes an input image, assign importance to various objects in it and able to identify differences between these objects


Kernel - The Kernel is the element that performs the convolution operation in the initial half of a Convolutional Layer


Pooling - The pooling layer is a layer that is responsible for reducing the spatial size of the convolved feature


Fully connected layer - Fully connected layer is where all of the inputs from one layer are connected to every activation unit of the next layer



Step3: Defining a Loss function and optimizer, again what is it?


Loss function - It is a function that evaluates on how our algorithm behaves on the dataset provided


Optimizer - It is a way to optimize or an algorithm to optimize different kinds of attributes such that it reduce the losses which will help in achieving the high accuracy



Step4: Training the network


Training is same as the word says i.e. training which means to train the algorithm in such a way that it finds the values in it given the provided attributes



Saving our trained model


Step5: Testing the network on test dataset


Testing as the name suggests it is the process of validating the algorithm on the test dataset



As we can see it gives us an accuracy of 56% on the test dataset


Let's find out what is the accuracy for the 10 classes



So we are here. Why not try increasing the accuracy?

Let's try it out.


First plot a graph to visualize the accuracies for all the 10 different classes



Consider the above training as Experiment 1.


We will start the Experiment 2 by tuning the above given code


Experiment 2

In this experiment we are changing the epoch from 2 to 10, instead of adding a new convolution layer and let's train it



Plotting a horizontal bar plot to visualize the results



Let's visualize and compare it with the previous experiment



Cool! This has increased the overall accuracy from 56% to 60%.


Why not try with a new experiment?


Experiment 3

In this experiment we will add two new convolution layers in the first experiment and see how it goes.



Let's train it.



It looks like accuracy has been decreased. This is not good.


Time to visualize it with a comparison plot.



We can see here that this experiment has decreased the accuracy from 56% to 53%.


Here come a new experiment.


Experiment 4

In this experiment we are adding 8 more new convolution layers in experiment 1, so in total of 10 convolution layers are added.


Let's train it.



It has provided us an increase in 1 % from the previous experiment but this is not what we are looking for. Isn't it?


Ok, let's do one last experiment.


Experiment 5

In this experiment we will try some different combination of things.


Here we are changing the width of our CNN for first convolution layer from 6 to 100, and updating this 100 value to input of second layer of convolution



Also, increase epoch from 2 to 50.



Time to train it.


It will consume a lot of time to train, so let's take a nap.



It's still running!!



Finally, after almost 3 hours runtime we can see the magic.


Woah! The overall accuracy has reached from 56% to 66%. Isn't it amazing?




Let's visualize the comparison of all the experiments we did along with the accuracies provided by each experiment.



We can also see the training time comparison for all the experiments we did, starting from some few minutes to hours.





My contribution:

  • Change in width of CNN and epoch which took time to train but increased the accuracy from 56% to 66%


  • Multiple experiments performed that involves addition of different convolution layers, epoch


  • Visualization using different types of plot diagrams and comparison of various experiments, accuracies and their training time from few minutes to hours



Challenges faced:

  • Increase the accuracy - I was thinking on how to increase the accuracy as this was the major challenge, so I tried running multiple experiments to see which one can provide me an increasing amount of accuracy for this dataset and that's how I solved it


  • Unavailability of GPU - While I was running with the last experiment, it stopped in between multiple times with an error message that I have reached my usage limit for GPU. In the end, I changed it to CPU which worked


  • Experiments, time comparison and visualization - I was not sure how to compare the experiments and while searching I couldn't find some important information to use other than how to plot a graph, so I thought to try storing it in the variable and it worked



This Notebook can be found here: Notebook



Note - We have used the same code provided at this link. We have modified it whenever needed to perform different experiments in a way that increases the accuracy and plotted graph for the same.


References:













 
 
 

Titanic tutorial from Kaggle

  • Writer: Vinay Anant
    Vinay Anant
  • Sep 24, 2021
  • 3 min read

Overview

This Titanic tutorial attempts to estimate who will survive and who will die based on the data provided.


Dataset

There are three csv files provided: train.csv, test.csv and gender_submission.csv.


Train.csv: This file consists of onboarded passengers information


Test.csv: This file contains information from which we have to predict whether or not the passenger will survive


Gender_submission.csv: This file serves as an example of how a submission should appear


Files location

This code displays the location of csv files on Kaggle.



Data reading

Now that we know where the data is stored, we can begin reading the files.

This code reads "train.csv" with pandas, and the head() function returns the first five rows of "train.csv."


Similarly, this code reads "test.csv" using pandas, and the head() function provides the first five rows of "test.csv."


Analyse pattern

We have "gender submission.csv" here, which assumes that only female passengers survived and that male passengers did not.



This code displays the percentage of women who survived the disaster as well as the percentage of men who survived the disaster.

According to the data, 74.2 percent of women and 18.9 percent of men survived.


Random forest model

The random forest model is a machine learning model that is useful for classification and regression tasks. This Titanic problem is a classifier task because we are attempting to determine whether or not the passenger survived, so we can apply this model here.


The random forest model is made up of several trees that examine each passenger's data and vote on whether or not the passenger can survive. The model then makes a democratic decision, in which the outcome with the most votes wins.



We import Random Forest Classifier and search for patterns in the four columns "Pclass", "Sex", "SibSp", and "Parch". It builds trees in the random forest model based on "train.csv" patterns and then generates predictions in "test.csv."

Then it saves all of the predictions in "submission.csv" and prints the message "Your submission was successfully saved!"


Submission

This is the final step in the process, in which we submit the code to the competition.



This demonstrates an accuracy of 0.77511, or 77 percent accurate results.


Contribution

Now that we have an accuracy of 0.77511, we can try to improve it.


To improve accuracy, we can try a different model. We can experiment with SVC, or Support Vector Classification, which is a class of SVM (Support Vector Machine) algorithm to see if it can improve accuracy, as this can handle classifier tasks. The implementation of Support Vector Classification is based on libsvm. It can have a variety of parameters such as C, kernel, degree, gamma, probability, and many more. However, I have already checked with multiple parameters and the results did not reflect the changes in accuracy which we will get, so in this case we will proceed without any parameters.


We follow the same steps and code as described above in the Titanic tutorial, the only updates we will do are in the last cell which has been shown in the screenshot below, where we use SVM model instead of Random Forest Classifier.


First, we use "from sklearn.svm import SVC" to import SVC from Scikit Learn's SVM library (Please note that this line is updated from previous line we had in Titanic tutorial for Random Forest i.e. "from sklearn.ensemble import RandomForestClassifier")

The model is then updated with "model = SVC()" (Please note that this line is updated from the previous line we had in Titanic tutorial for Random Forest i.e. "model =RandomForestClassifier(n_estimators=100, max_depth=5, random_state=1)")


The remaining steps and lines of code will remain the same and no other changes are needed.


Now we run it to see if there are any errors.



Once successful execution is done we can proceed with submission to the competition.



Voila! The accuracy has been increased to 0.77751 from 0.77511, a 0.0024 increase.



This Notebook can be found here:




Note - For this exercise below tools and libraries are used:

Jupyter Notebook (on Kaggle), Pandas, Numpy, Scikit learn


References



 
 
 
bottom of page