Overview
CIFAR-10 is a dataset which consists of 60,000 32x32 colour images in 10 classes. They were collected by Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton.
We are using the same code provided by pytorch.org at this link.
Step1: Importing packages, dataset and then normalizing it
We are importing essential packages to load CIFAR-10 dataset using torchvision.
Now we are loading CIFAR-10 dataset into trainset and testset. This dataset is downloading from this link.
Let's load some of the images to see how it looks.
Step2: Defining a Convolution Neural Network, but what is it?
A Convolutional Neural Network is an algorithm in deep Learning that takes an input image, assign importance to various objects in it and able to identify differences between these objects
Kernel - The Kernel is the element that performs the convolution operation in the initial half of a Convolutional Layer
Pooling - The pooling layer is a layer that is responsible for reducing the spatial size of the convolved feature
Fully connected layer - Fully connected layer is where all of the inputs from one layer are connected to every activation unit of the next layer
Step3: Defining a Loss function and optimizer, again what is it?
Loss function - It is a function that evaluates on how our algorithm behaves on the dataset provided
Optimizer - It is a way to optimize or an algorithm to optimize different kinds of attributes such that it reduce the losses which will help in achieving the high accuracy
Step4: Training the network
Training is same as the word says i.e. training which means to train the algorithm in such a way that it finds the values in it given the provided attributes
Saving our trained model
Step5: Testing the network on test dataset
Testing as the name suggests it is the process of validating the algorithm on the test dataset
As we can see it gives us an accuracy of 56% on the test dataset
Let's find out what is the accuracy for the 10 classes
So we are here. Why not try increasing the accuracy?
Let's try it out.
First plot a graph to visualize the accuracies for all the 10 different classes
Consider the above training as Experiment 1.
We will start the Experiment 2 by tuning the above given code
Experiment 2
In this experiment we are changing the epoch from 2 to 10, instead of adding a new convolution layer and let's train it
Plotting a horizontal bar plot to visualize the results
Let's visualize and compare it with the previous experiment
Cool! This has increased the overall accuracy from 56% to 60%.
Why not try with a new experiment?
Experiment 3
In this experiment we will add two new convolution layers in the first experiment and see how it goes.
Let's train it.
It looks like accuracy has been decreased. This is not good.
Time to visualize it with a comparison plot.
We can see here that this experiment has decreased the accuracy from 56% to 53%.
Here come a new experiment.
Experiment 4
In this experiment we are adding 8 more new convolution layers in experiment 1, so in total of 10 convolution layers are added.
Let's train it.
It has provided us an increase in 1 % from the previous experiment but this is not what we are looking for. Isn't it?
Ok, let's do one last experiment.
Experiment 5
In this experiment we will try some different combination of things.
Here we are changing the width of our CNN for first convolution layer from 6 to 100, and updating this 100 value to input of second layer of convolution
Also, increase epoch from 2 to 50.
Time to train it.
It will consume a lot of time to train, so let's take a nap.
It's still running!!
Finally, after almost 3 hours runtime we can see the magic.
Woah! The overall accuracy has reached from 56% to 66%. Isn't it amazing?
Let's visualize the comparison of all the experiments we did along with the accuracies provided by each experiment.
We can also see the training time comparison for all the experiments we did, starting from some few minutes to hours.
My contribution:
Change in width of CNN and epoch which took time to train but increased the accuracy from 56% to 66%
Multiple experiments performed that involves addition of different convolution layers, epoch
Visualization using different types of plot diagrams and comparison of various experiments, accuracies and their training time from few minutes to hours
Challenges faced:
Increase the accuracy - I was thinking on how to increase the accuracy as this was the major challenge, so I tried running multiple experiments to see which one can provide me an increasing amount of accuracy for this dataset and that's how I solved it
Unavailability of GPU - While I was running with the last experiment, it stopped in between multiple times with an error message that I have reached my usage limit for GPU. In the end, I changed it to CPU which worked
Experiments, time comparison and visualization - I was not sure how to compare the experiments and while searching I couldn't find some important information to use other than how to plot a graph, so I thought to try storing it in the variable and it worked
This Notebook can be found here: Notebook
Note - We have used the same code provided at this link. We have modified it whenever needed to perform different experiments in a way that increases the accuracy and plotted graph for the same.
References:
Comments