Coding a more complex Convolutional Neural Network in Python

by Darko Medin, the Creator of bioaiworks.com and AI platform for Life Science

The goal of this tutorial is to learn to design and code a more complex Convolutional Neural Network (CNN) in Python. In previous edition https://www.linkedin.com/pulse/simple-neural-network-code-i-convolutional-ann-darko-medin-imsnf we explored the simple Neural Network design.

This time we will use complex architecture and try to improve the accuracy. Do remember, after 90% it usually become more and more difficult to improve accuracy, so if we can improve it by several percent, that would be a super achievement! You may find full code for this project here https://github.com/DarkoMedin/Complex_NeuralNet/blob/main/complex_neural_network.py

In this tutorial we will be using the same data processing code as in previous one and will also be using the MNIST dataset. https://github.com/zalandoresearch/fashion-mnist

However, this time we will be adding new and more complex blocks/layers in the Neural Networks. We will be using the 'Block' design. Neural Network will be designed from multiple blocks and each block will have specialized layers of neurons to improve the performance of the Artificial Neural Network model.

To install required packages in your terminal or command prompt after activating the environment. If you use Google Colab, use the ! mark before pip.

pip install numpy

pip install tensorflow

pip install scikit-learn

pip install matplotlib

pip install pandas

Ok now to the coding part!

First we need to load the libraries and perform the data processing, exactly as in the previous tutorial.

We will be using an approach which is called the 'Block Sequential Design' of the Neural Network. We will have new layers which we haven't discussed before, the Batch Normalization, the MaxPoolinig2D and the Dropout layers.

Here is how 1 block would look like...

What are these layers for?

Conv2D - We discussed this one in the previous tutorial. It serves for 2D processing of the image data.

Batch Normalization ( ) will stabilize and speed up the training of deep neural network by reducing what's called the internal covariate shift. What does this mean? We will have more stable training, batch effects / noise will be reduced from the data.

Maxpool2D - Max pool layer will enable neural network to focus on stronger signals, effectivelly reducing overfitting. Also maxpool layers will perform dimensionality reduction, also reducing the computational price for training the neural network, while not reducing its effectiveness and accuracy.

Dropout layer will enable drooping some of the neural pathways and neuron information to reduce overfitting. Dropout layer, combined with Maxpool layers will make our neural network not only more accurate on training data, but also more accurate on unseen data by increasing generalization. This process is called the regulatization of the Neural Network.

Now, we said we will add these layers in blocks and use multuple blocks to constuct the neural network. Here is how a 4 block Neural Network will look like.

As yo can see now we have some more new layers such as Flatten and Dense.

A Dense Layer is a fully connected (FC) laye and in convolutional neural networks (CNN) serves to “connect” every neuron in its input to every neuron in its output through a learnable weight matrix, effectivelly integrating and extracting all information learned from the previous layers.

However, Dense layers accept information as 1-dimensional vector of features, so we need to flatten input from Conv2D to 1D using the Flatten layer we added on the beginning of the last block, called the Classifier head.

Classifier head will consolidate information learned from previous layers and prepare it for the classification task (sometimes regression tasks, in that case would be called Regression head)

Ok enough about schematics, lets skip to the coding part!

This is how the code for all explained before looks, 4 block complex Convolutional Neural Network for image recognition, As you can see we coded 3 blocks and a classifier head. What we have to do know is compile the Neural Network and start training and evaluating the Neural Network!

With this code the Neural Network training will start and you can follow its progress in the Terminal. If you call model.evaluate(testI, testL) after trainin it will provide you with metrics of testing on unseen data after...

As you can see the accuracy is just slightly bellow 92% and this is the test data accuracy on model unseen data. By making the Neural Network more complex, and using various layers with different functions, we made the Neural Network accuracy jump from 90 to 92% compared to previous tutorial (which is huge). Remember, increasing accuracy after 90% is very difficult. Every percent increase is huge.

If we continued training and made more hyperparameter tuning, the jump could go to 93-94% with this architecture. This was only for tutorial purposes where only 20 iterations were used. Also we made the Neural Network more generalizable, less overfit to the data which is very important aspect when training Neural Networks.

Thanks for reading! In the Next edition we will be talking about Advanced Neural Networks.