The Ultimate Guide to Convolutional Neural Networks for Beginners

Overview

Don’t you think it is incredible that self-driving cars can detect pedestrians, traffic signs, obstacles, and other objects on the road? How do they observe the surroundings and make decisions? The secret behind such an amazing feat lies in the powerful technique known as convolutional neural networks (CNNs), which serves as the backbone of many such computer vision applications.

Convolutional neural networks (CNNs) are not limited to computer vision but also various other domains such as natural language processing, speech recognition, audio analysis, medical image analysis, and many more. CNNs can handle complex and high-dimensional data better than traditional machine learning models. Moreover, CNNs are robust to noise and variations and have better generalization ability on new and unseen data.

This article will explore the background of convolutional neural networks, their various components, hyperparameters, and some popular architectures. So, let’s dive in and learn everything about CNNs in this amazing article.

Background Of Convolutional neural networks

It all started in the 1980s when Kunihiko Fukushima and his research group proposed the Neocognitron, which is a hierarchical network of simple and complex cells to recognize handwritten characters. However, in those days, it did not receive enough attention due to their computational burden and limited availability of data.  The applications of neural networks were limited to recognizing handwritten characters such as postal codes and bank checks.

However, a breakthrough happened in 2012 when Alex Krizhevsky and his colleagues developed a convolutional neural network model called AlexNet and won ImageNet (which was a large-scale image recognition competition). Their model achieved a remarkable performance by outperforming the previous state-of-the-art by a large margin. 

With the breakthrough of AlexNet, great interest was renewed in convolutional neural networks and deep learning. Researchers around the world understood the significance and potential of convolutional neural networks to handle complex and diverse tasks, such as object detection, face recognition, natural language processing, and many others.

Understanding Convolutional Neural Networks

Convolutional Neural Networks (CNN) is a type of deep learning algorithm for analyzing images.  It has multiple layers (shown in the figure below) each serving a specific purpose in the feature extraction and pattern recognition process. The section below discusses various components of CNN in detail.

Convolutional Layer

In the convolutional layer, filters are applied to the input image to extract specific features such as edges, shapes, and patterns. The extracted features are also known as feature maps, which indicate the locations and strengths of the features in the image. The process of extracting important features from images is also known as the convolution operation.

In the convolution operation, a filter (which is also known as a kernel and is typically a 3×3 or 5×5 matrix) slides over the input matrix repetitively, producing the dot product (element-wise multiplication and summation) of the input pixels and the filter values. The dot product is stored in an output matrix, also known as a feature map.

In the next step, the filter is shifted by a certain number of pixels, called the stride, and the above process is repeated until the filter has swept across the entire image.

Let us try to visualize the convolution operation with the help of the figure below. In the figure, we have a filter of size 3×3 matrix that slides over the input image, which is a 5×5 matrix. In the first step, the filter starts from the top-left corner of the input image and multiplies each element of the filter with the corresponding element of the input image. Then, it adds all the products and stores the sum in the feature map.  The value of the feature map is found to be 22 in this case.

In the second step, the filter is moved one pixel to the right, and after the convolution operation, the value of the feature is found to be 15.

This process needs to be repeated until the filter covers the entire input image, resulting in a 3×3 feature map. This is how the convolution operation between the input images and filters takes place.

Activation Layer

In convolutional neural networks, an activation layer such as ReLU (Rectified Linear Unit) is generally applied after each convolutional layer.  It adds non-linearity to the model by transforming the output of the convolutional layer. By doing so, the network learns complex patterns between the features in the image and generalizes better. 

The activation layer also helps to alleviate the vanishing gradient problem, which may impede the training of deep neural networks.

Pooling layer

The pooling layer is used to reduce the spatial dimension of the feature map by applying the pooling operation. Similar to the convolution layers, the pooling layers also use filters to scan the input.  However, the pooling layers don’t have learnable parameters, unlike convolution layers.

Instead, they compute the maximum or the average value of the pixels within the filter and populate an output array using those values. There are two common types of pooling operations: max pooling and average pooling.

Max pooling

The max pooling selects several small subregions in the feature map, finds the maximum in those regions, and uses it to create a pooled (downsampled) feature map. There are several advantages of max pooling:

• Max pooling reduces the number of parameters and computations in the model, making it computationally efficient.

• Max pooling improves the generalization ability by making the model less sensitive to the location and orientation of features in the input.

• Max pooling helps to avoid overfitting by suppressing noise and irrelevant details in the input, which leads to better detection of dominant features.

Average pooling

The average pooling selects several small subregions in the feature map, computes the average of the pixels in those regions, and uses it to create a pooled (downsampled) feature map. There are several advantages of average pooling:

• Average pooling improves the robustness of the model by reducing the effect of background noises and outliers in the feature map.

• Average pooling improves the computational performance by reducing the number of parameters in the model.

• Average pooling smooths the feature map and reduces the variance, which helps to extract more global features from the input.

Read More: https://paravisionlab.co.in/convolutional-neural-networks/

Hashtags: #ConvolutionalNeuralNetworks

• #DeepLearning

• #ArtificialIntelligence

• #ComputerVision

• #MachineLearning

• #AIForBeginners

• #TechEducation

• #NeuralNetworks

• #DataScience