Mastering Transfer Learning with TensorFlow Part: 1

Transfer Learning

If we want to build a system using deep learning, we will need a lot of data. A significant amount of time will be spent cleaning, optimizing, and performing other tasks on that data. One simple way to solve this problem is transfer learning. Transfer learning is the process of applying a neural network, trained on a large dataset, to a new dataset. It’s the method of using a pre-trained model in your own project.

What was software development like in the early days? Every company had to write every part of their software line by line. How is it done now? Many libraries and plugins are already developed. Open-source libraries are available, and by using the relevant one for your project, more than half of the work is reduced. Some projects don’t even require writing any code. With just proper configuration, everything can be done.

Similarly, the same approach can be applied to machine learning through transfer learning. We might not need to write any code; just customize an open-source model according to our needs, and the work will be done. Writing the entire model from scratch could take a lot of time, and it might not even achieve the desired accuracy. Additionally, it might not be feasible to collect enough data to train the model.

For the problem we are working on, it might turn out that someone has already done similar work and made the training data publicly available. This makes our task easier. We can take the open-source model and integrate it into our project. By retraining that model with our own data, we can achieve better accuracy. The point of explaining all this is that machine learning is easy. Anyone can learn it if they want to.

Pre Trained Models

A trained network can typically be saved for future use. A saved network trained on a large dataset is called a pre-trained model. This model can be used as it is, meaning it can be employed for the same purpose it was originally trained for. Alternatively, the input and output layers can be customized to suit a new purpose.

There are many pre-trained models available for transfer learning. For example, for images, there are models like VGG19, Inceptionv3 (GoogLeNet), ResNet50, and EfficientNet. For natural language processing, there are models like Google’s BERT, Microsoft’s CodeBERT, and OpenAI’s GPT-3. Apart from these, there are many other pre-trained models. You need to find the one that is most suitable for your project.

TensorFlow Hub

TensorFlow Hub offers a variety of pre-trained models. We can choose the one that is most appropriate for our project.

Using Pre-trained Models in Your Project Pre-trained models can be used in two ways:

1. Use them as they are, without any modification.
2. Modify them according to your specific requirements before using them.

1) Using a Pre-trained Model Without Modification

First, we'll look at how to use a pre-trained model exactly as it is, without any modifications. This is the simplest process in transfer learning. First, you need to load the pre-trained model, and then it's ready for any prediction task. As we’ve mentioned, there are many pre-trained models available. For simplicity, we'll use MobileNet. Since it’s optimized for mobile devices, it doesn't take much time to run. You can use any other model if you prefer, and all of them are available on TensorFlow Hub.

To load the model, you can write something like the following code. Typically, instructions on how to load a specific model can be found on the model’s page.

Example code to load the model:

model = tf.keras.Sequential([
hub.KerasLayer("https://tfhub.dev/google/tf2-preview/mobilenet_v2/classification/4", output_shape=[1001])
])        

You could say the work is almost done at this point. Now, we can predict any image using the model. First, we'll need to download an image. Since MobileNet requires input images to be of size 224x224 pixels, we need to resize the downloaded image accordingly. After that, the image needs to be normalized.

IMAGE_RES = 224
 
predict_img = tf.keras.utils.get_file('image.jpg','https://upload.wikimedia.org/wikipedia/commons/thumb/4/43/AD2009Aug07_Natrix_natrix_01.jpg/640px-AD2009Aug07_Natrix_natrix_01.jpg')
predict_img = Image.open(predict_img).resize((IMAGE_RES, IMAGE_RES))
predict_img
 
predict_img = np.array(predict_img)/255.0 # data normalization
predict_img.shape        

Now the image is ready for prediction. TensorFlow models take input in the form of a batch of images, so we need to convert the single image into a batch image. This can be done as follows:

result = model.predict(predict_img[np.newaxis, ...])
result.shape        

The output we get from the model will be a probability vector, indicating the likelihood of the image belonging to each of the 1001 classes. From this vector, we can take the maximum value to identify the index of the class to which the image most likely belongs.

predicted_class = np.argmax(result[0], axis=-1)
predicted_class        

Finally, from the index value, we can determine what the image represents by mapping the index to the corresponding label name. To do this, we need to download the label index file, which contains the mapping of class indices to human-readable names.

# get level name from prediction
 
labels_path = tf.keras.utils.get_file('ImageNetLabels.txt','https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt')
imagenet_labels = np.array(open(labels_path).read().splitlines())
 
plt.imshow(predict_img)
plt.axis('off')
predicted_class_name = imagenet_labels[predicted_class]
_ = plt.title("Prediction: " + predicted_class_name.title())        

Great! You now have the predicted name of the image. Below is the complete code that encapsulates everything we've discussed for using transfer learning with a pre-trained MobileNet model in Google Colab:

import matplotlib.pylab as plt
 
import tensorflow as tf
import tensorflow_hub as hub
from tensorflow.keras import layers
 
 
import numpy as np
import PIL.Image as Image
 
 
model = tf.keras.Sequential([
    hub.KerasLayer("https://tfhub.dev/google/tf2-preview/mobilenet_v2/classification/4", output_shape=[1001])
])
 
 
 
IMAGE_RES = 224
 
predict_img = tf.keras.utils.get_file('image.jpg','https://upload.wikimedia.org/wikipedia/commons/thumb/4/43/AD2009Aug07_Natrix_natrix_01.jpg/640px-AD2009Aug07_Natrix_natrix_01.jpg')
predict_img = Image.open(predict_img).resize((IMAGE_RES, IMAGE_RES))
predict_img 
 
predict_img = np.array(predict_img)/255.0 # data normalization 
predict_img.shape
 
# convert single image array to batch dimension, cause models always want a batch of images to process
 
result = model.predict(predict_img[np.newaxis, ...]) 
result.shape
 
predicted_class = np.argmax(result[0], axis=-1)
predicted_class
 
# get level name from prediction
 
labels_path = tf.keras.utils.get_file('ImageNetLabels.txt','https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt')
imagenet_labels = np.array(open(labels_path).read().splitlines())
 
plt.imshow(predict_img)
plt.axis('off')
predicted_class_name = imagenet_labels[predicted_class]
_ = plt.title("Prediction: " + predicted_class_name.title())