Demo: Training a custom machine learning model

We've looked at training a machine learning model using Python code and Python based data science libraries. But did you know there are low code and no code options to train a machine learning model using Amazon SageMaker? You don't need deep technical experience, just your data and an AWS account. The scenario we'll explore now will teach a machine to classify images of flowers. You can easily use any image you like though, cats, dogs, people, fruit, cars, shoes, and more. The list goes on. Image classification and machine learning falls under the computer vision umbrella, which allows machines to identify items, people, places, and things in an image or video. More specifically, image classification is a supervised learning problem where labeled images are used to train a model to recognize those labels. Our data contains images of five types of flowers, daisies, dandelions, roses, sunflowers, and tulips. Since the training data are images, the features are just RGB image pixels. We'll feed the data to Amazon SageMaker's image classification algorithm, which is a supervised learning algorithm that supports multi-label classification. It takes an image as input and outputs one or more labels assigned to that image. Under the hood, it's a convolutional neural network, CNN, which is a network architecture for deep learning that learns directly from structured arrays of data-like images. I encourage you to learn more about deep learning, which is the next step in your learning journey by coming up to speed on neural networks. The algorithm needs to know the image labels since we're using supervised learning. In order to do that, you'll provide the labels via two text files in LST format. Here's a look at an LST file. You'll need to manually create the LST files based on the images you're using. There's also a tool that you can use to generate the files for you. I've included my LST files in the GitHub repo for your reference. LST file is a tab separated file with three columns that contain the list of images. The first column specifies the image index. The second column specifies the class label index for the image. And the third column specifies the relative path of the image file. The image index in the first column must be unique across all of the images. The set of class label indices are numbered successively and the numbering should start with zero. For example, zero here is the daisy class. One here is for the rose class. Three is for dandelion and two is for tulip. The next step is to create a training job via the AWS console. Here on Amazon SageMaker, you scroll down under training jobs and you click on training jobs, and it will bring you to a screen like this, where you are able to enter the information for your job. Here, you give a job name. It will default to an IAM role, which is related to security. we are going to use the Amazon SageMaker built-in image classification algorithm. You come down and you choose an algorithm. Here I've selected the image classification that uses MXNet. Scroll down. We're going to use file mode because we'll upload image files. Here you have a quick look at the metrics calculated accuracy. Here, you will select the server type, the instance count, and any additional storage that you need. Let's scroll down and look at the hyperparameters. Notice here, you'll set the hyperparameters with several of them being defaulted. One important hyperparameter that you want to set is a used pre-trained model. We would change that to 1. The one indicates the use of transfer learning. Transfer learning allows you to add your data on top of a pre-trained model, allowing you to train a new model that inherits the learnings from the previous model. This is useful because you leverage previous learnings and avoid starting from scratch. Transfer learning is like inheritance in the Java programming language. You gain pre-existing functionality for reusability and efficiency. Another important hyperparameter is num classes. In this case, we're using five because that's the number of flowers. Another important hyperparameter is the number of training samples. This is simply the number of images. Then also, let's find the epics here. We want the machine learning algorithm to make 10 passes over our data. And then lastly, let's find Resize here. I've set this to 224, and I needed to use this because some images had a shorter width and height, and we need to resize them to all be the same. Scroll down, you'll create what's called channels to point to the training and validation LST files. Before setting up the channels, make sure that you've uploaded the LST files and images to an S3 bucket that you create. We have one for train. We have the train LST file. We have another channel for validation and then the validation LST file. Once you enter that information, you can store a checkpoint and you can place the location for your output data. And then you create the training job. Now this training job, because of the amount of data, actually takes about 45 minutes to train. Once the job completes successfully, you'll be able to go to the CloudWatch log to look for the validation accuracy. So let's scroll here. Notice for Epic one, the accuracy is at 0.54. Let's scroll down all the way to the last Epic. Notice the accuracy did improve and it ended at 0.70. The more training cycles you have or epochs, the better your accuracy. Now that you have trained a model, you can deploy the model through Amazon SageMaker Hosting, making the model available via an API endpoint. This means you can easily integrate this model into existing applications. Now that you've seen this low-code machine learning example, let's explore classification metrics like accuracy in more detail.