HPE and NVIDIA empowering AI and IoT

Editor's Notes

• #7: Let’s take a look at how deep learning actually works in practice… Say you want to automatically identify various types of animals, which is a classification task. The first step is to assemble a collection of representative examples to be used as a training dataset which will serve as the experience from which a deep neural network will learn. If you only need to classify cats vs. dogs, then you only need to have cat images and dog images in the training dataset. In this case you’ll need several thousand images, each with a label indicating whether it is a cat image or a dog image. To ensure the training dataset is representative of all the pictures of cats and dogs that exist in the world, it must include a wide range of species, poses, and environments in which dogs and cats may be observed. [next]
• #8: The next thing you need is a deep neural network model. Typically, this will be an untrained neural network designed to perform a general task like detection, classification, or segmentation on a specific type of input data like images, text, audio, or video. Here you can see a very simple model of an untrained neural network. At the top of the model, there is a row or “layer” that has 4 input nodes, and at the bottom there is a layer that has 2 output nodes. Between the input layer and the output layer are a few so-called “hidden” layers with several nodes each. The white lines show which nodes in the input layer share their results with nodes in the first hidden layer, and so on all the way down to the output layer. You may hear the nodes referred to as “artificial neurons” or “perceptrons” since their simple behavior is inspired by the neurons in the human brain. A real deep neural network model would have many hidden layers between the input layer and the output layer - which is why it’s called “deep” - but a simplified representation works fine for this example. Just keep in mind that the design of the neural network model is what makes it suitable for a particular task. For example, the best models for image classification are very different from the best models for speech recognition. And the differences can include the number layers, the number of nodes in each layer, the algorithms performed in each node, and the connections between nodes. It’s worth noting that there are readily available deep neural network models for image classification, object recognition, image segmentation and several other tasks – but it is often necessary to modify these models to achieve high levels of accuracy for a particular dataset. So, the computation required to train one version of the model is multiplied by the number of model variations that need to be evaluated. All of this processing requires a tremendous amount of computation, much of which can be performed in parallel, which makes deep learning an ideal workload for GPUs to accelerate. ---------- simple version ---------- So, if the goal is to train a deep neural network to distinguish cats vs. dogs, you would use a neural network model that is designed to be good at image classification. ---------- detailed version -------- For the image classification task of distinguishing images of cats vs. dogs, we would probably use a convolutional neural network, such as AlexNet, which is comprised of nodes that implement simple generalized algorithms. 1. Convolution layers (linear, element-wise matrix multiplication and addition) 2. Non-linearity function layers (ReLU - rectified linear unit, which replaces negative values with zero) 3. Pooling layers (subsampling/downsampling, reduces dimensionality while retaining the most important information, max/avg/sum) Using these simple, generalized algorithms is a key difference and advantage for deep learning vs. earlier approaches to machine learning, which required many custom data-specific feature extraction algorithms to be developed by specialists for each dataset and task. ------------------------------------ [next]
• #9: Once you have assembled a training dataset and selected a neural network model, a deep learning framework (such as Caffe2, MXNET, or TensorFlow) is used to feed the training dataset through the neural network. For each image that is processed through the neural network, each node in the output layer reports a number that indicates how confident it is that the image is a dog or a cat. In this case there are only two options, so the model needs just two nodes in the output layer – one for dogs and one for cats. When these final outputs are sorted most-confident to least-confident, the result is called a confidence vector. The deep learning framework then looks at the label for the image to determine whether the neural network guessed (or “inferred”) the correct answer. If it inferred correctly, the framework strengthens the weights of the connections that contributed to getting the correct answer. And vice-versa, if the neural network inferred the incorrect result, the framework reduces the weights of the connections that contributed to getting the wrong answer. After processing the entire training dataset once, the neural network will generally have enough experience to infer the correct answer a little more than half of the time (slightly better than a random coin toss) but it will require several additional “epochs” (meaning rounds of processing the entire training dataset) to achieve higher levels of accuracy. There are many deep learning frameworks to choose from, each with its own set of strengths, programming language interfaces, etc. but they’re all GPU-accelerated using the CUDA Deep Neural Networks library (cuDNN) and other libraries in the NVIDIA Deep Learning SDK. You can learn more about all the deep learning frameworks and the NVIDIA Deep Learning SDK on our developer web site at developer.nvidia.com. [next]
• #10: So, now that the model has been trained on a large representative dataset, it’s very good at distinguishing between cats vs. dogs. But if you showed it a picture of a racoon it would be very confused since it has no previous experience with racoons, and would probably report a low confidence number for both dog and cat. If you needed the ability to classify racoons as well as dogs and cats, you would simply modify the design topology of the model to add a third node to the output layer, expand the training dataset to include thousands of representative images of racoons, and use the deep learning framework to re-train the model. There’s no need to manually write a racoon feature extraction algorithm and figure out how to integrate it into your deep neural network model, just modify the network topology a bit and re-train the model with your new dataset. [next]
• #11: Once the model has been trained, much of the generalized flexibility that was necessary during the training process is no longer needed, so it’s possible to optimize the model for significantly faster runtime performance. [Story: How Cooper learned to classify cars, “tucks”, busses, motorcycles, etc.] Common optimizations include fusing layers to reduce memory and communication overhead, pruning nodes that do not contribute significantly to the results, and other techniques supported in the NVIDIA TensorRT runtime. [next]
• #12: Your fully trained and optimized model is then ready to be integrated into an application that will feed it new data, in this case images of cats and dogs that it hasn’t seen before, and it will be able to quickly and accurately infer the correct answer based on its training. And your application can be deployed on a GPU-accelerated platform in your datacenter, in the cloud, on a local workstation, in a robot, a smart camera, or even a self-driving car.
• #17: There are a wide range of GPU-accelerated platforms you can use to accelerate deep learning training and inference application workloads. If you want a fully-integrated solution, we recommend the DGX-1 supercomputer in a box which delivers the performance equivalent of 100s of CPU-only servers using 8 world-class Tesla GPUs, or its little brother the DGX Station, which is powered by 4 Tesla GPUs and runs whisper-quiet next to your desk. If you just want to get started on a prototype using your existing workstation, the TitanV includes new TensorCores designed specifically for deep learning that deliver up to 12x higher peak TFLOPs for training. In the datacenter, the Tesla P100 and V100 with NVLink Technology deliver strong scaling support for mixed workloads across both HPC applications and Deep Learning training & inference applications. And for scale-out inference workloads the Tesla P4 (GP104) supports high efficiency (perf/watt) low-latency performance. And, of course, if you need to deploy deep learning applications in automotive or embedded applications, NVIDIA offers the DrivePX and Jetson platforms. ====================== NVIDIA also provides a wide range of GPU-accelerated platforms you can use to accelerate deep learning training and inference application workloads. If you want a fully-integrated solution, we recommend the DGX-1 supercomputer in a box which delivers the performance equivalent of 250 CPU-only servers, or it’s little brother the DGX Station, which runs whisper-quiet next to your desk. If you just want to get started on a prototype using your existing workstation, the Titan X Pascal supports fast 32-bit floating point (FP32) and 8-bit integer (INT8) performance for deep learning applications. In the datacenter, the Tesla P100 and V100 with NVLink Technology deliver strong scaling support for mixed workloads across both HPC applications and Deep Learning training & inference workloads (using FP64, FP32, and FP16). And for scale-out inference workloads the Tesla P4 (GP104) supports high efficiency (perf/watt) low-latency performance with fast FP32 and INT8. And, of course, if you need to deploy deep learning applications in automotive or embedded applications, NVIDIA offers the DrivePX and Jetson platforms. ====================== What happens after development. NGC is tuned for all these platforms, but what happens next is you productionize your hard work, you created this awesome AI solution, there is a seamless deployment path to a cloud based microservices in the form of the NGC TensorRT container, or you can take it to the the embedded devices… you want to productize your research or your solution, this is the path. Take these models into Jetpack and Driveworks for robotics, drones, autonomous vehicles, etc.