Efficient Neural Network Architecture for Image Classfication
Download as PPTX, PDF5 likes2,252 views
The document outlines the objectives, methodology, and work accomplished for a project involving designing an efficient convolutional neural network architecture for image classification. The objectives were to classify images using CNNs and design an effective CNN architecture. The methodology involved designing convolution and pooling layers, and using gradient descent to train the network. Work accomplished included GPU configuration, designing CNN architectures for CIFAR-10 and MNIST datasets, and tracking training loss, validation loss, and accuracy over epochs.
![References
[1] A. D. J. J. J. B. Eugenio Culurciello, “An Analysis of the Connections Between Layers of Deep Neural
Networks,” arXiv, 2013.
[2] B. K. A.-r. M. B. R. Tara N. Sainath, “Learning Filter Banks within a Deep Neural Network Framework,”
in IEEE, 2013.
[3] A.-r. M. G. H. Alex Graves, “Speech Recognition with Deep Recurrent Neural Networks,” University of
Toronto.
[4] A. Graves, “Generating Sequences with Recurrent Neural Networks,” arXiv, 2014.
[5] Q. V. Oriol Vinyals, “A Neural Conversational Model,” arXiv, 2015.
[6] J. D. T. D. J. M. Ross Grishick, “Rich Features Hierarchies for accurate object detection and semantic
segmentation.,” UC Berkeley.
[7] A. Karpathy, “CS231n Convolutional Neural Networks for Visual Recognition,” Stanford University,
[Online]. Available: http://cs231n.github.io/convolutional-networks/.
[8] I. Sutskever, “Training Recurrent Neural Networks,” University of Toronto, 2013.
[9] “Convolutional Neural Networks (LeNet),” [Online]. Available:
http://deeplearning.net/tutorial/lenet.html.
[10] I. S. E. H. Alex Krizhevsky, “ImageNet Classification with Deep Convolutional Neural Networks,” 2012.](https://image.slidesharecdn.com/mid-term-presentation-151008061313-lva1-app6892/85/Efficient-Neural-Network-Architecture-for-Image-Classfication-22-320.jpg)
![References
[11] R. F. Matthew D Zeiler, “Visualizing and Understanding Convolutional Networks,” arXiv, 2013.
[12] A. K. a. L. Fie-Fie, “Deep Visual Alignment for Generating Image Descriptions,” Standford University,
2014.
[13] A. T. S. B. D. E. O. Vinyals, “Show and Tell: A Neural Image Caption Generator.,” Google Inc., 2014.
[14] J. M. G. H. IIya Sutskever, “Generating Text with Recurrent Neural Networks,” in 28th International
Conference on Machine Learning, Bellevue, 2011.
[15] M. A. Nielsen, “Neural Networks and Deep Learning,” Determination Press, 2014.
[16] J. Martens, “Deep Learning via Hessian-Free Optimization,” in Procedings of 27th International
Conference on Machine Learning, 2010.](https://image.slidesharecdn.com/mid-term-presentation-151008061313-lva1-app6892/85/Efficient-Neural-Network-Architecture-for-Image-Classfication-23-320.jpg)
Efficient Neural Network Architecture for Image Classfication
- 1. Efficient Convolutional Neural Network Architecture for Image Classification Yogendra Tamang MSCS-070-670 Supervisor: Prof. Dr. Sashidhar Ram Joshi Presented By:
- 2. Outline • Background • Convolutional Neural Network • Objectives • Methodology • Work Accomplished • Work Remaining • References
- 3. Background • Learning • Supervised • Unsupervised • AI Tasks • Classification and Regression • Clustering Machine Learning Problem Supervised RegressionClassfication Unsupervised Clustering
- 4. Background • Classification • Classifies data into one of discrete classes • Eg. Classifying digits • Cost Function for Classification Task may be Logistic Regression or Log- likelihood • Regression • Predicts continuous real valued output • Eg. Stock Price Predictions • Cost function for regression type problem are MSE(Mean Squared Error)
- 5. Multi Layerd Perceptrons (MLPs) Input Layer Hidden Layer Output Layer
- 6. Convolutional Neural Networks • One or more convolutional layer • Followed by one or more fully connected layer • Resulting in easy to train networks with many fewer parameters.
- 7. Objectives • To classify images using CNN • To design effective architecture of CNN for image classification task.
- 8. Convolutional Neural Networks • Receptive fields(RFs) • Apply filter to image. • Pooling and subsampling layers
- 11. Methodology • Convolution Layer Design
- 12. Methodology • Pooling Layer Design
- 13. Methodology Example CNN Architecture Learning a Classifier • Gradient Descent Algorithm • Calculate Cost Function or Lost Function J(s) • Calculate Gradient 𝜕𝐽(w)/𝜕w • Update Weights • Stochastic Gradient Descent: Updates Adjust after example. • Minibatch SGD: Updates after batches.
- 14. Learning a Classifier- Negative Log likelihood 𝑁𝐿𝐿 𝜃, 𝒟 = − 𝑖=0 |𝒟| log 𝑃(𝑌 = 𝑦(𝑖)|𝑥 𝑖 , 𝜃) Where 𝒟 is Dataset 𝜃 is weight parameter (𝑥 𝑖 , 𝑦 𝑖 ) is ith training data. Y is target data.
- 15. Work Accompolished 1. GPU Configuration to support CUDA.
- 16. 2. CNN Architecture for CIFAR-10 dataset
- 17. 3. CNN Architecture for MNIST-10 dataset INPUT-> CONV ->MAXPOOL-> CONV -> MAXPOOL-> FULL -> OUTPUT
- 18. MNIST Dataset Training and Output
- 19. Training Loss, Validation Loss, Validation Accuracy on MNIST Dataset 0 0.2 0.4 0.6 0.8 1 1.2 1 2 3 4 5 6 7 8 9 10 TrainingLoss/Validation Loss/ValidationAccuracy Epochs CNN running over mnist dataset Training Loss Validation loss Validation accuracy
- 20. Work Remaining • Dropout Implementation • Parameter Changing
- 21. Time Schedule
- 22. References [1] A. D. J. J. J. B. Eugenio Culurciello, “An Analysis of the Connections Between Layers of Deep Neural Networks,” arXiv, 2013. [2] B. K. A.-r. M. B. R. Tara N. Sainath, “Learning Filter Banks within a Deep Neural Network Framework,” in IEEE, 2013. [3] A.-r. M. G. H. Alex Graves, “Speech Recognition with Deep Recurrent Neural Networks,” University of Toronto. [4] A. Graves, “Generating Sequences with Recurrent Neural Networks,” arXiv, 2014. [5] Q. V. Oriol Vinyals, “A Neural Conversational Model,” arXiv, 2015. [6] J. D. T. D. J. M. Ross Grishick, “Rich Features Hierarchies for accurate object detection and semantic segmentation.,” UC Berkeley. [7] A. Karpathy, “CS231n Convolutional Neural Networks for Visual Recognition,” Stanford University, [Online]. Available: http://cs231n.github.io/convolutional-networks/. [8] I. Sutskever, “Training Recurrent Neural Networks,” University of Toronto, 2013. [9] “Convolutional Neural Networks (LeNet),” [Online]. Available: http://deeplearning.net/tutorial/lenet.html. [10] I. S. E. H. Alex Krizhevsky, “ImageNet Classification with Deep Convolutional Neural Networks,” 2012.
- 23. References [11] R. F. Matthew D Zeiler, “Visualizing and Understanding Convolutional Networks,” arXiv, 2013. [12] A. K. a. L. Fie-Fie, “Deep Visual Alignment for Generating Image Descriptions,” Standford University, 2014. [13] A. T. S. B. D. E. O. Vinyals, “Show and Tell: A Neural Image Caption Generator.,” Google Inc., 2014. [14] J. M. G. H. IIya Sutskever, “Generating Text with Recurrent Neural Networks,” in 28th International Conference on Machine Learning, Bellevue, 2011. [15] M. A. Nielsen, “Neural Networks and Deep Learning,” Determination Press, 2014. [16] J. Martens, “Deep Learning via Hessian-Free Optimization,” in Procedings of 27th International Conference on Machine Learning, 2010.
Editor's Notes
- #17: Different Layers in CNN are: Input Layer Convolution Layer Pooling Layer Dense Layer Output Layer Input layer is 32x32 images of 3 colour channel i.e 3x32x32. Convolution of layer consists of 20 5x5 patches with padding size 2 and stride equal to 1. Pooling layer after first convolution consists of max function with 2x2 patch size. Hence it will reduce the image to 20x16x16. Second convolution layer consists of same properties as that of first convolution layer and the image it produces will be of size 20x16x16 Max pooling layer after second convolution again reduces image to size 20x8x8. The fully connected layer after pooling consists of 1000 units. And it is connected to output layer with only 10 units, for each class.