Deep learning with TensorFlow
8 likes1,610 views
This document provides an agenda for a presentation on deep learning with TensorFlow. It includes: 1. An introduction to machine learning and deep networks, including definitions of machine learning, neural networks, and deep learning. 2. An overview of TensorFlow, including its architecture, evolution, language features, computational graph, TensorBoard, and use in Google Cloud ML. 3. Details of TensorFlow hands-on examples, including linear models, shallow and deep neural networks for MNIST digit classification, and convolutional neural networks for MNIST.
Deep learning with TensorFlow
- 1. An Introduction Deep Learning with TensorFlow
- 2. • How Am I? • Machine Learning & Deep Networks • What is ML ? • Neural Network Example • What is Deep Learning? • TensorFlow (v1.0) • What is TF? • TF Architecture • TF evolution • TF language features through an example • The Computational Graph • TensorBoard • TF in Google Cloud ML • TF High Level API • TF Serving • TensorFlow Hands-On • TensorFlow Examples • Linear Model • Shallow MNIST • Deep MNIST • ConvNet MNIST Agenda 1
- 3. How Am I ? • Ndjido Ardo BAR : Data Scientist @ Davidson Consulting UAE • Background : Research In Mathematics • Now: Working in @DU Telecom as a Consultant • Past: • Worked @AXA (Paris): Develop A Credit Scoring Model with H2O • Worked @BearingPoint Hypercube: Did consulting for different companies: • Hypercube (Paris): A web-based Machine Learning Platform • Jonson & Jonson (Florida): Lens Design Process Optimisation (Vistakon) • Keolis: Setup a Big Data Platform and Developed a Fraud Detection Model • And many more … • Co-Founder of a StartUp (MLouma) • Worked @Pasteur Institute: Involved in BioStatistical Research @ndjido 2
- 5. What Machine Learning? - Machine Learning is a field of Computer Science that aims at giving computers the ability to learn complex patterns without being explicitly programmed. - Compared to Statistical models, ML does not make strong hypothesis on the distribution of the variables in a model. Random Forest Neurale Network K-means Few examples of ML Algorithm 4
- 6. What Machine Learning? Detailed Example a ML Algorithm: Neural Network Y(real) Y(estimated) Error BACK PROPAGATION FORWARD PROPAGATION 1 2 activation function 5
- 7. What Machine Learning? Example a ML Algorithm: Neural Network Activation Functions FORWARD PROPAGATION Neurone ErrorOutput loss function 6
- 8. What Machine Learning? Example a ML Algorithm: Neural Network Error =Output BACK PROPAGATION Layer (L)Layer (L-1) Chain Rule Update (Gradient Descent) learning rate 1 2 7
- 9. What is Deep Learning ? Deep Learning is one of the field of AI (Artificial Intelligence) using sophisticated techniques to train multi-layers Neural Networks with complex data format without the need to perform Feature Engineering. 8
- 10. What is Deep Learning ? Why does it even matter ? - New techniques to easier build better Neural Networks: - solutions to vanishing or exploding gradient - Representational learning (source: UBC) 9
- 11. What is Deep Learning ? Why does it even matter ? Performance Amount of Data Classical Algorithms Deep Learning - Hardware Performances with GPU and distributed computation - Overwhelming and more complex amount of data (Big Data): - sensors data - Video, - Voice - Outperforms classical methods with complex data 10
- 12. What is Deep Learning ? Basic Deep Learning Architectures Convolutional Network Recurrent Neural Network Recursive Neural Network Deep Belief Network AutoEncoder 11
- 13. Deep Learning Library Zoo 12
- 15. TensorFlow What is TensorFlow ? - An Open Source Machine Learning tool started by the Google Brain Team in 2015 - TF was create to build complex Neural Networks but is general enough to build a wide variety of ML algorithm or computation systems - A library for numerical computation using Data Flow Graphs - TF is highly scalable. It can be used on desktop, mobile devices, and Data centres with both CPU and GPU - TF can be used by Researchers, Data Scientists and Developers. Data Flow Graph This is a Tensor 14
- 16. TensorFlow TensorFlow Architecture TensorFlow Core Execution System CPU GPU Android IOS … TF supports a lot of front-end API : c++, Python, … 15
- 17. TensorFlow TensorFlow, the most active ML Open Source tool ! (source: Google Next) ~1000 commits / month 16
- 18. TensorFlow Features Anatomy of a TF Model: The Babylonian Algorithm 1 2 3 4 5 6 7 8 1 Constant value 2 Type casting -> float32 3 Variable in the Graph 4 Updating a Variable 5 Adding 2 Variables 6 Session Object 7 Initialise All Variables 8 Running the Model 17
- 19. TensorFlow Features TF Placeholder and Feed Dictionary 1 Placeholder for input value A 2 Feeding the graph with inputs A & B 1 2 Notice there no need for initialisation 18
- 20. TensorFlow Features TF Computational Graph A TF program performs 2 main steps: 1) Assembling a ”lazy” Graph 2) Running the computation Result After Running in a tf.Session() Computational Graph 19
- 21. TensorFlow Features TF Variable Scope & TensorBoard Massive deep neural network are notoriously hard to debug and optimise. TensorBoard just make it easier! You can use TensorBoard to visualise your TensorFlow graph, plot quantitative metrics about the execution of your graph, and show additional data like images that pass through it. When TensorBoard is fully configured, it looks like this: Demo with a Linear Regression Decorate TF programs with tf.name_scope 20
- 22. TensorFlow Features Run TF in Cloud ML Cloud StorageCloud ML setup.py submit to Cloud ML with gcloud ml-engine package uploaded1 2 3 4 4 data fetched train + model stored Demo on Cloud ML 21
- 23. TensorFlow Features TensorFlow High Level API • Different Level of abstraction for Researchers, Data Scientist and Developers • With ML Toolkit TF is now a serious competition to Scikit-Learn (source: Google Next) 22
- 24. TensorFlow Features TensorFlow Serving : Serving Pipeline Data TF Cluster TF Model Storage Training + Storage Model #1 Model #2 Model #n … TensorFlow Serving Client Client Client gRPC 23
- 25. Hands-on TensorFlow 24 https://github.com/ndjido/tensorflow-dubai-meetup-hands-on
- 26. TensorFlow Hands-On . . . 784 = 28 x 28 pixels (.8, .2, 0, 0, 0, 0, 0, 0, 0, 0) (1, 0, 0, 0, 0, 0, 0, 0, 0, 0) Predicted One-hot encoding 0 1 2 3 4 5 6 7 8 9 flattened Shallow MNIST . . . 25
- 27. 784 = 28 x 28 pixels 0 1 2 3 4 5 6 7 8 9 . . . . . . . . . . . . TensorFlow Hands-On Deep MNIST 200 100 60 30. . . 10 26
- 28. TensorFlow Hands-On ConvNet MNIST 28 28 28 x 28 x 6 14 x 14 x 12 7 x 7 x 24 200 0 1 2 3 4 5 6 7 8 9 27