Startup Jungle Cambodia | How to Build your First Machine Learning Application
Download as PPTX, PDF3 likes1,368 views
The document outlines a workshop on building a machine learning application, encompassing topics such as artificial intelligence, machine learning types, and algorithms. It discusses the evolution of AI through three waves, providing examples like image recognition and training processes. Furthermore, it shares insights on various machine learning libraries and resources for further learning.
Startup Jungle Cambodia | How to Build your First Machine Learning Application
- 1. AI Machine Learning Workshop (build your first ML application) December 6th 2017 STARTUP JUNGLE
- 2. MACHINE LEARNING 2 TABLE OF CONTENT Artificial Intelligence introduction Machine Learning and Neural Networks Build your first Machine Learning application
- 3. MACHINE LEARNING 3 TABLE OF CONTENT Artificial Intelligence introduction Machine Learning and Neural Networks Build your first Machine Learning application
- 4. MACHINE LEARNING 4 Product Foundry. For innovation teams and ambitious startups/SMEs. Focus: deep tech. Venture Builder. Serial entrepreneurs. Slash Ventures is where we build our own lean projects. Offices in Singapore and Phnom Penh
- 5. MACHINE LEARNING 5 Learning Community. Learn from the best. Build great products, and bring them to market. Meetup.com/Startup-Jungle/ STARTUP JUNGLE
- 6. MACHINE LEARNING 6 STARTUP JUNGLE You can ... 1. Join us every month for great (free) sessions 2. Meet awesome people in tech & startups 3. Mentor and share your knowledge - we are RECRUITING great companies to join!
- 11. MACHINE LEARNING From Artificial Power . . . . . . to Artificial Intelligence 11
- 12. MACHINE LEARNING WHAT IS ARTIFICIAL INTELLIGENCE? “the science of engineering and making intelligent machines” 12
- 13. MACHINE LEARNING 2 TYPES OF AI WEAK AI - Narrow tasks - Very powerful tool STRONG AI - General intelligence - Outperform humans on all tasks 13
- 14. MACHINE LEARNING 2 TYPES OF AI WEAK AI - Narrow tasks - Very powerful tool STRONG AI - General intelligence - Outperform humans on all tasks FOCUS OF THIS TALK 14
- 15. MACHINE LEARNING 3 WAVES OF AI 15
- 16. MACHINE LEARNING WAVE 1: EXPERT SYSTEMS + Expert systems are rule-based - Very narrow, no learning capability, poor in real-world 16
- 17. MACHINE LEARNING 2004: 0 finish 2005: 5 finish Limits of expert systems: DARPA car challenge 17
- 18. MACHINE LEARNING WAVE 2: MACHINE LEARNING 95% of ‘hyped’ AI applications as of 2017 18
- 19. MACHINE LEARNING WAVE 2: MACHINE LEARNING + Probabilistic, statistical learning techniques + Good at supervised learning, classifying and predicting - Needs lots of data to learn - Limited ability to understand context and reason - Statistically impressive, individually unreliable 19
- 20. MACHINE LEARNING WAVE 3: (FUTURE) CONTEXTUAL ADAPTATION + AI construct explanatory models + Seed AI require much less training data 20
- 21. MACHINE LEARNING EXAMPLE: CAT IMAGE RECOGNITION 21
- 22. MACHINE LEARNING 3 WAVES OF AI 22
- 23. MACHINE LEARNING From AI-assisted Humans . . . . . . To Human-assisted AI’s Will Humans evolve into supervisors of AI’s or bots? 23
- 24. MACHINE LEARNING TABLE OF CONTENT Artificial Intelligence introduction Machine Learning and Neural Networks Build your first Machine Learning application 24
- 29. MACHINE LEARNING Learn from experiences Follow instructions WHAT IS MACHINE LEARNING? 29
- 30. MACHINE LEARNING Learn from experiences DATA WHAT IS MACHINE LEARNING? Learn from experiences Follow instructions 30
- 31. MACHINE LEARNING BITCOIN PRICE PREDICTION 31
- 32. MACHINE LEARNING 1007 6416 (100,1007) (1100,6416) ? BITCOIN PRICE PREDICTION 32
- 34. MACHINE LEARNING Error : ERROR CALCULATION 34
- 35. MACHINE LEARNING Error : ERROR CALCULATION 35
- 36. MACHINE LEARNING Error : ERROR CALCULATION 36
- 37. MACHINE LEARNING Data : (x , y) , x=input=feature , y = output f(x) : w*x + b , algorithm Error : cost function , square error number of iteration : loop, number of repeat LINEAR REGRESSION 37
- 38. MACHINE LEARNING Ex: we want differentiate between apple and watermelon. HOW TO TEACH A MACHINE? or Apple Watermelon 38
- 39. MACHINE LEARNING Simple training data: 2 dimensions (weight, size) Label Weight (g) 70 100 1000 1500 1200 Size (cm) 7.6 8.3 31 40 36 IDENTIFY PHYSICAL CHARACTERISTICS OF FRUITS 39
- 40. MACHINE LEARNING Label Weight (g) 70 100 1000 1500 1200 Size (cm) 7.6 8.3 31 40 36 IDENTIFY PHYSICAL CHARACTERISTICS OF FRUITS 40
- 41. MACHINE LEARNING ● Regression ● Bayesian ● Decision Tree ● Neural Networks ● Deep Learning ● Clustering MACHINE LEARNING ALGORITHMS 41
- 42. MACHINE LEARNING ● Regression ● Bayesian ● Decision Tree ● Neural Networks ● Deep Learning ● Clustering MACHINE LEARNING ALGORITHMS 42
- 43. MACHINE LEARNING 70 7.6 162.8 Weight Size w1= 2 w2= 3 f(x) = Weight * w1 + Size * w2 70 * 2 + 7.6 * 3 = 162.8 100 * 2 + 8.3 * 3 = 224.9 1000 * 2 + 31 * 3 = 2093 1500 * 2 + 40 * 3 = 3120 1200 * 2 + 36 * 3 = 2508 Label Weight (g) 70 100 1000 1500 1200 Size (cm) 7.6 8.3 31 40 36 NEURAL NETWORK - BEHIND THE SCENE 43
- 44. MACHINE LEARNING Label ~ Y Weight (g) ~X1 70 100 1000 1500 1200 Size (cm) ~ X2 7.6 8.3 31 40 36 f(x) = Weight * w1 + Size * w2 70 * 0.01 + 7.6 * -0.32 = -1.73200 100 * 0.01 + 8.3 * -0.32 = -1.65600 1000 * 0.01 + 31 * -0.32 = 0.08 1500 * 0.01 + 40 * -0.32 = 2.2 1200 * 0.01 + 36 * -0.32 = 0.48 70 7.6 -1.73 w1= 0.01 w2= -0.32 NEURAL NETWORK - BEHIND THE SCENE Weight Size 44
- 45. MACHINE LEARNING Pixel Values NEURAL NETWORK - BEHIND THE SCENE 45
- 46. MACHINE LEARNING f(x)=y = w1* pixel1 + w2* pixel2 + … + wn * pixeln 255 155 y pixel1 pixel2 w 1 w 2 55 255 w 3 w n pixel3 pixeln . . . . . . NEURAL NETWORK - BEHIND THE SCENE 46
- 47. MACHINE LEARNING SURVEY Why are you interested in AI? 47
- 48. MACHINE LEARNING 10min BREAK Pizzas and beer ☺ 48
- 49. MACHINE LEARNING TABLE OF CONTENT Artificial Intelligence introduction Machine Learning and Neural Networks Build your first Machine Learning application 49
- 51. MACHINE LEARNING Weight Size W1 W2 Output Output = Weight * W1+ Size * W2 NEURAL NETWORK ARCHITECTURE 51
- 52. MACHINE LEARNING Input Layer Hidden Layer Output Layer Weights Weights . . . . . . Pixel 1 Pixel 2 Pixel N NEURAL NETWORK ARCHITECTURE 52
- 53. MACHINE LEARNING ● Get better using Data. ● Update Weights. ● 1 Example 1 Prediction. LEARNINGS SO FAR? 53
- 54. MACHINE LEARNING iteration 1: NN(cat_image) = cat : 0.2, dog : 0.7 iteration 2: NN(cat_image) = cat : 0.4, dog : 0.5 iteration n : NN(cat_image) = cat: 0.92, dog:0.05 EXAMPLES 54
- 55. MACHINE LEARNING ● Classification is the operation that is used in production. ● We are going to create a prediction using those weights. CLASSIFICATION 55
- 56. MACHINE LEARNING MACHINE LEARNING CYCLE 56
- 57. MACHINE LEARNING ● Epochs (number of iterations) ● Alpha (learning rate) ● Size of the Neural Network. => Warning: there is no rule of thumb to tune them. TUNE WITH HYPERPARAMETERS 57
- 58. MACHINE LEARNING ● How many times you input the data ● How many times you update the weights EPOCHS 58
- 59. MACHINE LEARNING ● By how much you update the weights ● Speed of training ● weight += alpha * update ALPHA 59
- 60. MACHINE LEARNING Iterations : 10 000 alpha : 0.001 1 layer of size 100 Time = 44s GOOD CONFIGURATION 60
- 61. MACHINE LEARNING Iterations : 10 000 alpha : 0.001 1 layer of size 1 Time = 34s SMALL NETWORK 61
- 62. MACHINE LEARNING Iterations : 10 000 alpha : 0.001 5 layers of size 1000 Time = 25mn 25s BIG NETWORK 62
- 63. MACHINE LEARNING Iterations : 10 000 alpha = 0.0000005 1 layer of size 100 LOW ALPHA 63
- 64. MACHINE LEARNING Iterations : 10 000 alpha = 0.3 1 layer of size 100 BIG ALPHA 64
- 65. MACHINE LEARNING ● Training = optimize weights ● Classification = Application. ● Better Training = Better Accuracy. SUMMARY 65
- 66. MACHINE LEARNING MACHINE LEARNING LIBRARIES 66
- 67. MACHINE LEARNING Pros: • High level API based on tensorflow • Easy for beginners • Fast results • Huge documentation Cons: • Small community • Huge documentation TFLearn 67
- 68. MACHINE LEARNING Pros: ● Great Documentation. ● Great feature from dataset to deep learning algorithm. ● It’s really easy to get into this library. ● Dynamic Graph. ● Debug is easy as it should be. Cons: ● New library so the community is small. ● Can have bug as it’s quite new. PyTorch 68
- 69. MACHINE LEARNING Classification of sentences in English and Khmer ! TEXT CLASSIFICATION 69
- 70. MACHINE LEARNING VISA APPLICATION CHATBOT A chatbot is a computer program which conducts a conversation via auditory or textual methods. 70
- 72. MACHINE LEARNING ● Visa Chatbot ● Speaker Recognition (prototype) ● Speech recognition (prototype) ● Newscrawler OUR JOURNEY 72
- 73. MACHINE LEARNING ● No knowledge ● Small objectives ● Read ● Implement OUR JOURNEY 73
- 74. MACHINE LEARNING ● Python (But if you already programmed a lot it should be easy) ● Linear Algebra i.e Matrix operation. ● Datastructure REQUIREMENTS TO START 74
- 75. MACHINE LEARNING ● http://iamtrask.github.io/2015/07/12/basic-python-network/ ● http://iamtrask.github.io/2015/07/27/python-network-part2/ ● https://www.youtube.com/watch?v=vOppzHpvTiQ&list=PL2-dafEMk2A7YdKv4XfKpfbTH5z6rEEj3 ● http://www.wildml.com/2015/09/implementing-a-neural-network-from-scratch/ ● https://www.coursera.org/learn/machine-learning ● https://ayearofai.com/ - For great free dataset use www.kaggle.com - Quora. - Medium. https://bitbucket.org/ventureslash/neural-network-lib (example to build from scratch a Neural Network) https://bitbucket.org/ventureslash/naivebayes (example to build from scratch a Naive Bayes) https://bitbucket.org/ventureslash/demos (demos source code) USEFUL RESOURCES 75
Editor's Notes
- #25: What is Machine Learning & its process. Team members introduction
- #26: Simple Gathering data Preparing data Choosing models Train Evaluation Hyperparameter Tuning Prediction
- #27: What the main different between human and machine learning? Here is the human learning, we have a beautiful lady is teaching her students.
- #28: In this image you will see a very cute machine teacher who is teaching his machine students. Is that call machine learning ?
- #29: The answer no. lolz
- #30: So what is the machine learning. This is the real world human and machine, what is the main differences between human and machine is , human learn from past experiences while machine need to tell what to do, they need to be programed, they follow the instructions. Another question is, can we can get machine to learn from past experience too? The answer yes we can. That’s what machine learning is .
- #31: for machines, the past experiences have a name , we called data. So in the next few minutes, i’m going to show you few examples in which we can teach machine to learn from previous data, more importantly i’m going to show you that, these algorithms are actually pretty easy and the machine learning is nothing to fear.
- #32: https://www.desmos.com/calculator/crzk7slwsb This example you cannot apply for investing bitcoin, unless you need to research more about it. You can improve it later Let’s have a look the first example, let’s say we studying the bitcoin price market, our task is to predict a price of the bitcoin by given the date. The graph is the price of the bitcoin in 2017 from January to November. Now we want to predict a price in December. How can we do?
- #33: To help us out we collect some previous data of the bitcoin price,Let put data in the grid, the x-axis represents the date of the bitcoin’s price and the y-axis represents the price of the bitcoin in dollars.. To make it easy and calculable I transform the date to numbers. So 100 represent as the month of January and 200 represent as February and so on. Can you guess the bitcoin price in december or (equal to 1200) by given these data? What is the best estimate for the price of the bitcoin? 5000, 6000 or 7000 ?
- #34: to help us out we can see the green points kind of form a line so we can draw a line that best fit data. So we can draw a line that best fit the data. Now in this line we can say our best guess for the price of bitcoin in December is 5616$. You may ask how can i find this line.
- #35: Let look at simple example these three points, we are going to try to find the best fit line that fit with those three points. Since we are teaching computer how to do it , computer cannot eyeball the line so you have to get it to draw a random line and see how bad of this line is. In order to see how bad of this line is. We calculate the error. So we gonna calculate the error look at the lengths of the distances from the line to these three points. We are going just to simply say that error is the sum of those red lines.
- #36: What we gonna to do is next, move line around and see if we can reduce the error. Assuming we move the line to this direction and we calculate the error. And we see the error become smaller and smaller.
- #37: Let say we take that step to closer our solution. If we continue doing this procedure several times, we always decreasing the error and we will finally arrive to a good solution in the form of this line. https://www.desmos.com/calculator/2zunvwck21 https://www.desmos.com/calculator/0dej2z5luc
- #38: In data , we have x and y , x is the date of the bitcoin price we called input or feature , and y is bitcoin price we called an output. every machine learning algorithm , it has its own formula or we can called function. In the formula, it consist of weight and bias, To make our formula or algorithm more accurate, we need to have a good weight value and bias value. To judge it is a good weight and bias or not , we need to calculate the error or we called cost function or square error. To try several time to find a good weight we need to set number of iteration or a loop, how many time we need to repeat and find a good weight.
- #40: Assuming i collect 4 fruits from the market, 2 are apple and 2 are watermelon, i weight and size all the fruits and i note it in the table. And my mom buy me another fruit, but i don’t know what the fruit is, she just gave me the weight and size of the fruit, and i try to guess what is that fruit. To guess it , i try to observe the number that i’ve already weight and size it. I notice that , watermelon seems to be bigger and heavier. So now i am able to guess now, it should be a watermelon. This technique is not working for the big dataset. We cannot observe hundred of data points.
- #41: Let try another technique , you take all of those data points that i was collected. And plot it in the graph. With this graph you will able to guess or predict very quickly , that is a watermelon. Because the new data is close to watermelon. So now how to find an algorithm to solve with this problem?
- #42: Let talk a little bit with some machine learning algorithm, We have a lot of algorithms out there. Here are some of those.
- #43: Now we choose Neural Networks to present today. You may ask, why we choose Neural Networks to solve the problem? Because it works like a human brain. Neural Networks are the biologically inspired simulations performed on the computer to perform certain specific tasks like clustering, classification, pattern recognition etc. Some examples of the pattern are — fingerprint image, a handwritten word, human face or speech signal.
- #45: https://www.desmos.com/calculator/xezilnxrbz https://www.desmos.com/calculator/coknirwubg
- #46: Back to the slide 38 and 39, If you go further for the machine learning, all input data is digital, we cannot weigh and size by image with those digital data, so we need to extract the feature from the image that is called pixel value.
- #47: Each pixel belong to one node
- #52: Reminder of what kevin’s said And explain that this kind of network is not sufficient
- #53: Explain that the hidden layer is used to have more learnable parameter More than one hidden layer is possible We will explain how to configure that
- #57: https://towardsdatascience.com/the-7-steps-of-machine-learning-2877d7e5548e Explain about evaluate and training set
- #59: How long will be your training
- #73: Visa chatbot using neural networks from scratch. Better to support Khmer language. Mor dataset for accuracy Speaker from scratch and with pytorch same result ~70% accuracy. Speech Recognition from scratch. NewsCrawler using semi-supervised learning and naive bayes(not neural network but use probability laws). It’s good to propose the interesting articles to the users.
- #74: From no knowledge began with some really really easy tutorial and tried to code a neural network from scratch. Read many articles and implement them at the same time. One thing we did not do but can be great use kaggle dataset and try to “solve it”
- #75: Python is easy to learn. It has many libraries for machine learning. Linear Algebra Matrix Operation or transformation Vectors and spaces Alternate Coordinate Systems