Machine learning from a software engineer's perspective - Marijn van Zelst - Codemotion Amsterdam 2017
2 likes1,277 views
The document is a presentation by Marijn van Zelst on machine learning from a software engineer's perspective, focusing on practical applications and understanding of machine learning concepts. It discusses neural networks, high-level APIs like TensorFlow, and provides insights into training models and evaluating their performance. The presentation aims to equip software developers with the knowledge to experiment with machine learning without deep mathematical expertise.
![from tensorflow.contrib.learn import infer_real_valued_columns_from_input, DNNClassifier
from tensorflow.examples.tutorials import mnist
mnist_data = mnist.input_data.read_data_sets("MNIST_data/")
train_data, test_data = mnist_data.train.images, mnist_data.test.images
train_labels, test_labels = mnist_data.train.labels.astype('int32'),
mnist_data.test.labels.astype('int32')
estimator = DNNClassifier(
feature_columns=infer_real_valued_columns_from_input(mnist_data.train.images),
hidden_units = [2500, 1000, 1500, 2000, 500],
n_classes = 10)
estimator.fit(x = train_data, y = train_labels, steps = 1000)
print(estimator.evaluate(x = test_data, y = test_labels)['accuracy'])
# 97.5% accuracy in ~15min
Handwritten digit classifier in TensorFlow](https://image.slidesharecdn.com/tr5vi0dwt9qtt25pv36d-signature-564dbe09c3073c373a17b41a3283b35f54ad8b43cddac524b095f4aac841bba3-poli-170529092239/85/Machine-learning-from-a-software-engineer-s-perspective-Marijn-van-Zelst-Codemotion-Amsterdam-2017-41-320.jpg)
Machine learning from a software engineer's perspective - Marijn van Zelst - Codemotion Amsterdam 2017
- 1. Machine learning from a software engineer’s perspective Marijn van Zelst AMSTERDAM 16 - 17 MAY 2017
- 2. About me @marijnvanzelst Marijn van Zelst • Software engineer at Luminis Amsterdam
- 3. Goal: • Practical knowledge about machine learning • What kind of problems it can solve • Do your own experiment
- 4. Why is machine learning interesting for Software Developers? • New opportunities • Builds on existing knowledge • Open-source communities and libraries
- 5. Tensorflow • Developed by Google • Open-source • Has api’s for most major programming languages • High level api
- 6. Machine learning Learn from examples
- 7. Handwritten digit classification MNIST dataset
- 8. What is the difference between a 2 and a 3?
- 9. Neural networks Image recognition Speech recognition Natural language recognition Churn prediction Fraud detection Anomaly detection
- 10. Neural networks Pixel 1 Pixel 2 Pixel 3 Pixel 784 Zero Two One Nine
- 11. Neurons x3 x1 x2 w1 w2 w3 b ((x1 * w1) + (x2 * w2) + (x3 * w3)) * b
- 12. Activation function x3 x1 x2 w1 w2 w3 ((x1 * w1) + (x2 * w2) + (x3 * w3)) * b b Input Output (activation)
- 13. Learning algorithm • While not done • pick a training example (input data, class) • run it through the neural network which produces a prediction • modify the weights in the neural network so that the prediction is closer to the actual class
- 14. Agile approach to machine learning • Theory takes a lot of time to explain and learn • Using high level api’s you can do experiments without knowing all the mathematical details • When you have your first results and want to improve • Learn more • Ask for expert help
- 15. Using a high level api • Which features do I use? • Do I need more data? • How many neurons and layers? • How many training cycles?
- 16. Distinguish 0s and 1s
- 18. Two dimensional feature space white black black 0 1 y x y x y x
- 24. Neural nets intuition • Input: • Features (coordinates in feature space) • Output: • A predicted class at every coordinate in the feature space white black black y x
- 26. Two dimensional feature space white black black 0 1 y x y x y x 0 & 1
- 27. Neural nets intuition • The combination of several features make classes distinguishable • Training on more examples doesn’t always work • Neural networks can handle thousands of features white black black 0 1 y x 0 & 1
- 29. y x y x
- 30. Neural nets intuition Adding a feature can make other features more useful
- 32. 50 steps 600 steps5 steps train - 72.5% train - 88.8% train - 92.2%
- 34. 50 steps 600 steps5 steps train - 72.5% train - 88.8% train - 92.2% test - 76.5% test - 82.6% test - 80.3%
- 36. Neural nets intuition • Keep a test set to verify the accuracy of your training algorithm • Keep an eye on the test set accuracy during training to decide when to stop
- 37. 78% 88% 71%
- 38. 78%88% 71%
- 39. Neural nets intuition • How many neurons? • Depends on the complexity of the data • Start small, gradually ramp up • Check for overfitting
- 40. Insights • High dimensional • Can handle and find correlations between lots of features • They do probabilistic predictions • They require a training set and a test set for validation
- 41. from tensorflow.contrib.learn import infer_real_valued_columns_from_input, DNNClassifier from tensorflow.examples.tutorials import mnist mnist_data = mnist.input_data.read_data_sets("MNIST_data/") train_data, test_data = mnist_data.train.images, mnist_data.test.images train_labels, test_labels = mnist_data.train.labels.astype('int32'), mnist_data.test.labels.astype('int32') estimator = DNNClassifier( feature_columns=infer_real_valued_columns_from_input(mnist_data.train.images), hidden_units = [2500, 1000, 1500, 2000, 500], n_classes = 10) estimator.fit(x = train_data, y = train_labels, steps = 1000) print(estimator.evaluate(x = test_data, y = test_labels)['accuracy']) # 97.5% accuracy in ~15min Handwritten digit classifier in TensorFlow
- 42. Conclusion • You can use machine learning using a high level api • It can make complex decisions on lots of features • Using high level api’s you can do experiments without knowing all the mathematical details • When you have your first results and want to improve • Learn more • Ask for expert help
- 43. Resources on machine learning • Tensorflow high level api quickstart: https://www.tensorflow.org/get_started/tflearn • Other resources: • My blog on the text classification with some more in depth code: https://github.com/luminis-ams/ blog-text-classification • For a theoretical understanding of the backpropagation algorithm: Machine learning course Coursera - Andrew Ng • More practical deep learning course: Udacity deep learning course • Visualization of a neural network (used in the presentation) • http://playground.tensorflow.org
- 44. Dijkstra probably hates me - Linus Torvalds, in kernel/sched.c ?Thank you Any Questions?