Machine learning by example
3 likes1,536 views
The document discusses machine learning, covering types such as supervised, unsupervised, semi-supervised, and reinforcement learning, alongside their applications including voice recognition and self-driving cars. It outlines the machine learning process from data gathering to model evaluation, as well as improvement techniques like k-fold cross-validation and parameter tuning. Additionally, it provides references and resources for further exploration in the field of machine learning.
Machine learning by example
- 1. Machine learning by example Michał Matłoka @mmatloka
- 2. –Alan Turing (1950) “Can machines think?”
- 4. • Make machines “think” like humans • Learn from data and make predictions What is ML?
- 5. • Supervised learning ◦ Classification ◦ Regression • Unsupervised learning ◦ Clustering ◦ Dimensionality Reduction • Semi-supervised learning • Reinforcement learning ◦ E.g. AlphaGo Learning types
- 10. SEMI SUPERVISED
- 12. • Voice recognition • Fraud analysis • Face detection • Ads click-through rate prediction • Spam detection • Shop recommendations • Photos description • Self-driving cars • Healthcare • ... Use cases
- 13. 1. Data gathering 2. Data cleaning & feature engineering 3. Dataset -> training & test set 4. Learning -> Model 5. Evaluation -> Accuracy 6. New observation -> Prediction Learning process
- 14. EXAMPLE
- 15. Classify conference talk abstracts into tracks
- 16. • RDD (Resilient Distributed Dataset) - map, filter, count etc - DataFrame • Spark SQL • MLib • GraphX • Spark Streaming • API: Scala, Java, Python, R*
- 18. • Bigger data set • Smarter tokenizer • Stemming & lemmatization • IDF - Inverse Document Frequency • K-fold Cross-Validation • Parameters tuning (grid- search) What can be improved?
- 19. “I will present Big Data topics” “I”, “will”, “present”, “Big”, “Data”, “topics” What can be improved? (Tokenizer)
- 20. “I will present Big Data topics” “I will”, “will present”, “present Big”, “Big Data”, “Data topics” What can be improved? (n-grams)
- 22. “communities”, community” “commun” What can be improved? (Porter Stemmer)
- 23. “communities”, community” “community” What can be improved? (Lemmatization)
- 24. TF - Term Frequency (HashingTF!) - number of times term occurs in given document IDF - Inverse Document Frequency - occurs many times in documents set What can be improved? (TF & IDF)
- 25. What can be improved? (K-fold Cross-Validation)
- 27. Articles & references • https://www.csee.umbc.edu/courses/471/papers/turing.pdf • http://spark.apache.org/ • https://databricks.com/try-databricks • https://research.googleblog.com/2016/09/introducing- open-images-dataset.html • https://arstechnica.co.uk/information-technology/2017/05/ deepmind-alphago-go-ke-jie-china/ • https://www.kaggle.com • https://github.com/dylanmei/docker-zeppelin • https://github.com/databricks/spark-corenlp • https://www.coursera.org/learn/scala-spark-big-data