Machine Learning
0 likes260 views
This document contains notes from a course on machine learning taught by Carlos J. Costa. It discusses different approaches to machine learning, including analogizers, Bayesians, connectionists, evolutionaries, and symbolists. It also covers topics like supervised learning, unsupervised learning, reinforcement learning, regression, classification algorithms, logistic regression, random forests, and cluster analysis.
Machine Learning
- 1. 2018/19 - 1 Carlos J. Costa (ISEG) MACHINE LEARNING (2019/2020) Carlos J. Costa
- 2. 2019/20 - 2 Carlos J. Costa (ISEG) Machine Learning • It is as a subset of artificial intelligence. • It is the scientific study of algorithms that computer systems use to perform a specific task without using explicit instructions • Study and construction of algorithms that can learn from and make predictions on data
- 3. 2019/20 - 3 Carlos J. Costa (ISEG) Machine Learning
- 4. 2019/20 - 4 Carlos J. Costa (ISEG) Analogizers Analogizers are interested in mapping to new situations. They are influenced by psychology. Analogizers use k-nearest neighbor, and support vector machines
- 5. 2019/20 - 5 Carlos J. Costa (ISEG) Bayesians Bayesians use probabilistic inference. They are influenced by statistics. Bayesians use Hidden Markov Models, graphical models, and causal inference.
- 6. 2019/20 - 6 Carlos J. Costa (ISEG) Connectionists Connectionists use neural networks. They are influenced by neuroscience. Connectionists rely on deep learning technologies, including CNN (Convolutional Neural Networks), RNN(Recurrent Neural Networks), and deep reinforcement learning
- 7. 2019/20 - 7 Carlos J. Costa (ISEG) Evolutionaries Evolutionaries are interested in evolving structure. They are influenced by biology. Evolutionaries use genetic algorithms, evolutionary programming, and evolutionary game theory.
- 8. 2019/20 - 8 Carlos J. Costa (ISEG) Symbolists Symbolists use formal systems. They are influenced by computer science, linguistics, and analytic philosophy. Symbolists use decision trees, production rule systems, and inductive logic programming
- 9. 2019/20 - 9 Carlos J. Costa (ISEG) Machine Learning
- 10. 2019/20 - 10 Carlos J. Costa (ISEG) Machine Learning
- 11. 2019/20 - 11 Carlos J. Costa (ISEG) Machine Learning Supervised learning It is the machine learning task of learning a function that maps an input to an output based on example input-output pairs (Hinton & Sejnowski,1999) Classification Regression
- 12. 2019/20 - 12 Carlos J. Costa (ISEG) Machine Learning Unsupervised learning The goal of unsupervised learning is to extract an efficient internal representation of the statistical structure implicit in the inputs. (Hinton & Sejnowski,1999) Clustering Dimensional Reduction
- 13. 2019/20 - 13 Carlos J. Costa (ISEG) Machine Learning Reinforcement Learning (RL) There are 3 main components: Agent, Environment Actions (performed by the agent) The purpose of RL is to train an intelligent agent that is capable of navigating its environment and performing actions that arrives at the end goal. Actions changes the state of the environment and the agent receives rewards or punishments The challenge of the agent is to maximize the amount of rewards at the end of a specific period
- 14. 2019/20 - 14 Carlos J. Costa (ISEG) Machine Learning Train- Validate-Test Step 1: Making the model examine data. Step 2: Making the model learn from its mistakes. Step 3: Making a conclusion on how well the model performs
- 15. 2019/20 - 15 Carlos J. Costa (ISEG) Machine Learning Data Processing and Machine Learning Libraries: Numpy, Pandas, statsmodels, sklearn, networkx Tools: IDE – Jupiter IDE: Integrated Development Environment
- 16. 2018/19 - 16 Carlos J. Costa (ISEG) REGRESSIONS Carlos J. Costa
- 17. 2019/20 - 17 Carlos J. Costa (ISEG) Regression Is a set of statistical processes for estimating the relationships among variables. Dependent variable,outcome variable, target Independent variables, predictor, covariates, or features
- 18. 2019/20 - 18 Carlos J. Costa (ISEG) Regression simple regression/multivariate regression
- 19. 2019/20 - 19 Carlos J. Costa (ISEG) Regression .Linear/non linear
- 20. 2019/20 - 20 Carlos J. Costa (ISEG) Regression
- 21. 2018/19 - 21 Carlos J. Costa (ISEG) CLASSIFICATION Carlos J. Costa
- 22. 2019/20 - 22 Carlos J. Costa (ISEG) Classification Supervised learning approach Categorizing some unknown items into discrete set of categories or “classes” The target attribute is a categorical variable To solve a classification problem identify the target or class, which is the variable to predict. the target balancing is mandatory choose the best training strategy to train classification models.
- 23. 2019/20 - 23 Carlos J. Costa (ISEG) Classification Churn (not churn rate) depends from several characteristics of the client, product and communication.
- 24. 2019/20 - 24 Carlos J. Costa (ISEG) Classification What is the best drug according to specific characteristics of the patient
- 25. 2019/20 - 25 Carlos J. Costa (ISEG) Classification Classification algorithms in machine learning: Decision Trees Naive Bayes Linear Discriminate Analysis K -Near Neighbor (KNN) Logistic Regression Neural Networks Support Vector Machines (SVM)
- 26. 2019/20 - 26 Carlos J. Costa (ISEG) Logistics Regression • A regression that having binary dependent variable • in its basic form, uses a logistic function to model a binary dependent variable
- 27. 2019/20 - 27 Carlos J. Costa (ISEG) Random Forest • are an ensemble learning method for classification, regression and other tasks • operates by constructing a multitude of decision trees at training time • outputting the class that is the mode of the classes (classification) or mean prediction (regression) of the individual trees.
- 28. 2018/19 - 28 Carlos J. Costa (ISEG) CLUSTERS ANALYSIS Carlos J. Costa
- 29. 2019/20 - 29 Carlos J. Costa (ISEG) Cluster Analysis • is a multivariate method • aims to classify a sample of subjects (or objects) into several different groups such that similar subjects are placed in the same group • based on a set of measured variables
- 30. 2019/20 - 30 Carlos J. Costa (ISEG) K-means Clustering 1.Select K (i.e. 2) random points as cluster centres called centroids 3. Determine the new cluster centre by computing the average of the assigned points 2. Assign each data point to the closest cluster by calculating its distance with respect to each centroid 4. Repeat steps 2 and 3 until none of the cluster assignments change
- 31. 2019/20 - 31 Carlos J. Costa (ISEG) WCSS Within-Cluster-Sum-of-Squares (WCSS)- Implicit objective function in k- Means measures sum of distances of observations from their cluster centroids. Yi is centroid for observation Xi. Given that k-Means has no in-built preference for right number of clusters, following are some of the common ways k can be selected: Domain Knowledge Rule of Thumb Elbow-Method using WCSS Cluster Quality using Silhouette Coefficient
- 32. 2019/20 - 32 Carlos J. Costa (ISEG) References Albon, Ch. (2018) Machine Learning with Python Cookbook. O’Reilly Domingos, P. (2015) The Master Algorithm, Penguin Books Hinton, J.; Sejnowski, T.(1999). Unsupervised Learning: Foundations of Neural Computation. MIT Press Morgan; P. (2019) Data Science from Scratch with Python, AI Science Murphy, K. P. (2012). Machine Learning: A Probabilistic Perspective (1 edition). Cambridge, MA: The MIT Press. Otte, E.; Rousseau, R. (2002). "Social network analysis: a powerful strategy, also for the information sciences". Journal of Information Science. 28 (6): 441–453. doi:10.1177/016555150202800601. Stuart J. R., Norvig, P. (2010) Artificial Intelligence: A Modern Approach, Third Edition, Prentice Hall ISBN 9780136042594.