large scale Machine learning
Download as PPTX, PDF1 like202 views
The document outlines the framework for machine learning algorithms, discussing definitions, and types such as supervised, unsupervised, and reinforcement learning. It explains the concepts of training sets, validation and test sets, and methods like batch and online learning. Additionally, it covers foundational models like perceptrons and support vector machines, detailing their training processes and practical applications in tasks like spam classification.
![Perceptron
8
A perceptron is a linear binary classifier. Its input is a vector
x = [x1,x2,...,xd]
with real-valued components. Associated with the perceptron is a vector of weights
w = [w1,w2,...,wd]
also with real-valued components.
Each perceptron has a threshold θ. The output of the perceptron is +1 if w.x > θ,
and the output is −1 if w.x < θ. The special case where w.x = θ will always be
regarded as “wrong,”](https://image.slidesharecdn.com/machinelearningpresentation-200105095418/85/large-scale-Machine-learning-8-320.jpg)
![Parallel Implementation of Perceptron's
12
The training of a perceptron is an inherently sequential process. If the
number of dimensions of the vectors involved is huge, then we might
obtain some parallelism by computing dot products in parallel
The Map Function: Each Map task is given a chunk of training examples, and each Map
task knows the current weight vector w. The Map task computes w.x for each feature vector
x = [x1,x2,...,xk]
in its chunk and compares that dot product with the label y, which is +1 or −1, associated with
x.
The Reduce Function: For each key i, the Reduce task that handles key i adds all the
associated increments and then adds that sum to the ith component of w.](https://image.slidesharecdn.com/machinelearningpresentation-200105095418/85/large-scale-Machine-learning-12-320.jpg)
large scale Machine learning
- 2. machine-learning framework In this section we introduce the framework for machine-learning algorithms and give the basic definitions.
- 3. Machine learning Tom Mitchel : “A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.” ▪ Supervise learning >> 1)Regression 2)Classification ▪ Unsupervised learning >> 1)Clustering 2)Assuciation Rules ▪ Reinforcement learning 3
- 4. Training set The data to which a machine-learning (often abbreviated ML) algorithm is applied is called a training set. A training set consists of a set of pairs (x,y), called training examples, where • x is a vector of values, often called a feature vector, • y is the class label, or simply output, the classification value for x. A B C D 1 122 78 23 1 2 12 64 65 1 3 65 257 82 2 4
- 5. Validation and Test set One general issue regarding the handling of data is that there is a good reason to withhold some of the available data from the training set. The remaining data is called the test set. In some cases, we withhold two sets of training examples, a validation set, as well as the test set. 5
- 6. Batch VS On-line Learning That is, the entire training set is available at the beginning of the process, and it is all used in whatever way the algorithm requires to produce a model once and for all. The alternative is on-line learning, where the training set arrives in a stream and, like any stream, cannot be revisited after it is processed. 1. Deal with very large training sets, because it does not access more than one training example at a time. 2. Adapt to changes in the population of training examples as time goes on. For instance, Google trains its spam-email classifier this way, adapting the classifier for spam as new kinds of spam email are sent by spammers and indicated to be spam by the recipients. 6
- 7. Feature Selection Sometimes, the hardest part of designing a good model or classifier is figuring out what features to use as input to the learning algorithm. For example, spam is often generated by particular hosts, either those belonging to the spammers, or hosts that have been coopted into a “botnet” for the purpose of generating spam. Thus, including the originating host or originating email address into the feature vector describing an email might enable us to design a better classifier and lower the error rate. 7
- 8. Perceptron 8 A perceptron is a linear binary classifier. Its input is a vector x = [x1,x2,...,xd] with real-valued components. Associated with the perceptron is a vector of weights w = [w1,w2,...,wd] also with real-valued components. Each perceptron has a threshold θ. The output of the perceptron is +1 if w.x > θ, and the output is −1 if w.x < θ. The special case where w.x = θ will always be regarded as “wrong,”
- 9. Training a Perceptron 9 The following method will converge to some hyperplane that separates the positive and negative examples, provided one exists. 1. Initialize the weight vector w to all 0’s. 2. Pick a learning-rate parameter η, which is a small, positive real number. The choice of η affects the convergence of the perceptron. If η is too small, then convergence is slow; if it is too big, then the decision boundary will “dance around” and again will converge slowly, if at all. 3. Consider each training example t = (x,y) in turn. (a) Let y′ = w.x. (b) If y′ and y have the same sign, then do nothing; t is properly classified. (c) However, if y′ and y have different signs, or y′ = 0, replace w by w + ηyx. That is, adjust w slightly in the direction of x.
- 10. Example 10 and Viagra the of nigeria y 1 1 1 1 1 1 +1 2 0 1 0 0 1 -1 3 0 0 1 0 0 +1 4 1 0 0 1 0 -1 5 1 0 1 1 1 +1 η = 1/2,
- 11. Convergence of Perceptron's 11 As we mentioned at the beginning of this section, if the data points are linearly separable, then the perceptron algorithm will converge to a separator. However, if the data is not linearly separable, then the algorithm will eventually repeat a weight vector and loop infinitely.
- 12. Parallel Implementation of Perceptron's 12 The training of a perceptron is an inherently sequential process. If the number of dimensions of the vectors involved is huge, then we might obtain some parallelism by computing dot products in parallel The Map Function: Each Map task is given a chunk of training examples, and each Map task knows the current weight vector w. The Map task computes w.x for each feature vector x = [x1,x2,...,xk] in its chunk and compares that dot product with the label y, which is +1 or −1, associated with x. The Reduce Function: For each key i, the Reduce task that handles key i adds all the associated increments and then adds that sum to the ith component of w.
- 13. SVM Support Vector Machine 13 As we mentioned at the beginning of this section, if the data points are linearly separable, then the perceptron algorithm will converge to a separator. However, if the data is not linearly separable, then the algorithm will eventually repeat a weight vector and loop infinitely.
- 14. SVM Support Vector Machine 14
- 15. How to Calculate this Distance ? 15
- 16. Test 16 1. What's On-line learning has the advantages? 2. Explain common tests for Perceptron termination. 3. Let us consider training a perceptron to recognize spam email. The training set consists of pairs (x,y) where x is a vector of 0’s and 1’s, with each component xi corresponding to the presence (xi = 1) or absence (xi = 0) of a particular word in the email. The value of y is +1 if the email is known to be spam and −1 if it is known not to be spam. While the number of words found in the training set of emails is very large, we shall use a simplified example where there are only five words: “and,” “Viagra,” “the,” “of,” and “Nigeria.” Figure gives the training set of six vectors and their corresponding classes. and Viagra the of Nigeria y 1 1 0 0 1 0 +1 2 1 1 1 0 1 -1 3 0 0 1 1 0 +1 4 1 1 0 0 0 -1