![3
1-Supervised
Learning
[By Amina Delali]
Machine LearningMachine Learning
Fit
Observed Data: described
by a set of features
Machine learning
Build
Predict
Mathematical Model with Tunable parameters
New Data](https://image.slidesharecdn.com/aaa-ped-10-190413160855/85/Aaa-ped-10-Supervised-Learning-Introduction-to-Supervised-Learning-3-320.jpg)
![4
1-Supervised
Learning
[By Amina Delali]
Supervised LearningSupervised Learning
Fit
Labeled Observed Data
Supervised learning
Build
Predict the Labels
Mathematical Model with Tunable parameters
Fitting == minimize the cost function
( difference between True and Predicted labels )
Predicting == Apply the obtained mathematical model on the New Data](https://image.slidesharecdn.com/aaa-ped-10-190413160855/85/Aaa-ped-10-Supervised-Learning-Introduction-to-Supervised-Learning-4-320.jpg)
![5
2-Classifcation
[By Amina Delali]
Unsupervised LearningUnsupervised Learning
●
Fit
Not Labeled Observed Data
Unsupervised learning
Build
Predict the Labels
Mathematical Model with Tunable parameters
Predicting = Extract information about New Data](https://image.slidesharecdn.com/aaa-ped-10-190413160855/85/Aaa-ped-10-Supervised-Learning-Introduction-to-Supervised-Learning-5-320.jpg)
![6
2-Classifcation
[By Amina Delali]
ClassifcationClassifcation
●
Fit
Labeled Observed Data: The Labels are categories
Supervised learning: Classification
Build
Predict
Mathematical Model with Tunable parameters
Predict the known categories for the New Data](https://image.slidesharecdn.com/aaa-ped-10-190413160855/85/Aaa-ped-10-Supervised-Learning-Introduction-to-Supervised-Learning-6-320.jpg)
![7
2-Classifcation
[By Amina Delali]
Scikit-LearnScikit-Learn
●
Scikit-learn is a python library used for data mining and data
analysis
●
It is build on :”Numpy”, “Scipy”, and “Matplotlib” libraries.
●
It will be used for both classifcation and regression by importing
the corresponding modules.
●
It will also be used for feature extraction
using :”sklearn.feature_extraction” module
●
The full documentation about scikit-learn, is available at its
homepage: http://scikit-learn.org/stable/index.html](https://image.slidesharecdn.com/aaa-ped-10-190413160855/85/Aaa-ped-10-Supervised-Learning-Introduction-to-Supervised-Learning-7-320.jpg)
![8
3-Regression
[By Amina Delali]
Classifcation ProblemsClassifcation Problems
●
Fit
Observed Data: x-ray images. Labels == yes or no.
Yes == Presence of Tumor, No == Absence of Tumor
Classifying images
Predict if there is a tumor or not
New Data: Not labeled x-ray images.
Classifying text
Predict if the category of the word
New Data: Not labeled words
Fit
Observed Data: words in sentences. Labels == verb, subject, or
adjective](https://image.slidesharecdn.com/aaa-ped-10-190413160855/85/Aaa-ped-10-Supervised-Learning-Introduction-to-Supervised-Learning-8-320.jpg)
![9
3-Regression
[By Amina Delali]
RegressionRegression
●
Fit
Labeled Observed Data: The Labels are continuous
quantities
Supervised learning: Regression
Build
Predict
Mathematical Model with Tunable parameters
Predict the New quantities for the New Data](https://image.slidesharecdn.com/aaa-ped-10-190413160855/85/Aaa-ped-10-Supervised-Learning-Introduction-to-Supervised-Learning-9-320.jpg)
![10
3-Regression
[By Amina Delali]
Regression ProblemsRegression Problems
●
Fit
Observed Data: houses described by a set of characteristics.
Labels == prices of the houses
Photometric redshift
Predict the distance
New galaxies without the distance information
Fit
Observed Data: galaxies described by their brightness at several
wavelengths. Labels == distances of the galaxies
House pricing
Predict houses price
New Data: a set of houses described by the same set of characteristics
(with different values), but they don’t have the price indicated.](https://image.slidesharecdn.com/aaa-ped-10-190413160855/85/Aaa-ped-10-Supervised-Learning-Introduction-to-Supervised-Learning-10-320.jpg)
![11
4-Features
[By Amina Delali]
IntroductionIntroduction
● Fit
Observed Data: the data must be described by a set of
characteristics: numerical values
Machine Learning
Predict if there is a tumor or not
The new Data: must be described by the same set but with
different values
These characteristics, aren’t always in a numerical format. They can be:
Categories Text Images
They have to be transformed in a numerical format](https://image.slidesharecdn.com/aaa-ped-10-190413160855/85/Aaa-ped-10-Supervised-Learning-Introduction-to-Supervised-Learning-11-320.jpg)
![12
4-Features
[By Amina Delali]
Categorical featuresCategorical features
Categories One-hot-encoding
The Feature 3 is
transformed into 3 other
features each one
representing the presence
of a category belonging
to feature 3](https://image.slidesharecdn.com/aaa-ped-10-190413160855/85/Aaa-ped-10-Supervised-Learning-Introduction-to-Supervised-Learning-12-320.jpg)
![13
4-Features
[By Amina Delali]
Text featuresText features
● Text
Word count
TF – IDF:
term frequency–inverse
document frequencyWord count
Each line will represent
a row. And each word
will represent a feature.
Features values in each
row will be the count of
the corresponding word
in the corresponding
line
Appears 0 times in line 0
Appears 2 times in line 1](https://image.slidesharecdn.com/aaa-ped-10-190413160855/85/Aaa-ped-10-Supervised-Learning-Introduction-to-Supervised-Learning-13-320.jpg)
![14
4-Features
[By Amina Delali]
Text featuresText features
● Text
Word count
TF – IDF:
term frequency–inverse
document frequencyTF – IDF
We didn’t normalize the
results
tf −idf (t ,d)=tf (t ,d)∗idf (t)
tf (t ,d)=number of ×the termt occurs∈document t
idf (t)=ln(
1+nd
1+df (d ,t)
)+1
df (d ,t)=number of documents that containthe termt
== 1 *
(ln[(1+2)/1+1]+1 )
== 1.405465
nd=total number of documents0](https://image.slidesharecdn.com/aaa-ped-10-190413160855/85/Aaa-ped-10-Supervised-Learning-Introduction-to-Supervised-Learning-14-320.jpg)
![15
4-Features
[By Amina Delali]
ImagesImages
● Image
Use pixel values as
features values:
number of features ==
number of pixels
Is just a
reshape
of images](https://image.slidesharecdn.com/aaa-ped-10-190413160855/85/Aaa-ped-10-Supervised-Learning-Introduction-to-Supervised-Learning-15-320.jpg)
![16
5-LinearRegression
[By Amina Delali]
Simple Linear Regression (with 1 Feature)Simple Linear Regression (with 1 Feature)
Fit
Labeled Observed Data: described by 1 feature: x
The labels are y values. Fitting == find a and b that minimize the
difference between the real labels: y and the estimated ones.
Simple Linear Regression : 1 feature
Build
Predict
Mathematical Model== a
Line, modelized by:
Y = a x + b
with
a: the slope
b: the intercept
Predict the New y for the New not labeled x values](https://image.slidesharecdn.com/aaa-ped-10-190413160855/85/Aaa-ped-10-Supervised-Learning-Introduction-to-Supervised-Learning-16-320.jpg)
![17
5-LinearRegression
[By Amina Delali]
ExampleExample
●
The model to train
Initialization of a
linear regressor
instance
Train the model
Predict y for new
values: actually this
new x values are
used to draw the
model
Parameters of the model
The line
representing the
model](https://image.slidesharecdn.com/aaa-ped-10-190413160855/85/Aaa-ped-10-Supervised-Learning-Introduction-to-Supervised-Learning-17-320.jpg)
![18
6-PolynomialRegression
[By Amina Delali]
Linear regression basis Polynomial function (1 feature)Linear regression basis Polynomial function (1 feature)
●
Fit
Labeled Observed Data: described by 1 feature: x
From that x, new polynomial features are generated: x2
,x3
, …, xn.
Linear regression basis Polynomial function (1 feature)
Build
Predict
Mathematical Model== a Linear
curve, modelized by:
Y = a1
x +a2
x2
+...+an
xn
+b
with
The parameters: a1
, a2
, …., an
,b
N: the degree of the polynomial model
The new x values must be transformed first into
polynomial feature, before applying the model.](https://image.slidesharecdn.com/aaa-ped-10-190413160855/85/Aaa-ped-10-Supervised-Learning-Introduction-to-Supervised-Learning-18-320.jpg)
![19
6-PolynomialRegression
[By Amina Delali]
ExampleExample
●
The model is the same, we will just generate
new features: polynomial features](https://image.slidesharecdn.com/aaa-ped-10-190413160855/85/Aaa-ped-10-Supervised-Learning-Introduction-to-Supervised-Learning-19-320.jpg)
![20
6-PolynomialRegression
[By Amina Delali]
Example with a pipelineExample with a pipeline
●
The newX values are
generated in a way that
the model can be plotted
in a linear form :
it describes the model
This pipeline will:
1- generate polynomial
features from the data.
2- apply the regression
model to the new data](https://image.slidesharecdn.com/aaa-ped-10-190413160855/85/Aaa-ped-10-Supervised-Learning-Introduction-to-Supervised-Learning-20-320.jpg)
Aaa ped-10-Supervised Learning: Introduction to Supervised Learning
- 1. Supervised Learning: Introduction to Supervised Learning AAA-Python Edition
- 2. Plan ● 1- Supervised Learning ● 2- Classifcation ● 3- Regression ● 4- Features ● 5- Linear Regression ● 6- Polynomial Regression
- 3. 3 1-Supervised Learning [By Amina Delali] Machine LearningMachine Learning Fit Observed Data: described by a set of features Machine learning Build Predict Mathematical Model with Tunable parameters New Data
- 4. 4 1-Supervised Learning [By Amina Delali] Supervised LearningSupervised Learning Fit Labeled Observed Data Supervised learning Build Predict the Labels Mathematical Model with Tunable parameters Fitting == minimize the cost function ( difference between True and Predicted labels ) Predicting == Apply the obtained mathematical model on the New Data
- 5. 5 2-Classifcation [By Amina Delali] Unsupervised LearningUnsupervised Learning ● Fit Not Labeled Observed Data Unsupervised learning Build Predict the Labels Mathematical Model with Tunable parameters Predicting = Extract information about New Data
- 6. 6 2-Classifcation [By Amina Delali] ClassifcationClassifcation ● Fit Labeled Observed Data: The Labels are categories Supervised learning: Classification Build Predict Mathematical Model with Tunable parameters Predict the known categories for the New Data
- 7. 7 2-Classifcation [By Amina Delali] Scikit-LearnScikit-Learn ● Scikit-learn is a python library used for data mining and data analysis ● It is build on :”Numpy”, “Scipy”, and “Matplotlib” libraries. ● It will be used for both classifcation and regression by importing the corresponding modules. ● It will also be used for feature extraction using :”sklearn.feature_extraction” module ● The full documentation about scikit-learn, is available at its homepage: http://scikit-learn.org/stable/index.html
- 8. 8 3-Regression [By Amina Delali] Classifcation ProblemsClassifcation Problems ● Fit Observed Data: x-ray images. Labels == yes or no. Yes == Presence of Tumor, No == Absence of Tumor Classifying images Predict if there is a tumor or not New Data: Not labeled x-ray images. Classifying text Predict if the category of the word New Data: Not labeled words Fit Observed Data: words in sentences. Labels == verb, subject, or adjective
- 9. 9 3-Regression [By Amina Delali] RegressionRegression ● Fit Labeled Observed Data: The Labels are continuous quantities Supervised learning: Regression Build Predict Mathematical Model with Tunable parameters Predict the New quantities for the New Data
- 10. 10 3-Regression [By Amina Delali] Regression ProblemsRegression Problems ● Fit Observed Data: houses described by a set of characteristics. Labels == prices of the houses Photometric redshift Predict the distance New galaxies without the distance information Fit Observed Data: galaxies described by their brightness at several wavelengths. Labels == distances of the galaxies House pricing Predict houses price New Data: a set of houses described by the same set of characteristics (with different values), but they don’t have the price indicated.
- 11. 11 4-Features [By Amina Delali] IntroductionIntroduction ● Fit Observed Data: the data must be described by a set of characteristics: numerical values Machine Learning Predict if there is a tumor or not The new Data: must be described by the same set but with different values These characteristics, aren’t always in a numerical format. They can be: Categories Text Images They have to be transformed in a numerical format
- 12. 12 4-Features [By Amina Delali] Categorical featuresCategorical features Categories One-hot-encoding The Feature 3 is transformed into 3 other features each one representing the presence of a category belonging to feature 3
- 13. 13 4-Features [By Amina Delali] Text featuresText features ● Text Word count TF – IDF: term frequency–inverse document frequencyWord count Each line will represent a row. And each word will represent a feature. Features values in each row will be the count of the corresponding word in the corresponding line Appears 0 times in line 0 Appears 2 times in line 1
- 14. 14 4-Features [By Amina Delali] Text featuresText features ● Text Word count TF – IDF: term frequency–inverse document frequencyTF – IDF We didn’t normalize the results tf −idf (t ,d)=tf (t ,d)∗idf (t) tf (t ,d)=number of ×the termt occurs∈document t idf (t)=ln( 1+nd 1+df (d ,t) )+1 df (d ,t)=number of documents that containthe termt == 1 * (ln[(1+2)/1+1]+1 ) == 1.405465 nd=total number of documents0
- 15. 15 4-Features [By Amina Delali] ImagesImages ● Image Use pixel values as features values: number of features == number of pixels Is just a reshape of images
- 16. 16 5-LinearRegression [By Amina Delali] Simple Linear Regression (with 1 Feature)Simple Linear Regression (with 1 Feature) Fit Labeled Observed Data: described by 1 feature: x The labels are y values. Fitting == find a and b that minimize the difference between the real labels: y and the estimated ones. Simple Linear Regression : 1 feature Build Predict Mathematical Model== a Line, modelized by: Y = a x + b with a: the slope b: the intercept Predict the New y for the New not labeled x values
- 17. 17 5-LinearRegression [By Amina Delali] ExampleExample ● The model to train Initialization of a linear regressor instance Train the model Predict y for new values: actually this new x values are used to draw the model Parameters of the model The line representing the model
- 18. 18 6-PolynomialRegression [By Amina Delali] Linear regression basis Polynomial function (1 feature)Linear regression basis Polynomial function (1 feature) ● Fit Labeled Observed Data: described by 1 feature: x From that x, new polynomial features are generated: x2 ,x3 , …, xn. Linear regression basis Polynomial function (1 feature) Build Predict Mathematical Model== a Linear curve, modelized by: Y = a1 x +a2 x2 +...+an xn +b with The parameters: a1 , a2 , …., an ,b N: the degree of the polynomial model The new x values must be transformed first into polynomial feature, before applying the model.
- 19. 19 6-PolynomialRegression [By Amina Delali] ExampleExample ● The model is the same, we will just generate new features: polynomial features
- 20. 20 6-PolynomialRegression [By Amina Delali] Example with a pipelineExample with a pipeline ● The newX values are generated in a way that the model can be plotted in a linear form : it describes the model This pipeline will: 1- generate polynomial features from the data. 2- apply the regression model to the new data
- 21. References ● Joshi Prateek. Artifcial intelligence with Python. Packt Publishing, 2017. ● Jake VanderPlas. Python data science handbook: essential tools for working with data. O’Reilly Media, Inc, 2017.
- 22. Thank you! FOR ALL YOUR TIME