EKON22 Introduction to Machinelearning
2 likes124 views
This document outlines an introduction to machine learning, covering various topics such as data reduction, regression, clustering, and classification with practical Python scripts. It provides examples of algorithms, discusses the importance of PCA for dimensionality reduction, and details a machine learning process from data collection to classification. Additionally, it highlights the use of libraries like scikit-learn and TensorFlow for implementing machine learning tasks.
![14
14
What's behind test ? (backend pattern, crossentropy)
60000/60000 [==============================] - 426s 7ms/step - loss: 0.4982 - acc: 0.8510 -
val_loss: 0.0788 - val_acc: 0.9749
Using TensorFlow backend.
INFO - MNIST-Convnet4 - Result: 0.9749
INFO - MNIST-Convnet4 - Completed after 0:07:27
Test loss: 0.0788029053777
Test accuracy: 0.9749
59392/60000 [============================>.] - ETA: 5s - loss: 0.0571 - acc: 0.9829
59520/60000 [============================>.] - ETA: 3s - loss: 0.0572 - acc: 0.9829
59648/60000 [============================>.] - ETA: 2s - loss: 0.0572 - acc: 0.9829
59776/60000 [============================>.] - ETA: 1s - loss: 0.0572 - acc: 0.9829
59904/60000 [============================>.] - ETA: 0s - loss: 0.0573 - acc: 0.9829
60000/60000 [==============================] - 513s 9ms/step - loss: 0.0573 - acc:
0.9829 - val_loss: 0.0312 - val_acc: 0.9891
INFO - MNIST-Convnet4 - Result: 0.9891
INFO - MNIST-Convnet4 - Completed after 0:33:28
Test loss: 0.0311644290059
Test accuracy: 0.9891](https://image.slidesharecdn.com/ekon22introductionmachinelearning-181009160223/85/EKON22-Introduction-to-Machinelearning-14-320.jpg)
![15
15
What's behind code ? (Classification Summary)
from sklearn import datasets, metrics
from sklearn.model_selection import train_test_split
>>> iris = datasets.load_iris()
>>> X = iris.data[0:, 1:3]
>>> y = iris.target
>>> X_train,X_test, y_train, y_test =
train_test_split(X, y,test_size=0.3, random_state=20)
>>> from sklearn import svm
>>> classifier = svm.SVC(kernel='linear', C=1.0)
>>> classifier.fit(X_train, y_train)
>>> y_pred = classifier.predict(X_test)
>>> from sklearn import metrics
>>> print ("Test - Accuracy SVC:", metrics.accuracy_score(y_test, y_pred))
Test - Accuracy SVC: 0.9555555555555556
https://www.programcreek.com/python/example/103267/keras.datasets.mnist.load_data](https://image.slidesharecdn.com/ekon22introductionmachinelearning-181009160223/85/EKON22-Introduction-to-Machinelearning-15-320.jpg)
![16
16
What's behind code II ? (Check for duplicates)
print(y_test)
array([0, 1, 1, 2, 1, 1, 2, 0, 2, 0, 2, 1, 2, 0, 0, 2, 0, 1, 2, 1, 1, 2,
2, 0, 1, 1, 1, 0, 2, 2, 1, 1, 0, 0, 0, 2, 1, 0, 1, 2, 1, 2, 0, 1, 1])
>>> unique, counts = np.unique(y_test, return_counts=True)
>>> dict(zip(unique, counts))
{0: 13, 1: 18, 2: 14}
>>> Xyt = np.column_stack((X_test, y_test))
>>> csort = Xyt[Xyt[:,2].argsort()]
>>> dfiris = pd.DataFrame(csort)
>>> dfiris[0:13].groupby([0,1]).size()
3.0 1.1 1
1.4 2
3.1 1.5 2
3.2 1.2 1 - 3.4 1.4 1 1.6 1 1.7 1 -3.5 1.4 1 - 3.7 1.5 1
1.4 1
3.8 1.6 1
>>> sum(dfiris[0:13].groupby([0,1]).size()>1) 2](https://image.slidesharecdn.com/ekon22introductionmachinelearning-181009160223/85/EKON22-Introduction-to-Machinelearning-16-320.jpg)
![20
THE TEST OVERVIEW
20
Double Trouble with ML →
Stackexchange,
Stackoverflow
No. of URLs removed 76,732,515
+ No. of robots.txt
requests
3,675,634
- No. of excludedURLs 3,050,768
= No. of HTTP requests 77,357,381
HTTP requests not
respond
1,763850
Status Description
QUEUED
The run was just
queued
and not run yet
RUNNING
Currently running (but see below)
COMPLETED
Completed successfully
FAILED
The run failed due to an exception
INTERRUPTED
The run was cancelled with a
KeyboardInterrupt
TIMED_OUT
The run was aborted using a TimeoutInterrupt
[custom]
A custom py:class:
~sacred.utils.SacredInterrupt
occurred
File "C:UsersmaxAppDataLocalProgramsPythonPython36libsite-
packagessklearnmetricsclusterunsupervised.py", line 254, in
calinski_harabaz_score
intra_disp += np.sum((cluster_k - mean_k) ** 2) MemoryError
File
"C:UsersmaxAppDataLocalProgramsPythonPyt
hon36libsite-
packagessklearnmetricsclusterunsupervised.py",
line 254, in calinski_harabaz_score
intra_disp += np.sum((cluster_k - mean_k) ** 2)
MemoryError
https://stats.stackexchange.com/](https://image.slidesharecdn.com/ekon22introductionmachinelearning-181009160223/85/EKON22-Introduction-to-Machinelearning-20-320.jpg)
EKON22 Introduction to Machinelearning
- 1. Introduction to Machine Learning Max Kleiner 10.2018
- 2. 2 ML Agenda 2 http://www.softwareschule.ch/examples/machinelearning.jpg 4 Cases with 5 Scripts Data Reduction - EKON22_PCA_1.py Regression - EKON22_REG_2.py Clustering - EKON22_CLU_3.py Classification - EKON22_CLA_4.py Decision Tree, Random Forest - EKON32_DET_5.py Cluster & Classify with different inputs, algos ,config Define label, features or topic ratings, hyper-parameters, tests – assumed/implicit labels, predict versus target, random state Conclusions/ ML Process Summary/
- 3. 3 3 https://www.springboard.com/blog/data-mining-python-tutorial/ PCA (Principal Component Analysis) C:maXboxEKON22EKON22_scriptsEKON22_PCA_1.py http://playground.tensorflow.org_maXbox2 Visualizing 2 or 3 dimensional data is not that challenging. However, even Iris dataset used 4 dim. Use PCA to reduce 4 dim data into 2 or 3 dim so that you can plot & understand data better. Use StandardScaler for features onto unit scale (mean = 0, variance = 1) which is a requirement for optimal performance.
- 4. 4 4 https://www.springboard.com/blog/data-mining-python-tutorial/ Topic IRIS Classify Task https://sebastianraschka.com/images/blog/2015/principal_component_analysis_files/iris.png
- 5. CASSANDRASystem 5 Regression and Correlation @C:maXboxEKON22EKON22_scriptsEKON22_REG_2.py 2. C:maXboxmX46210ntwdblib.dllUnsharpDetector-masterUnsharpDetector-masterinference_gui.py
- 6. 6 From Correlation to 4 Dim Cluster 6https://www.soovle.com/ https://answerthepublic.com/reports/ Finding the question is often more important than finding the answer - John Tukey
- 7. CASSANDRASystem 7 Clustering from module import class 0 1 2 3 4 5 6 37 81 9 @C:maXboxEKON22EKON22_scriptsEKON22_CLU_3.py
- 8. 8 GEO Cluster Story 8 An agent or probe that collects threat data from the security sensor and correlation middleware. A console and associated database for managing the solution and its alerts. https://www.esecurityplanet.com/views/article.php/1501001/Security-Threat-Correlation-The-Next-Battlefield.htm
- 9. CASSANDRASystem 9 IRIS Classification Concept 0 1 2 3 4 5 6 4 3 4 from sklearn import datasets, tree iris = datasets.load_iris() clf = tree.DecisionTreeClassifier() clf = clf.fit(iris.data, iris.target) y_pred = clf.predict(iris.data) print('Train accuracy_score: ') metrics.accuracy_score(iris.target,y_pred) Demo in VSCode /maXbox4 C:maXboxsoftwareschuleMT-HS12- 05mentor_xmlcasra2017crawlerplot_iris_dataset_mx.py @C:maXboxEKON22EKON22_scriptsEKON22_CLA_4.py
- 12. CASSANDRASystem 12 MongoDB My Cluster sacred.runs & completed
- 13. 13 13 Task II
- 14. 14 14 What's behind test ? (backend pattern, crossentropy) 60000/60000 [==============================] - 426s 7ms/step - loss: 0.4982 - acc: 0.8510 - val_loss: 0.0788 - val_acc: 0.9749 Using TensorFlow backend. INFO - MNIST-Convnet4 - Result: 0.9749 INFO - MNIST-Convnet4 - Completed after 0:07:27 Test loss: 0.0788029053777 Test accuracy: 0.9749 59392/60000 [============================>.] - ETA: 5s - loss: 0.0571 - acc: 0.9829 59520/60000 [============================>.] - ETA: 3s - loss: 0.0572 - acc: 0.9829 59648/60000 [============================>.] - ETA: 2s - loss: 0.0572 - acc: 0.9829 59776/60000 [============================>.] - ETA: 1s - loss: 0.0572 - acc: 0.9829 59904/60000 [============================>.] - ETA: 0s - loss: 0.0573 - acc: 0.9829 60000/60000 [==============================] - 513s 9ms/step - loss: 0.0573 - acc: 0.9829 - val_loss: 0.0312 - val_acc: 0.9891 INFO - MNIST-Convnet4 - Result: 0.9891 INFO - MNIST-Convnet4 - Completed after 0:33:28 Test loss: 0.0311644290059 Test accuracy: 0.9891
- 15. 15 15 What's behind code ? (Classification Summary) from sklearn import datasets, metrics from sklearn.model_selection import train_test_split >>> iris = datasets.load_iris() >>> X = iris.data[0:, 1:3] >>> y = iris.target >>> X_train,X_test, y_train, y_test = train_test_split(X, y,test_size=0.3, random_state=20) >>> from sklearn import svm >>> classifier = svm.SVC(kernel='linear', C=1.0) >>> classifier.fit(X_train, y_train) >>> y_pred = classifier.predict(X_test) >>> from sklearn import metrics >>> print ("Test - Accuracy SVC:", metrics.accuracy_score(y_test, y_pred)) Test - Accuracy SVC: 0.9555555555555556 https://www.programcreek.com/python/example/103267/keras.datasets.mnist.load_data
- 16. 16 16 What's behind code II ? (Check for duplicates) print(y_test) array([0, 1, 1, 2, 1, 1, 2, 0, 2, 0, 2, 1, 2, 0, 0, 2, 0, 1, 2, 1, 1, 2, 2, 0, 1, 1, 1, 0, 2, 2, 1, 1, 0, 0, 0, 2, 1, 0, 1, 2, 1, 2, 0, 1, 1]) >>> unique, counts = np.unique(y_test, return_counts=True) >>> dict(zip(unique, counts)) {0: 13, 1: 18, 2: 14} >>> Xyt = np.column_stack((X_test, y_test)) >>> csort = Xyt[Xyt[:,2].argsort()] >>> dfiris = pd.DataFrame(csort) >>> dfiris[0:13].groupby([0,1]).size() 3.0 1.1 1 1.4 2 3.1 1.5 2 3.2 1.2 1 - 3.4 1.4 1 1.6 1 1.7 1 -3.5 1.4 1 - 3.7 1.5 1 1.4 1 3.8 1.6 1 >>> sum(dfiris[0:13].groupby([0,1]).size()>1) 2
- 17. 17 17https://github.com/pinae/Sacred-MNIST/blob/master/train_convnet.py What's behind Python: PIP3 Install pip3 install sacred Collecting sacred Downloading https://files.pythonhosted.org/packages/2d/86/7be3af a4d4c1c0c76a5de03e5ff779797ab2654e377685255c11c13c0e a5/sacred-0.7.3-py2.py3-none-any.whl (82kB) Collecting pymongo Downloading https://files.pythonhosted.org/packages/46/39/b9bb7fed3e3a0ea621a1 512a938c105cd996320d7d9894d8239ca9093340/pymongo-3.6.1-cp36-cp36m- win_amd64.whl (291kB) 100% |¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦| 296kB 728kB/s Installing collected packages: pymongo Successfully installed pymongo-3.6.1
- 18. 18 Machine Learning Process Chain • Collab (Set a control thesis, understand the problem, get resources Python etc.) • Collect (Scrapy data, store, data mining, filter data, inconsistent, incomplete) • Consolidate or Clean data (normalization and aggregation, PCA data reduction, Regression, Filters, slice out irrel. Or ambigous data or char unicode map prob.) • Cluster (kmeans for category, collocates for N-keywords) algorithm – unsupervised) • Classify (SVM, Sequential, Bayes – supervised) • Conclude and Control (Predict or report context thesis and drive data to decision) http://www.softwareschule.ch/examples/machinelearning.jpg https://maxbox4.wordpress.com/code/
- 19. CASSANDRASystem 19 https://www.kaggle.com/ similarity of doc a to doc b = '' )',( ),( )',( ),( ),( word ' 2 ' 2 BA jbv jbv jav jav basim i jj
- 20. 20 THE TEST OVERVIEW 20 Double Trouble with ML → Stackexchange, Stackoverflow No. of URLs removed 76,732,515 + No. of robots.txt requests 3,675,634 - No. of excludedURLs 3,050,768 = No. of HTTP requests 77,357,381 HTTP requests not respond 1,763850 Status Description QUEUED The run was just queued and not run yet RUNNING Currently running (but see below) COMPLETED Completed successfully FAILED The run failed due to an exception INTERRUPTED The run was cancelled with a KeyboardInterrupt TIMED_OUT The run was aborted using a TimeoutInterrupt [custom] A custom py:class: ~sacred.utils.SacredInterrupt occurred File "C:UsersmaxAppDataLocalProgramsPythonPython36libsite- packagessklearnmetricsclusterunsupervised.py", line 254, in calinski_harabaz_score intra_disp += np.sum((cluster_k - mean_k) ** 2) MemoryError File "C:UsersmaxAppDataLocalProgramsPythonPyt hon36libsite- packagessklearnmetricsclusterunsupervised.py", line 254, in calinski_harabaz_score intra_disp += np.sum((cluster_k - mean_k) ** 2) MemoryError https://stats.stackexchange.com/
- 21. 21 QUESTIONS ? 17:45 - 18:30 Machine Learning II Art. Neural Network Best Book in my opinion: Mastering Machine Learning with Python in Six Steps A Practical Implementation Predictive Data Analytics 21