Machine learning interviews day5
- 1. Machine Learning Interviews – Day 5 Arpit Agarwal
- 2. Practical Considerations for Selecting Learning Algorithms • Large Number of Samples in Dataset? – Don’t worry about variance, but worry about computational time – Can use Random Forests, kernel-SVM with SGD, SMO, Logistic Regression with SGD, Perceptron – Don’t use k-NN – The problem with SVM is that too many parameters, so computational time is large
- 3. Practical Considerations for Selecting Learning Algorithms • Very Small Dataset – Worry about variance of your classifier – Naïve Bayes works very well with less amout of data – Can use SVM with linear kernel with regularization, Logistic Regression with regularization – Don’t use decision trees
- 4. Practical Considerations for Selecting Learning Algorithms • Very low Dimensionality? – Worry about high bias – Need to use powerful kernel methods like Kernel SVM, Kernel LR subject to that we have large enough data – Useful to collect more features • Very Large Dimensionality? – Don’t worry about high bias, worry about computational time – SVM with linear kernel, Random forests can be used – Can’t use Decision Trees
- 5. Practical Considerations for Selecting Learning Algorithms • Want probability estimates? – Logistic Regression is good, (SVM with Platt scaling) – Might not want to use Random Forests, kNN unless you have large amount of data • Working with Text Data? – Naïve Bayes works very well • Want to constantly update your model with new data? – Difficult to use Random Forests – Can use Logistic Regression, Perceptron, kNN
- 6. Practical Considerations for Selecting Learning Algorithms • Categorical Attributes? – Can work with Decision Trees and Random Forests • Don’t want any parameter tuning? – Use Naïve Bayes, Random Forests – Don’t use SVM • Can have large training time but want less prediction time? – Use SVM, Neural Networks – Don’t Use kNN,
- 7. Practical Considerations for Selecting Learning Algorithms • The underlying data is to complex? – SVM with powerful kernel, Neural Networks • Want to parallelize your algorithm? – Random Forests bit easy to parallelize, SVM can also be parallelized
- 8. Linear Regression • On Board
- 9. Perceptron • On Board
- 10. SVD • Any real m x n matrix A can be decomposed uniquely: • U is m x n and column orthonormal (UTU=I) • D is n x n and diagonal – σi are called singular values of A – It is assumed that σ1 ≥ σ2 ≥ … ≥ σn ≥ 0 • V is n x n and orthonormal (VVT=VTV=I)
- 11. SVD • If m=n, then: • U is n x n and orthonormal (UTU=UUT=I) • D is n x n and diagonal • V is n x n and orthonormal (VVT=VTV=I)
- 12. SVD • The columns of U are eigenvectors of AAT • The columns of V are eigenvectors of ATA for square matrices: A=PΛP-1 • If λi is an eigenvalue of ATA (or AAT), then λi =σi 2
- 13. U = (u1 u2 . . . un) V = (v1 v2 . . . vn) D
- 14. Relation with PCA • On board
- 15. Disclaimer: This crash course was just to aid your preparation not to replace your preparation.
- 16. All the very best for you placements!