Data Science Competition
6 likes1,428 views
The document outlines the structure and purpose of data science competitions on Kaggle, detailing elements such as training data, submission requirements, and public/private leaderboard scoring. It emphasizes the benefits of participating, including fun, experience, learning, and networking opportunities. Additionally, it discusses best practices in data science competitions like feature engineering, algorithms, cross-validation, and ensemble methods.
Data Science Competition
- 1. DATA SCIENCE COMPETITION Conversion Logic @ Whisper 2. 1. 2017
- 3. ATTRIBUTION
- 4. DATA SCIENCE COMPETITION Since 1997 2006 - 2009 Since 2010 4
- 5. COMPETITION STRUCTURE Training Data Test Data Feature Label Provided Submission Public LB Score 5
- 6. COMPETITION STRUCTURE Training Data Test Data Feature Label Provided Submission Public LB Score Private LB Score 5
- 7. KAGGLE • 237 competitions since 2010 • 500K+ users • 50K+ competitors • $3MM+ prize paid out 6
- 8. KAGGLE 7
- 9. KAGGLE 8
- 11. WHY COMPETITION • For fun • For experience • For learning • For networking 10
- 12. FUN 11
- 13. FUN • Competing with others 11
- 14. FUN • Competing with others • Incremental improvement 11
- 15. EXPERIENCE 12
- 16. LEARNING 13
- 17. LEARNING 13
- 18. LEARNING 13
- 19. LEARNING 13
- 20. LEARNING 13
- 21. LEARNING 13
- 22. LEARNING 14
- 23. NETWORKING 15
- 24. NETWORKING 15
- 25. NETWORKING 15
- 26. NETWORKING 15
- 27. 16
- 29. BS ON COMPETITIONS • No ETL • No EDA • Not worth it • Not for production 18
- 30. NO ETL? 19 • Deloitte Western Australia Rental Prices
- 31. NO ETL? 20 • Outbrain Click Prediction 2B page views. 16.9MM clicks. 700MM users. 560 sites
- 32. NO ETL? 21
- 33. NO EDA? • Most of competitions provide actual labels - typical EDA • Anonymized data - more creative EDA • People decode age, states, time intervals, income, etc. 22
- 34. NO EDA? • Anonymized data - more creative EDA 23
- 35. NOT WORTH IT? • Performance matters • You can walk easier once you know how to run 24
- 36. NOT FOR PRODUCTION? • Kaggle Kernel • Max execution time:10 minutes • Max file output: 500MB • Memory limit: 8GB 25
- 37. ENSEMBLE PIPELINE AT CL 26
- 39. BEST PRACTICES • Feature Engineering • Algorithms • CrossValidation • Ensemble 28
- 40. FEATURE ENGINEERING • Numerical - Log, Log(1 + x), Normalization, Binarization • Categorical - One-hot-encode,TF-IDF (text),Weight-of-Evidence • Timeseries - Stats, FFT, MFCC, ERP (EEG) • Numerical/Timeseries to Categorical - RF/GBM* • Dimensionality Reduction - PCA, SVD,Autoencoder * http://www.csie.ntu.edu.tw/~r01922136/kaggle-2014-criteo.pdf 29
- 41. ALGORITHMS Algorithm Tool Note Gradient Boosting Machine XGBoost, LightGBM The most popular algorithm in competitions Random Forests Scikit-Learn, randomForest Extremely RandomTrees Scikit-Learn Neural Networks/ Deep Learning Keras, MXNet Blends well with GBM. Best at image recognition competitions, NLP. Logistic/Linear Regression Scikit-Learn,Vowpal Wabbit Fastest. Good for ensemble. SupportVector Machine Scikit-Learn FTRL Vowpal Wabbit Competitive solution for CTR estimation competitions Factorization Machine libFM Winning solution for KDD Cup 2012 Field-aware Factorization Machine libFFM Winning solution for CTR estimation competitions (Criteo,Avazu) 30
- 42. CROSSVALIDATION Training data are split into five folds where the sample size and dropout rate are preserved (stratified). 31
- 44. ENSEMBLE * for other types of ensemble, see http://mlwave.com/kaggle-ensembling-guide/33
- 46. WHY COMPETITION • For fun • For experiences • For learning • For networking 35
- 47. 36