Machine Learning with Hadoop
Download as PPTX, PDF30 likes7,694 views
The document discusses the implementation of a pluggable machine learning platform on Hadoop at Huffington Post, focusing on supervised classification for comment moderation and article classification. It highlights the need for various machine learning algorithms and the challenges of managing numerous combinations of parameters, which are addressed through parallel processing using tools like Mahout and Pig. The approach allows for the rapid training and selection of numerous models, significantly enhancing adaptability to changes in user-generated content.
Machine Learning with Hadoop
- 1. Training on a pluggable machine learning platformMachine Learning on Hadoop at Huffington Post | AOL
- 2. A Little Bit about UsCore Services Team at HPMG | AOL Thu Kyaw (thu.kyaw@teamaol.com)Principal Software EngineerWorked on machine learning, data mining, and natural language processingSang Chul Song, Ph.D. (sangchul.song@teamaol.com)Senior Software EngineerWorked on data intensive computing – data archiving / information retrieval
- 3. Machine Learning:Supervised Classification1. Learning PhaseModelTrain“Business”2. Classifying Phase“Entertainment”ModelResultClassifycapital gains to be taxed …“Politics”
- 4. Two Machine Learning Use Cases at HuffPost | AOLComment ModerationEvaluate All New HuffPost User Comments Every DayIdentify Abusive / Aggressive CommentsAuto Delete / Publish ~25% Comments Every DayArticle ClassificationTag Articles for AdvertisingE.g.: scary, salacious, …
- 5. Our Classification Tasksabusivenon-abusivenon-abusivescarysexynon-abusivenon-abusiveabusiveComment ModerationArticle Classification
- 6. In Order to Meet Our Needs,We Require…Support for important algorithms, includingSVMPerceptron / WinnowBayesianDecision TreeAdaBoost …Ability to build tons of models on regular basis, and pick the bestBecause, in general, it’s difficult to know in advance what algorithm / parameter set will work best
- 7. However,N algorithms, K parameters each, L values in each parameter There are N x LK combinations!, which is often too many to deal with sequentially.For example, N=5, K=5, L=10 500K
- 8. So, we parallelize on HadoopGood news: Mahout, a parallel machine learning tool, is already available.There are Mallet, libsvm, Weka, … that support necessary algorithms.Bad news: Mahout doesn’t support necessary algorithms yet. Other algorithms do not run natively on Hadoop.
- 9. Therefore, we do…We build a flexible ML platform running on Hadoop that supports a wide range of algorithms, leveraging publicly available implementations.On top of our platform, we generate / test hundred thousands models, and choose the best.We use Pig for Hadoop implementation.
- 10. Our ApproachOUR APPROACH More algorithms (thus better model), and faster parallel processing AdaBoost, SVM, Decision Tree,Bayesian and a Lot OthersTrain RequestReturnCONVENTIONAL1000s Models(one for each param set)Best ModelTraining DataSelectTrain (sequential)
- 11. What Parallelization?Training TaskTraining TaskTraining TaskTraining TaskTraining Task
- 12. General Processing FlowTrainingDocsPreprocessVectorizedDocsTrainModelPreprocess ParametersStopword use, n-gram size, stemming, etc.Train ParametersAlgorithm and algorithm specific parameters(e.g. SVM, C, Ɛ, and other kernel parameters)
- 13. Our Parallel Processing FlowModelVectorizedDocsModelModelTrainingDocsVectorized DocsModelModelModelModelVectorized DocsModelModel
- 14. Preprocessing on Hadoop(see next slide)Preprocessing on Hadoopbusiness Investments are taxed as capital gains.....business It was the overleveraged and underregulatedbanks …none I am afraid we may be headed for …none In the famous words of Homer Simpson, “it takes 2 to lie …”…Vector 1Training DataVector 2Vector 3Vector 4279 68ngram_stem_stopword 1snowballtrue279 68 ngram_stem_stopword2 snowball true279 68 ngram_stem_stopword3 snowball true279 68 ngram_stem_stopword 1 porter true279 68 ngram_stem_stopword2porter true279 68 ngram_stem_stopword3none false…Vector 5Preprocessing Request (a parameter set per line)Vector k
- 15. Preprocessing on HadoopBig PictureVector 1Through UDF CallVector 2UDFpar = LOAD param_file AS par1, par2, …;run = FOREACH par GENERATE RunPreprocess(par1, par2, …);STORE run ..;RunPreprocess()……..Preprocessors (Pluggable Pipes)StemmerTokenizerStopwordFilterVector kVectorizerFeatureSelector
- 16. Training on Hadoop010101101020101100010101110100010101011100…010111010100010100100010101011100110110101…011101011010101011101011011010001010010101…010010111010100010101010001010111010101010…111010110001110101011010100101011010001011…Model 1Training on Hadoop(see next slide)VectorsModel 2Model 3Model 473 923 balanced_winnow 5 1 10…73 923 balanced_winnow 5 210…73 923 balanced_winnow 5 310…73 923 balanced_winnow 5 1 20 …73 923 balanced_winnow 5 2 20 …73 923 balanced_winnow 5 320……Model 5Train Request (a parameter set per line)Model kMahout, Weka, Malletor libsvm
- 17. Training on HadoopBig PictureModel 1Through UDF CallModel 2UDFRunTrainer()par = LOAD param_file AS par1, par2, …;run = FOREACH par GENERATERunTrainer(par1, par2, …);STORE run ..;…….MalletAdaBoost (M2)
- 18. Bagging
- 19. Balanced Winnow
- 20. C45
- 21. Decision Tree
- 22. …MahoutBayesian
- 24. …WekaAdaBoostM1
- 25. Bagging
- 27. …Model klibsvmSVMTraining on Hadoop : Trick #1Each model can be generated independently an easy parallelization problem (aka ‘embarrassingly parallel’)But, how do we achieve parallelism with Pig?par = LOAD param_file AS par1, par2, …;run = FOREACH par GENERATE RunTrainer(par1, par2, …);STORE run ...;par = LOAD param_file AS par1, par2, …;grp = GROUP par BY (par1, par2, …) PARALLEL 50fltn = FOREACH grp GENERATE group.par1 AS par1, …;run = FOREACH fltn GENERATE RunTrainer(par1, …);STORE run …;
- 28. Training on Hadoop: Trick #2We call ML functions from UDF.Some functions can take too long to return, and Hadoop will kill the job if they do.RunTrainer()“Pig Heartbeat” ThreadMain Thread
- 29. As a result, we now see…We are now able to build tens of thousands of models within an hour and choose the best.Previously, the same task took us days.As we can generate more models more frequently, we become more adaptive to the fast-changing Internet community, catching up with newly-coined terms, etc.
- 30. Useful ResourcesMahout: http://mahout.apache.org/Mallet: http://mallet.cs.umass.edu/Weka: http://www.cs.waikato.ac.nz/ml/weka/libsvm: http://www.csie.ntu.edu.tw/~cjlin/libsvm/OpenNLP: http://incubator.apache.org/opennlp/Pig: http://pig.apache.org/
- 31. Thank You!