Big data mlb analyze system
Download as PPTX, PDF1 like777 views
1) The document describes a baseball game prediction system that analyzes player and team statistics to predict game outcomes. 2) It uses Extract-Transform-Load (ETL) processes to extract raw baseball data from websites, transform and load it into databases for analysis. 3) Logistic regression models are used to analyze relationships between various stats like OBP, SLG, and OPS and game results, and to generate prediction probabilities. The models are tested on historical game data for accuracy.
Big data mlb analyze system
- 1. 資策會數位教育研究所 國際人才養成班 YB102企業智慧系統設計結業專題 老師 郭惠民 組員 簡俊能 MLB ANALYZE SYSTEM
- 2. THE BEGGING ~I READ A BOOK I study basin again I watch baseball game in I study any data about baseball game I want analyze the secret in the big data I want proof he is true I began .. I find that …. Prediction baseball can make fans and players have more fun in this game because is it too complex to Prediction
- 3. ETL RDB R Matlab Web Server ML Mahout Hadoop eco pig RDD Spark HDFS NOSQLDB HIVE Client UI R hadoop SYSTEM ARCHETECH
- 4. RAWDATA ETL EXTRACT CRAWLall team data 2010-15 Change link by Set dictionary
- 8. ETLLOAD JDBC PYTHON TO ORACLE 25000 games in 5 years PYTHON TO ORACLE
- 9. ETLLOAD BEFORE on the web AFTERinmyDB Sep1->0902 W->1 136-> WSN20140902--136 BE FILTER ETL
- 10. ETL EXTRACT CRAWLMLB Website DATA ACCOUNT 25000games*100 BAT EACH GMAE More than 2,500,000
- 11. 將資料進行轉換為介於0～1 間的資料型態 最後模式產生輸出變數，其值可為0 或1 ， 常代表選擇量度中的「是」與「否」或 「成功」與「失敗」 LOGISTICREGRESSION
- 12. 羅吉斯迴歸分析是最普遍的迴歸技術，邏輯斯迴歸模型可用來計 算資料中單一類別或兩類別的分類機率，可描述諸多預測因子 （predictor）變項 X1、X2、X3…對於二分類依變項Y之間的關係 通常Y編碼為（0,1）兩分類（Kleinbaum, D. G.,1998） ANALYZE MODEL
- 13. Forward stepwise正向逐步選擇法:即在截 距模型基礎上，將符合鎖定顯著水準的 自變數一次一個加入模型 Backward stepwise反向逐步選擇法:在模 型包括所有候選變數基礎上，將不符合 保留要求顯著水準的自變數一次一個地 刪除掉 Combined stepwise混和逐步選擇法 ANALYZE STEP
- 14. • IT doesn’t have advantages playing at home (主場不一定有優勢) ANALYZEPROCESS
- 15. ANALYZEPROCESS IT is less difference between each team by OBP (MLB各隊OBP差異比你想像的小很多) • MLB players are pretty amazing(MLB球員的數 據世界級水準)
- 18. LOGISTIC PREDICTIONMODEL ~R library(RJDBC) <-load RJDBC drv=JDBC("oracle.jdbc.driver.OracleDriver","C:/mylib/ojdbc14.jar",id entifier.quote="'") <-load driver con=dbConnect(drv,"jdbc:oracle:thin:@localhost:1521:XE","user1"," u111") <-get connection options(max.print=10000000) <-set list view res=dbReadTable(con, "game") <-readtable YEAR=substr(ex1$GDATE,4,7) <-subsitude string YEARSEQ=as.data.frame(YEAR) <-transform data type colnames(YEARSEQ)=c("YEAR") <-named colnume exp2=cbind(ex1,YEARSEQ) <-combine exp2$YEAR=as.character(exp2$YEAR) <-transform data type exp2$YEAR=as.numeric(exp2$YEAR) <- transform data type Load data From Oracle ETL IN R
- 20. LOGISTIC PREDICTIONMODEL ~R pred=predict(logyear,newdata=esortyear, type="response") tab=table(Y=newdata$GRESULT,Ypred=pred) a=100*sum(diag(tab))/sum(tab) TEST PREDICTION TEAM:ARI Duration:1-60 Year:2014
- 21. LOGISTIC PREDICTIONMODEL ~R pred=predict(logyear,newdata=esortyear, type="response") tab=table(Y=newdata$GRESULT,Ypred=pred) a=100*sum(diag(tab))/sum(tab) TEST PREDICTION TEAM:ARI Duration:50-110 Year:2014
- 22. LOGISTIC PREDICTIONMODEL ~R pred=predict(logyear,newdata=esortyear, type="response") tab=table(Y=newdata$GRESULT,Ypred=pred) a=100*sum(diag(tab))/sum(tab) TEST PREDICTION TEAM:ARI Duration:100-162 Year:2014
- 23. LOGISTIC PREDICTIONMODEL ~R TESTPREDICTION YEAR 1-50 series 50-100 series 100-160 series 2014 58.33 63.93 68.85 2013 64.00 60.78 54.10 2012 62.00 66.67 59.02 2011 55.00 67.21 63.93 2010 58.33 62.75 57.38 TEAM:ARI YEAR:2010-15 Game:810 SERIES:1-50/50-100/100-160
- 24. LOGISTIC PREDICTIONMODEL ~R TEAM:ARI YEAR:2010-15 Game:810 OPP:30 TEST PREDICTION TEST ANALYZE Logistic is sign in 5 years 9 teams have NOT enough games to run logistic model 對每個球隊
- 25. LOGISTIC PREDICTIONMODEL ~R TEAM:ARI YEAR:2010-15 Game:810 STARTER:254 TEST PREDICTION Total 254 starter to ARI Few has enough games to run logistic model 對每個先發投手
- 26. ANALYZECONCLUTION • 若OBP+SLG+OPS 可估出七成勝率那其他三成? 一支穩定得分的球隊在大聯盟即可以預估近七成的勝率那意外的是?? • 投手被打爆 • 實力接近 • 對手超水準演出 • 團隊士氣 • 教練戰術 • 愛國裁判 • 其他
- 27. VISULIZATION
- 28. VISULIZATION
- 29. VISULIZATION
- 30. VISULIZATION
- 31. Q&A THANKS YOU FOR ATTENTION