Looking for Something Special -- Outlier Detection in R
- 1. Ágnes Salánki salanki.agnes@gmail.com Budapest BI Forum 2015
- 2. PhD student in Computer Engineering Fault Tolerant Systems Research Group Availability of 99.99% 2011: „We need a method to detect erroneous observations.”
- 3. PhD. student in Computer Engineering Fault Tolerant Systems Research Group Availability of 99.99% 2011: „We need a method to detect erroneous observations.”
- 4. PhD. student in Computer Engineering Fault Tolerant Systems Research Group Availability of 99.99% 2011: „We need a method to detect erroneous observations.”
- 5. PhD. student in Computer Engineering Fault Tolerant Systems Research Group Availability of 99.99% 2011: „We need a method to detect erroneous observations.”
- 6. „An outlier is an observation which deviates so much from the other observations as to arouse suspicions that it was generated by a different mechanism” (Hawkins 1980)
- 7. „An outlier is an observation which deviates so much from the other observations as to arouse suspicions that it was generated by a different mechanism” (Hawkins 1980)
- 8. „An outlier is an observation which deviates so much from the other observations as to arouse suspicions that it was generated by a different mechanism” (Hawkins 1980)
- 9. „An outlier is an observation which deviates so much from the other observations as to arouse suspicions that it was generated by a different mechanism” (Hawkins 1980)
- 11. isodepth mve db lof 1970 1980 1990 2000 2010 mcd bacon s-h-esd fast-mcd
- 12. isodepth mve db lof 1970 1980 1990 2000 2010 mcd bacon s-h-esd fast-mcd
- 13. PISA 2012 results Children’s math and reading scores by country
- 14. PISA 2012 results Children’s math and reading scores by country
- 15. PISA 2012 results Children’s math and reading scores by country China- Shanghai Quatar Peru Japan Indonesia Colombia
- 20. 1970 1980 1990 2000 2010 isodepth mve db lof China- Shanghai Kazakhstan Japan Costa Rica Colombia
- 35. Hans Rosling’s TED talk in 2006 Still one of the most popular talks (as of Oct. 2015)
- 37. Happy families are all alike; every unhappy family is unhappy in its own way. /Anna Karenina/
- 41. Fault Tolerant Systems Research Group Outliers: high communication workload Only planned system maintenance with moving lots of data
- 42. isodepth mve db lof 1970 1980 1990 2000 2010 mcd bacon s-h-esd fast-mcd Algorithm R Scikit-learn Rapidminer WEKA ELKI isodepth MVE DB LOF salankia
- 43. R packages: depth, fields, robustX, DMwR Pictures Forest Gump, Judit Polgár, Garry Kasparov Outlier detection applications in finance, security, medicine, police surveillance 1977 and 1987 pictures Github code: https://github.com/salankia/OutlierDetection- Budapest-BI-2015