TDA for feature selection
2 likes838 views
The document discusses topological data analysis (TDA) and its use cases. It first describes how TDA can be used for data visualization. It then provides three examples of using TDA for insights discovery in Titanic passenger data, energy consumption data, and high-dimensional customer data. Finally, it discusses how different methods including TDA, PCA, and MRMR can be evaluated on a energy consumption modeling task using metrics like MAPE and F1 score. TDA was able to achieve high reduction rates while maintaining good performance.
TDA for feature selection
- 1. Topological Data Analysis (TDA) and Use Cases Kim Hee (kimheekimi@gmail.com)
- 2. Outline 1. Visualization by TDA 2. Insights Discovery & Feature Selection 3. Evaluate the Insights 22.01.2016 Kim Hee, “Topological Data Analysis and Use Cases” 2
- 3. Visualization 22.01.2016 Kim Hee, “Topological Data Analysis and Use Cases” 3 Raw Data Filter/Filter&Metric NodesDivision with Redundancy Point Cloud e f g h A 3 7 10 12 B 4 8 11 13 C 5 9 8 10 D 13 11 8 4 Network Node A Edge Node B Node A Node B L2(A,B)= 4 𝒃𝒚 𝐴𝑏𝑠 3 − 4 2 + 𝐴𝑏𝑠 7 − 8 2 + 𝐴𝑏𝑠 10 − 11 2 + 𝐴𝑏𝑠 12 − 13 2 L2(A,C)= 16 𝒃𝒚 𝐴𝑏𝑠 3 − 5 2 + 𝐴𝑏𝑠 7 − 9 2 + 𝐴𝑏𝑠 10 − 8 2 + 𝐴𝑏𝑠 12 − 10 2 L2(A,D)= 180 𝒃𝒚 𝐴𝑏𝑠 4 − 13 2 + 𝐴𝑏𝑠 8 − 11 2 + 𝐴𝑏𝑠 11 − 8 2 + 𝐴𝑏𝑠 13 − 4 2 cos(∠AOB)=0.999 𝒃𝒚 (𝟑×𝟒)+(𝟕×𝟖)+(𝟏𝟎×𝟏𝟏)+(𝟏𝟐×𝟏𝟑) 𝟑2+𝟕2+𝟏𝟎2+𝟏𝟐2 × 𝟒2+𝟖2+𝟏𝟏2+𝟏𝟑2 = 334 334.275 cos (∠AOC)=0.974 𝒃𝒚 (𝟑×𝟓)+(𝟕×𝟗)+(𝟏𝟎×𝟖)+(𝟏𝟐×𝟏𝟎) 𝟑2+𝟕2+𝟏𝟎2+𝟏𝟐2 × 𝟓2+𝟗2+𝟖2+𝟏𝟎2 = 278 285.552 cos(∠AOD)= 0.757 𝒃𝒚 (𝟒×𝟏𝟑)+(𝟖×𝟏𝟏)+(𝟏𝟏×𝟖)+(𝟏𝟑×𝟒) 𝟒2+𝟖2+𝟏𝟏2+𝟏𝟑2× 𝟏𝟑2+𝟏𝟏+𝟖2+𝟒2 = 280 370 Euclidean Distance, 𝐿2 𝑋, 𝑌 𝑖=1 𝑁 𝑋𝑖 − 𝑌𝑖 2 CosineSimilarity, cos θ 𝑖=1 𝑁 𝑋𝑖 × 𝑌𝑖 𝑖=1 𝑁 𝑋𝑖 2 × 𝑖=1 𝑁 𝑌𝑖 2 X, Y: data sample, Xi, Yi: each attribute, N: number of attributes 1. Visualization 2. Insights Discovery 3. Evaluation
- 4. Insights Discovery Case 1 – Titanic 22.01.2016 Kim Hee, “Topological Data Analysis and Use Cases” 4 1. Visualization 2. Insights Discovery 3. Evaluation
- 5. Insights Discovery Case 2 – Energy Consumption Problem Domain » Detect features that has correlation to the energy consumption Data Description » Energy consumption history data in U.K. given by power plant » 1,096 rows * 8 attributes » Label attribute is volume, other are weather/calendar events Apply TDA →Discovered insights: Volume is correlated to day_type and school_holiday 1. Visualization 2. Insights Discovery 3. Evaluation
- 6. 22.01.2016 Kim Hee, “Topological Data Analysis and Use Cases” 6 1. Visualization 2. Insights Discovery 3. Evaluation Insights Discovery Case 3 – High Dimensional Data Problem Domain » Detect features that can predict customers who may terminate service Data Description » Customer data given by Orange telecom » 50,000 rows * 233 attributes » Label attribute is churn (binary) » Other attributes are anonymous Apply TDA Column Name Value Hypergeometric p-value churn 1 1.00E-12 Var202 PXLV 3.78E-04 Var199 Gai9lEF2Fr 4.19E-04 Var198 Z4hPoJV 4.82E-04 Var222 xiJRusu 4.82E-04 ⋮ ⋮ ⋮ Var220 Af96s0w 0.047965 Var220 rDm3DH0 0.047965 Var197 yMvB 0.049324 49 underlying features are captured (p-value that smaller than 0.05) The result of group comparison
- 7. Time to evaluate the insights… 22.01.2016 Kim Hee, “Topological Data Analysis and Use Cases” 7
- 8. Evaluation Framework label sample 1 Y sample 2 N sample 3 Y sample 4 Y sample 5 N sample 6 Y sample 7 Y sample 8 Y sample 9 N sample 10 Y Method Selected Features Reduction Accuracy - all - 66% PCA 7 features 22.22% 0% RF 4 features 55.56% 33% TDA 2 features 77.78% 100% Sample Comparison Result 1. Visualization 2. Insights Discovery 3. Evaluation prediction 1 label result Y Y N Y Y Y prediction 3prediction 2 ModelingEvaluation Decision Tree FeatureSelection PCA TDAMRMR Model 1 Model 2 Model 3 Model 4 label result Y N N Y Y N label result Y N N N Y N label result Y Y N N Y Y test data (30%) Training data (70%) prediction 4
- 9. Energy Consumption 22.01.2016 Kim Hee, “Topological Data Analysis and Use Cases” 9 Evaluation 1. Visualization 2. Insights Discovery 3. Evaluation Modeling Dimensional reduction All PCA MRMR TDA Reduction rate (no. of selected features) 0 % (0) 92.70 % (17) 57.08 % (100/default) 83.26 % (39) Evaluation (F1 Score) Model by Naïve Bayes 0.147 0.005 0.146 0.147 Evaluation (F1 Score) Model by Decision tree 0.016 0.002 0.023 0.036 Modeling Dimensional reduction All PCA MRMR TDA Reduction rate 0 % 66.67 % 88.89 % 77.78 % Selected features all winter, solar_rad, temp day_type day_type, sch_holiday Evaluation (MAPE) Model by Neural Network 3.0546 % 11.1026 % 5.7003 % 3.6406 % Model by SVM 10.9843 % 11.0649 % 10.6166 % 10.7778 % High Dimensional Data
- 10. References Used tool: Ayasdi, http://www.ayasdi.com/ Open source: Mapper, http://danifold.net/mapper/ PCA: https://en.wikipedia.org/wiki/Principal_component_analysis SVM: https://en.wikipedia.org/wiki/Support_vector_machine MRMR: http://penglab.janelia.org/proj/mRMR/ MAPE: https://en.wikipedia.org/wiki/Mean_absolute_percentage_error F1 Score: https://en.wikipedia.org/wiki/F1_score 22.01.2016 Kim Hee, “Topological Data Analysis and Use Cases” 10