Generalized Low Rank Models
5 likes1,133 views
This document discusses generalized low rank models, which provide a compressed representation of data tables by approximating them as the product of two smaller numeric tables. This reduces storage space and improves prediction speed while maintaining accuracy. Two examples are described: one where low rank models are used to visualize important stances from walking data, and another where they compress zip code data to predict compliance violations.
![Generalized Low Rank Models
Anqi Fu
Machine Learning Scientist, H2O.ai
anqi@h2o.ai
October 29, 2015
Based on work by Stephen Boyd, Madeleine Udell, Corinne Horn and Reza Zadeh
Anqi Fu (H2O.ai) Generalized Low Rank Models
October 29, 2015[1ex] Based on work b
/ 8](https://image.slidesharecdn.com/h2oworldglrmdemo1-151030180054-lva1-app6891/85/Generalized-Low-Rank-Models-1-320.jpg){kind=icon}
![What is a Low Rank Model?
Given: Data table A with m rows and n columns
Find: Compressed representation as numeric tables X and Y , where
# cols in X = # rows in Y = small user-specified k max(m, n)
m
A
n
≈ m
X
k
Y k
n
Row of Y = archetypal feature created from cols of A
Row of X = row of A in reduced feature space
Can approximately reconstruct A from product XY
Anqi Fu (H2O.ai) Generalized Low Rank Models
October 29, 2015[1ex] Based on work b
/ 8](https://image.slidesharecdn.com/h2oworldglrmdemo1-151030180054-lva1-app6891/85/Generalized-Low-Rank-Models-2-320.jpg){kind=icon}
![Why use Low Rank Models?
Reduce storage space, e.g. 10 GB compressed to 100 MB
Increase prediction speed, e.g. 10x speed-up with no accuracy loss
Identify and visualize important features
Impute missing data
Anqi Fu (H2O.ai) Generalized Low Rank Models
October 29, 2015[1ex] Based on work b
/ 8](https://image.slidesharecdn.com/h2oworldglrmdemo1-151030180054-lva1-app6891/85/Generalized-Low-Rank-Models-3-320.jpg){kind=icon}
![Example 1: Visualizing Walking Stances
time forehead (x) forehead (y) · · · right toe (y) right toe (z)
t1 1.4 2.7 · · · -0.5 -0.1
t2 2.7 3.5 · · · 1.3 0.9
t3 3.3 -.9 · · · 4.2 1.8
...
...
...
...
...
...
A contains 151 rows (observations over time) by 124 columns
(location of body parts)
Build a low rank model X, Y with rank k = 10
Rows of Y are principal stances person takes while walking
Rows of X decompose each bodily position into combination of
principal stances
Anqi Fu (H2O.ai) Generalized Low Rank Models
October 29, 2015[1ex] Based on work b
/ 8](https://image.slidesharecdn.com/h2oworldglrmdemo1-151030180054-lva1-app6891/85/Generalized-Low-Rank-Models-4-320.jpg){kind=icon}
![Example 2: Compressing Zip Codes
repeat violator (y) ZCTA state · · · violations compensation
N/A 70525 LA · · · 9 8100
R 75189 TX · · · 6 935
RW 95621 CA · · · 4 1155
...
...
...
...
...
...
Train: U.S. Labor Department Wage & Hour Division (WHD)
compliance actions contains 208,806 rows (cases) and 252 cols
(violation info)
Response: Was firm a repeat and/or willful violator?
Predictors: ZCTA, state, number of violations, civil penalties, etc
Anqi Fu (H2O.ai) Generalized Low Rank Models
October 29, 2015[1ex] Based on work b
/ 8](https://image.slidesharecdn.com/h2oworldglrmdemo1-151030180054-lva1-app6891/85/Generalized-Low-Rank-Models-5-320.jpg){kind=icon}
![Example 2: Compressing Zip Codes
ZCTA associate degree bachelor’s degree · · · welsh west indian
01001 1584 1953 · · · 34 57
01002 510 3098 · · · 332 181
01003 27 49 · · · 40 134
...
...
...
...
...
...
American Community Survey (ACS) data contains 32,989 rows by
150 columns (population info)
Each row represents a unique zip code tabulation area (ZCTA)
Build a low rank model X, Y with rank k = 10 and regularization to
sparsify features
Rows of Y are demographic archetypes
Rows of X map ZCTAs into combination of demographic archetypes
Anqi Fu (H2O.ai) Generalized Low Rank Models
October 29, 2015[1ex] Based on work b
/ 8](https://image.slidesharecdn.com/h2oworldglrmdemo1-151030180054-lva1-app6891/85/Generalized-Low-Rank-Models-6-320.jpg){kind=icon}
![Example 2: Compressing Zip Codes
Train =
y ZCTA ···
N/A 70525 · · ·
...
... · · ·
R 01002 · · ·
X =
ZCTA archetypes
01001 x1
01002 x2
...
...
70525 x24128
Replace ZCTA col of training data with low rank model (X) of ACS
Predict if firm will be a repeat violator using modified training data
repeat violator (y) archetypes state · · · violations compensation
N/A x24128 LA · · · 9 8100
...
...
...
...
...
...
R x2 MA · · · 2 225
Anqi Fu (H2O.ai) Generalized Low Rank Models
October 29, 2015[1ex] Based on work b
/ 8](https://image.slidesharecdn.com/h2oworldglrmdemo1-151030180054-lva1-app6891/85/Generalized-Low-Rank-Models-7-320.jpg){kind=icon}
![References
M. Udell, et al (2014), Generalized Low Rank Models
Example 1: Visualizing Walking Stance
Walking Gait Data
Walking Gait Data with Missing Values
Example 2: Compressing Zip Codes
Wage and Hour Division Data
American Community Survey Data
Anqi Fu (H2O.ai) Generalized Low Rank Models
October 29, 2015[1ex] Based on work b
/ 8](https://image.slidesharecdn.com/h2oworldglrmdemo1-151030180054-lva1-app6891/85/Generalized-Low-Rank-Models-8-320.jpg){kind=icon}
Generalized Low Rank Models
- 1. Generalized Low Rank Models Anqi Fu Machine Learning Scientist, H2O.ai anqi@h2o.ai October 29, 2015 Based on work by Stephen Boyd, Madeleine Udell, Corinne Horn and Reza Zadeh Anqi Fu (H2O.ai) Generalized Low Rank Models October 29, 2015[1ex] Based on work b / 8
- 2. What is a Low Rank Model? Given: Data table A with m rows and n columns Find: Compressed representation as numeric tables X and Y , where # cols in X = # rows in Y = small user-specified k max(m, n) m A n ≈ m X k Y k n Row of Y = archetypal feature created from cols of A Row of X = row of A in reduced feature space Can approximately reconstruct A from product XY Anqi Fu (H2O.ai) Generalized Low Rank Models October 29, 2015[1ex] Based on work b / 8
- 3. Why use Low Rank Models? Reduce storage space, e.g. 10 GB compressed to 100 MB Increase prediction speed, e.g. 10x speed-up with no accuracy loss Identify and visualize important features Impute missing data Anqi Fu (H2O.ai) Generalized Low Rank Models October 29, 2015[1ex] Based on work b / 8
- 4. Example 1: Visualizing Walking Stances time forehead (x) forehead (y) · · · right toe (y) right toe (z) t1 1.4 2.7 · · · -0.5 -0.1 t2 2.7 3.5 · · · 1.3 0.9 t3 3.3 -.9 · · · 4.2 1.8 ... ... ... ... ... ... A contains 151 rows (observations over time) by 124 columns (location of body parts) Build a low rank model X, Y with rank k = 10 Rows of Y are principal stances person takes while walking Rows of X decompose each bodily position into combination of principal stances Anqi Fu (H2O.ai) Generalized Low Rank Models October 29, 2015[1ex] Based on work b / 8
- 5. Example 2: Compressing Zip Codes repeat violator (y) ZCTA state · · · violations compensation N/A 70525 LA · · · 9 8100 R 75189 TX · · · 6 935 RW 95621 CA · · · 4 1155 ... ... ... ... ... ... Train: U.S. Labor Department Wage & Hour Division (WHD) compliance actions contains 208,806 rows (cases) and 252 cols (violation info) Response: Was firm a repeat and/or willful violator? Predictors: ZCTA, state, number of violations, civil penalties, etc Anqi Fu (H2O.ai) Generalized Low Rank Models October 29, 2015[1ex] Based on work b / 8
- 6. Example 2: Compressing Zip Codes ZCTA associate degree bachelor’s degree · · · welsh west indian 01001 1584 1953 · · · 34 57 01002 510 3098 · · · 332 181 01003 27 49 · · · 40 134 ... ... ... ... ... ... American Community Survey (ACS) data contains 32,989 rows by 150 columns (population info) Each row represents a unique zip code tabulation area (ZCTA) Build a low rank model X, Y with rank k = 10 and regularization to sparsify features Rows of Y are demographic archetypes Rows of X map ZCTAs into combination of demographic archetypes Anqi Fu (H2O.ai) Generalized Low Rank Models October 29, 2015[1ex] Based on work b / 8
- 7. Example 2: Compressing Zip Codes Train = y ZCTA ··· N/A 70525 · · · ... ... · · · R 01002 · · · X = ZCTA archetypes 01001 x1 01002 x2 ... ... 70525 x24128 Replace ZCTA col of training data with low rank model (X) of ACS Predict if firm will be a repeat violator using modified training data repeat violator (y) archetypes state · · · violations compensation N/A x24128 LA · · · 9 8100 ... ... ... ... ... ... R x2 MA · · · 2 225 Anqi Fu (H2O.ai) Generalized Low Rank Models October 29, 2015[1ex] Based on work b / 8
- 8. References M. Udell, et al (2014), Generalized Low Rank Models Example 1: Visualizing Walking Stance Walking Gait Data Walking Gait Data with Missing Values Example 2: Compressing Zip Codes Wage and Hour Division Data American Community Survey Data Anqi Fu (H2O.ai) Generalized Low Rank Models October 29, 2015[1ex] Based on work b / 8