Larson.defense
- 1. disaggregated hot and cold water sensing with minimal calibration semi-supervised training for infrastructure mediated sensing Eric C. Larson UbiComp Lab electrical engineering computer science and engineering University of Washington
- 2. how can indirect sensing and machine learning be used to reduce our environmental footprint?
- 3. 3
- 6. we are using water faster than it is being replenished Pacific Institute for Studies in Development, Environment, and Security, 2011 6
- 7. we are using water faster than it is being replenished Pacific Institute for Studies in Development, Environment, and Security, 2011 6 image: weiku.com
- 8. $2,994.83 7
- 9. 8
- 10. water usage is vastly misunderstood
- 11. eco-feedback 10
- 12. eco-feedback 11 image: gardena, inc
- 13. eco-feedback 11 image: gardena, inc image: showersmart image: iSave
- 14. eco-feedback Geographic Comparisons Dashboards Metaphorical Unit Designs Recommendations 12
- 15. eco-feedback 13
- 16. eco-feedback 14
- 17. eco-feedback 15
- 18. eco-feedback 16
- 19. eco-feedback 17 video: courtesy Jon Froehlich
- 20. what are the potential water savings?
- 22. 0% 5% 10% 15% 20% 1 2 3 4 5 Untitled 1 20% 12% 9.2%8.4% 6.8% 3.8% Enhanced Billing Web Based Daily Feedback Realtime Feedback Appliance Level + Personalized Feedback Annual%Savings Based on 36 studies between 1995-2010. Summarized by Ehrhardt-Matinez et al. >20% reduction: Gardner et al. (2008) and Laitner et al. (2009) Appliance Level eco-feedback in electricity aggregate disaggregated Courtesy: Sidhant Gupta 20
- 23. how can we sense water usage?
- 26. meters flow rate fixture flow inline water 23
- 27. meters flow rate fixture flow inline water water pressure pressure sensor machine learning estimated
- 28. • central sensing point • easy to install • low cost • can observe every fixture HydroSense 25
- 29. • central sensing point • easy to install • low cost • can observe every fixture HydroSense 25 40# 50# 60# 70# 80# Cold Line Pressure (Hose Spigot) 0 94.5 time (s) psi open close
- 30. HydroSense 26
- 31. kitchen sink upstairs toilet template matching unknown event 27 downstairs toilet
- 32. kitchen sink upstairs toilet template matching unknown event 27 downstairs toilet
- 33. feasibility study • 10 homes • staged calibration • ~98% accuracy Froehlich, J., Larson, E., Campbell, T., Haggerty, C., Fogarty, J., and Patel, S.N. HydroSense: infrastructure-mediated single-point sensing of whole-home water activity. Proceedings of the 11th ACM international conference on Ubiquitous computing, (2009), 235–244. Larson, E., Froehlich, J., Campbell, T., et al. Disaggregated water sensing from a single, pressure- based sensor: An extended analysis of HydroSense using staged experiments. Pervasive and Mobile Computing, (2010). 28
- 34. 70 50 30 pressure(psi) 70 50 30 pressure(psi) initial study: staged events kitchen sink kitchen sink 29
- 36. how well does HydroSense work in a natural setting?
- 38. 33
- 39. 34
- 40. 35
- 41. 36
- 42. totals days 33 33 30 27 33 156 events 2374 3075 4754 2499 2578 14,960 events/day 71.9 93.2 158.5 92.6 78.1 95.9 compound 22.2% 21.8% 16.6% 32% 21.3% 21.9% data collection Larson, E., Froehlich, J., Saba, E., et al. A Longitudinal Study of Pressure Sensing to Infer Real- World Water Usage Events in the Home. Pervasive Computing, Springer (2011), 50–69. most comprehensive labeled dataset of hot and cold water ever collected 37
- 43. bathroom sink natural water usage 70 50 30 pressure(psi) 70 50 30 pressure(psi) 8AM 2 minutes 38
- 44. kitchen sink kitchen sink toilet bathroom sink natural water usage 70 50 30 pressure(psi) 70 50 30 pressure(psi) 8AM 2 minutes 38
- 45. kitchen sink kitchen sink toilet bathroom sink natural water usage 70 50 30 pressure(psi) 70 50 30 pressure(psi) template matching: 98% 74% 10 fold cross validation 35% 8AM 2 minutes minimal 38
- 46. need a more realistic approach templates feature vectors matching parametric model minimize training 39
- 47. need a more realistic approach templates feature vectors matching parametric model minimize training 39
- 48. 10 psi 7.32 psi 15 Hz 200 ms feature vectors: dense features 43
- 49. 10 psi 7.32 psi 15 Hz 200 ms feature vectors: dense features xd 1 44
- 50. feature vectors: sparse features unlabeledinstances xd 1 45
- 51. feature vectors: sparse features codebook x1 x56 x132 x240 0 0... 0 0... 0 0... 0 0... xd 1 46
- 52. feature vectors: sparse features codebook xd 1 xs 1
- 53. 70 50 30 pressure(psi) 70 50 30 pressure(psi) xd 1 xs 1 xs 2 xd 2 xd 3 xs 3 xd 4 xs 4 xd 5 xs 5 xd 6 xs 6 xs 7 xd 7 feature vectors: sequence
- 54. templates matching parametric model feature vectors Traditional Methods KNN SVM Decision Trees CRF DBN (i.e., HMM) Ensemble Methods KNN-subspace Bagged Trees Stacking Methods TB+CRF SVM+CRF TB+DBN 51
- 55. minimal training set 0 10 20 30 40 50 60 70 80 90 100 NN KNN TM HMM KNN-sub SVM HMM-TM CRF TB SVM+CRF TB+CRF valvelevelaccuracy(%) error bars=std err. 1-2 labels per valve 52 densefeatures sparsefeatures
- 57. fixture level confusions KitchenSink MasterBathroomSink SecondaryBathroomSink SecondaryBathroomToilet MasterBathroomToilet dishwasher laundry MasterBathroomBath/Shower MasterBathroomBath/Shower WashingMachine Dishwasher 54
- 58. accuracy trusted not trusted how accurate should the system be? how can we be sure the user trusts the system? highly critical noticeable Lim, B. and Dey, A. Investigating intelligibility for uncertain context-aware applications. Proceedings of the 13th international conference on ubiquitous computing, (2011), 415. ~80% ~99% 55
- 61. 0 0.5 1 1.5 2 2.5 3 3.5 4 46 48 50 52 54 56 58 60 hours psi finding the dishwasher 0.5 1 1.5 2 2.5 3 3.5 4 hours template 59 dishwasher
- 62. 0 0.5 1 1.5 2 2.5 3 3.5 4 46 48 50 52 54 56 58 60 hours psi finding the dishwasher 0.5 1 1.5 2 2.5 3 3.5 4 hours template X n tn 59 dishwasher
- 63. 0 0.5 1 1.5 2 2.5 3 3.5 4 46 48 50 52 54 56 58 60 hours psi finding the dishwasher 0.5 1 1.5 2 2.5 3 3.5 4 hours template X n tn X n pn 59 dishwasher
- 64. 0 0.5 1 1.5 2 2.5 3 3.5 4 46 48 50 52 54 56 58 60 hours psi finding the dishwasher 0 0.5 1 1.5 2 2.5 3 3.5 4 hours template pressure difference time difference X n tn X n pn 59 dishwasher
- 65. 0 0.5 1 1.5 2 2.5 3 3.5 4 46 48 50 52 54 56 58 60 hours psi finding the dishwasher 0 0.5 1 1.5 2 2.5 3 3.5 46 48 50 52 54 56 58 60 hours psi template pressure difference time difference X n tn X n pn 59 dishwasher
- 66. 0 0.5 1 1.5 2 2.5 3 3.5 4 46 48 50 52 54 56 58 60 hours psi finding the dishwasher pressure difference time difference X n tn X n pn 59 dishwasher
- 67. 0 0.5 1 1.5 2 2.5 3 3.5 4 46 48 50 52 54 56 58 60 hours psi finding the dishwasher pressure difference time difference X n tn X n pn X n tcyclelaundry machine 59 laundry dishwasher
- 68. laundry dishwasher true positives false alarms precision 73% 15 90% 75% 16 89% 43% 58 85% 39% 119 82% 75% of all showering category 60
- 70. leveraging unlabeled data labeled unlabeled classifier classifier feature set 2 high confidence high confidence agree? feature set 1 self labeled multi-view classification 62
- 71. training 48% 49% labeled unlabeled multi-view classification self labeled TB SVM+CRF 55% 53% feature split: hot sensor vs. cold sensor 88% 90%TB SVM+CRF 99% dense features sparse features 63 training training training
- 72. 0 5 10 15 20 46 48 50 52 54 56 58 60 time of day (hours) psi kitchen sink, hot master bathroom toilet multi-view classification 64
- 73. semi-supervised learning rule based classifier 0 0.5 1 1.5 2 46 48 50 52 54 56 58 60 hourspsi self labels expert knowledge virtual evidence 65
- 74. A B C Dvirtual evidence 1 semi-supervised learning virtual evidence kitchen sink kitchen sink P=1 bath sink bath sink P=1 toilet toilet P=1 P=0.01 otherwise 66 arg max A,B P(A)P(B|A)P(C = c|A)P(D = d|B)P(ve|A, B)
- 80. semi-supervised learning results 0 10 20 30 40 50 60 70 80 90 100 TM HMM SVM TB SVM+CRF TB+CRF HMM-VE-Co valvelevelaccuracy(%) 10 fold cross validation 72
- 81. room shower. 57 6.6 0.6 1.1 4.2 56 1.7 0.8 6.4 4.6 31 0.8 82 1.5 9.1 4.3 14 1.5 2.6 4.2 2.0 6.1 1.1 8.8 1.5 28 2.0 81 10 9.1 2.0 15 1.0 3.0 22 6.1 6.7 8.5 2.0 6.5 0.6 33 3.3 1.0 0.8 0.5 0.7 17 5.0 39 1.0 1.5 0.6 0.8 24 4.4 0.7 2.5 6.8 1.1 11 6.5 65 2.4 5.4 0.8 3.1 7.9 2.5 6.0 0.9 7.8 5.7 14 3.0 68 1.0 1.6 3.5 1.5 3.9 1.6 9.5 4.8 4.3 1.4 83 0.8 1.4 17 1.9 3.1 3.5 1.9 91 2.5 0.8 11 17 6.4 42 2.4 60 4.3 1.0 3.4 4.8 7.9 0.6 3.7 7.5 1.6 86 1.6 8.4 1.5 1.7 0.7 4.5 1.7 5.7 1.0 6.8 0.6 3.0 3.1 1.9 85 6.4 2.2 6.1 0.7 66 3.2 1.1 38 14 9.1 43 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 r close,1 open,2 k close,3 open,4 r close,5 open,6 k close,7 open,8 t close,9 open,10 close,11 open,12 close,13 open,14 open,15 close,16 open,17 dishwasher laundry dishwasher laundry 73
- 82. room shower. 57 6.6 0.6 1.1 4.2 56 1.7 0.8 6.4 4.6 31 0.8 82 1.5 9.1 4.3 14 1.5 2.6 4.2 2.0 6.1 1.1 8.8 1.5 28 2.0 81 10 9.1 2.0 15 1.0 3.0 22 6.1 6.7 8.5 2.0 6.5 0.6 33 3.3 1.0 0.8 0.5 0.7 17 5.0 39 1.0 1.5 0.6 0.8 24 4.4 0.7 2.5 6.8 1.1 11 6.5 65 2.4 5.4 0.8 3.1 7.9 2.5 6.0 0.9 7.8 5.7 14 3.0 68 1.0 1.6 3.5 1.5 3.9 1.6 9.5 4.8 4.3 1.4 83 0.8 1.4 17 1.9 3.1 3.5 1.9 91 2.5 0.8 11 17 6.4 42 2.4 60 4.3 1.0 3.4 4.8 7.9 0.6 3.7 7.5 1.6 86 1.6 8.4 1.5 1.7 0.7 4.5 1.7 5.7 1.0 6.8 0.6 3.0 3.1 1.9 85 6.4 2.2 6.1 0.7 66 3.2 1.1 38 14 9.1 43 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 r close,1 open,2 k close,3 open,4 r close,5 open,6 k close,7 open,8 t close,9 open,10 close,11 open,12 close,13 open,14 open,15 close,16 open,17 DW Shower Shower CW dishwasher laundry dishwasher laundry 73
- 83. semi-supervised learning leveraging the homeowner which labels are needed most? can we leverage multi-view models? 74
- 84. semi-supervised learning leveraging the homeowner can we leverage multi-view models? • select low confidence examples • ask homeowner for label 75
- 85. semi-supervised learning leveraging the homeowner can we leverage multi-view models? • select low confidence examples • ask homeowner for label AT&T LTEAT&T LTE 5:23 PM did you just use water? YesNo 75
- 86. semi-supervised learning leveraging the homeowner can we leverage multi-view models? • select low confidence examples • ask homeowner for label AT&T LTEAT&T LTE 5:23 PM did you just use water? YesNo 75 AT&T LTEAT&T LTE 5:23 PM Fixture Selection Do Not Disturb Select Notification Times OFF Master Toilet Recently Used: Half Bath Toilet Dishwasher Master Sink Select Temp. Kitchen Sink Select Temp. Master Shower Select Temp. More Half Bath Sink Select Temp.
- 87. simulating labels from homeowner AT&T LTEAT&T LTE 5:23 PM did you just use water? YesNo • ask for two labels every other day • one morning and one evening • only from 8AM-9PM • randomly ask for previous event 76 AT&T LTEAT&T LTE 5:23 PM Fixture Selection Do Not Disturb Select Notification Times OFF Master Toilet Recently Used: Half Bath Toilet Dishwasher Master Sink Select Temp. Kitchen Sink Select Temp. Master Shower Select Temp. More Half Bath Sink Select Temp.
- 88. 10 15 20 25 30 35 40 45 0.65 0.7 0.75 0.8 0.85 Co−Labeling in H1 Number of Labels ValveLevelAccuracyofCoLabel−HMM Co−Labeling Random Labeling co-labeling random labeling iteration 1 iteration 3 iteration 5 iteration 10 simulating labels from homeowner co-labeling for H1minimaltrainingset totals days 33 33 30 27 33 156 events 2374 3075 4754 2499 2578 14,960 events/day 71.9 93.2 158.5 92.6 78.1 95.9 compound 22.2% 21.8% 16.6% 32% 21.3% 21.9% 77
- 89. totals totals 0 1 2 3 4 5 6 7 8 9 10 11 12 13 60% 70% 80% 90% 100% valve fixture category error bars=std err. co-label iteration accuracy 78
- 90. usion is the secondary bathroom shower for the master bathroom shower. -374 8.5 1.5 -3 6.1 72 1.1 0.8 5.0 0.7 1.5 0.6 -7 18 92 0.6 5.4 2.6 4.9 3.4 2.3 3.5 0.9 5.1 -7 18 0.9 92 1.4 4.9 0.7 4.4 1.4 1.4 7.3 1.0 0.7 3.2 -753 2.1 -1 4.2 67 0.6 2.4 0.8 31 2.5 5.2 -5 1.2 5.0 0.6 18 4.2 88 1.8 7.5 1.5 1.6 5.7 1.0 -5 1.2 0.5 5.3 4.6 10 1.8 89 1.7 1.7 1.4 0.7 3.2 -2 2.9 0.6 81 12 -2 0.8 94 0.6 8.1 -1 6.7 68 -2 2.8 42 -1 1.9 2.9 0.5 25 0.8 95 0.7 -1 2.2 3.1 10 1.3 96 6.6 -290 -3 1.1 0.7 59 18 -3 0.9 0.9 15 60 er close,1 open,2 k close,3 open,4 er close,5 open,6 k close,7 open,8 et close,9 open,10 r close,11 open,12 k close,13 open,14 open,15 e close,16 open,17 dishwasher laundry dishwasher laundry 79
- 91. implications for homeowner week one • homeowner installs system • 1-2 examples per fixture • show sparse classes: dishwasher, shower, laundry 80
- 92. implications for homeowner week one • homeowner installs system • 1-2 examples per fixture • show sparse classes: dishwasher, shower, laundry week two • 2-4 labels, every 2 days • fixture category: 85% 80
- 93. implications for homeowner week one • homeowner installs system • 1-2 examples per fixture • show sparse classes: dishwasher, shower, laundry week two • 2-4 labels, every 2 days • fixture category: 85% week three • 9-12 more examples • fixture: 82% 80
- 94. implications for homeowner week one • homeowner installs system • 1-2 examples per fixture • show sparse classes: dishwasher, shower, laundry week two • 2-4 labels, every 2 days • fixture category: 85% week three • 9-12 more examples • fixture: 82% end of week three • fixture: 87% • valve: 80% 80
- 95. summary contributions • comprehensive disaggregated dataset • multi-view classification • expert knowledge • compressed sensing • framework for virtual evidence in IMS • co-labeling with multi-view • idea: inception to industry ready 81
- 96. how can indirect sensing and machine learning be used to reduce our environmental footprint?
- 97. disaggregated hot and cold water sensing with minimal calibration semi-supervised training for infrastructure mediated sensing UbiComp Lab electrical engineering computer science and engineering University of Washington eclarson.com eclarson@uw.edu @ec_larson Eric C. Larson
- 98. disaggregated hot and cold water sensing with minimal calibration semi-supervised training for infrastructure mediated sensing UbiComp Lab electrical engineering computer science and engineering University of Washington eclarson.com eclarson@uw.edu @ec_larson Eric C. Larson