Multimodal augmented homeostasis 200925 final
Download as PPTX, PDF3 likes549 views
The document discusses the concept of multimodal augmented homeostasis, which aims to improve personal health through technologies that enhance human capability and integrate various data sources for real-time health monitoring. It highlights the shift in global health challenges towards managing chronic diseases and the need for a predictive, preventive, personalized, and participatory (P4) healthcare approach. Additionally, it introduces the idea of a personal health navigator that utilizes continuous health data to help individuals maintain optimal health and address chronic conditions effectively.
Multimodal augmented homeostasis 200925 final
- 1. Multimodal Augmented Homeostasis Ramesh Jain Department of Computer Science UCI Future Health (https://futurehealth.ics.uci.edu/) University of California, Irvine jain@ics.uci.edu With Several Collaborators
- 2. Multimodal Augmented Homeostasis: Agenda • What is Multimodal Augmented Homeostasis? • Health as a ‘Grand Challenge’ for Multimodal computing. • Personal Health Navigator for Augmenting Homeostasis. • Progress and Challenges. 2
- 3. Papers that are related to the ideas presented here. 1. Ramesh Jain, Laleh Jalali: Objective Self. IEEE MultiMedia 21(4): 100-110 (2014) 2. N. Nag and R. Jain, “A navigational approach to health: Actionable guidance for improved quality of life,” IEEE Comput., vol. 52, no. 4, pp. 12–20. April 2019. 3. Weiqing Min, Shuqiang Jiang, Linhu Liu, Yong Rui, Ramesh Jain, “Food Computing”, ACM Computing Surveys, 52(5):1-36, September, 2019 4. Weiqing Min, Shuqiang Jiang, and Ramesh Jain, ‘Food Recommendation: Framework, Existing Solutions, and Challenges’, IEEE Trans on Multimedia, November 2019. 5. Nitish Nag, Vaibhav Pandey, Ramesh Jain: Health Multimedia: Lifestyle Recommendations Based on Diverse Observations. ICMR 2017: 99-106 6. Hyungik Oh, Ramesh Jain: From Multimedia Logs to Personal Chronicles. ACM Multimedia 2017: 881-889 7. Nitish Nag, Vaibhav Pandey, Preston J. Putzel, Hari Bhimaraju, Srikanth Krishnan, Ramesh Jain: Cross- Modal Health State Estimation. ACM Multimedia 2018: 1993-2002 8. Hyungik Oh, Ramesh C. Jain: Detecting Events of Daily Living Using Multimodal Data. CoRR abs/1905.09402 (2019) 9. Nitish Nag: Health State Estimation. CoRR abs/2003.09312 (2020) 10. Ali Rostami, Vaibhav Pandey, Nitish Nag, Vesper Wang: Personal Food Model. ACM Multimedia 2020. 11. Laleh Jalali and Ramesh Jain: Event Mining for Explanatory Analysis (Book), Coming from ACM Books. 3
- 4. Augment • “technologies that enhance human productivity or capability, or that somehow add to the human body” • to make greater, more numerous, larger, or more intense. • make (something) greater by adding to it; increase.
- 6. Multimedia • Essential for understanding real world •No single mode provides complete information in a real situation. • Different modes must be combined for understanding a situation. • Different modes have complementary information.
- 7. Multimodal is the future of Multimedia • Multimedia • GPS • Heartrate • Temperature • Accelerometer • … 7
- 8. Big Data is Multimedia Data Multimedia Realtime Credibility
- 9. Dominant Applications of Multimedia • Image Search • Video on demand • Video-conferencing • Shopping • Social Media • The Best is yet to come – just now becoming popular. 9
- 10. The Most Important Application of Multimedia Computing? Health
- 11. BUT, Healthcare is broken in every country. Can Multimedia Community Help?
- 12. What is Homeostasis? • Homeostasis is the state of steady internal, physical, and chemical conditions maintained by living systems. • Maintain optimal conditions for • Temperature • Glucose level • pH • Oxygen • … • Maintains the constancy of the internal environment in which the cells of the body live and survive. • Works every moment of life.
- 13. Important Turning Point in Health In the mid-20th century, the primary causes of death worldwide shifted from infections to chronic conditions.
- 14. The Trend in Global Health is Clear.
- 15. * Source: CDC The Effect of Chronic Diseases, 2016* 7 of 10 Deaths Due to chronic diseases 86% of Costs Driven by treating chronic diseases Heart Disease Cancer Type 2 Diabetes Brain Diseases Lung Diseases Bone Diseases We all are likely to face Chronic Diseases.
- 16. Why are Chronic Diseases so Common? • All bodily functions deteriorate with age. • When this dysfunction affects Homeostasis, the Onset of the Chronic Disease starts. • Usually by the time, this is detected, it is too late. • P4 (predictive, preventive, personalized and participatory) is the new Paradigm in Healthcare 16
- 17. Health Factors This needs attention.
- 18. Homeostasis is Nature’s Engineering Homeostasis: any self-regulating process by which biological systems tend to maintain stability while adjusting to conditions that are optimal for survival. 18
- 19. Cybernetics: Feedback revolutionizes system design 1. Sensors to detect State 2. Model – of the person 3. Controller: Logic and Inputs
- 20. Basic Systems Theory Where x, u, and y are state, input, and output vectors; A, B, C, and D are appropriate matrices for transformation. Observability: Determine state using measurements. Controllability: Get to desirable state.
- 21. Type 1 Diabetes • Your pancreas doesn't make insulin or makes very little insulin. • Treatment aims at maintaining normal blood sugar levels through regular monitoring, insulin therapy, diet, and exercise. 21
- 22. Impact: It has been a life- saver for many patients and their families. ‘Artificial Pancreas’ for Type 1 Diabetes. Cybernetics is now Used for Augmenting Homeostasis Miracle for Type 1 Diabetes Patient
- 23. Why Are These Birds Huddling? By Blalonde - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=37172033
- 24. Augmenting Homeostasis: Want to help yourself! 24
- 25. Continuous Augmentation 25 An additional layer of homeostasis!!!
- 26. Augmented Homeostasis: • Health State Estimation. • A disease is just an undesirable region in health state space. • Guidance: life-style, environment, and medication. • Actuation • Personal Model • Goal State • Close-loop perpetual operation Self-regulating external process by which human systems achieve health goals to maximize their quality of life.
- 27. Augmented Homeostasis Architecture Controller Continuous Observations Episodic Observations Personicle Event Mining Lifestyle Actions, Meds Health State Person Person Model Cloud Goal External Factors 27 Control Signals
- 28. Traditional Episodic Health Cycle How can I get more attention?
- 29. Getting to a destination: 30 Years Ago.
- 30. Digitalization made getting directions easy. Maps Points of Interest Starting Point Destination Mapping System Getting to a destination: 20 Years Ago.
- 31. Getting to a destination: 10 Years Ago. Maps Points of Interest Current State : GPS Current Traffic: Waze Destination Cybernetic loop Navigation system Knowing Current State is Magical.
- 32. Measure Collect all data Perpetual Health Guidance Guidance Estimate (Health State) Influence and Compliance How do we close the loop? Inspiration: Navigation Systems.
- 33. When do people get best healthcare? When they are about to die – in ICU.
- 34. Personal Health Navigator •Guides you to better health. •Uses real-time information to redirect you around a disease. •Provides step-by-step directions to get you to your optimal health state, whether you’re a professional athlete or a taxi driver.
- 36. Personal Health Navigator: Diabetes Activity Food Medication Glucose Level Activity Food Medication Glucose Level in last 3 hours Sensitivity to different foods. Sensitivity to medication. Activity Checks. Glucose Level Checks.
- 37. • Ping Wang and Ramesh Jain co-founded, then joined by other people • Licensed UCI technology • A start-up company formed (Team of 7 now, raised seed funding) • App available on iPhone Personal Diabetes Navigator Dianavi is building Personal Diabetes Navigator. Their application is now available on iPhone. Disclaimer: I am a co-founder.
- 38. Sensors to Estimate Health State • Two major classes: • Low cost, continuous, unobtrusive (LCU) – Used in Wellness • High cost, episodic, intrusive (HEI) – Used in Diagnostics • Currently LCU is used for ‘Wellness’ and HEI for ‘Disease Control’. • Though both are related and complementary, they are treated in different silos. • Both are to improve the quality of Life of a person.
- 39. Low Cost, Continuous, Unobtrusive Sensors (LCU) Observing you all the time.
- 40. High Cost, Episodic, Intrusive (HEI) Doctors use for better and precise diagnostics.
- 41. General and Personal Health State Space General Health State Space Personal Health State Space
- 42. Heart Failure Olympic Athlete Coronary Disease High Blood Pressure Good Hiker Healthy Pregnancy Health State: Multidimensional Space Health state is what matters, disease is just a subset of it.
- 43. Person :== Genome + Lifestyle + Exposome ‘Omics’ Personal Model Exposome Personicle AI + Learning • Propensity to diseases • Rule Based Health Behavior • Your behavior and health profile Could be for Whole Life Recent Past After a particular health Condition (such as heart attack)
- 44. From data streams to Personal Chronicle (Personicle)
- 45. Add other personal streams as NEEDED. RAMESH JAIN@Imaging and Multimedia Analytics 45Jan 29, 2020
- 46. Personicle: Personal Chronicle 46 Personicle is a complete chronological log of person- centered events in life (health, social, environmental) captured objectively but augmented using subjective and diagnostic information as needed. Continuous Observations Episodic Observations Personicle Person • All continuous sensors work 24/7. • Episodic data when suggested by • Either continuous data tracking or • By doctors.
- 47. Open Source Personicle 47 • Being built by Clearsense – Thanks, Charles and the team!
- 48. Pattern Mining High-level Pattern Formulation Pattern Query … Data Streams Event Streams Semi-interval Event Sequences Data-Driven Analysis Hypothesis-Driven Analysis While air pressure is high, pollution starts increasing gradually, within T time units asthma outbreak happens. ((pollution_inc_steadily ;ωT asthma_outbreak ) || airpressure_stayshigh) Example: Interactive Visualization Interactive Event Mining: Correlation and Causality Bottom Up Top Down
- 49. Input to the System • Food • Exercise/activities • Sleep • Stress • Environment • Medicine 49
- 50. Food is the most important input. 50What I like to eat is not what my body likes!
- 51. Food Recommendation •Who: Person. •What: Food Items. •When: Context. Food Recommendation System Food Items Context Personal Model List and volumes of Food Items and How to get them.
- 52. Food Logging is important application. 52 Information in a Food Log for Personal Food Model Learn from Activity and sleep Logging. Can we develop a platform for easier logging?
- 53. Building Food Model: Health 53
- 54. Two Challenges to Solve the Problem! • Creating a detailed Food Chronicle for a person. • Creating a detailed World Food Atlas.
- 55. Event Mining for Rule-Based Actionable Insights Having difficulty sleeping well on several nights. Warn him when he is at a Sichuan restaurant. Personicle Food Stream Sleep Spicy food and 2 glasses of wine result in sleepless nights.
- 56. Homeostasis is essential for Life. • Sensing for health state is multimodal. • Multimedia community is expert in information extraction from multimodal data streams. • Augmented Homeostasis is a new grand challenge for our community. 56
- 57. My dream: Empowering Personal Health For every living person in the world. Multimodal Computing will bring it. We can make it happen. Should we start? Now?
- 58. Thanks for your attention and Time Time for Questions and Discussions Continue after the session: jain@ics.uci.edu 58