Cognitive Computing for Aging Society
- 1. Eldercare Research Cognitive Computing for Aging Society Hiro Takagi STSM, IBM Research - Tokyo Aging Strategic Initiative lead, IBM Research Human Computer Interaction area lead, IBM Research Academy of Technology member
- 2. 62 87 28% 27% Dementia 15% + MCI 13% estimation by Japanese Government, 2012 of people over the age of 65 are at risk of dementia. People over the age of 65, 2017. 70%of people want to work over the age of 70 (and forever). Life expectancy of women in 1950 and 2015. * numbers in Japan
- 3. 2017
- 4. We need a new model for active life-long society.
- 5. © 2016 IBM Corporation 2003 “Elderly Vision Simulator” 2007 Eclipse.org Open source 2011 Official tool for JIS (Japanese Industrial Standard) 2011 Japanese Government Science Grant “IT for Active Aging Society” 2009 Open Collaborative Research University of Tokyo “Elderly Accessibility” 2013 Japanese Government Pilot Grant “Remote community collaboration” 2015 Sekisui House JDA 2017 Japan Post Official Service “Conversation as Sensors” GTO 2016 Demographic Shift 2015 Japan Post Announcement 2015 – 2016 Japan Post Pilot “Voice QA” 2005 2010 2015 2016 2017 Tokyo Research Aging Research History Research Aging Strategic Initiative2014 KDDI “Senior Smartphone” Government and University projects Customer projects Internal activities
- 6. Power of IT – Five Long-term Focus Areas Social Intelligent assist Sensing Accessibility Brain
- 8. Use case barrier Psychological barrier User interface barrier Price Barrier Small fonts, small buttons Don’t know how to operate Fear, uncertainty Cannot imagine use cases Barriers for Senior Citizens to Start Using Information Technologies
- 9. Target Actual (average) Fingertip angle + Parallax Landed on Took off (px) (px) Touch duration + Tremor Unintentional Motion Accessibility: Touching a target location
- 10. Gesture Analysis for Novice Elderly Assist (2014) Press release & images (Japanese only) IBM Japan Press Release: http://bit.ly/1gavLuN KDDI R&D Labs Press Release: http://bit.ly/1hK1B2K KDDI Corporation Press Release: http://bit.ly/1f5zbTx Provided appropriate guidance to users based on user's level of smartphone operation proficiency (skill and operational misstep) by analyzing smartphone operation logs with mathematical analysis technology
- 11. Improved confidence and subjective operation speed 33 0 3 -- 7 0 -6 067 ,3 -- 7 6 067 ,3 2 7 - 2 1 172 17 You entered “one-hundred thousand” yen Accessibility: Voice-over Effectiveness
- 13. Elderly Skill Matching System – J-scouter Senior profile datasets (Free-form text) Job / Activity Descriptions (Free-form text) Find best persons Find best jobs/activities A Prime minister initiative “Work-style transformation.” “Senior workforce” is one of focuses. A senior worker with rich experience in legal compliance is working as an advisor in a start up company Supported by Japan Science and Technology Agency Senior HR company 10,000 seniors registered from TV BS11 By Kyodo News
- 14. “Reimagining Life” “Target 5 million users.” http://www-03.ibm.com/press/us/en/pressrelease/46740.wss 2015
- 15. Japan Post Project • Pilot with 1000 elderlies completed (Oct. 2015 – Oct. 2016) • Medicine reminders, video phones, shopping assist, photo sharing, etc. Video
- 16. Global Apple+IBM Apps for Elder Services Support Worker Elder at Home Family / Loved Ones
- 18. ) ( ) ( - - (- ( ( ) ) - ( - - - ( Understand ambiguous questions Pick up appropriate answers Learn user feedback - -
- 19. Video
- 21. IBM Accessibility Research IoT, Health and Aging Ecosystem Partners / Enriched Data Watson IoT & Health Platform Sensors & Devices Client’s Data Personality Insights & Social Interaction Data Other Data Sources Applications Data & Platforms 01 0110 0010 001001 Cognitive Services for Elders Empowered Living Empowered Social Empowered Care Knowledge Reactor “Infrastructure Providers”
- 23. • Voice-base natural user interface for daily watchover • Analyze life patterns, feelings, interests and issues from daily conversation (e.g. cognitive decline detection) • Share information with families and care givers • Provide support that can keep the elderly self-sufficient Conversation as Sensors
- 25. Intent of Each Questionnaire Questionnaire Intent “What did you have last night?” Dementia (Neuro Cognitive Disease) Risk Assessment “Did you go out today?” Activity checking “You mentioned about gardening in diary. Are you interested in community gardening?” Recommendation of activities ”Which city do you want to travel?” Marketing “The shop, you mentioned last week, is now on sale!” Promotion “We delivered a package, but you were not there. Please contact post office" Notification (business related) 25
- 26. Emotion and Physical Condition Recognition Voice data Features 988 acoustic features for emotion recognition. Intensity Loudness 12 MFCC Pitch (F0) Probability of voicing, F0 envelope, 8 LSF, Zero-Crossing Rate Labeled voice dataset ML-based Regression Estimation Feeling good or bad 40 0 100 Physical condition Ex: Good night *Each circle represents one greeting. *One participant data Emotion NegativePositive Confusion FrustrationDelight Flow Physical Condition Emotion Yorktown – Speech team Tokyo Research Happy Sad
- 27. Dashboard for Family and Care Givers 27 Condition Emotion (voice sound analysis) Personality analysis (Watson personality insights) Daily activities Eating habits Dementia questionnaire scores
- 28. Dashboard Visualization Examples 28 “what did you do today?” = Variety of daily activities. practice, songs, poems, musical instruments, sessions, friends, music hall, …. House (largest), cleaning, washing, room, study….. Smaller variety and words are not so active.Wider variety, and words are active. “what did you have today?” = Variety of nutrition Wider variety, and healthy meals Vegetables, salad, radish, juice, natto, miso, eggs, …. Rice (largest), bento, sandwich, …. Smaller variety and ready-made meals
- 29. Feature extraction of gait • Gait speed & its variability • Step frequency • Stride time variability • Step-length & its variability • Foot swing velocity • Stance and stride time Detection • Motor function • Fall risks • Cognitive decline • Episodic memory • Executive function • Diseases • Parkinson disease • MCI • Alzheimer’s disease Gait and Cognitive Decline
- 30. Video
- 32. Brain Simulation: Understanding Cognitive Deficit (2016) Video
- 33. Power of IT – Five Long-term Focus Areas Social Intelligent assist Sensing Accessibility Brain