Brain, Body, and Bytes CHI 2010 Workshop Presentations
9 likes3,452 views
The 'Brain, Body, and Bytes' workshop at CHI 2010 aimed to foster collaboration between physiological computing and brain-computer interface disciplines to explore new HCI methodologies and identify key research questions. The workshop included presentations from various participants on topics like psychophysiological responses, application in games, and methods for using biosignals. Participants engaged in discussions and networking to enhance their understanding and application of these technologies in human-computer interaction.
Brain, Body, and Bytes CHI 2010 Workshop Presentations
- 2. BBB Workshop Overview Audrey Girouard, Erin Treacy Solovey, Regan Mandryk, Desney Tan, Lennart Nacke, Robert J.K. Jacob
- 3. BBB Workshop: Bringing Together BCI& Physiological Computing Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 3
- 4. Workshop Goals 1. Provide a platform for creating synergies between two related and emerging HCI disciplines (PC and BCI). 2. Kick-start BBB research in HCI by identifying key research questions and application areas 3. Enhancing HCI methodologies by adding these new methods and techniques to the toolbox of HCI researchers Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 4
- 5. About Us Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 5
- 6. About you • 25 Participants • X EEG papers • Y fNIRS papers • Z EMG papers • 9 Brain papers • A GSR • 19 Body papers • B Eye tracking Brain Body 32% 68% Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 6
- 7. Workshop Discussion Questions <on your tables> Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 7
- 8. Workshop Discussion Questions What terms/acronyms should we use to describe our work? Psychophysiological computing seems a bit long… BBB? BBI? Other suggestions? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 8
- 9. Today’s Schedule 09:00-10:30 Introduction of organizers & participants, CHI madness style 10:30-11:00 Break 11:00-12:30 Group Discussion (assign groups + topic) 12:30-14:00 Lunch 14:00-15:30 Small group discussion (1h of discussion + 30 minutes of reporting) 15:30-16:00 Break 16:00-17:00 Report from small groups, recap, group discussion – barriers to publication in HCI Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 9
- 10. Staying connected Website http://www.eecs.tufts.edu/~agirou01/workshop/ Twitter http://twitter.com/bbbCHI2010 Twitter #bbbCHI2010 Hashtag Facebook http://www.facebook.com/home.php#/pages/Atlanta-GA/Brain-Body-and- Bytes-Psychophysiological-User-Interaction-at-CHI-2010/136581171786 Flickr http://www.flickr.com/groups/bbbchi/ Slide Share http://www.slideshare.net/group/brain-body-and-bytes-psychophysiological- user-interaction Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 10
- 11. And now… 2 Minute Introductions • About your research: – Application or domain space? (e.g. video games) – Signal you are measuring? (e.g. fNIRS) – Device/hardware you are using? (e.g. ISS, Inc) • What would you like to get out of workshop? – What problems are you facing? – What do you see in common with others here? – What device would you like to know more about? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 11
- 12. List of Presentations 1. Adriane Randolph 16. Katarzyna Wac 2. Krzysztof Z. Gajos 17. Feng Tian 3. Saraswathi Bellur 18. Veronica Zammitto 4. Danny Plass-Oude Bos 19. Johanna Octavia 5. Kai Kuikkaniemi 20. Bob Wray 6. Mitchel Benovoy 21. Marjolein van der Zwaag 7. Yee Chieh Chew 8. Martha Crosby 9. Jan Kallenbach 10. Eva Oliveira 11. Lucia Filgueiras 12. Marc Grootjen 13. Melody Moore Jackson 14. Johann Schrammel 15. Daniel Sjölie Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 12
- 13. Brain-Computer Interfaces as Child’s Play: Using Mental Training to Improve Motor Skills in Children Adriane B. Randolph, Ph.D. Ashley Ingraham Kennesaw State University
- 14. Outline • Research Position – Examine ties between sensorimotor area and physical ability to uncover a training tool to aid children’s motor skill development • Pilot Study – 3 able-bodied adults (2 Male, 1 Female) – Acquire baseline Wii tennis score – Conduct mental training with BCI2000 mu training protocol for 5 days – Retest Wii tennis score for improvements Participant Session One Session Five – Mixed results AZ 98 60 CM 70 64 MW 103 118 • Future Work – Extend study with 20 able-bodied adults (currently in progress) – Analyze link between training and baseline ability – Test with elementary-aged children Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 14
- 15. Screening for Mu Ashley prepares participant to use mental imagery of hand and foot movements to determine his best channels for use in mu BCI training before he plays a match of Wii Tennis. Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 15
- 16. Thank you Adriane B. Randolph, Ph.D. Kennesaw State University BrainLab (770) 423-6083 brainlab@kennesaw.edu http://coles.kennesaw.edu/brainlab/ Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 16
- 17. Understanding How to Design Complex Brain-Controlled Applications Krzysztof Z. Gajos
- 18. How to Design Brain-Controlled Applications? • Basic operations Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 18
- 19. How to Design Brain-Controlled Applications? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 19
- 20. How to Design Brain-Controlled Applications? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 20
- 21. How to Design Brain-Controlled Applications? • Basic operations Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 21
- 22. How to Design Brain-Controlled Applications? • Interaction between primary task and BCI performance Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 22
- 23. How to Design Brain-Controlled Applications? • Error prevention, recovery, and actions with side effects Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 23
- 24. Thank you Krzysztof Gajos kgajos@eecs.harvard.edu http://www.eecs.harvard.edu/~kgajos/ Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 24
- 25. Psychophysiological Responses to Media Interfaces Saraswathi Bellur S. Shyam Sundar Media Effects Research Lab http://www.psu.edu/dept/medialab Pennsylvania State University
- 26. Biopac MP 35 & MP 150 ACQknowledge software Skin Conductance (EDA) Heart Rate (ECG) Facial EMG Brain Wave (EEG) Eye Movement (EOG) Application Areas Interface presentation Information Processing Usability Testing & Evaluation User Experience Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 26
- 27. Physiological Measures: Affect, Arousal & Attention 3. ECG: Heart Rate recording 1. EEG: Bipolar recordings Occipital Alpha Blocking Orienting Response via BPM change 2. Skin Conductance (EDA) Tonic arousal levels 4. Facial EMG: Corrugator & Zygomatic recordings Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 27
- 28. Theoretical Frameworks • The Media Equation: Mediated Life = Real Life (Reeves & Nass, 1996). • Structural aspects of media technologies: Automatic or Controlled processing? Voluntary or Involuntary responses? • Specific cues triggered via interactions with four types of media affordances, MAIN model (Sundar, 2008) Animation M: MODALITY Customization Heuristics A: AGENCY Information Perceptual Scent Bandwidth I: INTERACTIVITY N: NAVIGABILITY Contingency UGC & Sourceness Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 28
- 29. Thank you Saras Bellur (saras@psu.edu) S. Shyam Sundar (sss12@psu.edu) Media Effects Research Lab http://www.psu.edu/dept/medialab Pennsylvania State University Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 29
- 30. BCI for Games Danny Plass-Oude Bos Boris Reuderink, Bram van de Laar, Hayrettin Gürkök, Christian Mühl, Mannes Poel, Anton Nijholt, Dirk Heylen HMI, University of Twente
- 31. Target Group: the general population Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 31
- 32. BCI: issues Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 32
- 33. BCI: new information, new features Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 33
- 34. Acceptance: Usability & Realistic Settings Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 34
- 35. Subjects Users Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 35
- 36. Thank you d.plass@ewi.utwente.nl http://hmi.ewi.utwente.nl/~oudebos Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 36
- 37. Designing Biofeedback for Games and Playful Applications Kai Kuikkaniemi Helsinki Institute for Information Technology
- 38. Emoshooter Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 38
- 39. Emoshigeru - cinema games Real-time Public display Biosignal adaptation Massive co-located Shooter / Strategy Part of Shigeru-games local wlan Public display Connected to game server Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
- 40. Presemo – presentation engine Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
- 41. Emolisten – biosignal sonification Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
- 42. Research statement • Social Games and Playful Applications, which Utilize Biosignals • Grey Area between Adaptation and Biofeedback – Implicit and Explicit Biofeedback • Biosignal Synchrony (Inter-Personal) Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 42
- 43. Our team and projects Professor Marko Turpeinen Professor Timo Saari Ilkka Kosunen, Toni Laitinen, Petri Lievonen, Pauli Ojala Fun of Gaming (EU FP6) (2006-2009) Enactive Social Media and Games (Aalto) (2009) Emokeitai (Tekes – National Innovation Agency) (2009-2011) Thanks: CKIR / Niklas Ravaja Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 43
- 44. Psychophysiological signal analysis and classification Mitchel Benovoy Centre for Intelligent Machine, McGill University
- 45. Outline • Signals • Hardware • Application domains Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 45
- 46. Signals Galvanic Skin Response Phalange Temperature 1.5 ∆ Cond. (µS) 1 0.5 0 -0.5 0.5 1 1.5 2 2.5 Sample 4 x 10 Electromyogram 42 Blood Volume Pulse – HR & Amp. Respiration – Rate & Amp. 40 2.2 2 38 Refl. Exp. 1.8 36 1.6 34 1.4 32 1.2 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 7 1 0.8 Amp. Amp. 6 0.6 0.4 5 0.2 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 105 30 HR (bpm) RR (bpm) 100 25 95 20 90 15 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 46
- 47. Hardware • Thought Technology ProCom Infinity Images via www.thoughttechnology.com Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 47
- 48. Application Domains • Affective Computing – Interaction evaluation (UI, devices, latency) – Music & emotion research – Psychoneurology Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 48
- 49. Thank you benovoym@cim.mcgill.ca Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 49
- 50. BCI and Creativity Yee Chieh (Denise) Chew Georgia Institute of Technology
- 51. Outline • fNIR and Creative Expression • EEG and Music Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 51
- 52. Character Templates for fNIR Created with a mouse, these are templates that subjects traced using fNIR Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 52
- 53. Thank you ychew@gatech.edu Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 53
- 54. Physiological Measures Used for Identification of Cognitive States and Continuous Authentication Curtis S. Ikehara & Martha E. Crosby University of Hawaii at Manoa
- 55. Outline • Overview of our Research Using Physiological sensors – Assessing Cognitive States – Continuous Authentication • Possible Applications Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 55
- 56. Multi-modal Physiological Sensors System Eye Tracker (Gaze Position, Fixation Number, Fixation Duration, Repeat Fixations, Search Patterns, MTF Pupil Size, Blink Rate, Blink Duration) (Moving Target Fractions) HUMAN Pressure Mouse (Pressures on Mouse Button and Body) Laptop (Data Collection, Analysis & Display) EDA / TSA Finger Sensors (Electrodermal and Temperature Sensor Activity) Oximeter (Pulse, Blood Oxygen) Custom Electronics (Analog to Digital Converter) Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 56 56
- 57. A Composite of Over 20 Mouse Clicks of Three People (Click Signature) US Patent # 7,245,218 Subject 4 Subject 5 Subject 7 57 Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
- 58. Possible Applications • The student’s confidence can be assessed by evaluating the distortions of the mouse click signature. • High Value Vehicle Control – Pressure and other physiological sensors can be attached to a steering (e.g. aircraft yoke or steering wheel) . – Sensors can be used to indicate that: • The authorized pilot or driver is in control. • The cognitive state of the pilot is consistent with the vehicle’s activity. Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 58
- 59. Thank you Curtis S. Ikehara Martha E. Crosby cikehara@hawaii.edu crosby@hawaii.edu University of Hawaii at Manoa 1680 East-West Road - POST 317 Honolulu, Hawaii 96822 USA Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 59
- 60. Measuring Interaction Experiences: Integration of Multiple Psychophysiological Methods Jan Kallenbach Aalto University Helsinki, Finland
- 61. Outline User Experience = Choice & Interaction + Cost-Benefit Evaluations Measuring Interaction Experiences? Software Framework Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 61
- 62. Multimodal Measurements Client Application for UX Research Sampling frequency: max 1000Hz Psychophysiological Measures: Gaze, Pupil Size, EDA, EMG, EEG, … Behavioral Measures: Keyboard, Mouse, Slider, … System Measures: Window Events Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 62
- 63. Thank you Jan Kallenbach Research Scholar Aalto University School of Science & Technology Department of Media Technology P.O. Box 15500 FIN - 00076 Aalto Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 63
- 64. Towards enhanced video access and recommendation through emotions Eva Oliveira Nuno Ribeiro Teresa Chambel IPCA | LASIGE
- 65. We Want • To access to videos based on emotions – user actual emotional (psychophysiological) state – video emotional content – director’s perspective • To recommend videos based on emotions – create a set of rules from every affective classification perspective (user and video, director) Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 65
- 66. Affective video recommendation system requirements () • Every video must be classified affectively from the user and from the content perspective. • Every user has an emotional profile constituted by all his implicit (physiological) classification of every classified movie. • User physiological signals can change the way we watch the videos Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 66
- 67. User Classification Issues • Automatic Classification – Users emotional state • emotional model – Classification techniques • Result output • Accuracy • Lab contexts Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 67
- 68. Current State • Capturing users phychophysiological signals (BIOPAC System) while watching movies This is my exciting picture • Next step: several classification techniques to convert signal result into emotional categories Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 68
- 69. Thank you • Eva Oliveira : • eva.oliveira@gmail.com Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 69
- 70. Your users can’t talk! How can you tell they liked your system? Lucia Filgueiras and Claudia Tambascia Escola Politecnica Universidade de São Paulo
- 71. What we want to do • Develop products for cerebral palsy users • Perform participatory design • Understand non-verbal communication Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 71
- 72. Direct measures and usability evaluation HRV and workload measures Eye tracking in iTV apps in control room operability evaluation evaluation Comparison of eye tracker NASA TLX, SWAT, Heart Rate findings to conventional Variability usability tests results Variabilidade da frequência cardíaca OP13 LF nu / HF nu LSC Média 18,0 Operador analisa sistema após vários Índice do Balanço Simpático? Parassimpático Ação do operador para alarmes: temperatura alta (10:29); Ação do operador para 16,0 correção nível alto (10:32) e segurança correção 14,0 (10:32; 10:37; 10:43) 14,1 12,0 Alarme de 11,7 temperarura alta: 10,0 10:15 9,1 8,3 8,5 8,0 7,3 6,6 6,0 5,9 5,2 5,3 5,6 4,6 5,0 4,0 3,9 3,8 4,0 4,4 4,1 2,9 3,3 2,6 2,6 2,6 2,3 2,4 2,0 0,0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 :3 :4 :4 :5 :5 :0 :0 :1 :1 :2 :2 :3 :3 :4 :4 :5 :5 :0 :0 :1 :1 :2 :2 :3 :3 09 09 09 09 09 10 10 10 10 10 10 10 10 10 10 10 10 11 11 11 11 11 11 11 11 HORA Heart Rate Variability of an operator in a unstable unit situation Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
- 73. Our questions: • How personalized are body expressions for cerebral palsy patients? • Are there general body expressions that can mean the same for everyone? • Is it possible to determine a generic way of measuring these expressions? • Is there any other liaison besides gestures and facial expressions , like an unconscious connection of limbic systems that can be responsible for the mutual sensitive communication? • Is there a reference body signal that can be used as a reliable parameter to evaluate body expressions? • How can we establish a correlation between body expressions and patient´s opinion of approval or disapproval of a design solution? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 73
- 74. Thank you! Contact us: Lucia V. L. Filgueiras Escola Politecnica, Universidade de São Paulo lucia.filgueiras@poli.usp.br Claudia A.Tambascia Fundação CPqD claudiat@cpqd.com.br http://lts-i.pcs.usp.br Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 74
- 75. Human Computer Symbiosis by Mutual Understanding Marc Grootjen EagleScience-TNO
- 76. Outline • Symbiosis & mutual understanding • Information models • Interfaces Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 76
- 77. Maximum task performance Symbiotic entity Disturbance Mutual human understanding computer Tasks (goals) Human-computer symbiosis with mutual understanding as key issue for maximum task performance Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 77
- 78. Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 78
- 79. Domesticating Brain- Computer Interfaces: Systems for the Home Melody Moore Jackson BrainLab School of Interactive Computing Georgia Tech
- 80. Extended Home-based BCI study • fNIR-based BCI for communication • 32 People with ALS (so far) over 1 year • 78% (25) could operate the device on initial visit with accuracy > 70% • Accuracy to 100%, average 76.5% • Issues: – Motivation – Data Collection – Remote Support Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 80
- 81. The Kokoro Gatari Brain Computer Interface based on Functional Near Infrared imaging Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 81
- 82. Thank you Melody Moore Jackson melody@cc.gatech.edu 404-277-1324 Ian McClendon imclend@cc.gatech.edu 404-641-7712 Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 82
- 83. Get up, move on! Using Electromyography to Explore the Relationship of Experience and Motion Johann Schrammel Centre for Usability Research and Engineering
- 84. Outline • Compare experiences and emotions in conditions with and without movements with Electromyography(EMG) and other methods Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 84
- 85. Outline • Can EMG successfully be used to estimate a user‘s emotional experience in motion contexts (artefacts, feasibility, etc.)? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 85
- 86. Outline • Is a more embodied way of controlling interaction actually amplifying experience? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 86
- 87. Results • EMG is suitable for motion contexts. • Both EMG and self-report data indicate that natural body movement intensifies emotions. • It’s crucial that body movements are in accordance with activity. Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 87
- 88. Thank you schrammel@cure.at Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 88
- 89. Reality-Based Brain- Computer Interaction Daniel Sjölie Umeå University, Umeå, Sweden
- 90. Reality-Based Brain-Computer Interaction • Complex, reality-based, interaction with the ability to adapt to the user in real-time – Using brain measurements to evaluate • Direct brain-computer interaction – The user can be unaware • We’re using a combination of virtual reality (VR) and functional MRI (fMRI) Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 90
- 91. HCI and cognition • An improved understanding of the brain in this interactive and complex context is needed • Build on theories of cognition as a foundation for the development of advanced HCI – As interaction moves towards being reality-based we move closer to “interacting with reality”, and thus to general cognition • Cognitive neuroscience and computational models of the brain are important tools Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 91
- 92. Working with fMRI and Virtual Reality Working with fMRI requires specialist competence and forces us to deal with powerful magnetic fields and restricted mobility, but the ability to measure the whole brain is an invaluable research tool. Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 92
- 93. Thank you Daniel Sjölie, daniel@cs.umu.se Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 93
- 94. emoBAN: Improving Quality of Life via Psychophysiological Mobile Computing Katarzyna Wac, Anind K. Dey Carnegie Mellon University, HCII
- 95. emoBAN idea • “what always speaks silently is the body” (Brown, 1990) – measure QoL via psychophysiology Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 95
- 96. emoBAN: Usable Useful Functional • Body Area Network – sensors • vital signs, e.g., ECG, temp, resp • context, e.g., location, time, activity – actuators • feedback via, e.g., AV, tactile, light Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 96
- 97. Thank you Dr Katarzyna Wac Carnegie Mellon University Human-Computer Interaction Institute katewac@cmu.cs.edu www.cui.unige.ch/~wac Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 97
- 98. Leveraging Psychophysical Data in Monitoring and Analyzing the States of Badminton Players Feng Tian, Wencan Luo, Jing Dai, Reza Naima, Yongmin Cheng, Hongan Wang, Guozhong Dai Institute of Software, Chinese Academy of Sciences Physical Education Collage, Beijing Sport University Berkeley Institute of Design, UC. Berkeley China Badminton Team, General Administration of Sport of China 4/20/2010
- 99. Traces and physical actions - Camera, accelerometers Psychophysical data - Berkeley Tricorder ECG, EMG, Pulse Oximetry, BioImpedance Respiration Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
- 100. Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
- 101. Exploring Quantitative Methods for Evaluating Sports Games Veronica Zammitto Simon Fraser University Magy Seif El-Nasr Simon Fraser University Paul Newton Electronic Arts Canada
- 102. Current Study: NBA Live 10 • Eye tracking – Eye Tech TM3 • Psycho-physiological data – ProComp5 Infiniti – EMG (corrugator supercilii and zygomatic major), EKG (heart rate), GSR and BVP • Gameplay Telemetry Data – event-based RPC telemetry collection system • Triangulation Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 102
- 103. What was the user the Data Triangulation - NBA Live 10 emotional response? looking at? Game user experience Event, when Was positive? Where user’s Was negative? attention was? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 103
- 104. Data Triangulation - NBA Live 10 Game user experience Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 104
- 105. Thank you Veronica Zammitto vzammitt@sfu.ca Magy Seif El-Nasr magy@sfu.ca Paul Newton pnewton@ea.com Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 105
- 106. Exploring Psycho-physiological Measures for the Design and Behavior of Socially-Aware Avatars in Ubicomp Environments Johanna Renny Octavia Hasselt University, Belgium
- 107. Avatars in Ubicomp Environment User’s representation Mediator between system and user (companion) To design a socially-aware avatar that adapts its behavior according to user’s emotion Any incorrect actions of the avatar can lead to loss of identification of the user Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 107
- 108. Our informal experiment ProComp Infiniti device, measured GSR, EMG and BVP Higher physiological values when playing against a friend How different social relationships between players influence those values? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 108
- 109. Thank you Expertise Centre for Digital Media (EDM) Web: www.edm.uhasselt.be Email: johanna.octavia@uhasselt.be Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 109
- 110. Tailoring Practice for Individualized Learning via Cognitive State Bob Wray Soar Technology, Inc.
- 111. Why I need Psychophysiological Computing • Practice environments are becoming common-place • Adaptive tailoring of practice is important for individualized experience/learning • Need more than traditional adaptations (e.g., scaffolding) – Adapting to engage learner / maintain interest – Adaptation to reducing “gaming” in environment – Adaptation to elicit learner affect/arousal (“real”) Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 111
- 112. Research Testbed Narrative, conversational, inter-cultural training application Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 112
- 113. Thank you Bob Wray Soar Technology wray@soartech.com, 919.967.5079 Brian Magerko Georgia Institute of Technology magerko@git.edu Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 113
- 114. Guidelines to biosignal driven HCI Marjolein van der Zwaag Philips Research Europe, University of Groningen
- 115. Guidelines to biosignal driven HCI Marjolein van der Zwaag Egon L. van den Broek Joris H. Janssen
- 116. Outline • Signal(s) measured: psycho-physiological • Application domain: Affective computing • Device used: Nexus 10, mind media • Guidelines for processing biosignals – Triangulation – Identification of users – Biosignal characteristics – Temporal construction Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 116
- 117. Unobtrusive affective computing Music direct your mood concept Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 117
- 118. Thank you Marjolein.van.der.zwaag@philips.com Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 118
- 120. Workshop Discussion Questions What is brain/body computing good for? How can we think systematically about this space? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 120
- 121. Workshop Discussion Questions What are the major themes of HCI that can be explored using brain/body interaction? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 121
- 122. Workshop Discussion Questions What specific brain & body sensing modalities are interesting and how can we use them? What are most innovative technologies in this space? How do these compare with existing practices (NASA-TLX, Likert-style questionnaires, etc)? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 122
- 123. Workshop Discussion Questions How can BCI and PC be defined within an HCI context? How is a HCI context different from neuropsychology, biomedicine or biomedical engineering? How does HCI contribute to psychophysiological computing (or more specifically neural engineering, bio- engineering, etc)? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 123
- 124. Workshop Discussion Questions If we could measure anything on brain and body, what would we want to measure and why? What cognitive states and emotions have been investigated with sensors? What methodologies are best for what signals? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 124
- 125. Workshop Discussion Questions How can accurate physiological, psychological and neurophysiological measurements improve HCI? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 125
- 126. Workshop Discussion Questions With what software techniques and hardware technologies could these signals be optimally processed? How can software account for the combination of user actions taken to accomplish a task versus actions that would surprise the user, yet optimize their experience? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 126
- 127. Workshop Discussion Questions What are the issues that must be considered when using brain/body computing for direct control, passive sensing, adaptation, etc? Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 127
- 128. Groups • How to go from raw data to something else – Emotions: TABLE 1 – Data Fusion: TABLE 4 • How to use these things in interactive interfaces: TABLE 3 Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 128
- 129. BBB Workshop: Bringing Together BCI& Physiological Computing Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 129
- 130. Staying connected Website http://www.eecs.tufts.edu/~agirou01/workshop/ Twitter http://twitter.com/bbbCHI2010 Twitter #bbbCHI2010 Hashtag Facebook http://www.facebook.com/home.php#/pages/Atlanta-GA/Brain-Body-and- Bytes-Psychophysiological-User-Interaction-at-CHI-2010/136581171786 Flickr http://www.flickr.com/groups/bbbchi/ Grouplist http://groups.google.com/group/brain-body-interfaces/ Slide Share http://www.slideshare.net/group/brain-body-and-bytes-psychophysiological- user-interaction Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 130
- 131. BBB at CHI2010 Brains and Brawn session: Tuesday 11:30-1pm Making Muscle-Computer Interfaces More Practical T. Scott Saponas, Desney S. Tan, Dan Morris, Jim Turner, James A. Landay University of Washington, USA Microsoft Corporation, USA A Novel Brain-Computer Interface Using a Multi-Touch Surface Beste F. Yuksel, Michael Donnerer, James Tompkin, Anthony Steed. University College London, UK The Influence of Implicit and Explicit Biofeedback in First-Person Shooter Games Kai Kuikkaniemi, Toni Laitinen, Marko Turpeinen, Timo Saari, Ilkka Kosunen, Niklas Ravaja, Helsinki Institute for Information Technology, Finland Center for Knowledge and Innovation Research, Finland Effects of Interactivity and 3D-motion on Mental Rotation Brain Activity in an Immersive Virtual Environment Daniel Sjölie, Kenneth Bodin, Eva Elgh, Johan Eriksson, Lars-Erik Janlert, Lars Nyberg Umeå University, Sweden Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 131
- 132. THANK YOU • Keep in touch! Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 132