Analyzing gaming-simulations using video based techniques
The document discusses analyzing group dynamics and interactions during gaming simulations using video-based techniques. It describes an energy policy simulation game involving 3 small groups representing different energy departments. The game aims to create awareness and abstract reality using game elements to maximize learning. Video recordings of the game would allow quantitative and qualitative analysis of group behaviors like discussions, negotiations, and influences between individuals and groups. The document proposes using technologies like audio-visual sensors, signal processing, and machine learning to automatically track behaviors from multimodal data and provide summarized perceptions and feedback on interactions like speaking patterns, eye gaze, and gestures.
![+
Behavior Tracking and Perception
[Human Human Interaction]
Behavior Perception
Cue Summarization
Multimodal
Signal
Processing
Speaking,
Looking
Gesturing
style
Who speaks,
Looks at
Whom, who nods
Feedback/Analysis
Machine
Learning
13
Sensing
P(Perception/Behavior)
Behavior Tracking
VIDEO](https://image.slidesharecdn.com/dineshseriousgames-teem2014-141014073054-conversion-gate01/85/Analyzing-gaming-simulations-using-video-based-techniques-13-320.jpg)
![+
Conclusion
n Serious game [Face-to-face] design
n Need for automatic observation (summarization and
querying)
n Automatic behavior analysis
n Speech – who speaks when to whom, interruption,
negotiation, word usage, dialog acts (question, answer)
n Video – who looks at whom, who gestures what
n Challenges: Canceling speech interference from other
groups, occlusion.
16](https://image.slidesharecdn.com/dineshseriousgames-teem2014-141014073054-conversion-gate01/85/Analyzing-gaming-simulations-using-video-based-techniques-16-320.jpg)
Analyzing gaming-simulations using video based techniques
- 1. + Analyzing gaming simulations using video based techniques Dinesh Babu Jayagopi, Assistant Professor, IIIT Bangalore Co authors: Bharath and Harsha, Fields of View, India TEEM 2014, Gamification Ecosystems, 2 Oct 2014
- 2. + Energy game: a sample session 2 Goal serious Game designer: • Use game elements to maximize learning • Create awareness • Abstract reality Goal of the group: Energy policy for the country Observe 3 small groups (representing departments – coal, atomic, renewable) Conflict/ Cooperation among groups Conflict/ Cooperation across groups
- 3. + Role of game observation 3
- 4. + Quantitative data 4
- 5. + Qualitative data: game observation 5 n Group 1 (Dept of Atomic Energy) had heavy discussions n Individual 2 in Group 3 was negotiating with Group 2 and was quite influential n Individual 3 in Group 2 was dominating and disturbing Group 1 Group 2 Group 3 Behavior Analysis of Group dynamics
- 6. + Behavior (Multimodal / audio-visual) analysis of group interactions
- 7. + Human Communication: Several channels of info. 7
- 8. + Behavior Coding: extracting cues 8 Source: Adam Kendon Conducting Interaction
- 9. + Technological trends: Human-centered Sensing 9 Microcone – Array microphone Kinect – Image + Depth Camera Sociometer – Wearable badge Google glasses
- 10. + Technological trends: Human-centered Signal Processing 10
- 11. + Technological trends: Human-centered Signal Processing © CHIL 11
- 12. + Technological trends: Machine Learning/Probabilistic Formulations Label Observations Hidden Variable Observations 12
- 13. + Behavior Tracking and Perception [Human Human Interaction] Behavior Perception Cue Summarization Multimodal Signal Processing Speaking, Looking Gesturing style Who speaks, Looks at Whom, who nods Feedback/Analysis Machine Learning 13 Sensing P(Perception/Behavior) Behavior Tracking VIDEO
- 14. + Example: Gaze tracking VFOA: Visual Focus of Attention State space: Location, scale, eccentricity - continuous Pose – discrete 14 Observation: Texture (HOG) & color features
- 15. + 15 Example: Analysis HHI speaking activity inferred focus head pose tracking person ID side-view cameras VIDEO
- 16. + Conclusion n Serious game [Face-to-face] design n Need for automatic observation (summarization and querying) n Automatic behavior analysis n Speech – who speaks when to whom, interruption, negotiation, word usage, dialog acts (question, answer) n Video – who looks at whom, who gestures what n Challenges: Canceling speech interference from other groups, occlusion. 16