Bodily Interaction
                                  Lecture 1, 24.03.2011
                                       Ferhat Şen




Friday, 25 March 2011, Week
Course

                    • develop gesture-based Natural User
                              Interfaces (NUI)
                    • in various interactive application platforms
                              for sonic, musical, 2D or 3D interfaces




Friday, 25 March 2011, Week
Course In General
                    • the course is not to teach Processing, Pure
                              data or Unity
                    • to develop applications involving bodily
                              interaction by using these programming
                              environments and Kinect sensor.
                    • prerequisite is to know basics of one of
                              them. you will learn by doing more about
                              them

Friday, 25 March 2011, Week
Schedule
                •       24th Mar 2011 12-14: Introduction

                •       28th Mar 2011 13-15: Kinect with Processing (Joint session with Processing Club)

                •       31st Mar 2011 12-14: Kinect with Pure Data (4th floor Mac Classroom)

                        WS WEEK (Optional Sessions on Unity)

                •       11th Apr 2011 13-15: Skeletal Tracking with Processing and Pure Data

                •       14th Apr 2011 12-14: Small Project Presentation/Concept Idea Presentation/Group
                        Formation

                •       18th Apr 2011 13-15: Hands on Project Work/Tutoring

                •       21st Apr 2011 12-14: Hands on Project Work/Tutoring

                •       28st Apr 2011 12-14: Hands on Project Work/Tutoring

                •       2nd May 2011 13-15: Final Project Presentation (Group-work)

                        Paja is booked for Week 16-17-18 for you btwn 9-17. (Except 18.04.2011)


Friday, 25 March 2011, Week
Small Project
                    • Very simple project
                    • Done in Processing, PD, Unity or QC
                    • Some interaction with body data
                    • Individually
                    • Presented in the class


Friday, 25 March 2011, Week
Final Project
                    •
                                                                   Virtual Reality
                              Concept Development                  Augmented Reality
                                                                   Puppetry


                    •
                                                                   Multiuser interactive environment
                              Sonic/Musical, 2D, 3D applications   Augmented interactive dance performance
                                                                   Gestural musical instrument
                              (from image viewer to media art)     Architectural Projection



                    •         Groups of 2 people (strongly suggested)

                    •         Exhibition

                          •     Public space in TaiK

                          •     Venue outside (public space outside)

                          •     Spring Demoday 26.05

Friday, 25 March 2011, Week
Programming
                    • Processing?
                    • Pure Data?
                    • Unity?
                    • Any other?

                    • Level of knowledge? (0-5)

Friday, 25 March 2011, Week
User Interface
             the system by which
             users (people) interact (communicate) with a machine.




Friday, 25 March 2011, Week
Components of UI
                    •         The user interface includes

                          •     hardware (physical)

                          •     software (logical) components.

                    •         Provide a means of:

                          •     Input, allowing the users to manipulate a system,
                                and/or

                          •     Output, allowing the system to indicate the
                                effects of the users' manipulation.


Friday, 25 March 2011, Week
Interaction
                    •         indicates the means by which user inputs changes
                              to the system and the feedback supplied by the
                              system




Friday, 25 March 2011, Week
Interaction


                                      • How do you do?
                                      • How do you feel?
                                      • How do you know?

Friday, 25 March 2011, Week
Timeline of UI’s


                Command-line Interface




Friday, 25 March 2011, Week
Friday, 25 March 2011, Week
Timeline of UI’s


                Command-line Interface   Graphical User Interface




Friday, 25 March 2011, Week
WIMP Paradigm
                              WINDOW
                              ICON
                              MENU
                              POINTER




Friday, 25 March 2011, Week
Friday, 25 March 2011, Week
Timeline of UI’s

                                                                    Post WIMP Era
                                                                    Tangible User Interface
                                             WIMP                   Gesture-Based Interface
                Command-line Interface   Graphical User Interface
                                                                    Natural User Interface




Friday, 25 March 2011, Week
tangible




                                         gestural




Friday, 25 March 2011, Week
Natural Interaction
                    •         Experience (Human-Computer --> Human-Human )

                          •     People communicate thru

                               •   gestures

                               •   expressions

                               •   movements

                          •     People discover by

                               •   looking around

                               •   manipulating physical stuff


Friday, 25 March 2011, Week
Why?
                    • Less cognitive load
                    • Simpler (for certain applications)
                     • typing
                     • changing slide, navigating in 3D VR
                    • Direct manipulation


Friday, 25 March 2011, Week
Bodily Interaction
      • whole body in context
      • multi-modality
         • human=multi-sensory                 ?
      • user as an input modality
         • direct input from user’s sensory
                        modalities
      • manipulate digital data with body
Friday, 25 March 2011, Week
Bodily Input Modalities
                    •         Skeletal: Hand,Fingers, Head, Leg, Feet
                    •         Sonic: Speech,Voice characteristics (volume,
                              frequency), Body-made sounds
                    •         Biofeatures: Breath, Sweat, Heartbeat, Blood
                              Pressure
                    •         Touch
                    •         Facial muscular activation
                    •         Presence in space
                    •         Multiple bodies


Friday, 25 March 2011, Week
Put-that-There, 1979
                    •         voice and gesture at the graphics interface
                              pioneering multimodal application combined
                              speech and gesture recognition.

                    •         Put-that-There, MIT, 1979

                    •         Put-that-there, MIT 1983




                    •         Richard A. Bolt "Put-That-There":Voice and Gesture at the Graphics Interface (pdf) SIGGRAPH '80




Friday, 25 March 2011, Week
Kinect Sensor


                              IR Laser Projector    IR Camera




Friday, 25 March 2011, Week
How Does it Work




                                 Watch the Video


Friday, 25 March 2011, Week
Kinect’s IR-Grid




                                 Watch the Video
Friday, 25 March 2011, Week
Friday, 25 March 2011, Week
Friday, 25 March 2011, Week
Two Methods to Track
                    • Depth Filtering
                    • Skeletal Tracking




Friday, 25 March 2011, Week
Depth Filtering




Friday, 25 March 2011, Week
Skeletal Tracking




                                       Watch the Video



Friday, 25 March 2011, Week
Friday, 25 March 2011, Week
Technical Specs of
                                   Kinect
      •       Field of View
              Horizontal field of view: 57 degrees
              Vertical field of view: 43 degrees
              Physical tilt range: ± 27 degrees
              Depth sensor range: 1.2m – 3.5m (10m)

      •       Skeletal Tracking System
              Tracks up to 6 people, including 2 active players
              Tracks 20 joints per active player

      •       Works in complete darkness


Friday, 25 March 2011, Week
Friday, 25 March 2011, Week
IR Projector   IR Camera




Friday, 25 March 2011, Week
Examples


                        http://nui.mlog.taik.fi/interesting-projects




Friday, 25 March 2011, Week

More Related Content

PDF
Multitasking in iOS - The Junction
PDF
Psychology-informed Design of Responsive Open (Personal) Learning Environments
PDF
Awe 2019 - Using AR and VR for Brain Synchroniztion
PDF
Advanced Methods for User Evaluation in Enterprise AR
PDF
Evolving connectivity
PDF
Introduction to Interaction Design
PDF
From Galapagos to Twitter: Darwin, Natural Selection, and Web 2.0
Multitasking in iOS - The Junction
Psychology-informed Design of Responsive Open (Personal) Learning Environments
Awe 2019 - Using AR and VR for Brain Synchroniztion
Advanced Methods for User Evaluation in Enterprise AR
Evolving connectivity
Introduction to Interaction Design
From Galapagos to Twitter: Darwin, Natural Selection, and Web 2.0

Similar to Bodily Interaction Lecture 1 Slides (20)

PDF
Lecture 1: Bodily Interaction Autumn2011
PPTX
JChueke_BCS_Mar 2012_PRINT
PDF
Blended Libraries (Harald Reiterer)
PPT
Tangible User Interface Showcase
KEY
Research2.0
PDF
Hci intro + task
PDF
Interaction Paradigms
PPT
Model-based engineering of multi-platform, synchronous & collaborative UIs
KEY
Research20
PDF
PDF
adsfasdf
PPT
If You Want To Be A Winner, Change Your VRDOLL Philosophy Now!
PDF
[I3 d]04 interactivity
PDF
Ubiquitous interactions
PPT
Multi-Touch Tangible Interface; HCI trends, projects, and development
PDF
Being human : Future of human computer interaction
PDF
Usability Engineering
PPT
Fundamental principles of Usability and User Centred Design
Lecture 1: Bodily Interaction Autumn2011
JChueke_BCS_Mar 2012_PRINT
Blended Libraries (Harald Reiterer)
Tangible User Interface Showcase
Research2.0
Hci intro + task
Interaction Paradigms
Model-based engineering of multi-platform, synchronous & collaborative UIs
Research20
adsfasdf
If You Want To Be A Winner, Change Your VRDOLL Philosophy Now!
[I3 d]04 interactivity
Ubiquitous interactions
Multi-Touch Tangible Interface; HCI trends, projects, and development
Being human : Future of human computer interaction
Usability Engineering
Fundamental principles of Usability and User Centred Design
Ad

Recently uploaded (20)

PPTX
B.Sc. DS Unit 2 Software Engineering.pptx
PPTX
Core Concepts of Personalized Learning and Virtual Learning Environments
PDF
Hazard Identification & Risk Assessment .pdf
PPTX
Virtual and Augmented Reality in Current Scenario
PPTX
ELIAS-SEZIURE AND EPilepsy semmioan session.pptx
PDF
medical_surgical_nursing_10th_edition_ignatavicius_TEST_BANK_pdf.pdf
PDF
Environmental Education MCQ BD2EE - Share Source.pdf
PPTX
A powerpoint presentation on the Revised K-10 Science Shaping Paper
PDF
ChatGPT for Dummies - Pam Baker Ccesa007.pdf
PPTX
What’s under the hood: Parsing standardized learning content for AI
PDF
FOISHS ANNUAL IMPLEMENTATION PLAN 2025.pdf
PDF
advance database management system book.pdf
PDF
1.3 FINAL REVISED K-10 PE and Health CG 2023 Grades 4-10 (1).pdf
PDF
Journal of Dental Science - UDMY (2021).pdf
PPTX
Introduction to pro and eukaryotes and differences.pptx
PDF
FORM 1 BIOLOGY MIND MAPS and their schemes
PDF
semiconductor packaging in vlsi design fab
PDF
LEARNERS WITH ADDITIONAL NEEDS ProfEd Topic
PDF
MBA _Common_ 2nd year Syllabus _2021-22_.pdf
PDF
CISA (Certified Information Systems Auditor) Domain-Wise Summary.pdf
B.Sc. DS Unit 2 Software Engineering.pptx
Core Concepts of Personalized Learning and Virtual Learning Environments
Hazard Identification & Risk Assessment .pdf
Virtual and Augmented Reality in Current Scenario
ELIAS-SEZIURE AND EPilepsy semmioan session.pptx
medical_surgical_nursing_10th_edition_ignatavicius_TEST_BANK_pdf.pdf
Environmental Education MCQ BD2EE - Share Source.pdf
A powerpoint presentation on the Revised K-10 Science Shaping Paper
ChatGPT for Dummies - Pam Baker Ccesa007.pdf
What’s under the hood: Parsing standardized learning content for AI
FOISHS ANNUAL IMPLEMENTATION PLAN 2025.pdf
advance database management system book.pdf
1.3 FINAL REVISED K-10 PE and Health CG 2023 Grades 4-10 (1).pdf
Journal of Dental Science - UDMY (2021).pdf
Introduction to pro and eukaryotes and differences.pptx
FORM 1 BIOLOGY MIND MAPS and their schemes
semiconductor packaging in vlsi design fab
LEARNERS WITH ADDITIONAL NEEDS ProfEd Topic
MBA _Common_ 2nd year Syllabus _2021-22_.pdf
CISA (Certified Information Systems Auditor) Domain-Wise Summary.pdf
Ad

Bodily Interaction Lecture 1 Slides

  • 1. Bodily Interaction Lecture 1, 24.03.2011 Ferhat Şen Friday, 25 March 2011, Week
  • 2. Course • develop gesture-based Natural User Interfaces (NUI) • in various interactive application platforms for sonic, musical, 2D or 3D interfaces Friday, 25 March 2011, Week
  • 3. Course In General • the course is not to teach Processing, Pure data or Unity • to develop applications involving bodily interaction by using these programming environments and Kinect sensor. • prerequisite is to know basics of one of them. you will learn by doing more about them Friday, 25 March 2011, Week
  • 4. Schedule • 24th Mar 2011 12-14: Introduction • 28th Mar 2011 13-15: Kinect with Processing (Joint session with Processing Club) • 31st Mar 2011 12-14: Kinect with Pure Data (4th floor Mac Classroom) WS WEEK (Optional Sessions on Unity) • 11th Apr 2011 13-15: Skeletal Tracking with Processing and Pure Data • 14th Apr 2011 12-14: Small Project Presentation/Concept Idea Presentation/Group Formation • 18th Apr 2011 13-15: Hands on Project Work/Tutoring • 21st Apr 2011 12-14: Hands on Project Work/Tutoring • 28st Apr 2011 12-14: Hands on Project Work/Tutoring • 2nd May 2011 13-15: Final Project Presentation (Group-work) Paja is booked for Week 16-17-18 for you btwn 9-17. (Except 18.04.2011) Friday, 25 March 2011, Week
  • 5. Small Project • Very simple project • Done in Processing, PD, Unity or QC • Some interaction with body data • Individually • Presented in the class Friday, 25 March 2011, Week
  • 6. Final Project • Virtual Reality Concept Development Augmented Reality Puppetry • Multiuser interactive environment Sonic/Musical, 2D, 3D applications Augmented interactive dance performance Gestural musical instrument (from image viewer to media art) Architectural Projection • Groups of 2 people (strongly suggested) • Exhibition • Public space in TaiK • Venue outside (public space outside) • Spring Demoday 26.05 Friday, 25 March 2011, Week
  • 7. Programming • Processing? • Pure Data? • Unity? • Any other? • Level of knowledge? (0-5) Friday, 25 March 2011, Week
  • 8. User Interface the system by which users (people) interact (communicate) with a machine. Friday, 25 March 2011, Week
  • 9. Components of UI • The user interface includes • hardware (physical) • software (logical) components. • Provide a means of: • Input, allowing the users to manipulate a system, and/or • Output, allowing the system to indicate the effects of the users' manipulation. Friday, 25 March 2011, Week
  • 10. Interaction • indicates the means by which user inputs changes to the system and the feedback supplied by the system Friday, 25 March 2011, Week
  • 11. Interaction • How do you do? • How do you feel? • How do you know? Friday, 25 March 2011, Week
  • 12. Timeline of UI’s Command-line Interface Friday, 25 March 2011, Week
  • 13. Friday, 25 March 2011, Week
  • 14. Timeline of UI’s Command-line Interface Graphical User Interface Friday, 25 March 2011, Week
  • 15. WIMP Paradigm WINDOW ICON MENU POINTER Friday, 25 March 2011, Week
  • 16. Friday, 25 March 2011, Week
  • 17. Timeline of UI’s Post WIMP Era Tangible User Interface WIMP Gesture-Based Interface Command-line Interface Graphical User Interface Natural User Interface Friday, 25 March 2011, Week
  • 18. tangible gestural Friday, 25 March 2011, Week
  • 19. Natural Interaction • Experience (Human-Computer --> Human-Human ) • People communicate thru • gestures • expressions • movements • People discover by • looking around • manipulating physical stuff Friday, 25 March 2011, Week
  • 20. Why? • Less cognitive load • Simpler (for certain applications) • typing • changing slide, navigating in 3D VR • Direct manipulation Friday, 25 March 2011, Week
  • 21. Bodily Interaction • whole body in context • multi-modality • human=multi-sensory ? • user as an input modality • direct input from user’s sensory modalities • manipulate digital data with body Friday, 25 March 2011, Week
  • 22. Bodily Input Modalities • Skeletal: Hand,Fingers, Head, Leg, Feet • Sonic: Speech,Voice characteristics (volume, frequency), Body-made sounds • Biofeatures: Breath, Sweat, Heartbeat, Blood Pressure • Touch • Facial muscular activation • Presence in space • Multiple bodies Friday, 25 March 2011, Week
  • 23. Put-that-There, 1979 • voice and gesture at the graphics interface pioneering multimodal application combined speech and gesture recognition. • Put-that-There, MIT, 1979 • Put-that-there, MIT 1983 • Richard A. Bolt "Put-That-There":Voice and Gesture at the Graphics Interface (pdf) SIGGRAPH '80 Friday, 25 March 2011, Week
  • 24. Kinect Sensor IR Laser Projector IR Camera Friday, 25 March 2011, Week
  • 25. How Does it Work Watch the Video Friday, 25 March 2011, Week
  • 26. Kinect’s IR-Grid Watch the Video Friday, 25 March 2011, Week
  • 27. Friday, 25 March 2011, Week
  • 28. Friday, 25 March 2011, Week
  • 29. Two Methods to Track • Depth Filtering • Skeletal Tracking Friday, 25 March 2011, Week
  • 30. Depth Filtering Friday, 25 March 2011, Week
  • 31. Skeletal Tracking Watch the Video Friday, 25 March 2011, Week
  • 32. Friday, 25 March 2011, Week
  • 33. Technical Specs of Kinect • Field of View Horizontal field of view: 57 degrees Vertical field of view: 43 degrees Physical tilt range: ± 27 degrees Depth sensor range: 1.2m – 3.5m (10m) • Skeletal Tracking System Tracks up to 6 people, including 2 active players Tracks 20 joints per active player • Works in complete darkness Friday, 25 March 2011, Week
  • 34. Friday, 25 March 2011, Week
  • 35. IR Projector IR Camera Friday, 25 March 2011, Week
  • 36. Examples http://nui.mlog.taik.fi/interesting-projects Friday, 25 March 2011, Week