SlideShare a Scribd company logo
Touching More than 3 Dimensions
Research Into Novel Interfaces – three
dimensions and above
Joaquim Jorge
INESC-ID & IST / U Lisboa
Portugal
IP / Leiria Nov 2015
Joaquim Jorge
Instituto Superior Técnico
Universidade de Lisboa
Visualization and Multimodal Interfaces
@ INESC-ID Lisboa
http://web.ist.utl.pt/jorgej
ACM/SIGGRAPH Small Conferences
Committee Chair
Editor Computers & GraphicsResearch Interests:
Calligraphic Interaction,
Multimodal Interfaces,
Graphical Modeling
IS TONY STARK HOME
IS TONY STARK HOME ?
WE HAVE SEEN THE FUTURE…
Natural Interfaces ?
http://youtube.com/watch?v=oIOxT6PUgPA
Dassault Systèmes Vision
NATURAL INTERACTION
Ivan Sutherland – 52 years ago
Ivan Sutherland SketchPad Video
youtube.com/watch?v=495nCzxM9PI
Beyond 2D
+ Space Continuous Above
+ Hands Bimanual Interactions
+ Dimensions 3D UIs
+ Expressiveness Sound, Kicking
+ Proxemics Eery Space
INTERACT 2011
Interaction Techniques Unifying Touch and Gesture On and Above a
Digital Surface
The Continuous Interaction Space Video
http://youtu.be/A8kb7BV7j1Y
Exploring Bimanual Interaction on a
TableTop
www.shapeshop3d.com
Pedro Lopes, Daniel Mendes, Bruno Araújo, Joaquim A. Jorge
Combining bimanual manipulation and
pen-based input for 3D modeling, SBIM 2011
Key Points
Table Architecture
Combining Pen and Gestures
Asymetric Guiard Model
Table Architecture
MOVIE
See Movie at:
https://vimeo.com/152554942
Evaluation
Test Procedure
Free drawing
Evaluation
More sketching, less camera
Evaluation
Camera operations: more fluid
(seconds)
44% faster
Evaluation
Hand Role Distribution
9% 91%
3% 80% 17%
Evaluation
Users prefer touch for all manipulations
Imagem e benefits of direct vs indirect
Evaluation
Prefer Combined technique
Observations
Bimanual workspace and hand role
Observations
Natural resting position of NDH
Observations
Pen-only often forces re-arrangement
Observations
Pen-only causes too much DH motion
Observations
Rotate gesture was frail, swapped pen to NDH
Observations
Combined gestures were naturally used
Getting it together:
Mockup Builder
Setup
Multitouch 120Hz
Touch 16.25 DPI
3D SVGA 120Hz
Active Stereo
Kinect (30Hz)
Gametrak (120Hz)
Gametrak
User Model
Active User Kinect
Head Position Kinect
Hand Position (Right/Left) Kinect
Finger touches Table
4 Fingers tracked in Space (Gametrak)
Finger X from Hand Y on or above the surface
from user Z
BIMANUAL Asymmetric
DH : SketchingNDH : Manipulation
Modes  Hands (WIMP)
CONTINUOUS INTERACTION SPACE
NDH: Context Menus DH: Selects
ON & ABOVE THE SURFACE
Snapping
Discrete Constraint
Continuous Constraints
On the fly
Active as long as needed
Preliminary Results
expert architect Rhino3D
5’20”
2’45”
5’41”
3’46”
xpert User our modeler
Touching The 3D Dimension, Dagstuhl Feb. 2012
Modeling On and Above a Stereoscopic Multitouch
Display, (3DCHI): May, 2012.
Mockup Builder: Direct 3D Modeling On and Above
the Surface in a Continuous Interaction Space
(GI'2012), May 28 - May 30, 2012.
Special Section C&G (January 2013)
ACM Spatial User Interfaces Symposia (SUI)
Adding Sound to Multimodal
Interfaces
ITS 2011
Take-Home message
Touch signature Impact Intensity Interaction space Tangibles
Take-Home message
Touch signature Impact Intensity Interaction space Tangibles
Take-Home message
Touch signature Impact Intensity Interaction space Tangibles
Take-Home message
Touch signature Impact Intensity Interaction space Tangibles
Acoustics of Touch
Acoustics of Touch
Acoustic Sensing
Setup
Interaction Opportunities
https://vimeo.com/30997725
https://vimeo.com/30997725
See Video
Acoustic Sensing
Conclusions
Affordable Setup
Acoustic Sensing
Conclusions
Understand impact (amplitude)
Acoustic Sensing
Conclusions
Recognize contact parts (timbre)
Acoustic Sensing
Conclusions
Non-tracked Tangibles
Acoustic Sensing
Conclusions
Multiplicity of Input
Let’s Kick It
CHI 2014
Exploring New Interaction Space
Large Scale Displays
Interacting with the bottom of the screen
Beyond Situated Displays
Eery Space
Remote Proxemics: Person to Device Interactions
Social Interactions
Proxemics Distances [Hall, 1966]
Eery Space
Providing Awareness: Floor Circles
Issues
Working / Virtual Space
Simple Operators / Dialogues
Implicit Mode Changes
Continuous Transition
Asymmetric Bimanual Modes
Take Home Message
Extending Touch With Other Modalities
Bimanual Interaction
Explore Surface Above Table
Use Sound in Novel Ways
Explore New Modalities
Proxemics as tool for implicit interactions
Research Issues
Content Creation in VEs
Sketch Interpretation (3D)
New Modalities
Self Disclosure (Not!)
Understandability
Thank You!
Joaquim Jorge
http://web.ist.utl.pt/jorgej
Bruno Rodrigues De Araújo
Maurício de Sousa
Ricardo Jorge Jota Costa
Pedro Lopes
Daniel Mendes

More Related Content

PDF
COMP 4026 - Lecture1 introduction
PDF
Future Directions for Augmented Reality
PDF
COMP 4010 Lecture12 - Research Directions in AR and VR
PDF
2013 Lecture 5: AR Tools and Interaction
PDF
COMP 4010 - Lecture 7: Introduction to Augmented Reality
PDF
Designing Augmented Reality Experiences
PDF
2013 426 Lecture 2: Augmented Reality Technology
PDF
Ubiquitous Computer Vision in IoT
COMP 4026 - Lecture1 introduction
Future Directions for Augmented Reality
COMP 4010 Lecture12 - Research Directions in AR and VR
2013 Lecture 5: AR Tools and Interaction
COMP 4010 - Lecture 7: Introduction to Augmented Reality
Designing Augmented Reality Experiences
2013 426 Lecture 2: Augmented Reality Technology
Ubiquitous Computer Vision in IoT

What's hot (20)

PDF
2013 Lecture3: AR Tracking
PPTX
Human computer interaction
PDF
AR in Education
PDF
CHI 2013 DARE Course
PDF
COMP 4010: Lecture11 AR Interaction
PDF
Introduction to Augmented Reality
PDF
COMP 4010 - Lecture10: Mobile AR
PDF
Can You See What I See?
PPTX
Virtual Reality
PDF
426 lecture2: AR Technology
PDF
Augmented Reality: The Next 20 Years
PPTX
The UX of Mixed Reality
PDF
Augmented Human 2018
PDF
Comp 4010 2021 Lecture1-Introduction to XR
PDF
Introduction to Wearables
PPTX
Augmented Reality -A quick surface view
PDF
The Glass Class at AWE 2015
PDF
3D Content Development and AR/VR Authoring
PDF
CHI 2015: The Glass Class - Designing Wearable Interfaces
PPTX
User Interfaces and User Centered Design Techniques for Augmented Reality and...
2013 Lecture3: AR Tracking
Human computer interaction
AR in Education
CHI 2013 DARE Course
COMP 4010: Lecture11 AR Interaction
Introduction to Augmented Reality
COMP 4010 - Lecture10: Mobile AR
Can You See What I See?
Virtual Reality
426 lecture2: AR Technology
Augmented Reality: The Next 20 Years
The UX of Mixed Reality
Augmented Human 2018
Comp 4010 2021 Lecture1-Introduction to XR
Introduction to Wearables
Augmented Reality -A quick surface view
The Glass Class at AWE 2015
3D Content Development and AR/VR Authoring
CHI 2015: The Glass Class - Designing Wearable Interfaces
User Interfaces and User Centered Design Techniques for Augmented Reality and...
Ad

Viewers also liked (6)

PPTX
Levels of analysis
PPT
Towards A Human Dimensions Research Agenda For Parks
PPTX
Dimensions of open research: critical reflections on openness in the ROER4D p...
PPTX
Foreign Policy and Level of Analysis Problem
PPT
Definition and types of research
PPTX
Types of Research
Levels of analysis
Towards A Human Dimensions Research Agenda For Parks
Dimensions of open research: critical reflections on openness in the ROER4D p...
Foreign Policy and Level of Analysis Problem
Definition and types of research
Types of Research
Ad

Similar to Touching More than 3 Dimensions Research Into Novel Interfaces – three dimensions and above (20)

PDF
426 lecture6b: AR Interaction
PDF
COSC 426 lect. 4: AR Interaction
PPTX
hcid2011 - Gesture Based Interfaces: Jacques chueke (HCID, City University L...
PDF
Ux portfolio 2013 october final ivan pecorari
PPSX
NUI_jaydev
PPTX
Presentation on Tele-Immersion
PPTX
Surface computing ppt
PPTX
Sixth sense technology
PPT
Surfacecomputerppt 130813063644-phpapp02
PPTX
PPTX
JChueke_BCS_Mar 2012_PRINT
PDF
Prototyping GNOME UI for Gestural Input
PPTX
human computer interaction Lecture 2.pptx
PDF
COSC 426 Lect. 8: AR Research Directions
PPTX
BYO3D 2011: Welcome
PPTX
Visualization and Interactive Prototyping with Unity3D
PPTX
IoT Day Italy - Mixed Reality & IoT
PPTX
Multitouch Interaction
PPTX
Phd-Guidance-in-Augmented-Reality
PPTX
Next generation User interfaces
426 lecture6b: AR Interaction
COSC 426 lect. 4: AR Interaction
hcid2011 - Gesture Based Interfaces: Jacques chueke (HCID, City University L...
Ux portfolio 2013 october final ivan pecorari
NUI_jaydev
Presentation on Tele-Immersion
Surface computing ppt
Sixth sense technology
Surfacecomputerppt 130813063644-phpapp02
JChueke_BCS_Mar 2012_PRINT
Prototyping GNOME UI for Gestural Input
human computer interaction Lecture 2.pptx
COSC 426 Lect. 8: AR Research Directions
BYO3D 2011: Welcome
Visualization and Interactive Prototyping with Unity3D
IoT Day Italy - Mixed Reality & IoT
Multitouch Interaction
Phd-Guidance-in-Augmented-Reality
Next generation User interfaces

More from Joaquim Jorge (20)

PDF
Dagstuhl Seminar 25031 Approaches and Challenges to Virtual and Augmented Rea...
PPTX
AI+Labor Markets Presentation to CSM-16-may-2024
PPTX
Artificial Intelligence: Facts and Myths
PDF
Matera Summer School AI&XR 2023.pdf
PPTX
Presentation at SMI 2023
PDF
How to Craft and Deliver Winning Presentations
PDF
Virtual and Augmented Reality in Health Care & Rehabilitation
PDF
Siggraph asia 2019 Course slides
PPTX
Siggraph asia 2019 panel pesentation ian hall
PPTX
Morie frontiers nov2019
PDF
Embodiment cognition
PDF
Empathic Extended Reality
PDF
Debevec vrai-s-asia2019
PDF
Glassner Puppies and Dragons vr+ai workshop
PDF
Siggraph Asia 2019 VR & AI workshop-wiles-handout
PDF
Augmented Reality and Artificial Intelligence: Friends with Benefits?
PDF
Empathic Computing
PDF
Augmented Reality and Artificial Intelligence: Friends with Benefits?
PPTX
Anatomy Studio: a Tool for Virtual Dissection Through Augmented 3D Reconstruc...
PPTX
top 10 ways to get your paper rejected at Computers and Graphics Journal
Dagstuhl Seminar 25031 Approaches and Challenges to Virtual and Augmented Rea...
AI+Labor Markets Presentation to CSM-16-may-2024
Artificial Intelligence: Facts and Myths
Matera Summer School AI&XR 2023.pdf
Presentation at SMI 2023
How to Craft and Deliver Winning Presentations
Virtual and Augmented Reality in Health Care & Rehabilitation
Siggraph asia 2019 Course slides
Siggraph asia 2019 panel pesentation ian hall
Morie frontiers nov2019
Embodiment cognition
Empathic Extended Reality
Debevec vrai-s-asia2019
Glassner Puppies and Dragons vr+ai workshop
Siggraph Asia 2019 VR & AI workshop-wiles-handout
Augmented Reality and Artificial Intelligence: Friends with Benefits?
Empathic Computing
Augmented Reality and Artificial Intelligence: Friends with Benefits?
Anatomy Studio: a Tool for Virtual Dissection Through Augmented 3D Reconstruc...
top 10 ways to get your paper rejected at Computers and Graphics Journal

Recently uploaded (20)

PPTX
Fundamentals of Mechanical Engineering.pptx
PPT
introduction to datamining and warehousing
PDF
A SYSTEMATIC REVIEW OF APPLICATIONS IN FRAUD DETECTION
PPT
Introduction, IoT Design Methodology, Case Study on IoT System for Weather Mo...
PPTX
Information Storage and Retrieval Techniques Unit III
PPTX
Artificial Intelligence
PPTX
Safety Seminar civil to be ensured for safe working.
PDF
Visual Aids for Exploratory Data Analysis.pdf
PPT
Occupational Health and Safety Management System
PDF
Abrasive, erosive and cavitation wear.pdf
PDF
PPT on Performance Review to get promotions
PDF
SMART SIGNAL TIMING FOR URBAN INTERSECTIONS USING REAL-TIME VEHICLE DETECTI...
PDF
Integrating Fractal Dimension and Time Series Analysis for Optimized Hyperspe...
PDF
Soil Improvement Techniques Note - Rabbi
PDF
R24 SURVEYING LAB MANUAL for civil enggi
PPTX
Nature of X-rays, X- Ray Equipment, Fluoroscopy
PPTX
Fundamentals of safety and accident prevention -final (1).pptx
PDF
Categorization of Factors Affecting Classification Algorithms Selection
PPTX
CURRICULAM DESIGN engineering FOR CSE 2025.pptx
PDF
Human-AI Collaboration: Balancing Agentic AI and Autonomy in Hybrid Systems
Fundamentals of Mechanical Engineering.pptx
introduction to datamining and warehousing
A SYSTEMATIC REVIEW OF APPLICATIONS IN FRAUD DETECTION
Introduction, IoT Design Methodology, Case Study on IoT System for Weather Mo...
Information Storage and Retrieval Techniques Unit III
Artificial Intelligence
Safety Seminar civil to be ensured for safe working.
Visual Aids for Exploratory Data Analysis.pdf
Occupational Health and Safety Management System
Abrasive, erosive and cavitation wear.pdf
PPT on Performance Review to get promotions
SMART SIGNAL TIMING FOR URBAN INTERSECTIONS USING REAL-TIME VEHICLE DETECTI...
Integrating Fractal Dimension and Time Series Analysis for Optimized Hyperspe...
Soil Improvement Techniques Note - Rabbi
R24 SURVEYING LAB MANUAL for civil enggi
Nature of X-rays, X- Ray Equipment, Fluoroscopy
Fundamentals of safety and accident prevention -final (1).pptx
Categorization of Factors Affecting Classification Algorithms Selection
CURRICULAM DESIGN engineering FOR CSE 2025.pptx
Human-AI Collaboration: Balancing Agentic AI and Autonomy in Hybrid Systems

Touching More than 3 Dimensions Research Into Novel Interfaces – three dimensions and above

Editor's Notes

  • #6: ACRESCENTAR VIDEO MINORITY REPORT
  • #7: Pinter accao natural 3D Modelling should rely on more natural interaction techniques and take more advantage of virtual environments not only for visualization Ambientes Virtuais e Estereo mas não so para visualizacao Mas tambem para modelacao O conteudo virtual deveria libertar restricoes fisicas das maquetes
  • #11: The Continuous Interaction Space: Interaction Techniques Unifying Touch and Gesture On and Above a Digital Surface
  • #22: Questionnaire Analysis
  • #23: Questionnaire Analysis
  • #34: Implicit Switch between creation and manipulation DH : Sketching NDH : Manipulation (T R S 1 or 2 fingers/hands)
  • #35: On and Above Surface Interaction On surface: Sketch Based Modeling Interface Above the surface: Push and Pull Modelling Metaphor Shapes created by extrusion Topological Features (Vertex, Edge, Faces) Movable Curvilinear Extrusion
  • #36: Contextual Simple Menu (2D layer) Selection DH Menu Location NDH Only Visible if NDH close to surface Fading Effect Dynamic Moving
  • #37: DH Modelling NDH Operation Attribute complement DH NDH Operation mode Snapping Clonning Constraint Snapping Switch between surface and space Discrete Constraint Position of Clonning Activate Mirror Plane for Sketches Limit Extrusion while Extruding Continuous Constraint Scaling Profile Factor while extruding (cone, frustum...) Constraint Definition
  • #56: Working/Virtual Space Co-location Simple Operators and Dialogues Data Fusion from several devices Benefits of on and above interaction Continuous transition surface and space Asymmetric Bimanual Interaction Implicit Mode Switch Good Not error prone for non familiar users Some relaxation needed Button vs Pinch gesture vs other modalities Physics Initially considered for Modeling Sketch based Beautification (measures needed)
  • #57: Also, As prof-of-concept for person to device interactions, We devised the role of the moderator of the meeting, when in close proximity to a wall display, the person assuming that role can perform actions that relate to every participant.
  • #58: People like to meet in person, and when they do, they communicate in different manners besides verbally. Edward Hall states that space and distance play key roles in the way people communicate. Therefore, Proxemics are a non-verbal type of communication and can be exploited in the context of virtual meetings.
  • #59: We use the Proxemic Distances model to provide the necessary knowledge to bring social interactions into virtual meetings. To provide, the means for people to implicitly and automatically start remote interactions just by getting together.
  • #60: Avatars and telepresence robots are a poor representation of people, and they contribute to the lack of presence Our model do not play by the same rules The question is not how well we represent people. The important part of our research is the representation of interactions occurring around the room.
  • #61: We use a ceiling mounted projector to display floor circles which account for the position of all eery space participants. In this way, every one becomes aware of their intimate space (the inner circle) and their personal space (the outer ring). The dimensions follow the proxemic distances proposed by Edward Hall. In this figure, we display two people forming a social bubble, and performing a collaborative task. Again, there is no need for high definition avatars or video for people to know about the presence of others.
  • #62: Working/Virtual Space Co-location Simple Operators and Dialogues Data Fusion from several devices Benefits of on and above interaction Continuous transition surface and space Asymmetric Bimanual Interaction Implicit Mode Switch Good Not error prone for non familiar users Some relaxation needed Button vs Pinch gesture vs other modalities Physics Initially considered for Modeling Sketch based Beautification (measures needed)