Sensor based interaction
Download as PPTX, PDF5 likes5,651 views
The document discusses gesture-based human-computer interaction (HCI) using Microsoft’s Kinect, emphasizing the limitations of traditional input devices like keyboards and mice. It outlines the functionalities of the Kinect sensor, including its RGB camera, infrared capabilities, and hardware/software requirements, while comparing glove-based and vision-based gesture recognition approaches. The summary highlights that Kinect enables intuitive control for various tasks but has limitations in tracking finger gestures and requires a specific distance for effective skeleton tracking.
Sensor based interaction
- 1. Gesture Based Human Computer-Interaction Using Kinect for Interaction with Computer for Gaming, Entertainment Prepared by: Mirza Akif Israr
- 2. Introduction • The development of ubiquitous computing and the need to communicate in a more natural, flexible, efficient but powerful way has rendered most current user interaction approaches which utilizes keyboard, mouse and pen insufficient. • Human-Computer-Interaction technology has seen significant changes over the years which range from text-based UI which relies heavily on the keyboard as an input device to 2-D graphical-based interfaces based on mice
- 3. Gesture Recognition • Gesture Recognition is a technology that achieves dynamic human-machine interactions without requiring physical, touch, or contact based input mechanisms. • The main goal of gesture recognition involves creating a system capable of interpreting specific human gestures via mathematical algorithms and using them to convey meaningful information or for device control.
- 4. Tracking Technologies • The main requirement for supporting gesture recognition is the tracking technology used for obtaining the input data. The approach used generally fall into two major categories: Glove-based approaches and Vision-based approaches.
- 5. Glove-based approaches The common technique for hand pose tracking is to instrument the hand with a glove. Sensors Send input to Computer about Hand Position Orientation, and flex of the fingers using magnetic or inertial tracking devices.
- 6. Glove-based approaches • While it is however easier to collect hand configuration and movement with this approach, the major drawback is that the required devices are quite expensive and cumbersome. Also, the ease and naturalness with which the user can interact with the computer controlled environment is hampered by the load of cables attached to the user.
- 7. Vision-based approaches • It offers a more natural way of HCI as it requires no physical contact with any devices. • Camera(s) placed at a fixed location or on a mobile platform in the environment are used for capturing the input image usually at a frame rate of 30 HZ or more. • To recognize human gestures, these images are interpreted to produce visual features which can then be used to interpret human activity.
- 9. Gesture Recognition System • Gesture recognition is a complex task which involves many aspects and while there may be variations on the methodology used and recognition phases depending on the application areas, a typical gesture recognition system involves such process as data acquisition, gesture modeling, feature extraction and gesture recognition.
- 10. Microsoft’s Kinect • Of recent, gesture recognition has been integrated in various consumer devices for the purpose of entertainment. An example of such device is Microsoft’s Kinect, which allows a user to use gestures that are typically intuitive and relatively simple to perform various tasks such as controlling games, starting a movie etc.
- 11. The Kinect Sensor • The Kinect sensor is a motion-sensing input device that was originally developed in November 2010 for use with the Xbox 360 but has recently been opened up for use with Windows PCs for commercial purposes.
- 12. • The Kinect sensor has the following properties and functions: • An RGB Camera that stores three channel data in a 1280x960 resolution at 30Hz. The camera’s field of view as specified by Microsoft is 43° vertical by 57° horizontal. The system can measure distance with a 1cm accuracy at 2 meters distance. • An infrared (IR) emitter and an IR depth sensor used for capturing depth image. • An array of four microphones to capture positioned sounds. • A tilt motor which allows the camera angle to be changed without physical interaction and a three-axis accelerometer which can be used to determine the current orientation of the Kinect.
- 13. One Microphone Three Microphone Status LED CMOS Color Sensor (RBG Imaging) Three axis Accelerometer Motorized tilting base CMOS IR Sensor IR Light Source Used for 3D Depth Sensing
- 14. Hardware and Software Requirements • According to Microsoft, the PC that is to be used with the Kinect sensor must have the following minimum capabilities: (a) 32-bit (x86) or 64-bit (x64) processors,(b) Dual-core, 2.66-GHz or faster processor,(c) USB 2.0 bus dedicated to the Kinect, and (d) 2 GB of RAM. • To access Kinect’s capabilities, the following software is also required to be installed on the developer’s PC: Microsoft Visual Studio 2010/2012 Express or other Visual Studio edition. The development programming languages that can be used include C++, C# (C-Sharp), and Visual Basic.
- 15. Limitations • Kinect has the following limitations which are primarily based on the optical lenses: • For a smooth skeleton tracking, the user distance from the sensor must be between 1mand 3m. • Kinect cannot perform Finger Detection. This is due to the low-resolution of the Kinect depth map which, although works well when tracking a large object such as the human body; it is however difficult to track human hand (which occupies a very small portion of the image) especially gestures made with fingers.