Kinect sensor
Download as PPTX, PDF2 likes1,913 views
The Kinect sensor is an input device by Microsoft that uses cameras and microphones to track body movements and recognize gestures and voices. It consists of an RGB camera, depth sensor using infrared light, and 4-microphone array. The depth sensor uses structured light to measure distances by projecting a pattern and analyzing its distortion. Kinect can track up to 20 joints of the human body in real-time using skeletal tracking. It has applications in 3D scanning, sign language translation, augmented reality, robot control, and virtual fitting rooms due to its low-cost depth sensing capabilities.
Kinect sensor
- 1. KINECT SENSOR Group members Bhoomit Morkar 14103180 Sachin Keniya 14103170 Vaishnavi Shah 14103182 Ears and Eyes of your application
- 2. Content Introduction • Basic Construction • Basic Specifications • Kinect’s Potential and applications Working of Kinect • Classification of Depth Measurement Techniques Triangulation Principles of TOF Imaging Principle of structured light • Skeletal Tracking Applications References
- 3. Introduction • It was Designed by Microsoft for the Xbox 360 video game console and windows PC’s. • While making the Kinect sensor , Microsoft’s aim was to enable interactions between users and a game without the need to touch a controller. • Kinect is an RGB-D sensor providing synchronized color and depth images. • It’s greatest differentials is depth sensing technology that could be extended far beyond gaming and at a much lower cost than traditional 3-D cameras.
- 4. Why Kinect is revolutionary ? • Kinect has proven to be an incredible piece of technology. • It is the first practical natural user interface for machines communicating with people. • It is combining acoustical and visual sense. • It is recognizing full body movements, even complicated ones. • It finds application not only in games but also in the field of medical sciences and robotics. • After selling a total of 8 million units in its first 60 days, the Kinect holds the Guinness World Record of being the "fastest selling consumer electronics device".
- 6. Power Comes from the Sum • The Kinect Sensor is the outcome of several devices(or sensors) working together. • It is comprised of a color camera (also known as RGB), a depth sensor, a 4-microphone array, and a motorized tilt. • The depth sensor, in itself, consists of an infrared (IR) projector and an IR camera. • Due to that Kinect being a motion sensing input device, able to capture, track, and interpret full-body movements(3D), and recognize gestures and voices.
- 7. • The depth sensor is a monochrome complementary metaloxide semiconductor (CMOS) sensor. • One of Kinect’s greatest differentials is its relatively low cost depth sensor. • The depth-sensing technology is licensed from the Israeli company PrimeSense. Although the exact technology is not disclosed, it is based on the structured light principle. • The IR projector is an IR laser that passes through a diffraction grating and turns into a set of IR dots. • The sensor has an RGB (red-green-blue) camera for color video. • It has four microphones that enable your application to respond to verbal input, in addition to responding to movement.
- 8. Basic Specifications Hardware • Depth resolution : 640x480 px • RGB resolution : 1600x1200 px • FrameRate : 60 FP Software • Depth; 30 FPS: 80x60 , 320x240 , 640x480 • Color; 12 FPS : 1280x980 RGB 15 FPS : Raw YUV 640x480 20 FPS : 640x480
- 9. Kinect’s Potential and applications Depth sensor Skeletal tracking Microphone array Background removal Object recognition Multi-user Easy Gesture Recognition Sound source detection Speech recognition
- 10. How it works ?
- 11. Audio input Microphone array Sound threshold How does Kinect hear ? Track me ! Or Speak louder .
- 12. what does it see ?
- 13. Near mode depth ranges Default mode depth ranges Angle of vision (RGB-D) Area under Observation
- 14. Classification of Depth Measurement Techniques
- 15. Triangulation • In trigonometry and geometry, triangulation is the process of determining the location of a point by forming triangles to it from known points. • Here rather than measuring distances to the point directly as in trilateration; we use of both angles and distance
- 16. Basic Mathematical model for Kinect
- 17. Principles of TOF Imaging • Time-of-Flight (TOF) Imaging refers to the process of measuring the depth of a scene by quantifying the changes that an emitted light signal encounters when it bounces back from objects in a scene.
- 18. Pulsed ModulationContinuous Wave Modulation
- 19. Principle of structured light • Project a known light pattern into the 3D scene, viewed by cameras. • Distortion of light pattern allows computing the 3D structure. • This pattern has to be unique per position in order to recognize each point in the pattern. • Kinect uses pseudo random pattern.
- 20. speckles Based on distance and orientation First Region High accurate depth. (0.8 – 1.2 m) Second Region Medium accurate depth. (1.2 – 2.0 m) Third Region Low accurate depth. (2.0 – 3.0 m)
- 21. Depthobserved pattern Calibration IR camera observes the scene known pattern projected 3D map of the beginning frame New 3D map of the scene Renew 3D map of the scene Between projector and camera & x-direction speckle shift
- 22. Precision Spatial x/y resolution 3mm @ 2m distance Depth Z resolution 1cm @ 2m distance 10 cm @ 4m distance operation range 0.8m ~ 4m | 0.5m ~ 3m
- 23. Skeletal Tracking In the beginning,it was just hands; • Head and hands, • Added shoulder and elbows. Today! • It can track 20 joints of our body and rans fast.
- 24. Process Modes
- 25. Applications • The Kinect fusion project allows a user to use an off-the-shelf Kinect camera as a 3D scanner for producing high-quality 3D scans of small or large objects. • Researchers in China, in collaboration with Microsoft Research, developed the Kinect Sign Language Translator, a system that can translate sign language into spoken and written language in near real time. • The Holoflector is an augmented reality mirror. Behind the mirror is a large LCD monitor and right above the mirror is the Kinect. • Combining a Kinect, a few hacking tools and a 3D environment, Kinectasploit allows a user to break into a computer’s security system using body gestures. The system works almost like a video game. In one demo, a hacker ‘shoots’ at Wi-Fi targets to break into them.
- 26. • UBI Interactive developed their software UBI which combines your Windows 8 computer to Kinect and projector to create an interactive touchscreen off any surface. • In another case of copying from the movies, many users have hacked their Kinect to allow users to control robots via body movement. • We can pick out cloths and try them all out virtually with the help of Fitnect, an interactive dressing room.
- 27. References http://mms.ecs.soton.ac.uk/2014/papers/2.pdf http://lshao.staff.shef.ac.uk/pub/KinectReview_TC2013.pdf https://www.microsoft.com/en-us/research/wp- content/uploads/2016/02/Microsoft20Kinect20Sensor20and20Its20Effect20-20IEEE20MM202012.pdf http://mms.ecs.soton.ac.uk/2014/papers/2.pdf http://campar.in.tum.de/twiki/pub/Chair/TeachingSs11Kinect/2011-DSensors_LabCourse_Kinect.pdf http://www.slideshare.net/MatteoValoriani/introduction-to-kinect-update-v-18?qid=e70a04cc-282d-4c11-a0d0- cff7f58c2435&v=&b=&from_search=9 http://www.slideshare.net/rojizofrio/xbox-one-kinect?qid=e70a04cc-282d-4c11-a0d0- cff7f58c2435&v=&b=&from_search=15
- 28. Videos https://youtu.be/uq9SEJxZiUg http://youtu.be/bRgEdqDiOuQ?t=10m16s http://www.google.com/patents/US20090 http://campar.in.tum.de/twiki/pub/Cha... (p. 33) http://en.wikipedia.org/wiki/Range_im... (Stereo triangulation) https://www.youtube.com/watch?v=-RELJD0LBg8&spfreload=5 https://youtu.be/uq9SEJxZiUg http://www.hongkiat.com/blog/innovative-uses-kinect/