Getmoving as3kinect
4 likes1,860 views
The document discusses the Kinect sensor technology, originally designed for Xbox, focusing on its capabilities such as 3D motion capture, facial recognition, and voice recognition. It details the sensor components, including RGB and depth cameras, as well as their functionalities in various applications, including gesture tracking and body-pose recognition. Additionally, it provides links to resources and open methods for using Kinect in development.
Getmoving as3kinect
- 1. Get Moving! Reading Kinect data from Actionscript
- 2. Kinect ๏ By Microsoft. Previously Natal. ๏ Sold to be used with the XBox. ๏ "Controller-free gaming and entertainment experience"
- 3. You don’t have to buy a XBox to do any of the stuff shown here. You can buy the camera on its own for about NZ$200.
- 4. 1 Sensors
- 5. Kinect Sensor(s) “The Kinect sensor is a horizontal bar connected to a small base with a motorized pivot and is designed to be positioned lengthwise above or below the video display. The device features an "RGB camera, depth sensor and multi-array microphone running proprietary software", which provide full-body 3D motion capture, facial recognition and voice recognition capabilities. The depth sensor consists of an infrared laser projector combined with a monochrome CMOS sensor, which captures video data in 3D under any ambient light conditions. The sensing range of the depth sensor is adjustable“ Source: http://en.wikipedia.org/wiki/Kinect
- 7. Sensors Depth magic chip (data stream) Infrared Laser Projector Image chip RGB Camera (video stream) Infrared Camera Multi-array mic Motorised tilt
- 8. Sensors, #1 RGB Image chip 480 @ 30 frames/sec ๏ The RBG camera can be used for recording 640 video and assist with Horizontal field of view: 57º Vertical field of view: 43º Physical tilt range: 27º tasks such as facial recognition.
- 9. Sensors, #2 Depth ๏ Project a pattern of dots with a near infrared laser over the scene. A detector establishes the parallax shift Depth magic chip of the dot pattern for each pixel. 480 12 bits (Data Array) @ 30 frames/sec 640 Range: 1.2m -3.5 m
- 10. Depth. Uh? Depth computed by comparing distance between Image 1 (uniquely identifiable) dot in image 1 and image 2. The more distant, the more depth. Magic Algorithm generates the pattern of dots that will be sent by the Infrared projector. In a sense a virtual image. Image 2 Infrared camera Depth magic chip Stereo Image from: http://gigantico.squarespace.com/ 336554365346/2007/6/21/virtual-reality-part-1.html
- 11. Sensors, #3 Infrared Image chip 480 @ 30 frames/sec ๏ Images from the random dot pattern projected by 640 the near-infrared laser. ๏ Switch off other infrared sources (hallogen, sun light) to see them.
- 12. Sensors, #4 Voice 16 bits @ 16kHz Not covered in this talk
- 13. 2 Open Kinect
- 14. OpenKinect
- 15. BW + RGB images ๏ By default, provides depth information as an RGB image. Each color represents a layer of depth. Red = close, blue = far away ๏ Good for Flash. Flash is good with picture but doesn’t excel at looping through numbers.
- 17. Demo ๏ Modified as3 server • video2rgba: rbg to rgba mapping (prefixing with 0xFF for alpha). • depth2heatmap : depth to rgba mapping. • depth2grayscale : depth to grayscale mapping, a scale from 0...2048 is reduced to one of 0...255. •
- 18. Resolution vs meters ๏ The depth data provided by kinect follow some logarithmic curve : • High resolution in the middle, low res close and far away from the camera ๏ A linear estimate, in centimeters, can be computed with: 0.1236 * tan(rawDisparity / 2842.5 + 1.1863) http://openkinect.org/w/index.php?title=Imaging_Information
- 19. Info / Resources
- 20. 3 Tuio Kinect
- 21. Hands tracking ๏ TuioKinect tracks simple hand gestures using the Kinect controller and sends control data based on the TUIO protocol. This allows the rapid creation of gesture enabled applications
- 22. Demo!
- 23. Using TUIO from AS https://github.com/silviopaganini/openKinect-as3-experiments Documentation, Parameters Note: getting 0 for z or Z in current implementation. http://nuigroup.com/?ACT=28&fid=33&aid=703_FKSuKMcUkTAJrwBGk2Qv
- 25. 4 Advanced
- 26. Body-Pose ๏ What MS uses for his games is Machine Learning “The idea was that we would teach the computer with lots of different people of lots of different shapes and sizes in lots of different poses and the computer will learn how to distinguish one part of your body from another part,” he said. Since the Kinect camera includes depth information, it can distinguish between big people a long way away and small people up close.” http://blogs.wsj.com/tech-europe/2010/11/08/key- kinect-technology-devised-in-cambridge-lab/ For publicly available methods: http://openkinect.org/wiki/Research_Material
- 27. Skeleton http://www.openni.org/downloadfiles/ http://www.keyboardmods.com/2010/12/howto- kinect-openninite-skeleton.html http://www.freenect.com/kinect-skeleton-detection- work-in-progress http://www.modmykinect.com/2010/11/29/skeleton- detection-part-2/
- 28. Frame differencing http://actionsnippet.com/?p=2820
- 29. Blob Detection
- 30. UX