Hacking the Kinect with GAFFTA Day 2
- 1. OPEN FRAMEWORKS + KINECT PART II @GAFFTA Thursday, March 7, 13
- 2. The Kinect for XBOX - released in November 2010 Thursday, March 7, 13
- 3. why it matters http://memo.tv/archive/kinect_why_it_matters Affordable Depth Sensor Calibrated Depth + RGB Sensor Nothing new technically, just making it accessible to a MUCH larger audience. Thursday, March 7, 13
- 4. Microphone Array has advanced voice capabilities Motorized tilt helps it to adjust to multiple environments RGB camera for other data ( Facial Tracking, Color tracking ) Thursday, March 7, 13
- 5. what does the kinect see ? http://www.youtube.com/watch?v=dTKlNGSH9Po Laser Projector casts a grid of infrared dots Infrared lens interprets the size of the dots as distance and turns it into a “depth map” This depth map makes it easy to separate parts of a scene Computer learning is based off of these depth maps and large data sets Thursday, March 7, 13
- 6. Hack The Kinect Method #1 libfreenect is the driver / library ofxKinect is the OF addon https://github.com/ofTheo/ofxKinect Simplest wrapper for kinect. ( also the first ) Good at getting raw image + point cloud data + motor control Extremely useful if combined with openCV cross platform - PC / Mac / Linux Thursday, March 7, 13
- 7. What is OpenCV ? Open Source Computer Vision Library A library that allows for real-time image manipulation and feature extraction. Including : Gesture recognition Facial recognition 2D + 3D features Motion Tracking Stereopsis Originally developed by Intel Labs in 1999 for CPU intensive programs. Released to the public in 2006. Now developed by Willow Garage Thursday, March 7, 13
- 8. OpenCV with a webcam Thursday, March 7, 13
- 9. OpenCV with a webcam First there is a color image from a camera. That color image is converted to grayscale so that it can be processed it faster. ( less channels = less data ) The background has a grayscale “snapshot” taken A difference is applied to new frames after the background snapshot OpenCV ContourFinder returns: a shape with many points, a bounding box, and centroid Thursday, March 7, 13
- 10. OpenCV with a kinect Thursday, March 7, 13
- 11. OpenCV with a kinect There is already a grayscale depth feed, that can be used to isolate a range using OpenCV Threshold Running OpenCV contour finder on this thresholded gray image gets us OpenCV blobs. In addition to blob tracking we also get the 3d position of each point in space. Also known as a “point cloud” Thursday, March 7, 13
- 12. LET’S START CODING https://github.com/benMcChesney/OF_Kinect_Tutorials Thursday, March 7, 13
- 13. ofxKinect + ofxUI Thursday, March 7, 13