The not so short introduction to Kinect
Download as PPT, PDF1 like1,333 views
The document provides an introduction to the Kinect sensor. It discusses Kinect hardware, including its RGB camera, IR depth sensor, and IR projector. It explains the Kinect's time-of-flight depth sensing technology. The document also discusses the OpenNI and NITE middleware for accessing Kinect sensor data. It provides code examples for retrieving depth maps and skeletal tracking data from the Kinect. Overall, the document aims to educate readers on developing applications using the Kinect by exploring its technical capabilities and available development platforms.
![How
•kinect.enableScene();
•sceneMap = kinect.sceneMap();
•if(sceneMap[i] == n)](https://image.slidesharecdn.com/thenotsoshort-120515025440-phpapp01/85/The-not-so-short-introduction-to-Kinect-92-320.jpg)
The not so short introduction to Kinect
- 1. T he N ot So Short I ntroduction to Kinect 一份不太简短的 Kinect介绍 Hang xie sonicisdreaming@gmail.com http://Xhbang.com
- 2. About M e Chongqing University bachelor （单身汉 ） A boy couldn't stop dreaming Bike travelling
- 3. About Him •Dr. •Sensor related •Poet （魔幻现实主义）
- 4. About the Lab •50% Idle •25% Computer •20% Programmer •5% Student/Paper
- 5. About You? Programming dummy? Linux || Windows? Understand 3D? Work with the Kinect?
- 6. T he Contents wha u will lea toda t rn y 体感操作， Kinect ， OPENNI 介绍 基本程序设计 Working with Depth Image Working with Point Clouds （ * ） Working with Skeleton Data & Tracking Kinect Hacks
- 7. 昨夜西风凋碧树 , 独上高楼 , 望尽天涯 路
- 10. Buy it!
- 16. Kinect Ha re rdwa •RGB camera •IR depth sensor •IR projector •Audio device
- 22. T a a closer look ke Noise at edges(shoulders) Depth shadows Misalignment between the color and depth images Frame rate(30 vs 60)
- 23. Behind Kinect
- 24. T OF •Time of Flight •以色列公司 3DV （ 09 年微软收购 ） •光飞行时间，飞秒 (1e−15) 级快门 •砷化镓，使其时钟频率提高到上百 •高成本，难以适应消费电子品
- 25. Light Coding （光编码） •连续的照明（而非脉冲） •普通的 CMOS 感光芯片（低成本） •结构光技术（三维纵深的“体编码” ） •激光散斑（ laser speckle ） http://www.freepatentsonline.com/7433024.pdf http://www.wipo.int/pctdb/en/wo.jsp?WO=2007043036 http://www.javaforge.com/wiki/103639
- 29. Audio •Something we forget •Supported by OpenNI, not by SensorKinect
- 30. M ore About Audio （题外话） •“Sound Wave” •微软研究院和华盛顿大学 •利用多普勒效应，使用笔记本内置的扬声器和麦 克风开发出类似Kinect的姿势感知系统 •低成本 •识别精度能达到 90% 以上 •用手势玩俄罗斯方块游戏（视频）
- 31. Concepts
- 35. Wii Remote
- 36. ASUS Xtion
- 37. Kinect
- 38. Project Glass
- 39. Development
- 45. OpenN I •Short for “Open Natural Interface” •Multi-langue •Cross-platform •Standard & Infrastructure
- 46. 向后兼容 使用 openNI 开发的程序在以后的 版本中都可以继续使用 , 因为开发 人员可以毫无顾虑的更新 openNI 到最新的版本。
- 48. 抽象分层视图 •Top: 代表软件，在 OpenNI 顶部，实现了自然交互的 应用程序。 •Middle: 代表 OpenNI ，提供通信接口，同时连接传感 器和中间件组件，后者分析来自传感器的数据。 •Bottom: 硬件设备，捕捉现场的视频和音频内容。
- 50. 通用的框架工具集 •一个 USB 抽象访问层 •一些基本数据类型的实现（包括列表，哈希，等等） •系统日志和转储 •内存和性能分析 •活动（授权要注册的回调函数到一个特定的事件） •任务调度 •不是标准 OpenNI 的一部分（ * ）
- 51. Ca ck func llba •函数指针 •注册 •事件响应
- 52. OpenN I の构成 Generator Metadata Capability
- 55. Sensor Related •Device •Depth Generator •Image Generator •IR Generator •Audio Generator
- 56. M iddlewa Rela re ted •Gesture Generator •Scene Analyzer •Hands Generator •User Generator •//provided by NITE
- 58. Matadata
- 59. Capability
- 60. N I T E の构成 Session Detector
- 61. M iddleware •Full body analysis •Hand point analysis •Gesture detection •Scene Analyzer
- 62. Session
- 63. Detector
- 66. 衣带渐宽终不悔，为伊消得人憔悴。
- 68. Softwa Prepa tion re ra •OpenNI+NITE •Avin’s SensorKinect •Processing(*) •SimpleOpenNI(*) https://github.com/avin2/SensorKinect
- 69. Here’s the code, too young too simple
- 70. import SimpleOpenNI.*; SimpleOpenNI kinect; void setup() { size(640*2, 480); kinect = new SimpleOpenNI(this); kinect.enableDepth(); kinect.enableRGB(); } void draw() { kinect.update(); image(kinect.depthImage(), 0, 0); image(kinect.rgbImage(), 640, 0); }
- 72. import SimpleOpenNI.*; SimpleOpenNI kinect; •Import the library •Declare the SimpleOpenNI object
- 73. void setup() { size(640*2, 480); kinect = new SimpleOpenNI(this); kinect.enableDepth(); kinect.enableRGB(); } •Declare the size of app •Instantiate the object •Call two method
- 74. void draw() { kinect.update(); image(kinect.depthImage(), 0, 0); image(kinect.rgbImage(), 640, 0); } •Get fresh data from kinect •Place the image
- 75. Working with depth map •R G B •纯黑代表无穷远，纯白代表无穷近 •Demo:ex03_basic_depth_plus_rgb
- 76. M ore about depth map •Projective •Realworld •Z 值相同，代表深度，单位 mm •ConvertProjectiveToRealWorld()
- 77. The Kinect coordinate system
- 79. 范围以及精确性 •最近距离： about 0.5(?) •最远距离： 10m •GetDeviceMaxDepth() •官方建议： 1.2 – 3.6 米 •近远精确性比较
- 81. Working with Point Clouds(* ) Welcome to the Third Dimension 3d reconstruction E.g:KinectFusion PCL http://pointclouds.org/
- 83. Working with Skeleton D ata •User Detection •Joints & Skeleton •Distance in 3D •Scene Map, Com •Tracking
- 84. User D etection •Calibration or not •Stages in Process
- 86. Joints & Skeleton •Joints •Distance between two joints in 3D
- 90. Scene M ap •Background remove •User pixels •Center of Mass
- 95. Kinect Hacks Apps out of box Create your own cool stuffs Living coding with VS & Cpp
- 96. M edia Art Demo: chDE_face_changing
- 98. Computer I nterface Demo:Minority Report(*) Demo:2D TrackPad Demo:Hotpoint 3D
- 99. FAAST •Key event mapping •Game playing http://projects.ict.usc.edu/mxr/faast/
- 100. Sex Ga Controller me •DEMO: 宅男妄想地带 •Microsoft: NO!!
- 101. Sorry, Bill
- 102. Control ema ) cs(* •Map(Joint position + gesture, key event)
- 103. Kinect Visua Ca (* ) l mera http://www.kinectdevs.com/forums/kinect-virtual-camera-kinect-webcam-f17
- 104. The following demo need a better computer for higher GPU support
- 105. kinect-ultra ) (* 一秒钟凹凸曼合体！完成你童年 时候的梦想 Video http://code.google.com/p/kinect-ultra/
- 106. kinect-ka meha ) meha (*
- 107. M ikuM ikuD ance(* ) 初音妹子 Video
- 108. What do you want to make? http://www.kinect-hacks.com/openni
- 109. reference •http://viml.nchc.org.tw •Visualization and Interactive Media Laboratory of NCHC •国家科學視算與互動媒體實驗室 •http://www.javaforge.com/wiki/103532 •Kinect for PC Wiki •中文
- 110. T hanks! You a the controller. re Q &A Hang xie sonicisdreaming@gmail.com http://www.xhbang.com
- 111. T his is ba templa sic te Do not delete this page