![Loading the Classifier
• Just call cvLoad
NSString
*path
=
[[NSBundle
mainBundle]
pathForResource:@"haarcascade_frontalface_default"
ofType:@"xml"];
CvHaarClassifierCascade
*cascade
=
(CvHaarClassifierCascade
*)cvLoad(
[path
cStringUsingEncoding:NSASCIIStringEncoding],
NULL,
NULL,
NULL);](https://image.slidesharecdn.com/accelerometerstoar-110813150546-phpapp01/85/Accelerometers-to-Augmented-Reality-17-320.jpg)
![Feature Matching
• SIFT, SURF, FLANN:
Salient points in an
image (a) (b)
...
Scale
(next
octave)
(c) (d)
Figure 5: This figure shows the stages of keypoint selection. (a) The 233x189 pixel original image.
(b) The initial 832 keypoints locations at maxima and minima of the difference-of-Gaussian function.
Keypoints are displayed as vectors indicating scale, orientation, and location. (c) After applying
a threshold on minimum contrast, 729 keypoints remain. (d) The final 536 keypoints that remain
following an additional threshold on ratio of principal curvatures.
Scale
(first
octave) As suggested by Brown, the Hessian and derivative of D are approximated by using dif-
ferences of neighboring sample points. The resulting 3x3 linear system can be solved with
minimal cost. If the offset x is larger than 0.5 in any dimension, then it means that the ex-
ˆ
tremum lies closer to a different sample point. In this case, the sample point is changed and
Difference of the interpolation performed instead about that point. The final offset x is added to the location
ˆ
Gaussian Gaussian (DOG) of its sample point to get the interpolated estimate for the location of the extremum.
The function value at the extremum, D(ˆ), is useful for rejecting unstable extrema with
x
Figure 1: For each octave of scale space, the initial image is repeatedly convolved with Gaussians to
produce the set of scale space images shown on the left. Adjacent Gaussian images are subtracted low contrast. This can be obtained by substituting equation (3) into (2), giving
to produce the difference-of-Gaussian images on the right. After each octave, the Gaussian image is
down-sampled by a factor of 2, and the process repeated. Image gradients 1 ∂D T Keypoint descriptor
D(ˆ) = D +
x x.
ˆ
2 ∂x
Figure 7: A keypoint descriptor is created by first computing the gradient magnitude and orientation
In addition, the difference-of-Gaussian function provides a close approximation to the at each image sample point in a region around the keypoint location,value of |D(ˆ)| less than 0.03 were
For the experiments in this paper, all extrema with a as shown onx left. These are
the
scale-normalized Laplacian of Gaussian, σ 2 2 G, as studied by Lindeberg (1994). Lindeberg weighted bydiscarded (as before,indicated by the overlaid circle. These samples are then accumulated
a Gaussian window, we assume image pixel values in the range [0,1]).
showed that the normalization of the Laplacian with the factor σ 2 is required for true scale into orientation histograms summarizing the contents over 4x4 subregions, as shown on the right, with too
Figure 5 shows the effects of keypoint selection on a natural image. In order to avoid](https://image.slidesharecdn.com/accelerometerstoar-110813150546-phpapp01/85/Accelerometers-to-Augmented-Reality-23-320.jpg)
Accelerometers to Augmented Reality
- 1. From Accelerometers to Augmented Reality Jonathan Blocksom @jblocksom iOSDevCamp 2011 August 13, 2011 Jonathan Saggau (@jonmarimba) present in spirit
- 2. The Theme • Mobile devices are not just for viewing information on a screen; they can be a gateway to interacting with the world around us. • What can we do with them and how do we do it?
- 3. iOS and the World Around Us Purpose Sensor SDK Device Movement Accelerometer, Gyros Core Motion Geolocation GPS, Magnetometer Core Location Video Camera AVFoundation AVFoundation, Audio Microphone Core Jose
- 4. Core Motion • High level interface to orientation and movement data • iOS 4+ • Accelerometer and Gyroscope • Sensor Fusion and Data Filtering
- 5. Motion Sensors: Accelerometer • Accelerometer • From iPhone 1 • Noisy Gravity Detector !"#$#%&'%()*+,%-#,.#/% "012334445+67+$58#93:;+,<3=9)><39<$)3=!?@A1BCDEE@4E&>% • 100Hz, +/- 2.3G
- 6. Motion Sensors: Gyroscopes • Gyroscope • iPhone 4, iPad 2 • Rotation Rate • 200/500/2500 dps
- 7. Demo: Vibration • Visualize Accelerometer Data • App Store, $4.99 http:// itunes.apple.com/ app/vibration/ id301097580
- 8. Demo 1: Core Motion Teapot
- 9. Using Core Motion • Poll for data or block handlers for updates • Data as Yaw, Pitch, Roll Quaternion 4x4 Transform • Timestamps included
- 10. Demo: Core Motion Viewer • https://bitbucket.org/jblocksom/ coremotionviewer
- 11. Core Motion Resources • Core Motion Framework Reference • Event Handling Guide for iPhone OS • “Core Motion” under “Motion Events” • WWDC ’10: CoreMotionTeapot • J. R. Powers Talk from VTM November ’10 • O’Reilly Basic Sensors in iOS
- 12. Computer Vision
- 13. Computer Vision: OpenCV • The SDK I love to hate
- 14. Building OpenCV on iOS • Follow this 20 step recipe: http://computer-vision-talks.com/ 2010/12/building-opencv-for-ios/ • Or go here: https://github.com/jonmarimba/ OpenCV-iOS
- 15. Face Detection • Use Haar Wavelet classification • Built in classifiers in OpenCV to find front and side facing faces • Not perfect, not too fast, but not bad • Video: http://vimeo.com/12774628
- 16. Haar classification • “Cascade of boosted classifiers working with haar- like features”
- 17. Loading the Classifier • Just call cvLoad NSString *path = [[NSBundle mainBundle] pathForResource:@"haarcascade_frontalface_default" ofType:@"xml"]; CvHaarClassifierCascade *cascade = (CvHaarClassifierCascade *)cvLoad( [path cStringUsingEncoding:NSASCIIStringEncoding], NULL, NULL, NULL);
- 18. Running the classifier CvSeq* faces = cvHaarDetectObjects(small_image, cascade, storage, 1.2f, 2, CV_HAAR_DO_CANNY_PRUNING, cvSize(20, 20)); • Image, Haar cascades, spare storage • 1.2f: Size inc. for features per stage • 2: Minimum rectangle neighbors • Canny Pruning: Throw out areas with too few / too many edges
- 19. What it’s Doing • Windows show where wavelets being checked • Overlapping rectangles are a detection
- 20. Defeating Face Detection • cvDazzle project • Can also just turn to the side
- 21. Demo: Face Detection • OpenCV based
- 22. Feature Matching • Feature Matching is the workhorse of modern computer vision • Panoramas • Image stabilization • Superresolution • 3D reconstruction
- 23. Feature Matching • SIFT, SURF, FLANN: Salient points in an image (a) (b) ... Scale (next octave) (c) (d) Figure 5: This figure shows the stages of keypoint selection. (a) The 233x189 pixel original image. (b) The initial 832 keypoints locations at maxima and minima of the difference-of-Gaussian function. Keypoints are displayed as vectors indicating scale, orientation, and location. (c) After applying a threshold on minimum contrast, 729 keypoints remain. (d) The final 536 keypoints that remain following an additional threshold on ratio of principal curvatures. Scale (first octave) As suggested by Brown, the Hessian and derivative of D are approximated by using dif- ferences of neighboring sample points. The resulting 3x3 linear system can be solved with minimal cost. If the offset x is larger than 0.5 in any dimension, then it means that the ex- ˆ tremum lies closer to a different sample point. In this case, the sample point is changed and Difference of the interpolation performed instead about that point. The final offset x is added to the location ˆ Gaussian Gaussian (DOG) of its sample point to get the interpolated estimate for the location of the extremum. The function value at the extremum, D(ˆ), is useful for rejecting unstable extrema with x Figure 1: For each octave of scale space, the initial image is repeatedly convolved with Gaussians to produce the set of scale space images shown on the left. Adjacent Gaussian images are subtracted low contrast. This can be obtained by substituting equation (3) into (2), giving to produce the difference-of-Gaussian images on the right. After each octave, the Gaussian image is down-sampled by a factor of 2, and the process repeated. Image gradients 1 ∂D T Keypoint descriptor D(ˆ) = D + x x. ˆ 2 ∂x Figure 7: A keypoint descriptor is created by first computing the gradient magnitude and orientation In addition, the difference-of-Gaussian function provides a close approximation to the at each image sample point in a region around the keypoint location,value of |D(ˆ)| less than 0.03 were For the experiments in this paper, all extrema with a as shown onx left. These are the scale-normalized Laplacian of Gaussian, σ 2 2 G, as studied by Lindeberg (1994). Lindeberg weighted bydiscarded (as before,indicated by the overlaid circle. These samples are then accumulated a Gaussian window, we assume image pixel values in the range [0,1]). showed that the normalization of the Laplacian with the factor σ 2 is required for true scale into orientation histograms summarizing the contents over 4x4 subregions, as shown on the right, with too Figure 5 shows the effects of keypoint selection on a natural image. In order to avoid
- 24. Application: Automatic Panoramas • Application: Panoramas
- 25. Tracking • Feature finding and matching is slow • Lower quality features can match faster with same results
- 28. Augmented Reality • Fuse live video with generated pixels based on device sensors • Geolocated • Marker Based • Commercial SDKs available
- 29. Geolocated AR • AR Based on GPS Location • Fuse rendered objects with real world locations
- 30. 3DAR Toolkit • http://spotmetrix.com/ • Drop in replacement for MKMapView • Shows AR view based on phone orientation • Free if branded • $5K for unbranded
- 32. Marker Based AR • Find a marker • Figure out camera transform to it • Render something on top of it • String SDK • Qualcomm AR SDK
- 33. Demo: Marker Based AR SDK License Notes NYAR GPL Old String Commercial $ http://poweredbystring.com Qualcomm http://developer.qualcomm.com/dev/ Commercial, No cost Still in beta augmented-reality
- 34. Qualcomm SDK • FLANN to find initial features • FAST to update after marker is found
- 35. That’s It! • Qualcomm AR SDK: http://developer.qualcomm.com/dev/ augmented-reality • String SDK: http://poweredbystring.com • Me: http://twitter.com/jblocksom/