![cvFindContours
Input needs to be 8-bit binary image.
Make a copy of the image before using cvFindContours
• The original image will be written over by
cvFindContours
OpenCV represents the contour hierarchy as an array:
[next, previous, first_child, parent]
0
1
2
3
4](https://image.slidesharecdn.com/05contourssegmatching-190218095702/85/05-contours-seg_matching-8-320.jpg)
05 contours seg_matching
- 2. Legal Notices and Disclaimers This presentation is for informational purposes only. INTEL MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY. Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. Check with your system manufacturer or retailer or learn more at intel.com. This sample source code is released under the Intel Sample Source Code License Agreement. Intel and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others. Copyright © 2018, Intel Corporation. All rights reserved. 2
- 4. Contours A contour is an assembled collection of edges that (hopefully) represent one object in the image. • Can be used to automatically segment the different objects present in an image.
- 5. Preprocessing for cv2.findContours Can use canny edges • Images created by cvCanny Can use thresholded image where edges are boundaries between white and black • Images created by cvThreshold or cvAdaptiveThreshold Call cv2.findContours on result of Canny or thresholding.
- 6. Contour Hierarchy Parent – Child Hierarchy 0 1 2 3 4 0 1 2 3 4
- 7. Contour Hierarchy cvContour can represent data as a contour tree. 0 and 2 are the contours represented at the root, or parent, nodes. Other contours are represented as children of 0 or 2. 0 1 2 3 4
- 8. cvFindContours Input needs to be 8-bit binary image. Make a copy of the image before using cvFindContours • The original image will be written over by cvFindContours OpenCV represents the contour hierarchy as an array: [next, previous, first_child, parent] 0 1 2 3 4
- 9. Method Variable: Arguments to Find Contours Method clarifies how the contour is being computed. CV_CHAIN_CODE Output = sequence of vertices (Freeman chain code) CV_CHAIN_APPROX_NONE All chain code points = contour points CV_CHAIN_APPROX_SIMPLE Endpoints of horizontal, vertical, diagonal segments CV_CHAIN_APPROX_TC89_L1 Teh_Chin chain algorithm CV_LINK_RUNS Links horizontal segments of 1s Limited to CV_RETR_LIST
- 10. Mode Variable Mode clarifies what contours should be found and desired format for return value. Finds contours and uses horizontal and vertical links to link the found contours. Mode options: CV_RETR_EXTERNAL CV_RETR_LIST CV_ RETR_CCOMP CV_RETR_TREE
- 11. CV_RETR_LIST All contours are put into a list This list does not have parent – child relationships Uses horizontal links All contours are on same level (no actual hierarchy) 0 1 2 3 4
- 13. CV_RETR_CCOMP All contours are used to generate a 2-level hierarchy • Top-level hierarchy = external • Bottom-level = internal • Horizontal and vertical linkers 0 1 2 3 4
- 14. CV_RETR_TREE All contours are used to generate a full hierarchy 0 1 2 3 4
- 15. Drawing Contours Lets look at a new image and draw contours with cv2.drawContours Original Edges Edges used to compute contours
- 16. Hull Borders Compute a convex hull cv2.convexHull()
- 17. Hull Borders Compute convexity defects cv2.convexityDefects Checks contours to see if it is convex.
- 18. Length and Area Length • cv2.ContourPerimeter Returns length of a contour You can summarize length and area as a bounding structure • Boxes • Circles • Ellipses
- 19. Bounding Boxes cv2.BoundingRect() • Returns rectangle that bounds the contour • Red box in image cv2.MinAreaRec2() • Returns smallest rectangular bound • Green box in image Note green box is rotated for best fit
- 20. Circles and Ellipses cv2.MinEnclosingCirlce() • Similar to cv2.BoundingRect() in that it bounds the contour, but not a best fit • Needs radius as input • Red circle in image cv2.FitEllipse2() • Allows us to get a best fit bound • Green circle image
- 21. Geometric Toolkits/Checks cv2.maxRect() • Two input rectangles are used to compute a new rectangle cv2.BoxPoints() • Computers corner points of cvBox2D structure cv2.PointPolygonTest() • Test: is point in polygon?
- 22. Contour Statistics 𝐴𝑠𝑝𝑒𝑐𝑡 𝑅𝑎𝑡𝑖𝑜𝑛 = 𝑊𝑖𝑑𝑡ℎ 𝐻𝑒𝑖𝑔ℎ𝑡 𝐸𝑥𝑡𝑒𝑛𝑡 = 𝑂𝑏𝑗𝑒𝑐𝑡 𝐴𝑟𝑒𝑎 𝐵𝑜𝑢𝑛𝑑𝑖𝑛𝑔 𝑅𝑒𝑐𝑡𝑎𝑛𝑔𝑙𝑒 𝐴𝑟𝑒𝑎 𝑆𝑜𝑙𝑖𝑑𝑖𝑡𝑦 = 𝐶𝑜𝑛𝑡𝑜𝑢𝑟 𝐴𝑟𝑒𝑎 𝐶𝑜𝑛𝑣𝑒𝑥 𝐻𝑢𝑙𝑙 𝐴𝑟𝑒𝑎 𝐸𝑞𝑢𝑖𝑣𝑎𝑙𝑒𝑛𝑡 𝐷𝑖𝑎𝑚𝑒𝑡𝑒𝑟 = 4 𝑥 𝐶𝑜𝑛𝑡𝑜𝑢𝑟 𝐴𝑟𝑒𝑎 𝜋 𝑂𝑟𝑖𝑒𝑛𝑡𝑎𝑡𝑖𝑜𝑛 𝐴𝑛𝑔𝑙𝑒 = 𝑐𝑣2. 𝑓𝑖𝑡𝐸𝑙𝑙𝑖𝑝𝑠𝑒(𝑐𝑛𝑡) 𝑥, 𝑦, 𝑤, ℎ = 𝑐𝑣2. 𝑏𝑜𝑢𝑛𝑑𝑖𝑛𝑔𝑅𝑒𝑐𝑡(𝑐𝑜𝑛𝑡) 𝑎𝑟𝑒𝑎 = 𝑐𝑣2. 𝑐𝑜𝑛𝑡𝑜𝑢𝑟𝐴𝑟𝑒𝑎(𝑐𝑜𝑛𝑡) ℎ𝑢𝑙𝑙 = 𝑐𝑣2. 𝑐𝑜𝑛𝑣𝑒𝑥𝐻𝑢𝑙𝑙(𝑐𝑜𝑛𝑡) ℎ𝑢𝑙𝑙_𝑎𝑟𝑒𝑎 = 𝑐𝑣2. 𝑐𝑜𝑛𝑡𝑜𝑢𝑟𝐴𝑟𝑒𝑎(ℎ𝑢𝑙𝑙)
- 23. Finding Extreme Points: Min/Max Min/Max is computed with NumPy. Extreme points of the cross are min/max left/right and up/down.
- 25. Foreground/Background Separation Separate foreground from background Send foreground on for further analysis • Cars in security camera • Skin within an image (reduces complexity of finding faces) Superpixels • Groups of pixels in same object/type of object
- 26. Methods: Image Pyramids Collection of images where each subsequent image is a downsampling of the previous. Gaussian pyramid Used to downsample Removes even rows and columns Will give you higher resolution Laplacian pyramid Used to upsample Works with lower-resolution images and allows for faster computations
- 27. Gaussian Pyramid Use cvPyrDown() with a 5x5 Guassian kernel • Start with G0 (original image) • Convolve G0 with Gaussian kernel • Gi is ¼ size of G0 because we removed even-numbered rows and columns from G0 • Repeat for all desired Gi+1
- 28. Laplacian Kernel Use cvPyrUp() Notice the relationship between Gausian and Laplacian In OpenCV we see this as: Lapi = Gausi – PyrUp(Gausi+1 ) Remember, we saw this in Week 3!
- 29. Methods to Segment an Image cvPyrSegmentation () • Generates an image pyramid • Associates pyramid layers to parent-child relationships • Map pixels between Gi+1 and Gi • Perform segmentation on lower-resolution images • Note: Start image must be divisible by 2 for each layer of the pyramid
- 30. Mean Shift Clustering Over Color pyrMeanShiftFiltering() Peak of color-spatial distribution over space Data dimensions are: Spatial (x, y) Color (blue, green, red) Parameters are spatialRadius, colorRadius, max_level, and CvTermCriteria.
- 31. Watershed Used to separate objects when you do not have a separate background image. Input is a grayscale image or a smoothed color image. Can cause oversegmentation or undersegmentation • If you make too many basins or too few Lines converted to peaks; uniform regions to catchment basins.
- 32. Watershed Method Take the gradient of intensity image. Marker points, or known areas of region of interest, are defined by user. Basins get connected to marker points to create segments.
- 33. Watershed: Example Step 1: Threshold a color image
- 34. Watershed: Example Step 2: Remove noise with cv2.morphologyEx
- 35. Watershed: Example Step 3: Sure background area with cv2.dilate()
- 36. Watershed: Example Step 4: Find sure foreground area with cv2.distanceTransform() and cv2.threshold()
- 37. Watershed: Example Step 5: Find unknown region with cv2.subtract()
- 38. Grabcut: Graph Cutting Techniques Pixel-based energy function Basic binary segmentation algorithm to identify object Iteratively recalculates the region statistics (Gaussians) to perform segmentation
- 39. GrabCut Procedure (1) Assign background and unknown Unknown becomes potential foreground (2) Create five probabilistic clusters (each) of foreground and background (3) Assign every background pixel to a cluster of background Do the same for foreground (4) Compute cluster statistics for all clusters
- 40. GrabCut Procedure (continued) (5) Create a graph where: Pixels are linked to their neighbors and to two special nodes Nodes represent foreground and background The weights of the links are related to neighbor similarity and class probability
- 41. GrabCut Procedure (continued) (6) Apply a classic computer science graph theory algorithm min-cut to the graph The net result cuts the connections from each pixel to one of the two special nodes. In turn, this gives us a new assignment for foreground/background to the pixels. (7) Until convergence, we loop back to Step 4 and repeat Classifications remain similar enough from one round to the next.
- 43. Contour Moments: Matching Contours Compare statistical moments of contours: In statistics, first moment is the mean, second moment is the variance, and so on. Moments are used to compare two contours (outlines) for similarity. Another technique: Compute rough contour characteristic computed by summation of all pixels over contour boundary: p: x-order q: y-order Normalized moments are better for comparing similar shapes of different sizes. 𝑚 𝑝,𝑞 = 𝑖=1 𝑛 𝐼 𝑥, 𝑦 𝑥 𝑝 𝑦 𝑞
- 44. Statistical Moments In classical statistics, we use mean, variance, skew, and kurtosis. cvMoments() cvGetCentralMoment() invariant with respect to translation cvGetNormalizedMoment() invariant with respect to scale cvGetHuMoments() invariant with respect to rotation
- 45. Normalized Moments Central moments normalized with respect to a function intensity. First need to calculate central moment: Then you can calculate normalized moments: 𝜂 𝑝,𝑞 = 𝜇 𝑝,𝑞 𝑚00 𝑝 + 𝑞 /2 + 1 𝜇𝑝, 𝑞 = 𝑥,𝑦 𝐼 𝑥, 𝑦 𝑥 − 𝑥𝑎𝑣𝑔 𝑝 𝑦 − 𝑦𝑎𝑣𝑔 𝑞
- 46. Hu Invariant Moments Linear combinations of centralized moments. Use central moments to get invariant functions • Invariant to scale, rotation, and reflection cvGetHuMoments() ℎ1 = 𝜂20 + 𝜂02 ℎ4 = (𝜂30+ 𝜂12)2 + (𝜂21 + 𝜂03)2
- 48. Template Matching Overview Use cvMatchTemplate() to find one image within another. Uses the actual image of matching object instead of a histogram • This is called an image patch Input image • 8-bit, floating point plane, or color image Output • Single-channel byte or floating point image
- 49. Template Matching Methods Squared difference CV_TM_SQDIFF_NORMED Correlation CV_TM_CCORR_NORMED Correlation coefficient CV_TM_CCOEFF_NORMED 𝑅 𝑠𝑞𝑑𝑖𝑓𝑓 𝑥, 𝑦 = 𝑅 𝑠𝑞𝑑𝑖𝑓𝑓(𝑥, 𝑦) 𝑍(𝑥, 𝑦) 𝑅 𝑐𝑐𝑜𝑟𝑟 𝑥, 𝑦 = 𝑅 𝑐𝑐𝑜𝑟𝑟(𝑥, 𝑦) 𝑍(𝑥, 𝑦) 𝑅 𝑐𝑐𝑜𝑒𝑓𝑓 𝑥, 𝑦 = 𝑅 𝑐𝑐𝑜𝑒𝑓𝑓(𝑥, 𝑦) 𝑍(𝑥, 𝑦)
- 50. Histogram Backprojection Uses a histogram model to match pixels (or patches of pixels) to approach object recognition.
- 51. Histogram Backprojection Better thought of as segmentation to foreground ROIs used for further analysis. If you have a histogram of the ROI, we can use that to find the object in another image. Use histogram from one image to match to histogram profile from another image Normalize to pixel number Find matching object, or ROI cvCalcBackProjectPatch()
- 52. Histogram Backprojection Given a color, we want to know the probability of foreground/background. Approximate probability that a particular color belongs to a particular object You can also make a ratio histogram where you de-emphasize colors not in the ROI. 𝑝 𝑜𝑏𝑗𝑒𝑐𝑡 𝑐𝑜𝑙𝑜𝑟 = 𝑝 𝑐𝑜𝑙𝑜𝑟 𝑜𝑏𝑗𝑒𝑐𝑡 ∗ 𝑝(𝑜𝑏𝑗𝑒𝑐𝑡) 𝑝(𝑐𝑜𝑙𝑜𝑟) ~ 𝑝 𝑐𝑜𝑙𝑜𝑟 𝑜𝑏𝑗𝑒𝑐𝑡
- 53. Backprojection Method 1. Calculate ROI histogram 2. Normalize histogram 3. Apply cv2.calcBackProject 4. Convolve with circular disc 5. Duplicate threshold
Editor's Notes
- #6: Recall: White is the object and black is the background.
- #7: Quick test for students: What would happen if there were a box around everything?
- #16: We have only drawn the contour for the last lightning bolt.
- #18: NOTE: Green points are the defects. Defects are points furthest from the outer contour. For example, the top-right green is furthest from the top-right red contour.
- #24: Calculated using NumPy, not built-ins.
- #34: Image url?
- #40: Probabilistic clusters are components of a Gaussian mixture model. The class node probabilities can be set to 100% FG/BG, which guarantees the pixel keeping its association with that class after min-cutting.
- #53: NOTE: Equality is from Bayes’ Theorem. Approximation is made by assuming p(o)/p(c) is a constant, which, if we normalize probabilities, can be dropped.