![of 15
What is a Feature?
Example: Possible features of a student:
• Number of eyes x {0, 1, 2}
• Hair color x {0 =Black, 1 =Blond, 2 =Red ,…}
• Wear glasses or not x {0, 1}
• Hair length [cm] x [0..100]
• Shoe size [U.S] x [3.5, 4, 4.5, .. ,14]
• Height [cm] x [40..240]
• Weight [kg] x [30..600]
Feature is a scalar x which is quantitatively
describes a property of the Object.](https://image.slidesharecdn.com/objectrecognition-140321230541-phpapp02/85/Object-Recognition-44-320.jpg)
![Alon Slapak of 15
What is a Pattern?
Pattern is a n-tuple X (vector) of N scalars xi
i [1,N], which are called the Features.
Conventional form of a pattern is:
1 2
, ,
T
N i
X x x x x X V
Where V is known as the Feature Space, and
N is the dimension of V.](https://image.slidesharecdn.com/objectrecognition-140321230541-phpapp02/85/Object-Recognition-46-320.jpg)
![of 15
Example
Assume we are using the
height and the weight of
each of the students in
the university as a
pattern.
120 130 140 150 160 170 180 190 200 210 220
30
40
50
60
70
80
90
100
110
120
height [cm]
weight[kg]
The height and the weight
are both features, which
span a feature space V of
dimension 2.
Each student is characterized by a
vector of two feature: (height,
weight).
Since the male students and the
female students differ from each
other in height and weight, we are
expected to have two separated
clusters.
Female
s
Males
Each of the students is
represented as a point in
the feature space. Patterns
of male students are
depicted in blue, and those
of female students – in red.](https://image.slidesharecdn.com/objectrecognition-140321230541-phpapp02/85/Object-Recognition-48-320.jpg)
![of 15
What is Classification?
Classification is a mathematical function or
algorithm which assigns a feature to one
of the classes.
Example:
We can draw a line
between the two
clusters in the gender
example, and every
student will be classified
as a female or male
according to this line.
120 130 140 150 160 170 180 190 200 210 220
30
40
50
60
70
80
90
100
110
120
height [cm]
weight[kg] Males
Females](https://image.slidesharecdn.com/objectrecognition-140321230541-phpapp02/85/Object-Recognition-50-320.jpg)
Object Recognition
- 1. Object Recognition Prepared By: Eman Abed AlWahhab
- 2. Outline • Introduction (Computer Vision) • History • Human Vision Vs. Computer Vision • Main Goal of Computer Vision • Significance of Computer Vision • Connections to other Disciplines • Key Stages in Digital Image Processing • Object Recognition • What is Object Recognition? • What is Pattern Recognition? • Approaches • Applications • Main Components • Gender Example 2
- 3. Outline • Fingerprint Recognition • Definition • Fingerprint Matching Using Ridge-End and Burification • Fingerprint Image • Binarization • Thinning • Minutiae Extraction • Car Number Plate Recognition • What is an ALPR System? • ALPR Procedure • Characters Recognition • Characters Segmentation • Normalization of Characters • New Innovations in Object Recognition • References 3
- 4. Brief History of Computer Vision • 1966: Minsky assigns computer vision as an undergrad summer project • 1960’s: interpretation of synthetic worlds • 1970’s: some progress on interpreting selected images • 1980’s: ANNs come and go; shift toward geometry and increased mathematical rigor • 1990’s: face recognition; statistical analysis in vogue • 2000’s: broader recognition; large annotated datasets available; video processing starts • 2030’s: robot uprising? Guzman ‘68 Ohta Kanade ‘78 Turk and Pentland ‘91 4
- 5. Vision is the process of discovering what is present in the world and where it is by looking Human Vision Computational algo implemented in this massive network of neurons; they obtain their inputs from retina, & produce as output an “understanding” of the scene in view But what does it mean to “understand” the scene? What algos & data representation are used by brain? 5
- 6. Computer Vision is the study of analysis of pictures and videos in order to achieve results similar to those as by humans Computer Vision Analogously, given a set of TV camera What computer architectures, data structures & algorithms should use to create a machine that can “see” as we do? 6
- 7. Human Vision VS Computer Vision What we see What a computer sees 7
- 8. Main Goal of Computer Vision Every picture tells a story!! * write computer programs that can interpret images * 8
- 9. Significance of Computer Vision Safety Health Security Comfort AccessFun 9
- 10. Connections to other Disciplines 10
- 11. What is Digital Image Processing? •The continuum from image processing to computer vision can be broken up into low-, mid- and high-level processes Low Level Process Input: Image Output: Image Examples: Noise removal, image sharpening Mid Level Process Input: Image Output: Attributes Examples: Object recognition, segmentation High Level Process Input: Attributes Output: Understanding Examples: Scene understanding, autonomous navigation 11
- 12. Key Stages in Digital Image Processing Image Acquisition Image Restoration Morphological Processing Segmentation Representation & Description Image Enhancement Object Recognition Problem Domain Colour Image Processing Image Compression 12
- 13. Key Stages in Digital Image Processing: Image Acquisition Image Acquisition Image Restoration Morphological Processing Segmentation Representation & Description Image Enhancement Object Recognition Problem Domain Colour Image Processing Image Compression ImagestakenfromGonzalez&Woods,DigitalImageProcessing(2002) 13
- 14. Key Stages in Digital Image Processing: Image Enhancement Image Acquisition Image Restoration Morphological Processing Segmentation Representation & Description Image Enhancement Object Recognition Problem Domain Colour Image Processing Image Compression ImagestakenfromGonzalez&Woods,DigitalImageProcessing(2002) 14
- 15. Key Stages in Digital Image Processing: Image Restoration Image Acquisition Image Restoration Morphological Processing Segmentation Representation & Description Image Enhancement Object Recognition Problem Domain Colour Image Processing Image Compression ImagestakenfromGonzalez&Woods,DigitalImageProcessing(2002) 15
- 16. Key Stages in Digital Image Processing: Morphological Processing Image Acquisition Image Restoration Morphological Processing Segmentation Representation & Description Image Enhancement Object Recognition Problem Domain Colour Image Processing Image Compression ImagestakenfromGonzalez&Woods,DigitalImageProcessing(2002) 16
- 17. Key Stages in Digital Image Processing: Segmentation Image Acquisition Image Restoration Morphological Processing Segmentation Representation & Description Image Enhancement Object Recognition Problem Domain Colour Image Processing Image Compression ImagestakenfromGonzalez&Woods,DigitalImageProcessing(2002) 17
- 18. Key Stages in Digital Image Processing: Object Recognition Image Acquisition Image Restoration Morphological Processing Segmentation Representation & Description Image Enhancement Object Recognition Problem Domain Colour Image Processing Image Compression ImagestakenfromGonzalez&Woods,DigitalImageProcessing(2002) 18
- 19. Key Stages in Digital Image Processing: Representation & Description Image Acquisition Image Restoration Morphological Processing Segmentation Representation & Description Image Enhancement Object Recognition Problem Domain Colour Image Processing Image Compression ImagestakenfromGonzalez&Woods,DigitalImageProcessing(2002) 19
- 20. Key Stages in Digital Image Processing: Image Compression Image Acquisition Image Restoration Morphological Processing Segmentation Representation & Description Image Enhancement Object Recognition Problem Domain Colour Image Processing Image Compression 20
- 21. Key Stages in Digital Image Processing: Colour Image Processing Image Acquisition Image Restoration Morphological Processing Segmentation Representation & Description Image Enhancement Object Recognition Problem Domain Colour Image Processing Image Compression ImagestakenfromGonzalez&Woods,DigitalImageProcessing(2002) 21
- 22. What is Object Recognition? • Last step in image processing • It is the task of finding and identifying objects in an image or video sequence Like human understanding, it includes : • Detection – of separate objects • Description – of their geometry and positions in 3D • Classification – as being one of a known class • Identification – of the particular instance • Understanding – of spatial relationships between objects 22
- 23. So what does object recognition involve?
- 24. Detection: are there people?
- 25. Verification: is that a lamp?
- 26. Identification: is that Potala Palace?
- 28. Scene and context categorization/Understanding • outdoor • city • … Slide credit Fei-Fei, Fergus, Torralba CVPR07 Short Course
- 29. Learning and Adaptation • Supervised learning – A teacher provides a category label or cost for each pattern in the training set • Unsupervised learning – The system forms clusters or “natural groupings” of the input patterns 29
- 30. The Design Cycle
- 31. What is Pattern Recognition? • A pattern is an object, process or event that can be given a name. • A pattern class (or category) is a set of patterns sharing common attributes and usually originating from the same source. • During recognition (or classification) given objects are assigned to prescribed classes. • A classifier is a machine which performs classification. “The assignment of a physical object or event to one of several prespecified categeries” -- Duda & Hart
- 32. 32 Basic Components of a Pattern Recognition System
- 33. 33 Components of Pattern Recognition (Cont’d) • Data acquisition and sensing • Pre-processing Removal of noise in data. Isolation of patterns of interest from the background. • Feature extraction Finding a new representation in terms of features. (Better for further processing)
- 34. 34 Components of Pattern Recognition (Cont’d) • Model learning and estimation Learning a mapping between features and pattern groups. • Classification Using learned models to assign a pattern to a predefined category • Post-processing Evaluation of confidence in decisions. Exploitation of context to improve performances.
- 35. 35 Examples of Pattern Recognition Applications
- 36. Pattern Representation • A pattern is represented by a set of d features, or attributes, viewed as a d-dimensional feature vector. 1 2 ( , , , ) T d x x xx
- 37. Basic concepts y x n x x x 2 1 Feature vector - A vector of observations (measurements). - is a point in feature space . Hidden state - Cannot be directly measured. - Patterns with equal hidden state belong to the same class. Xx x X Yy Task - To design a classifer (decision rule) which decides about a hidden state based on an onbservation. YX:q Pattern
- 38. Feature Extraction Task: to extract features which are good for classification. Good features: • Objects from the same class have similar feature values. • Objects from different classes have different values. “Good” features “Bad” features
- 39. Feature Extraction Methods k m m m 2 1 n x x x 2 11 φ 2 φ n φ k m m m m 3 2 1 n x x x 2 1 Feature extraction Feature selection Problem can be expressed as optimization of parameters of featrure extractor Supervised methods: objective function is a criterion of separability (discriminability) of labeled examples, e.g., linear discriminat analysis (LDA). Unsupervised methods: lower dimesional representation which preserves important characteristics of input data is sought for, e.g., principal component analysis (PCA). φ(θ)
- 40. Classifier A classifier partitions feature space X into class-labeled regions such that ||21 Y XXXX }0{||21 Y XXX and 1 X 3 X 2 X 1 X 1 X 2 X 3 X The classification consists of determining to which region a feature vector x belongs to. Borders between decision boundaries are called decision regions.
- 41. Representation of classifier A classifier is typically represented as a set of discriminant functions ||,,1,:)(f YX ii x The classifier assigns a feature vector x to the i-the class if )(f)(f xx ji ij )(f1 x )(f2 x )(f || xY maxx y Feature vector Discriminant function Class identifier
- 42. of 15 Block diagram Both definitions may be depicted by the following block diagram. Object Pattern Class / Category Class / Category Class / Category The process consists of two major operations: Feature extraction Classification
- 43. of 15 Example : Gender Assume an algorithm to recognize the gender of a student in a university, where the available input is several features of the students (of course, the gender cannot be one of the features). The student to be classified is the Object, The gender (Male or Female) are the Classes, and the input which is referred to the student is the Pattern.
- 44. of 15 What is a Feature? Example: Possible features of a student: • Number of eyes x {0, 1, 2} • Hair color x {0 =Black, 1 =Blond, 2 =Red ,…} • Wear glasses or not x {0, 1} • Hair length [cm] x [0..100] • Shoe size [U.S] x [3.5, 4, 4.5, .. ,14] • Height [cm] x [40..240] • Weight [kg] x [30..600] Feature is a scalar x which is quantitatively describes a property of the Object.
- 45. of 15 What is Feature Extraction? “When we have two or more classes, feature extraction consist of choosing those features which are most effective for preserving class separability”(Fukunaga p. 441) Assume we choose the shoe size of the student as a feature. The selection is heuristically and seems reasonable.
- 46. Alon Slapak of 15 What is a Pattern? Pattern is a n-tuple X (vector) of N scalars xi i [1,N], which are called the Features. Conventional form of a pattern is: 1 2 , , T N i X x x x x X V Where V is known as the Feature Space, and N is the dimension of V.
- 47. of 15 Possible patterns for the gender problem: We can use the shoe size alone to have: X Shoe size We can combine the height and the weight to have: , T X height weight We can even combine the height, weight and the shoe size to be on the safe side: , , T X height weight shoe size Or, we can use them all: # , , , , , , T X of eyes hair color glasses hair length height weight shoe size
- 48. of 15 Example Assume we are using the height and the weight of each of the students in the university as a pattern. 120 130 140 150 160 170 180 190 200 210 220 30 40 50 60 70 80 90 100 110 120 height [cm] weight[kg] The height and the weight are both features, which span a feature space V of dimension 2. Each student is characterized by a vector of two feature: (height, weight). Since the male students and the female students differ from each other in height and weight, we are expected to have two separated clusters. Female s Males Each of the students is represented as a point in the feature space. Patterns of male students are depicted in blue, and those of female students – in red.
- 49. of 15 What is a Class? “Class is a set of patterns that share some common properties” (Wang p.10) In our example, the Male students and the Female students are two classes of objects that share a common gender.
- 50. of 15 What is Classification? Classification is a mathematical function or algorithm which assigns a feature to one of the classes. Example: We can draw a line between the two clusters in the gender example, and every student will be classified as a female or male according to this line. 120 130 140 150 160 170 180 190 200 210 220 30 40 50 60 70 80 90 100 110 120 height [cm] weight[kg] Males Females
- 51. of 15 Clusters Separation Misclassifications are a consequence of the separation of the clusters. The separation of clusters is quantified using two major methods: 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100 Separable clusters 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100 Almost separable clusters 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100 Non-separable clusters 1. Mathematically: there are several separation criteria’s. 2. “Intuitively”: overlapping of the clusters.
- 52. of 15 Classification Quality WARNING!!! Although the idea is well illustrated, it is a bad habit to judge a classification quality according to the visual representation of clusters. The classification quality is strongly depends on the clusters separation The clusters separation strongly depends on the features selection Feature selection is of paramount importance in classification quality
- 53. 53
- 54. Fingerprint The popular Biometric used to authenticate person is Fingerprint which is unique and permanent throughout a person’s life Fingerprint is the pattern of ridges and valleys The ridges have characteristics, called minutiae, are the ridge ending and the ridge bifurcation Ridge ending is defined as the point where ridge ends abruptly Ridge bifurcation is defined as the point where a ridge forks into branch ridges 54
- 56. Ridge Ending and Bifurcation 56
- 57. Fingerprint Recognition Fingerprint recognition or fingerprint authentication refers to the method of verifying a match between two human fingerprint Fingerprint recognition techniques have the advantage to use low- cost standard capturing device However , recognition of the fingerprint becomes a complex computer vision problem , especially when dealing with noisy and low quality images A minutia matching is widely used for fingerprint recognition and can be classified as ridge ending and ridge bifurcation 57
- 58. Fingerprint Matching using Ridge-End and Bifurcation Points 58
- 59. Fingerprint Image • The input fingerprint image is the gray scale image of a person, which has intensity values ranging from 0 to 255 • A number of methods are used to acquire fingerprints • The inked impression method remains the most popular one • Inkless fingerprint scanners are also present 59
- 60. Inked method Inkless method 60
- 61. Binarization Binarization is used to convert gray scale image into binary image by fixing the threshold value The pixel values above the threshold are set to ‘1’ and the pixel values below the threshold are set to ‘0’ respectively 61
- 62. Original Fingerprint Binarized Fingerprint 62
- 63. Thinning The binarized image is thinned using Block Filter To reduce the thickness of all ridge lines to a single pixel width to extract minutiae points effectively Thinning does not change the location of minutiae points compared to original fingerprint 63
- 64. Binarized Fingerprint Image after Thining 64
- 65. Minutiae Extraction Classification of ridge-end and ridge bifurcation points is done by creating matrix Crossing Number is used to locate the minutiae points in fingerprint image Crossing Number is defined as half of the sum of differences between intensity values of two adjacent pixels 65
- 66. • If crossing Number is 1 minutiae points are classified as Termination • If crossing Number is 2 minutiae points are classified as Normal ridge • If crossing Number 3 or greater than 3 minutiae points are classified as Bifurcation 66
- 67. Crossing Number and Type of Minutiae 67
- 68. Gray-scale Fingerprint Minutiae points 68
- 69. Minutiae Matching Image Acquisition Computation of Points Location Detection of Points Amount and Location Matching 69
- 70. Image Acquisition Iimage.jpg = Input Image acquisition from reader. Timage.jpg = Template Image retrieve from database. 70
- 71. Computation of Points After the detection of minutiae points, matching algorithm require to calculate total number of available points in the fingerprint image separately To perform this computation two counter variables are used to count both ridge-end and bifurcation points 71
- 73. Location Detection of Points Each minutiae point in the fingerprint image has a specific location. This location information of particular point is significant to store for further matching of fingerprints. The location of every point in the digital image is given by pixel position, so that it can be taken and stored separately for both ridge-end and bifurcation points. 73
- 74. Minutiae Point Extracted in Input Image 74
- 75. Amount and Location Matching In the previous steps, all the required information about points is computed and stored Now, this is the matching step, here the algorithm compares the computed values with the stored values This algorithm first, compares the combination of both amounts of ridge-end and bifurcation points with stored data If the match occurs, the algorithm then compares the location of ridge points with stored location data 75
- 76. 76
- 77. 77
- 78. 78
- 79. • Some toll road requirements we encounter: – 99.9% image capture – 99% overall plate read accuracy – 99% OCR accuracy on 90% of capture plate images – number and state What is an ALPR System? 79
- 80. The procedure is based on extraction of plate region, segmentation of plate characters and recognition of characters Recognized character 80
- 81. 81
- 82. • In the segmentation of plate characters, the car number plate is segmented into its constituent parts to obtain its characters individually . Image filtering from unwanted spots and noise. • Dilation of image to separate characters from each others. Segmented plate number Characters Segmentation 82
- 83. It is done by finding starting and end points of characters in horizontal direction. Characters separated individually Separating the plate characters 83
- 84. Normalization of characters Normalization is to refine the characters into a block containing no extra white spaces (pixels) in all the four sides of the characters. Sometimes called contrast stretching. Then each character should be equal in size. 84
- 85. Below an example of normalized character where the character fill all the 4 sides Normalized character 85
- 86. 86
- 87. Supermarket scanner recognizes objects without barcodes 87
- 88. 88
- 89. Supermarket scanner recognizes objects without barcodes Uses object recognition to identify foods at the supermarket checkout line. The technology uses a camera that compares the food that is being scanned to a large, expandable database of products. That camera filters out background "noise" in its picture, so that it only sees objects held close to its lens against a neutral black background. The technology recognizes supermarket items at check out without requiring a bar code...making bar codes obsolete for check out purposes. It uses proprietary pattern recognition technology and claims it can operate at high speeds. This object recognition system requires a database that contains the information about the items in the supermarket. This system claims to be able to make very precise identification of produce. 89
- 90. Google patents new object recognition technology, likely has plans to use with YouTube 90
- 91. Google patents new object recognition technology, likely has plans to use with YouTube It’s known as “automatic large scale video object recognition.” It can actually recognize the difference between a variety of objects, not just human faces. After recognizing an object it then labels it with certain tags. If you are wondering how it does this, there is a special object name repository involved. This database would hold at least 50,000 object names, information and shapes that would allow for easy identification. 91
- 92. 92
- 93. Software that does this is usually only available to government agencies and research facilities. It also guesses celebrity names. This is the new version; it works very well, particularly with vehicles, products, brands, and well-known "things". Take a picture of a foreign t-shirt label... Android Eye will tell you the brand, and where the shirt is from. Take a picture of a tree... a ball... a person... the results are endless. Take a picture of a car... Android Eye will tell you the make and model of the car. Android Eye is an advanced Object Recognition app. Take a picture of any object, and Android Eye will tell you what it is. Android Eye 93
- 94. A Google Glass App 94
- 95. A Google Glass App Knows What You're Looking At An app for Google’s wearable computer Glass can recognize objects in front of a person wearing the device. Google has shown that the camera integrated into Google Glass, the company’s head-worn computer, can capture some striking video. They built an app that uses that camera to recognize what a person is looking at. The app was built at an employee hack session held by the company this month to experiment with ways to demonstrate their new image recognition service. The app can either work on photos taken by a person wearing Glass, or constantly grab images from the device’s camera. Those are sent to the cloud or a nearby computer for processing by AlchemyAPI’s image recognition software. The software sends back its best guess at what it sees and then Glass will display, or speak, the verdict. 95
- 97. 97
- 98. References • http://arxiv.org/ftp/arxiv/papers/1001/1001.4186.pdf • http://www4.comp.polyu.edu.hk/~csajaykr/myhome/teaching/biometrics/final_re port.pdf • http://www.google.com.lb/ Fingerprint Analysis and Representation.ppt • http://www.academia.edu/2537762/Fingerprint_Matching_using_Ridge- End_and_Bifurcation_Points • http://ai.pku.edu.cn/aiwebsite/research.files/collected%20papers%20- %20fingerprint/Minutiae%20detection%20algorithm%20for%20fingerprint%20reco gnition.pdf • http://en.wikipedia.org/wiki/Computer_vision 98
- 99. 99
Editor's Notes
- #5: Synthetic = artificial, fake
- #7: It is the process of extracting useful information from digital image process of extracting useful information from digital imagesFinding objects of interest in imagesProperties of objects (size, shape, color)Recognition of objectsAlso known as machine vision, robot vision, computational vision, or image understanding
- #9: Person, old, hair, write, notepad, pen, sweater, bookshelf, books.
- #11: Machine Vision = Robots!