![• Adopt the lightness and add the width of the fish
Fish xT = [x1, x2]
Pattern Classification, Chapter 1
19
Lightness Width](https://image.slidesharecdn.com/patternrecognition-230714080554-04201f3e/85/pattern-recognition-ppt-19-320.jpg)
![Emerging Applications
Interest in pattern recognition and classification has
grown due to emerging applications, which include :-
• Data mining (sifting through a large volume of data to
extract a small amount of relevant and useful information,
e.g., fraud detection).
• Biometrics (personal identification based on physical
attributes of the face, iris, fingerprints, etc.)
• Machine vision (e.g., automated visual inspection in an
assembly line)
• Character recognition [ (automated teller machines)]
• Document recognition (e.g., recognize whether an e-mail
is spam or not, based on the message header and
content)
27](https://image.slidesharecdn.com/patternrecognition-230714080554-04201f3e/85/pattern-recognition-ppt-27-320.jpg)
![The methods used have been developed in
various fields, often independently.
• In statistics, going from particular observations to general
descriptions is called inference, learning [i.e., using example
(training) data] is called estimation, and classification is known
as discriminant analysis.
• In engineering, classification is called pattern recognition
and the approach is nonparametric and much more empirical .
Other approaches have their origins in machine learning ,
artificial intelligence , artificial neural networks , and data
mining. Fig. 2 illustrated the Pattern recognition and related
fields
29](https://image.slidesharecdn.com/patternrecognition-230714080554-04201f3e/85/pattern-recognition-ppt-29-320.jpg)
pattern recognition.ppt
- 2. References • Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley & Sons, 2000 • Pattern Recognition and Classification, An Introduction, by Geoff Dougherty, springer Science Business Media New York 2013 2
- 3. What is a Pattern? • A set of instances that - share some regularities and similarities - is repeatable - is observable, sometimes partially, using sensors - May have noise and distortion 3
- 4. What is Pattern Recognition? • Pattern recognition (PR) is the scientific discipline that concerns the description and classification (recognition) of patterns (objects) • PR techniques are an important component of intelligent systems and are used for many application domains - Decision making - Object and pattern classification 4
- 5. Human Perception • Humans have developed highly sophisticated skills for sensing their environment and taking actions according to what they observe, e.g., I recognizing a face, I understanding spoken words, I reading handwriting, I distinguishing fresh food from its smell. I We would like to give similar capabilities to machines 5
- 6. • Every day, we recognize faces around us, but we do it unconsciously and because we cannot explain our expertise, we find it difficult to write a computer program to do the same. • Each person’s face is a pattern composed of a particular combination of structures (eyes, nose, mouth, . . .) located in certain positions on the face. By analyzing sample images of faces, a program should be able to capture the pattern specific to a face and identify (or recognize) it as a face (as a member of a category or class we already know); this would be pattern recognition 6
- 7. A class is a collection of objects that are similar, but not necessarily identical, and which is distinguishable from other classes. Fig.1 illustrates the difference between classification where the classes are known beforehand and classification where classes are created after inspecting the objects 7 Fig 1 Classification when the classes are (A) known and (B) unknown beforehand
- 8. Human and Machine Perception • We are often influenced by the knowledge of how patterns are modeled and recognized in nature when we develop pattern recognition algorithms. Research on machine perception also helps us gain deeper understanding and appreciation for pattern recognition systems in nature. Yet, we also apply many techniques that are purely numerical and do not have any correspondence in natural systems. 8
- 9. Machine Perception • Build a machine that can recognize patterns: • Speech recognition • Fingerprint identification • OCR (Optical Character Recognition) • DNA sequence identification Pattern Classification, Chapter 1 9
- 10. An Example • “Sorting incoming Fish on a conveyor according to species using optical sensing” Sea bass Species Salmon 10
- 11. • Problem Analysis • Set up a camera and take some sample images to extract features •Length •Lightness •Width •Number and shape of fins •Position of the mouth, etc… • This is the set of all suggested features to explore for use in our classifier! Pattern Classification, Chapter 1 11
- 12. • Preprocessing • Use a segmentation operation to isolate fishes from one another and from the background • Information from a single fish is sent to a feature extractor whose purpose is to reduce the data by measuring certain features • The features are passed to a classifier 12
- 13. Pattern Classification, Chapter 1 13
- 14. • Classification • Select the length of the fish as a possible feature for discrimination Pattern Classification, Chapter 1 14
- 15. The length is a poor feature alone! Select the lightness as a possible feature. Pattern Classification, Chapter 1 15
- 16. • Threshold decision boundary and cost relationship • Move our decision boundary toward smaller values of lightness in order to minimize the cost (reduce the number of sea bass that are classified salmon!) Task of decision theory Pattern Classification, Chapter 1 16
- 17. • Feature extraction Task: to extract features which are good for classification. Good features: • Objects from the same class have similar feature values. • Objects from different classes have different values. Pattern Classification, Chapter 1 17
- 18. Feature vector : x Є X - A vector of observations (measurements). - x is a point in feature space X Pattern Classification, Chapter 1 18 Therefore… Basic concepts Hidden state: y Є Y - Cannot be directly measured. -Patterns with equal hidden state belong to the same class. Task - To design a classifier (decision rule) q : X Y which decides about a hidden state based on an observation.
- 19. • Adopt the lightness and add the width of the fish Fish xT = [x1, x2] Pattern Classification, Chapter 1 19 Lightness Width
- 20. Pattern Classification, Chapter 1 20 An example of Industrial Inspection In our case…
- 21. Pattern Classification, Chapter 1 21 Linear (simple) decision boundary; Cost of misclassification ?
- 22. • We might add other features that are not correlated with the ones we already have. A precaution should be taken not to reduce the performance by adding such “noisy features” • Ideally, the best decision boundary should be the one which provides an optimal performance such as in the following figure: Pattern Classification, Chapter 1 22
- 23. Pattern Classification, Chapter 1 23
- 24. • However, our satisfaction is premature because the central aim of designing a classifier is to correctly classify novel input Issue of generalization! Pattern Classification, Chapter 1 24
- 25. Pattern Classification, Chapter 1 25 An example of Industrial Inspection
- 26. Pattern Classification, Chapter 1 26 An example of Industrial Inspection Overfitting and underfitting
- 27. Emerging Applications Interest in pattern recognition and classification has grown due to emerging applications, which include :- • Data mining (sifting through a large volume of data to extract a small amount of relevant and useful information, e.g., fraud detection). • Biometrics (personal identification based on physical attributes of the face, iris, fingerprints, etc.) • Machine vision (e.g., automated visual inspection in an assembly line) • Character recognition [ (automated teller machines)] • Document recognition (e.g., recognize whether an e-mail is spam or not, based on the message header and content) 27
- 28. • Computer-aided diagnosis [e.g., helping doctors make diagnostic decisions based on interpreting medical data such as mammographic images. • Medical imaging [e.g., classifying cells as malignant or benign based on the results of magnetic resonance imaging (MRI) scans . • Speech recognition (e.g., helping handicapped patients to control machines). • Bioinformatics (e.g., DNA sequence analysis to detect genes related to particular diseases). • Remote sensing (e.g., land use and crop yield). • Astronomy (classifying galaxies based on their shapes 28
- 29. The methods used have been developed in various fields, often independently. • In statistics, going from particular observations to general descriptions is called inference, learning [i.e., using example (training) data] is called estimation, and classification is known as discriminant analysis. • In engineering, classification is called pattern recognition and the approach is nonparametric and much more empirical . Other approaches have their origins in machine learning , artificial intelligence , artificial neural networks , and data mining. Fig. 2 illustrated the Pattern recognition and related fields 29
- 30. Fig. 2 Pattern recognition and related fields 30
- 31. Pattern Recognition Systems • Data acquisition and sensing: - Use of a transducer (camera or microphone) - Important issues: bandwidth, resolution, sensitivity, distortion, SNR, latency, etc. • Pre-processing: -Removal of noise in data. - Isolation of patterns of interest from the background • Segmentation and grouping Patterns should be well separated and should not overlap Pattern Classification, Chapter 1 31
- 32. Pattern Recognition Systems Pattern Classification, Chapter 1 32
- 33. Pattern Recognition Systems • Feature extraction Finding a new representation in terms of features - Discriminative features - Invariant features with respect to translation, rotation and scale. • Post Processing • Exploit context input dependent information other than from the target pattern itself to improve performance • Classification • Use a feature vector provided by a feature extractor to assign the object to a category 33
- 34. The Design Cycle • Data collection • Feature Choice • Model Choice • Training • Evaluation • Computational Complexity Pattern Classification, Chapter 1 34
- 35. Pattern Classification, Chapter 1 35
- 36. • Data Collection • Collect an adequately large and representative set of examples and divide them into (70%) training and (30%) testing the system. • Feature Choice • Depends on the characteristics of the problem domain. Simple to extract, invariant to irrelevant transformation insensitive to noise 36
- 37. • Model Choice Unsatisfied with the performance of one classifier, jump to another class of models • Training -Use sample data to train the classifier. - Use Many different procedures for training classifiers and choosing models : • Random Sub-sampling • Bootstrap • Cross-Validation Pattern Classification, Chapter 1 37
- 38. • Evaluation – Measure the error rate (performance) using different training methods. – Switch from one set of features to another, or from one model to another to improve accuracy, i.e. to minimize error rate. Pattern Classification, Chapter 1 38
- 39. Performance of PR Systems • Error rate (Prob. of misclassification) • Speed • Cost • Robustness • Reject option • Return on investment Pattern Classification, Chapter 1 39
- 40. • Computational Complexity • What is the trade-off between computational ease and performance? • How an algorithm scales as a function of the number of features, patterns or categories? Pattern Classification, Chapter 1 40
- 41. Limitation of PR Systems - Human have the ability to switch rapidly and seamlessly between different pattern recognition tasks. - It is very difficult to design a device that is capable of performing a variety of different classification tasks as human. 41
- 42. Learning and Adaptation • Supervised learning • A teacher provides a category label or cost for each pattern in the training set • Unsupervised learning • The system forms clusters or “natural groupings” of the input patterns 42
- 43. Summary • Pattern recognition is extremely useful and are now part of many crucial computer applications: • Pattern recognition is a very difficult problem and have many complex sub-problems. • Successful systems have been built in well constrained domains. • No single technique/model is suited for all pattern recognition problems • Use of object models, constraints, and context is necessary for identifying complex patterns • Careful sensor design and feature extraction can lead to simple classifiers 43