sign language recognition using HMM
- 1. Sign Language RecognitionUsingHidden Markov ModelPresented by:VipulAgarwal - 070905060
- 5. SKIN AND HAND DETECTION
- 7. FEATURE EXTRACTION FOR TRAINING DATA
- 8. HIDDEN MARKOV MODEL & ITS USE
- 10. DEMONSTRATIONIntroductionInteraction with computers may often not be a comfortable experience.
- 11. Computers should be able to communicate with people with body language.
- 12. Hand gesture recognition becomes important …Interactive human-machine interface and virtual environment
- 13. IntroductionTwo common technologies for hand gesture recognitionGLOVE-BASED METHODUsing special glove-based device to extract hand postureVISION-BASED METHOD3D hand/arm modelingAppearance modeling
- 14. Introduction3D hand/arm modelingHighly computational complexity Using many approximation processAppearance modelingLow computational complexityReal-time processing
- 15. Sign LanguageRely on the hearing societyTwo main elements:Low and simple level signed alphabet, mimics the letters of the spoken language.Higher level signed language, using actions to mimic the meaning or description of the sign.The project aim is to make the computer recognize low and simple level American Sign Language.
- 16. Sign LanguageAmerican Sign Language26 signs to denote the alphabets.10 signs to denote numbers
- 17. Pre - ProcessingThe video sequence used has a lot of noise due to:Low quality of the webcam Improper lighting conditionsBackground
- 18. Pre - ProcessingPre-processing involves reducing the noise and illumination problems.The morphological operations used for reducing the noise involves:DilationStatistical Elimination
- 19. Pre - ProcessingDILATION>A disc shaped region is traversed over every blob and the ones which do not fit the disc are removed completely.
- 20. Pre - ProcessingSTATISTICAL ELIMINATION>For every region the area is computed. Since hand is the one with the largest area, all blobs having less than a specified area are removed.
- 21. Hand DetectionFirst all the noise is removed in the pre-processing stage.Now we assume that the hand is the largest skin blob in our video sequence.We calculate the area of every blob and take the one with the largest area.We also calculate the bounding box of the region containing the hand for further analysis
- 22. Hand Detection
- 23. Optical Flow AnalysisDEFINITION:Optical flow is the pattern of apparent motion of objects, surfaces, and edges in a visual scene caused by the relative motion between an observer (an eye or a camera) and the scene.
- 24. Optical Flow AnalysisWhy Optical Flow Analysis?Till now the system is just able to detect the hand and follow the bounding box as the hand moves.The problem now is that we need to define a way to take a snapshot of the hand when the hand is not moving.
- 25. Optical Flow AnalysisUsing this technique we find the motion in the hand. When the hand has stabilized, we assume that the gesture is ready. We then take a snapshot of the hand and perform the recognition on that image.
- 26. Feature ExtractionFor training the network with test images we perform the following feature extraction technique:-Thresholding of the test handConverting to a binary imageFinding the centroid of the hand and orientation of the minor axis.Making feature vectors using a predefined number of features.
- 27. Feature ExtractionExtracting the intersection of the feature vectors with the boundary points.Finding the scalar length of the vectors from the centroid.Normalising the lengths in a scale of 1 to 100 to make it scaling invariant.
- 29. Hidden Markov Model (HMM)HMMs allow you to estimate probabilities of unobserved eventsGiven plain text, which underlying parameters generated the surface
- 30. HMMs and their UsageHMMs are very common in Computational Linguistics:GESTURE RECOGNITION (observed: image, hidden: alphabets)
- 31. Progress ReportWORK COMPLETED:Data CollectionPre-processing Skin And Hand DetectionOptical Flow AnalysisFeature Extraction For Training DataWORK REMAINING:Training The Hidden Markov Model
- 32. Any Questions …?