IRJET- Sign Language Converter using OpenCV
0 likes32 views
This document summarizes a project to develop a sign language converter using OpenCV. The system captures gestures performed by the user via a camera and compares them to a database of signs and their corresponding audio meanings. It uses techniques like Local Binary Pattern Histogram to recognize gestures and plays the matching audio file. The project aims to help mute people communicate more easily by converting their sign language to audio. It was developed using Python and OpenCV for real-time computer vision and gesture recognition.
IRJET- Sign Language Converter using OpenCV
- 1. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 08 | Aug 2019 www.irjet.net p-ISSN: 2395-0072 © 2019, IRJET | Impact Factor value: 7.34 | ISO 9001:2008 Certified Journal | Page 994 Sign Language Converter using OpenCV Harsh Patel Computer Engineering, A. D. Patel Institute of Technology, Vallabh Vidhyanagar, 388120 ---------------------------------------------------------------------***---------------------------------------------------------------------- Abstract –The main purpose of the project is to convertthe signs performed by the dumb people into its corresponding audio file. In order to do this the user has to perform his/her sings in front of the camera. The system would compare the recorded actions to the contents of its database and play the audio that speaks out what the user is actually trying to convey. Key Words: Sign language, Text to Speech, OpenCV, LBPH harrcascade. 1. INTRODUCTION The Project is a softwarethat convertstheactionsperformed by the mute people as of their sign language into its corresponding meaning and provide an equivalent output. This is project is made for the dumb people as well as regular people as one more feature of this project is that we can use this software to make a virtual screenwhichcangive fell of touch screen device on the device which doesn’t have the touch screen 2. MODELLING AND ANALYSIS USING SOFTWARE • Open CV: OpenCV (Open Source Computer Vision) is a library of programming functions mainly aimed atreal-timecomputer vision. Originally developed by Intel, it was later supported by Willow Garage (which was later acquired by Intel). The library is cross-platform and free for use under the open- source BSD license. OpenCV supports the deep learning frameworks like TensorFlow, Torch/PyTorch and Caffe. • Python (Programming Language) : Python is an interpreted high-level programming language for general-purpose programming. Created by Guido van Rossum and first released in 1991, Python has a design philosophy that emphasizes code readability, notably using significant whitespace. It provides constructs that enable clear programming on both small and large scales. Python features a dynamic type system and automatic memory management. It supports multiple programming paradigms, including object oriented, imperative, functional and procedural and has a large and comprehensive standard library. 3. Design for Use, Reuse and Sustainability •Design for use:- Reliability:- As this is project is converting the actionsintocorresponding audio file the person who can not speak can completely rely on this project as it will take the sings as input done by that person so he/she can easily convey what he/she is trying to say. And for those who will use this project as pointer device can also rely as the gesture is going as input by color segments which is the powerful tool to capture the gestures. Maintainability:- Maintenance of the project isassimpleasitsimplementation. A personhavingbasicknowledgeaboutpythonprogramming can easily maintain the whole project. •Design for Reuse:- Reusing of the different parts of the projectisalsoagreattool for developing the new project with already developed methods which can provide the proper documentation as well as the knowledgeaboutthetechniqueswhichareusedin previous project. We have been using predefined Local Binary Pattern Histogram (LBPH) technique for gesture recognition. •Design for Sustainability:- Design for sustainability is an approach that puts the well- being of people and the sustainability of the environment. In this project as it iscompletelysoftware based project itisnot going to affect the environment to that level.
- 2. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 08 | Aug 2019 www.irjet.net p-ISSN: 2395-0072 © 2019, IRJET | Impact Factor value: 7.34 | ISO 9001:2008 Certified Journal | Page 995 3. Prototyping 3.1 Test of Prototype Fig : Capturing the gesture using colour segments. Fig: Capturing the gesture using colour segment Fig : capturing the gesture using Haas-Cascade files Fig : Displaying the captured gesture Fig : Displaying the captured gesture 4. Working: • Capturing the Gesturing: Capturing the gesture is the first phase for implementing this project. Gesture can be captured by using two methods. Using Haarcascade files. Using color segments. Capturing the gesture using haarcasde files is the approach in which the haarcascade file is created by using positive andnegativeimages.Positive imagesare the one in which the object is present which has to be detected. Negative images are the one in which the object is not present.
- 3. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 08 | Aug 2019 www.irjet.net p-ISSN: 2395-0072 © 2019, IRJET | Impact Factor value: 7.34 | ISO 9001:2008 Certified Journal | Page 996 Capturing the object by using the color segments is the most popular technique as there is no prior work is requires in this techniques only the basic image processing tools are used in this techniques like converting the captured gesture into grayscale mode, applying the gussian blur on the grayscale image and then applying thethresholdingonthecapturedgesture to get the exact gesture which is required. • Gesture Recognition and Playing audio: For the recognition of the gesture a method is used which is known as Local Binary Pattern Histogram(LBPH) technique. In this technique it will create the histogram of the gesture by making the matrices of 3 x 3 and then compare value with the middle value of particular matrix and replacing the value with 1 if the cell value is greater else replace that value with 0, by following this method it will generate the matrix which only contains 0 or 1 as the cellvalues. Once this process is done it will convert that matrix value in to the equivalent decimal number. This process is applied on whole matrix till it’s get converted into it’s equivalent decimal value. Once this process is done the captured gesture is getting compared with the already taken and saved images from the databaseandplayingthecorrespondingaudio file. 5. Conclusion: Sign language converter provides dumb people an optimum way to open up and convey their messages with much less difficulties than what they usually face. The software would be a voice to those who could no longer speak. Hence the user would be at an ease with the help of this product.