IRJET- Smart Wheelchair with Object Detection using Deep Learning
0 likes53 views
This document summarizes a research paper on developing a smart wheelchair that can detect objects and be controlled through eye movement detection using deep learning. The wheelchair is intended to help disabled people with limited mobility. A camera is mounted on the wheelchair to capture the user's eye movements. Computer vision and deep learning techniques are used to analyze the eye images and detect the movement direction. Arduino then sends control signals to motors to move the wheelchair in the intended direction. Ultrasonic sensors are also used to detect any obstacles and avoid collisions while moving. The system aims to provide independent and obstacle-free mobility through a user-friendly eye-based interface without requiring physical strength.
![International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 06 Issue: 12 | Dec 2019 www.irjet.net p-ISSN: 2395-0072
© 2019, IRJET | Impact Factor value: 7.34 | ISO 9001:2008 Certified Journal | Page 748
SMART WHEELCHAIR WITH OBJECT DETECTION USING
DEEP LEARNING
Miss. Gaikwad Sayali. S1, Miss Bhagat Samiksha. N2, Miss. Shendkar Nikeeta. S3,
Miss. Lonkar Supriya. D4, Prof. Mr. Dhaigude T.A5
1,2,3,4,5Dept. of Computer Engineering, Someshwar Engineering College, Maharashtra, India
---------------------------------------------------------------------***----------------------------------------------------------------------
Abstract – This paper describes a new method to control
the wheelchair for disabled people using image processing.
The statistics propose that there are around 40 cases per
million suffering from quadriplegic and paraplegic diseases.
The great person Stephan Hawking have been experiencing
this phenomenon. Our project is helping to make lives from
this type of phenomenon. In our project, an eye control device
based on image processing is designed to be used as an
intelligent mobility aid for the paralyzed. In our proposed
system ultrasonic is used to detect any obstacle in the path of
the wheelchair. Depends on the eyeball movement the
wheelchair will be moved. The camera is mounted on a
wheelchair for capturing the eye movement. The capturedeye
images send to PC and controlled by MATLAB, which will send
to Arduino circuit to manipulate to control the motors and
allow to move the wheelchair in particular direction. The
proposed system mainly work on two different stages; first to
detect the eye movement and second, sending control a signal
for movement to the wheelchair.
Key Words: Arduino, Automatic wheelchair, eye
movement detection, servo motor, MATLAB.
1. INTRODUCTION
Eye gesture controlled wheelchairisa veryimportantdevice
for the physically disabled, paralyzed patient and mentally
handicapped people. The normal people driving wheelchair
is difficult but for the patient who is unable to walk or
handicapped are driving wheelchair is too difficult than
normal people, they required a large amountofenergyusing
physical strength, to turn the wheels.Eyegesture wheelchair
is the main device for the patient with severe disabilities or
also for the elderly. The ability to move freely in anywhere is
highly valued by all patient/people. Nowadays there are
many different techniques used for controlled wheelchair
such as Electrooculogram (EOG), Electrocardiogram (ECG)
and Electroencephalogram (EEG) based, eyeball sensing
techniques.
In Quadriplegia, paralysis patient can move his eyes only. In
order to help that type of people “Eye gesture controlled the
wheelchair using Arduino” comes into existence. With the
help of this techniques, different formsofthewheelchaircan
be controlled. To control the whole wheelchair system we
used Arduino controller with laptop or PC interface withthe
small camera is attached to the wheelchair for continuously
capture the eye movement.
2. PROBLEM STATEMENT
A clinical survey shows that in every year 9-10% of people
severely disabled in accident cases. The Disabled person
having some difficulties are comesinhandlingandoperating
the wheelchair. That means that they required a large
amount of energy, motor skill and strength to operate the
wheelchair that used in the existing system. Our main motto
is to design or to develop the product user-friendly, which
requires no training.
3. LITERATURE REVIEW
The existing system uses input devices such as keyboard,
mouse, joystick and other input devices for interaction with
automated devices need among hand gestures.Inthispaper,
an input device is a human eye only, which is proposed for a
handicapped person. The existing system and their
drawbacks for handicap person are categorized below;
(1)Eye gesture controlled intelligent wheelchair using
Electrooculography (EOG), which utilizes potential from
user’s body action. Inthissystem,intelligenttechniquesused
the movement of eyes retina and that resulting signal is
called electro-oculogram.
Limitation of this method is it requires human effort to
navigate devices which is difficult for the patient with
deformities.
(2)Voice Controlled Intelligent Wheelchair, This paper
developed the voice-controlled wheelchair which uses the
voice command as the interface.it has3part1]Runninguntil
next command is input
2] Turning or rotating
3] Stopping.
It uses two type of verification commands to prevent the
wrong reaction.
Limitation of this paper is the speed of speech,
background noise, speaking style, and accuracy.
(3)Implementation of Head and Finger movement Based
automatic Wheel Chair. This system usestheheadandfinger
to move the wheelchair. The type of artificial aid physically](https://image.slidesharecdn.com/irjet-v6i12105-200106062350/85/IRJET-Smart-Wheelchair-with-Object-Detection-using-Deep-Learning-1-320.jpg)
![International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 06 Issue: 12 | Dec 2019 www.irjet.net p-ISSN: 2395-0072
© 2019, IRJET | Impact Factor value: 7.34 | ISO 9001:2008 Certified Journal | Page 750
6. ADVANTAGES
⮚ Less expensive than the conventional wheelchair.
⮚ Obstacle detection using the Ultrasonic sensor.
⮚ Front wheels drivetoprovidehighmaneuverability.
⮚ Smooth operation.
7. CONCLUSIONS
In this system, we present an innovative idea in a regular or
ordinary wheelchairbyadding some newfeatureslikemotor
mechanism and make it easy to use for handicapped people
or quadriplegic and paraplegic diseases people. The main
Aim of this system is to help the handicapped people to
move everywhere freely, with the helpof eyegesturecontrol
wheelchair.
The future scope is to develop a mobile App based
wheelchair with some additional facility like disable person
can do their many works with the help of that mobile App
i.e., appliances operation getting up from his position.
REFERENCES
[1] Adarsh Rajesh and Megha Mantur “Eyeball Gesture
Controlled Automatic Wheelchair Using Deep Learning”,
Department of Electronics and CommunicationEngineering,
National Institute of Technology, Karnataka, IEEE 2017.
[2] S. Tameemsultana and Kali Saranya “Implementationof
Head and Finger movement based automatic Wheel Chair.”
Bonfring International Journal of power system and
integrated circuits, Vol. 1, Special Issue, December-2011
[3] Masato Nishimori, Takeshi Saitoh and Ryosuke Knishi
“Voice Controlled Intelligent Wheelchair”TottoriUniversity,
Tottori, Japan, 2007
[4] Satish Ravishankar, Govinda Ram Pingali, Niyanth
Krishna Polisetty, “Eye-Gesture Controlled Intelligent
Wheelchair Using Electro-Oculography”, instrumentation
technology department, Banglore, India.](https://image.slidesharecdn.com/irjet-v6i12105-200106062350/85/IRJET-Smart-Wheelchair-with-Object-Detection-using-Deep-Learning-3-320.jpg)
IRJET- Smart Wheelchair with Object Detection using Deep Learning
- 1. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 12 | Dec 2019 www.irjet.net p-ISSN: 2395-0072 © 2019, IRJET | Impact Factor value: 7.34 | ISO 9001:2008 Certified Journal | Page 748 SMART WHEELCHAIR WITH OBJECT DETECTION USING DEEP LEARNING Miss. Gaikwad Sayali. S1, Miss Bhagat Samiksha. N2, Miss. Shendkar Nikeeta. S3, Miss. Lonkar Supriya. D4, Prof. Mr. Dhaigude T.A5 1,2,3,4,5Dept. of Computer Engineering, Someshwar Engineering College, Maharashtra, India ---------------------------------------------------------------------***---------------------------------------------------------------------- Abstract – This paper describes a new method to control the wheelchair for disabled people using image processing. The statistics propose that there are around 40 cases per million suffering from quadriplegic and paraplegic diseases. The great person Stephan Hawking have been experiencing this phenomenon. Our project is helping to make lives from this type of phenomenon. In our project, an eye control device based on image processing is designed to be used as an intelligent mobility aid for the paralyzed. In our proposed system ultrasonic is used to detect any obstacle in the path of the wheelchair. Depends on the eyeball movement the wheelchair will be moved. The camera is mounted on a wheelchair for capturing the eye movement. The capturedeye images send to PC and controlled by MATLAB, which will send to Arduino circuit to manipulate to control the motors and allow to move the wheelchair in particular direction. The proposed system mainly work on two different stages; first to detect the eye movement and second, sending control a signal for movement to the wheelchair. Key Words: Arduino, Automatic wheelchair, eye movement detection, servo motor, MATLAB. 1. INTRODUCTION Eye gesture controlled wheelchairisa veryimportantdevice for the physically disabled, paralyzed patient and mentally handicapped people. The normal people driving wheelchair is difficult but for the patient who is unable to walk or handicapped are driving wheelchair is too difficult than normal people, they required a large amountofenergyusing physical strength, to turn the wheels.Eyegesture wheelchair is the main device for the patient with severe disabilities or also for the elderly. The ability to move freely in anywhere is highly valued by all patient/people. Nowadays there are many different techniques used for controlled wheelchair such as Electrooculogram (EOG), Electrocardiogram (ECG) and Electroencephalogram (EEG) based, eyeball sensing techniques. In Quadriplegia, paralysis patient can move his eyes only. In order to help that type of people “Eye gesture controlled the wheelchair using Arduino” comes into existence. With the help of this techniques, different formsofthewheelchaircan be controlled. To control the whole wheelchair system we used Arduino controller with laptop or PC interface withthe small camera is attached to the wheelchair for continuously capture the eye movement. 2. PROBLEM STATEMENT A clinical survey shows that in every year 9-10% of people severely disabled in accident cases. The Disabled person having some difficulties are comesinhandlingandoperating the wheelchair. That means that they required a large amount of energy, motor skill and strength to operate the wheelchair that used in the existing system. Our main motto is to design or to develop the product user-friendly, which requires no training. 3. LITERATURE REVIEW The existing system uses input devices such as keyboard, mouse, joystick and other input devices for interaction with automated devices need among hand gestures.Inthispaper, an input device is a human eye only, which is proposed for a handicapped person. The existing system and their drawbacks for handicap person are categorized below; (1)Eye gesture controlled intelligent wheelchair using Electrooculography (EOG), which utilizes potential from user’s body action. Inthissystem,intelligenttechniquesused the movement of eyes retina and that resulting signal is called electro-oculogram. Limitation of this method is it requires human effort to navigate devices which is difficult for the patient with deformities. (2)Voice Controlled Intelligent Wheelchair, This paper developed the voice-controlled wheelchair which uses the voice command as the interface.it has3part1]Runninguntil next command is input 2] Turning or rotating 3] Stopping. It uses two type of verification commands to prevent the wrong reaction. Limitation of this paper is the speed of speech, background noise, speaking style, and accuracy. (3)Implementation of Head and Finger movement Based automatic Wheel Chair. This system usestheheadandfinger to move the wheelchair. The type of artificial aid physically
- 2. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 12 | Dec 2019 www.irjet.net p-ISSN: 2395-0072 © 2019, IRJET | Impact Factor value: 7.34 | ISO 9001:2008 Certified Journal | Page 749 disturbs people in order to move or in order to guide the wheelchair. Limitation of this paper is various condition should be checked by patients, it proves theburdentotheuserandless accuracy. (4)Sensor Based Automated Wheelchair. This system uses an ultrasonic sensor to detect any obstacle in the path of the wheelchair. In this system, they used MPLAB IDE for programming to move the wheelchair in any direction. Limitation of this paper is the movement is depends on the accelerometer tilt section. Less accuracy. (5)Eyeball Gesture Controlled Automatic Wheelchair Using Deep Learning. In this system, they use eyeball gesture for the movement of the wheelchair with the helpofthecamera. A Camera captures the images andsendstothecontroller for movement in the direction to the servo motor. Limitation of this topic is moderate accuracy and misclassified cases. 4. DESIGN MODULE We use a small chassis as a wheelchair to test our system working start to end. Arduino controller is used to controlling this chassis. The system contains five main parts:- 1) A small webcam and PC interface is used for tracking the movement of the user’s eye. 2) The Arduino board, take input from PC and convert the digital output to the electric signal. 3) L293D, a dual half bridge motor driver IC, which uses the signal from Arduino to control DC motor. 4) A DC motor is attached to the wheels. 5) The ultrasonic sensor used to detect any obstaclein the path between the wheelchairs. Fig -1: Design Module In this system, there is mandatory to captureeyeimagesand extract the images into the frame and interface with our machine or PC. If the extracted image and the reference image matches then system sends the signal to the microcontroller for following the requested command. Motor driver can directly movethe wheelchairinthedesired direction and that control by Arduino. The image processing is done by MATLAB library is used to detect the eye movement. 4.1. HARDWARE DESCRIPTION A) Arduino Arduino is the main part or brain of our system. Arduino is a simple microcontroller board that used to make computers that drive all the function and creative projects alike. B) Camera The Camera is used for capture the eye images and sendthat data to the main system or in PC. C) USB Cable USB is a serial communication interface through which we can send or information transmission fromonetoanotherat a time. D) Ultrasonic Sensor As name indicates, it measure distance by using ultrasonic waves. Sensor emit the data from waves and reflect back to the target. Ultrasonic sensor measure the distance to the target by measuring the time between emission and reception. 5. PROPOSED METHODOLOGY According to the tracked location of the eye, in the direction is found to be based on motion. On the basis of this direction system determined, the command is sent to the motor control device via Arduino. With the help of below flow graph we can easily understand the work of this system:- Fig -2: Flow graph of the system
- 3. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 12 | Dec 2019 www.irjet.net p-ISSN: 2395-0072 © 2019, IRJET | Impact Factor value: 7.34 | ISO 9001:2008 Certified Journal | Page 750 6. ADVANTAGES ⮚ Less expensive than the conventional wheelchair. ⮚ Obstacle detection using the Ultrasonic sensor. ⮚ Front wheels drivetoprovidehighmaneuverability. ⮚ Smooth operation. 7. CONCLUSIONS In this system, we present an innovative idea in a regular or ordinary wheelchairbyadding some newfeatureslikemotor mechanism and make it easy to use for handicapped people or quadriplegic and paraplegic diseases people. The main Aim of this system is to help the handicapped people to move everywhere freely, with the helpof eyegesturecontrol wheelchair. The future scope is to develop a mobile App based wheelchair with some additional facility like disable person can do their many works with the help of that mobile App i.e., appliances operation getting up from his position. REFERENCES [1] Adarsh Rajesh and Megha Mantur “Eyeball Gesture Controlled Automatic Wheelchair Using Deep Learning”, Department of Electronics and CommunicationEngineering, National Institute of Technology, Karnataka, IEEE 2017. [2] S. Tameemsultana and Kali Saranya “Implementationof Head and Finger movement based automatic Wheel Chair.” Bonfring International Journal of power system and integrated circuits, Vol. 1, Special Issue, December-2011 [3] Masato Nishimori, Takeshi Saitoh and Ryosuke Knishi “Voice Controlled Intelligent Wheelchair”TottoriUniversity, Tottori, Japan, 2007 [4] Satish Ravishankar, Govinda Ram Pingali, Niyanth Krishna Polisetty, “Eye-Gesture Controlled Intelligent Wheelchair Using Electro-Oculography”, instrumentation technology department, Banglore, India.