Designing Gesture Interface for Automotive Environment

International Journal of Innovative Research in Advanced Engineering (IJIRAE) ISSN: 2349-2763
Issue 06, Volume 3 (June 2016) www.ijirae.com
_________________________________________________________________________________________________
IJIRAE: Impact Factor Value – SJIF: Innospace, Morocco (2015): 3.361 | PIF: 2.469 | Jour Info: 4.085 |
Index Copernicus 2014 = 6.57
© 2014- 16, IJIRAE- All Rights Reserved Page -148
Designing Gesture Interface for Automotive Environment
Anand G Buddhikot*
Nitin. M. Kulkarni Arvind.D. Shaligram
Department of Electronic Science Department of Electronic Science Department of Electronic Science
Kaveri College of Arts, Science and Commerce Fergusson College Savitribai Phule Pune University
Abstract— The continual growth in electronic technology signifies a wide scope for research in automotive domain.
Presently, the automotive environment consists of various electronic systems embedded for the comfort of end user.
The infotainment system is one of such embedded feature incorporated in advance automobiles. A typical
infotainment device consists of radio, media player and telephone features as its basic application. The growing
popularity of these infotainment systems implies the scope for development in the entire domain of product design for
generating a console application. The traditional method for interaction with these infotainment systems involves
manual operations assisted with a remote control. Thus, exploring infotainment systems for effective human machine
interface (HMI) techniques is to be done, aiming for a better solution. Gesture based interaction is a new possibility
for having communication with these infotainment system. The use of gesture interface provides touch less mode of
interaction. Thus, it becomes necessary to study the automotive environment to define the placement and usage of
interface system considering its users. The driver being the primary user gets the majority of dash board access, thus
developing an effective interface for dash board is the primary goal. Further designing a gesture based access control
for the passengers in automotive domain is a novel solution in minimizing driver efforts. In the present work design
and analysis of a prototype gesture interface for infotainment system is discussed considering its optimum usage for
the passenger accommodating rear seat in automobile. Additionally the implementation of the prominent interface is
carried in a view to characterize the constraints and define its usability quotient. The prototype setup for experimental
work consists of MATLAB based algorithms for interpreting the interface.
Keywords— Infotainment, HMI, Gesture, automotive, MATLAB
I. INTRODUCTION
In recent years the automotive domain has developed significantly with the growth in technology and rising user
demands. The user demands for embedding multimedia, internet and computing features has evolved into a totally new
family of product called Infotainment System. An infotainment system is designed to provide entertainment, connectivity
and useful information as its basic features. Thus embedding an infotainment device in automotive environment has
become a trend and many advance automobiles provide this feature. Accordingly varieties of infotainment devices are
available in market. These systems provide assistance to both driver and passengers. However, managing an infotainment
system is confined to manual operations and research is to be done for extending its features towards passengers
accommodating the rear seat. The developments in multimedia featured interactions have now made it possible to use
multimedia as a means for interaction. These interfaces consist of audio or vision based gesture movements as input
command to the machine. The audio based interaction has a poor performance in noisy environment and also is a
complex technique [1]. Thus vision based gesture technology is favourable in automotive domain. A gesture interface
system is based on algorithms for real time tracking, modelling and recognition of human gestures from video input.
Traditionally, the passenger accommodating rear seat communicates with the infotainment device through remote
control. A gesture based interface will provide an efficient solution in this situation providing free hand interaction. The
aim of present research work is focused on designing a gesture interface, for an infotainment system in automotive
environment which would provide comfort and luxury to the passenger seating behind. The developed interface is
analysed to emphasize its constraints and define its usability quotient. This will provide exact rating which would assist
in design, to surmount environmental constraints and provide its usability for automotive applications.
II. REVIEW OF WORK
In past, several efforts have been made to bring effective interface as solution into automotive domain. The main
objective of interface is to relief the visual load on the driver while maintaining a natural intuitive interface. Many of
methods proposed auditory, touch based and multi sensory technique. Recently, proximity sensing systems have shown
promising results. These technologies have been developed and integrated in automobiles. Some recent work introduced
the kinect to automotive. The kinect was mounted on car ceiling tracking gesture in gear shift area. The combination of
kinect rgb and depth image tracking provides a robust gesture tracking scheme and is tolerant in changing light
conditions. Aiming at further improvement for touch less interaction, a lot of research work is going on the vision based
design. The vision based gesture interaction represent a comfortable solution, without compromising recognition
accuracy under noisy driving conditions that is seen with speech recognition systems [2][3].

_________________________________________________________________________________________________
In most of proposed work the prototype setup consists of camera interfaced to infotainment system. The camera
is placed on the ceiling and focused over the gear shift area. This choice was made because the drivers arm can lie
comfortably on the arm rest, and also this area is not visible for other road users, which is necessary to avoid
misunderstandings in public traffic. Such hand tracking system despite of the complex software and expensive hardware
with infotainment device were unproductive for varying light conditions and noisy environment in automotive domain.
Also the computation speed considering the processors in infotainment gadgets has limitations for response time.
The major research work in gesture based interaction for automotive domain is done considering driver as the
only user in the scenario. Also some schemes have highlighted influence of the execution speed, spontaneity and
requirement of vocabulary for hand gesture in automotive environment.
III. DESIGN ASPECT
In the present work design of vision based gesture interface for infotainment systems is discussed for developing a
console application in automotive environment [4]. There are two main design aspects in this scenario, designing a better
interaction for secondary controls without compromising the primary function of driving. Secondly, allowing the
passengers to access the device through gesture interaction for minimizing diver’s interaction [5]. This at first requires a
systematic analysis of automotive environment for placing the gesture interface system to get optimized performance.
Further, in the current work a setup has been developed with gesture interface to interact with media player application,
considering a computer as infotainment system. Many advance automobiles have used mini-computer on dashboard.
Using a computer will enhance the connectivity and computing features. The recent development in laptops provides a
touch screen interface for interaction. The driver will interact through touch interface and will have a control to provide
access to the passenger. Considering gesture based interaction for passenger, the placement of camera is done on the
ceiling of passenger seat. The Fig. 1 is the snapshot of setup in automotive. Accordingly a Graphic user interface is
designed using MATLAB through which driver can interact with the system. The GUI has selection mode through which
diver allows access control for passenger In the present work initially the volume attribute of media player is considered
and effective gesture control is developed. Initially the experimentation work is carried out in lab a setup. The Fig.2
shows the snapshot of Lab Setup made for carrying experimentation.
IV.ALGORITHM & IMPLEMENTATION
In the present work hand gesture recognition method is discussed which utilizes skin detection for hand
segmentation. Further the information about palm position is utilized to trigger the attributes of Media player application.
There are numerous methods for skin color detection based on different color schemes. Although the main objective of
skin color detection is to distinguish skin and non skin pixel. The most commonly used color spaces for skin detection
are YCbCr, RGB, normalized RGB, and HSV. In contrast to RGB based schemes, the YCBCR color space is
luminescence independent, resulting in a better performance. In the work YCBCR color space is used for skin detection.
In this segmentation method the image pixels are classified according to specific threshold values. Once the area of
interest is segmented from image the resultant image is processed through series of transformation to extract the required
information. ]
Fig. 1 Snapshot of Setup In Automotive Fig. 2 Snapshot of Lab Setup
The developed lab setup prototype interface consists of a conventional web camera interfaced to a Laptop system
having specifications, Core 2 Duo 2.4GHz with 4 GB RAM.

_________________________________________________________________________________________________
The web camera, INTEX -306WC is attached to a stand whose height can be adjusted. The cable used for USB camera is
4 meter long. The MATLAB software environment is used throughout as it provides high level formalisms that allow the
modelling of such systems and also allows automatic code generation, test and implementation for such systems. The
image acquisition is the first process in the gesture based system. The flow of system designed to recognize hand
movement is described in the Fig. 3.
Fig.3. Flow of system
The image obtained from the camera is first flipped using
RGB = flipdim(data ,2);
The next step is converting the image to YCBCR
YCBCR = rgb2ycbcr(RGB);
Then the skin pixels are marked which represent hand segmentation. The finger tip is calculated and plotted on the screen.
Here the coordinates of finger tip is considered as the reference point for triggering the attributes[6][7].
V. RESULT & DISCUSSION
The algorithm was developed on the prototype lab set up. The GUI consists of “Menu” for file selection, start, stop and
pause as attributes of media player. A toggle button is used to set the “Camera ON” or “Camera OFF “.The results for
vertical gesture motion verses increasing volume level is plotted in Fig.4. The relation observed is linear. The camera
setting at distance 55Cm showed the range from 4Cm to 31Cm and at distance 30Cm the range was 2Cm to 16Cm.
Further variation in sound level was tabulated for increasing volume in ten steps from 0 to 10. Fig.5 shows linear relation
for increasing sound level controlled through gesture motion. The experimentation was carried out at light intensity 90
Lux. The LX 101A/LX-102A HTC instruments, Lux meter was used to measure light intensity. The sound level was
measured using LT SL-4010 digital sound meter.
Fig.4 Distance Vs Volume level Fig.5 Sound in db Vs Volume level
The proposed system is tested in automotive environment. The system showed exact response considering gray
background environment. The gestures interactions were carried at the distance of 30 Centimetres from camera. The
volume control response was effective with a resolution of 1 for the range of 0 to 10. The system provided linear
response.
Web cam
Interface
Converting
video to image
RGB to YCBCR
Transformation
Marking skin
pixel
Extracting
information
Triggering event

_________________________________________________________________________________________________
VI. CONCLUSION
Gesture interface with infotainment systems can acquire several advantages with the introduction of different
natural forms of device free communication. Even in automotive environment gesture interface can be effectively used.
In the present work gesture interaction for volume control is effectively carried out. The major aspects of the design
involved web camera interface with the system with the cable length of 4metes. Also the volume control interface
algorithm with MATLAB, which was successfully embedded. The design of gesture based effective access control for
the passengers in automotive domain is a novel solution in minimizing driver efforts.
In conclusion, the method fulfilled the initial criteria, by producing a system which could be implemented to
control media player volume control attribute of an infotainment system in automobile. In future, other attributes of
media player and infotainment system can be considered in the designing a prominent interface for automotive
environment.
REFERENCES
[1] Raine Kajastilan and Tapio Lokki, Eyes-free interaction with free-hand gestures and auditory menus, ELSEVIER,
2012.
[2] Eshed Ohn-Bar, Cuong Tran and Mohan Trivedi, Hand Gesture-based Visual User Interface for Infotainment,
Proceedings of the 4th International Conference on Automotive User Interfaces and Interactive Vehicular
Applications (Automotive '12), Portsmouth, NH, USA. October 17–19, 2012.
[3] S. Sakthivel and A. Anbarasi, Hand Gesture Recognition System for Human-Computer Interaction, International
Journal of Recent Trends in Electrical & Electronics Engg. ©IJRTE ISSN: 2231-6612 Feb. 2014.
[4] Minyoung Kim, Jae-Hyun Nam and Jong-Wook Jang, Implementation of Smart Car Infotainment System including
Black Box and Self-diagnosis Function”, International Journal of Software Engineering and Its Applications Vol.8,
No.1, pp.267-274,2014.
[5] Shruti Chavan, Jayashree Shiravale, Neeraj Vaid, Sohail Khan and Prof. Sayara Shikalgar, Hand Gestures to
control automotive interfaces and Infotainment equipments in car”, International Journal of Advanced
Technology in Engineering and Science www.ijates.com Volume No.03, Issue No. 02, February 2015 ISSN
(online): 2348 – 7550.
[6] Tarachand Saini and Savita Sivani, Real Time Vision Hand Gesture Recognition Based Media Control via LAN &
Wireless Hardware Control”, International Journal of Modern Engineering Research (IJMER) www.ijmer.com
Vol. 3, Issue. 5, Sep - Oct. 2013 pp-3129-3133 ISSN: 2249-6645.
[7] Shivanku Mahna , Ketan Sethi and Sravan Ch, Controlling Mouse using Hand Gesture Recognition, International
Journal of Computer Applications (0975 – 8887) Volume 113 – No. 5, March 2015.

Designing Gesture Interface for Automotive Environment

More Related Content

What's hot (19)

Viewers also liked (10)

Similar to Designing Gesture Interface for Automotive Environment (20)

More from AM Publications (20)

Recently uploaded (20)

Designing Gesture Interface for Automotive Environment