Autonomous Vehicle by using 3D LIDAR and 2D Camera

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 09 Issue: 06 | June 2022 www.irjet.net p-ISSN: 2395-0072
© 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 1725
Autonomous Vehicle by using 3D LIDAR and 2D Camera
Niveda V1, Sudharsan S A2, Pavithra Sanan K V3, Arunkumar S4
123Student, Department of Electrical And Electronics Engineering, Kumaraguru College of Technology, Coimbatore,
Tamil Nādu
⁴Assistant Professor, Department of Electrical And Electronics Engineering, Kumaraguru College of Technology,
Coimbatore ,Tamil Nādu
---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - A Increasing populace is that the important
thing problem of transportation nowadays. So, right here
we have got advanced an automated driving system which
drives the vehicle automatically. Our aim is to help in
preventing injuries and people’s time by means of basically
converting automobile use. we've were given advanced
generation for motors that drives it automatically. we have
got designed an automated automobile it truly is centered
on supplying automatic driving vehicle in experience to
human drivers. This automobile can experience the
environment, navigating and enjoyable the human
transportation talents and not using a human input. Lidar
is applied for sensing the environment. It constantly tracks
the encircling and if any impediment is detected
automobile senses and actions round and avoids the
impediment. the main advantage of an self sustaining
automobile are fewer site visitors collisions, multiplied
reliability, multiplied roadway capacity, decreased
impediment. We consider that the self driving automobile
is moreover a truth quickly and be a need of existence via
way of means of overcoming this obstacles, as human
existence have to be stable and safe, efficient, cost-effective,
and comfortable method of transport.
Key Words: Raspberry Pi; deep learning; Object
Detection
1. INTRODUCTION
Over a million injuries take place every 12 months
because of horrific human driving. The main motive is the
influence of using alcohol. Other reasons encompass
diversion of the driving force from roads, way to falling
asleep, or even texting. These incidences require a
progressive concept that could put in force avenue safety.
Here's in which a driverless vehicle may be helpful. This
assignment introduces a prototype that's state of the
artwork independent vehicle that drives with no human
intervention. A CNN primarily totally based SSD-MobileNet
method is proposed to locate the important world objects.
This finished fantastic effects of detection and monitoring
of the trained objects with 99.2% accuracy at 98.4% self-
belief level, which facilitates the robotic to keep away from
injuries. An impediment detection and avoidance robotic is
offered supported Hough transform set of rules of item
detection the usage of python programming and managed
through a Microcontroller.
1.1 OBJECTIVE
The Designing and implementation of an autonomous car
prototype for vehilces.In order to accomplish the overall
objective of this project, the following sub-objectives have
to be achieved
 To develop an autonomous car prototype using
Raspberry Pi 3 as a main processing chip and an
2D camera and with 3D LIDAR sensor.
 To develop an algorithm for the fusion of 2D
camera and 3D Lidar to work together.
 To ensure the entire system works in time and
efficiently thus by avoiding the objects in real time.
1.2 LITERATURE SURVEY
To finalize the work, reviews of literature have been taken,
In this paper 1-Chong Han et. Al [2008] Chong had
chip away on the Vehicle discovery by means of
camcorders is a standout among the maximum
encouraging new advances for remote largescale data
gathering and execution of reducing area motion
manipulate and management plans, for example,
automobile direction/route. In this paper we suggest a way
to address identify and encompass cars at a crossing point
continuous, by making use of a settled camera. In the wake
of distinguishing shifting objects images through basis
casing differencing, area location, disintegration and
widening operations are achieved to smother clamor. This
identity calculation offers an expected variety of cars. A
flexible hobby flag manipulate machine controls the motion
stream. The recreating comes approximately reveal an
remarkable productiveness of activity manage and
management plot by and by.
In this paper 2-Ray-Shine Run et. Al [2010] The usage of
GPS is growing brief as of late. In army and technology
purposes, in addition to in common utilize, GPS assumes a
crucial element in several digital frameworks. For example

the electronic route of automobile, the digital manual of
PDA, and so on. We attempt a study in this topic,GPS path
framework managed by means of 8-bit MCU.For example-
GPS managing, obstacle avoidance, motion planning and
faraway correspondence are incorporated. Because of
spending factor of confinement, we simply gifted the
version on a "toy auto". In any case, the revel in of this
assignment advised us to maintain for the subsequent
stride. We would really like to alternate the associated
innovation to feasible automobile quicker instead of later.
In this paper 3-Julian Balcerek et. Al [2014] They have
they used camera to get the red and green light changes
from red to green so the vehicle start moving when traffic
light turns from red to green without any delay. They also
used camera to get the information to change the lane from
one lane to another lane. That system helps in save time
and avoid the accident on changing from one lane to
another.
In this paper 4-Stephen M. Erlien et. Al [2015] Direct
by-wire innovation empowers vehicle protection
frameworks to impart manipulate to a driver through
increase of the driver's controlling summons. Propels in
detecting improvements have interaction the ones
frameworks assist with constant data about the encircling
condition. Utilizing the ones progressions in vehicle
incitation and detecting, the creators introduce a mutual
manage shape for obstruction evasion and soundness
manipulate the use of steady driving envelopes. One of
these envelopes is characterised by the automobile
searching after elements of confinement, on the identical
time as the alternative is characterised by spatial
regulations pressured by direction limits and boundaries. A
version prescient manage (MPC) plot makes a selection at
on every occasion step if the ebb and glide driving force
summon takes into attention a sheltered vehicle direction
interior those envelopes, mediating sincerely whilst the
type of direction does now now not exist. Along the ones
lines, the controller offers manipulate with the motive
force in an insignificantly intrusive manner whilst staying
farfar from boundaries and fending off loss of manage. The
best control problem essential the controller is normally
nonconvex but is described as an affiliation of curved.
In this paper 5-Jinghua Guo et. Al [2016]: This paper
examines dynamic manipulate plan for mechanized driving
of vision-primarily based totally independent vehicles, with
an unusual focus on the organized guiding and braking
control in disaster difficulty evasion. An independent
automobile is a complicated multi-input and multi-
output(MIMO) framework, which has the factors of
parameter vulnerabilities and stable nonlinearities, and the
coupled marvels of longitudinal and sidelong development
are obvious in a consolidated cornering and braking
maneuver. In this work, a effective planned control
framework for automatic using is proposed to control
those coupled and nonlinear factors and reject the
unsettling influences. Initial, a dream calculation is
constructed to recognize the reference manner and supply
the neighborhood area information among automobiles
and reference manner continuously. At that point, a unique
facilitated guiding and braking manage method is proposed
in view of the nonlinear back stepping manage speculation
and the flexible fluffy sliding-mode control strategy, and
the asymptotic union of the proposed composed control
framework is established via way of means of the
Lyapunov speculation. At long last, trial tests show that the
proposed manage approach has terrific following execution
and improvements the driving solace and dependability of
self-enough automobiles.
2. PROPOSED SYSTEM
In the proposed block diagram, Lidar Sensor,
camera, rasperry Pi, motor driver, motors, power supply
and GPS have been used. The Lidar sensor is used to get the
360-degree image of the vehicle surroundings by using its
laser beam. Camera is used to capture the real-world
moving objects and detect the traffic light and signs. Radar
sensor is used for measuring the nearby obstacles.
Raspberry pi is used as our main processing chip and it is
the main computer.
The Overall block diagram for the autonomous vehicle
prototype is shown in figure 2.1.
.
Figure 2.1: Block Diagram
2.1 SYSTEM ARCHITECTURE
A. Raspberry pi3
Raspberry pi may be a small chip of single board
computer. Raspberry Pi is a microcontroller which has
good processing power and many ports to connect various

devices, it has 40 pins (28 GPIO pins). Camera, GPS, GSM,
and various sensors are connected to the Raspberry pi,
which processes and monitors everything. There are many
different types of raspberry pi available in the market and
they all differ in memory capacity and hardware features
like Raspberry pi3 has inbuilt Bluetooth and Wi-Fi modules
whereas in previous versions these modules weren't
available. It has a 1.2 GHz 64-bit quadcore ARMv8 CPU with
1 GB of RAM.
B. Pi Camera
Camera is employed to require continual images to
urge the traffic signs and signals from the important world.
According to the pictures available through the camera we
will send these images to the raspberry pi to perform the
action to regulate the car. Compatible with the official
Raspberry Pi Camera Board V2
• 3280x2464 pixel resolution
• Supports 1080p 30 FPS video recording
C. Motor driver IC
L293D is a 16-Pin motor driver IC which will
control two DC motors at a time. It is a medium power
motor driver suitable for driving DC motor and stepper
motor. It can drive up to 12V and DC current up to
600mA.As this motor is perfectly suitable for robotics for
controlling motor from micro-controllers. Input 00 and 11
stop the motors whereas logic 01 and 10 starts the rotation
of the motors in clockwise and anticlockwise directions,
respectively.
D. LCD Display
LCD is Liquid Crystal Display, which uses liquid
crystals as a primary form of operation. An LCD has a
backlight that provides light source to individual pixels
arranged in a rectangular grid. Each pixel has a RGB (Red,
Green, and Blue) sub-pixel that can be turned on or off.
When all of a pixel’s sub-pixels are turned off, it appears
black. When all the sub-pixels are turned on 100%, it
appears white. By adjusting the individual levels of red,
green, and blue light, millions of color combinations are
obtained.
E. LIDAR Sensor
LIDAR stands for Light Detection and Ranging. It is
a key component used in autonomous vehicles, which
scans objects in 360-degree rotation by using its laser
beams. Obstacle detection is one of the important works
within the field of driverless vehicles. LiDAR is employed to
detect obstacles by launching laser, which isn't easy to be
interfered by the environment.
F. GPS Module
Gobal positioning system (GPS) is used to extract
the GPS information. It can be interfaced with raspberry bi
by using python language. It is used to measure and
compute the positions in the earth. It contains tiny
processors and antennas that directly receive data sent by
satellites through dedicated RF frequencies.
2.2 HARDWARE CONNECTION
The two wheels of the chassis are connected with
two motors. Motor driver IC l293d is used to control the
motor, one motor driver IC can control only two motors.
So, the proposed system used one l293d that is enough to
control the motors. So, the input for motor driver IC is
given by the Raspberry pi and the output pins of the motor
IC are connected to the motor of the chassis.
For movement of the car in forward and backward
direction system will rotate the wheels in equal speed
whereas to maneuver left or right system will hamper the
one wheel as compare to the opposite one consistent with
the turning points i.e., if system got to rotate the turn in left
direction system got to hamper the left wheel and if there's
need to turn in right direction system have to slow down
the right wheel. In the proposed system the input to the
motor driver IC from is given from the GPIO pins (2,3) for
driving the left motor and GPIO pins (9,10) are employed
to drive the right motor from l293d.
The lidar may be a sealed unit with a motor
hanging from one end. The motor drives a turret that
rotates at around 300 rpm. The turret contains a laser and
receive sensor and by spinning provides a 360-degree scan
of the encompassing area. The laser and receive sensor
have two optical ports out of the turret. A two-pin
connector provides power to the motor. A four-pin
connector provides 5V power to the control circuits and
therefore the 3V3 serial interface.

3. FLOWCHART
Figure 3.1 represents the workflow diagram of the object
detection
Figure 3.1
Workflow of object detection,
Firstly, the object is captured from the camera. From the
captured image by using feature extraction its filters the
various components. It classifies the objects based on the
probability ratio. If the condition satisfies, it displays the
results with the probability ratio and if the condition does
not satisfy means it goes to the first step and performs the
same
4.EXPERIMENTAL RESULTS
4.1. Ros-Based Obstacle Avoidance Bot:
Figure –4.1: Simulation of ROS-based obstacle avoidance
bot
4.2. Real time Object Detection using Python-OpenCV:
Figure–4.2: Object Detection using Python-OpenCV
Fig –4.3: Object Detection results
4. CONCLUSIONS
A multi-modal system for detecting, tracking and
classifying objects in an environment for autonomous
vehicles was presented in this paper. According to the
algorithm and different combination of sensors gives the
better results. Lidar sensor and camera both play a major
role in this vehicle and it gives better environment
presents for the vehicle. This Autonomous vehicle has to
updated according to the environment change.
5. FUTURE SCOPE
The work could be enhanced by improving the
algorithm by adding advanced machine learning to it. Using
advanced algorithms, we can improve Image processing

algorithms. Multi layered processors can be used for fast
processing.
REFERENCES
[1] Johann Borenstein & Yoram Koren, Obstacle
Avoidance with Ultrasonic Sensors, IEEE JOURNAL OF
ROBOTICS AND AUTOMATION, VOL. 4, NO. 2, APRIL I988,
pp. 213-218
[2] Yue Wanga, Eam Khwang Teoha & Dinggang Shenb,
Lane detection and tracking using B-Snake, Image and
Vision Computing 22 (2004) , available
at:www.elseviercomputerscience.com, pp. 269–280.
[3] H. Dahlkamp, A. Kaehler, D. Stavens, S. Thrun, and
G. Bradski. Self-supervised monocular road detection in
desert terrain. G. Sukhatme, S. Schaal, W. Burgard, and D.
Fox, editors& Proceedings of the Robotics Science and
Systems Conference, Philadelphia, PA, 2006.
[4] Joel C. McCall &Mohan M. Trivedi, Video-Based Lane
Estimation and Tracking for Driver Assistance: Survey,
System, and Evaluation, IEEE Transactions on Intelligent
Transportation Systems, vol. 7, no. 1, March 2006, pp. 20-
37.
[5] Tushar Wankhade & Pranav Shriwas, Design of Lane
Detecting and Following Autonomous Robot, IOSR Journal
of Computer Engineering (IOSRJCE) ISSN: 2278-0661
Volume 2, Issue 2 (July-Aug. 2012), pp. 45-48.
[6] Xiaodong Miao, Shunming Li & Huan Shen, On-Board
lane detection system for intelligent vehicle based on
monocular vision, International Journal on Smart Sensing
and Intelligent Systems, vol. 5, no. 4, December 2012, pp.
957-972.
[7] A. Bar Hillel, R. Lerner, D. Levi, & G. Raz. Recent
progress in road and lane detection: a survey. Machine
Vision and Applications, Feb. 2012, pp. 727–745
[8] Narathip Thongpan, & Mahasak Ketcham, The State
of the Art in Development a Lane Detection for Embedded
Systems Design, Conference on Advanced Computational
Technologies & Creative Media (ICACTCM’2014) Aug. 14-
15, 2014
[9] Narayan Pandharinath Pawar & Minakshee M. Patil,
Driver Assistance System based on Raspberry Pi,
International Journal of Computer Applications (0975 –
8887) Volume 95– No.16, June 2014, pp. 36-39.
[10] J.M.A. Alvarez, A.M. Lopez & R. Baldrich,
IlluminantInvariant Model-Based Road Segmentation.
Intelligent Transportation Systems, IEEE Transactions on,
12, 2008, pp 184–193.

Autonomous Vehicle by using 3D LIDAR and 2D Camera

More Related Content

Similar to Autonomous Vehicle by using 3D LIDAR and 2D Camera (20)

More from IRJET Journal (20)

Recently uploaded (20)

Autonomous Vehicle by using 3D LIDAR and 2D Camera