Localization of license plate number using dynamic image processing techniques and genetic algorithms

1

Abstract—In this research, a design of a new genetic algorithm
(GA) is introduced to detect the locations of the License Plate
(LP) symbols. An adaptive threshold method has been applied to
overcome the dynamic changes of illumination conditions when
converting the image into binary. Connected component analysis
technique (CCAT) is used to detect candidate objects inside the
unknown image. A scale-invariant Geometric Relationship
Matrix (GRM) has been introduced to model the symbols layout
in any LP which simplifies system adaptability when applied in
different countries. Moreover, two new crossover operators,
based on sorting, have been introduced which greatly improved
the convergence speed of the system. Most of CCAT problems
such as touching or broken bodies have been minimized by
modifying the GA to perform partial match until reaching to an
acceptable fitness value. The system has been implemented using
MATLAB and various image samples have been experimented to
verify the distinction of the proposed system. Encouraging results
with 98.4% overall accuracy have been reported for two different
datasets having variability in orientation, scaling, plate location,
illumination and complex background. Examples of distorted
plate images were successfully detected due to the independency
on the shape, color, or location of the plate.
Index Terms—Genetic algorithms, image processing, image
representations, license plate detection, machine vision, road
vehicle identification, sorting crossover.
I. INTRODUCTION
HE detection stage of the LP is the most critical step in an
automatic vehicle identification system [1]. A numerous
research has been carried out to overcome many problems
faced in this area but there is no general method that can be
used for detecting license plates in different places or
countries, because of the difference in plate style or design.
Manuscript received September 11, 2012. This paper was funded by the
Deanship of Scientific Research(DSR), King Abdulaziz University, Jeddah,
under grant No.(22-611- D1432). The authors, therefore, acknowledge with
thanks DSR technical and financial support.
G. Abo Samra is an associate professor in the Faculty of Computing and
Information Technology, King Abdulaziz University-Saudi Arabia (phone
00966-509189962; fax:00966-(02) 6951605;email: gabosamra@kau.edu.sa)
F. Khalefah is a lecturer in the Faculty of Computing and Information
Technology, King Abdulaziz University (f.khalefah@gmail.com).
Copyright (c) 2012 IEEE. Personal use of this material is permitted.
However, permission to use this material for any other purposes must
be obtained from the IEEE by sending a request to pubs-
permissions@ieee.org.
All the developed techniques can be categorized according to
the selected features upon which the detection algorithm was
based and the type of the detection algorithm itself. Color-
based systems have been built to detect specific plates having
fixed colors [2], [3], [4]. External-shape based techniques
were developed to detect the plate based on its rectangular
shape [5], [6], [7], [8]. Edge-based techniques were also
implemented to detect the plate based on the high density of
vertical edges inside it [9]-[11]. Researches in [12] and [13]
were based on the intensity distribution in the plate’s area with
respect to its neighborhood where the plate is considered as
Maximally Stable Extremal Region (MSER). Many
researchers have combined different features in their systems
[14], [15], [16], [17], [18]. The applied detection algorithms
ranged from window-based statistical matching methods [18]
to highly intelligent-based techniques that used neural
networks [19], [20] or fuzzy logic [21]. GAs have been used
rarely because of their high computational needs. Different
researches have been tried at different levels under some
constraints to minimize the search space of GAs. Researchers
in [22] based their GA on pixel color features to segment the
image depending on stable colors into plate and non plate
regions, followed by shape dependent rules to identify the
plate’s area. Success rate of 92.8% was recorded for 70 test-
samples. In [23], GA was used to search for the best fixed
rectangular area having the same texture features as that of the
prototype template. The used technique lacks invariability to
scaling because fixed parameters have been used for the size
of the plate’s area. In [24], GA was used to locate the plate
vertically after detecting the left and right limits based on
horizontal symmetry of the vertical texture histogram around
the plate’s area. The drawback of this method is its sensitivity
to the presence of model identification text or other objects
above or below the vehicle which can disturb the texture
histogram. GA was used in [25] to recognize the LP symbols
not to detect the LP. Another group of researchers tried to
manipulate the problem from the texture perspective to
differentiate between text and other image types [26], [27].
The main drawback of these segmentation techniques was
their intensive computational demand and also sensitivity to
the presence of other text such as bumper stickers or model
identification.
Detecting license text and at the same time distinguishing it
from similar patterns based on the geometrical relationship
between the symbols constituting the license numbers is the
Localization of License Plate Number Using
Dynamic Image Processing Techniques And
Genetic Algorithms
G. Abo Samra, F. Khalefah
T
IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTING VOL : 18 NO: 2 YEAR 2014

2
selected approach in this research. Consequently, a new
technique is introduced in this paper which detects LP
symbols without using any information associated with the
plate’s outer shape or internal colors to allow for the detection
of the license numbers in case of shape or color distortion
either physically or due to capturing conditions such as poor
lighting, shadows and camera position and orientation. To
search for the candidate objects and to allow for tolerance in
the localization process, a new genetic algorithm has been
designed with a new flexible fitness function. Image
processing is carried out at first to prepare for the GA phase. A
complete overview of the system is given in Section II. Image
processing stages are presented in Section III. In Section IV,
GA formulation is demonstrated. In Section V, modifications
to the GA stage to overcome most of the problems associated
with CCAT are summarized. Finally, the results are discussed
in section VI.
II. SYSTEM OVERVIEW
In this section, an overview of the system is introduced. The
proposed system is composed of two phases: image processing
phase and GA phase. Each phase is composed of many stages.
The flowchart in Fig. 1 depicts the various image processing
stages that finally produce image objects to the GA phase. GA
selects the optimum LP symbol locations depending on the
input GRM that defines the geometrical relationships between
the symbols in the concerned LP.
Color
Image
Adaptive Binarization
Morphological operations
Connected Component Analysis
Objects Size Filtering
GA
Binary Image
Binary Image
Array Of N Objects
Array Of M Filtered
Objects (M<N)
GRM for the
concerned
license plate
layout
License plate symbols
locations (L symbols)
Color to Grayscale Conversion
Grayscale Image
Fig. 1. The system’s overall flowchart for the localization of the LP symbols.
III. IMAGE PROCESSING PHASE
In this phase, an input color image is exposed to a sequence
of processes to extract the relevant two dimensional objects
that may represent the symbols constituting the LP. These
processes that are carried out in different stages, as depicted in
Fig. 1, will be presented in the following subsections.
A. Color to grayscale conversion
The input image is captured as a color image taking into
account further processing of the image to extract other
information relevant to the concerned vehicle. Color (RGB) to
grayscale (gs) conversion is performed using the standard
NTSC method by eliminating the hue and saturation
information while retaining the luminance as follows:
gs=0.299*R+0.587*G+0.114*B (1)
Fig. 2 shows an example of the output of this stage that will
be used as input to the next stage.
Fig. 2. Converted grayscale image.
B. Gray to binary using a dynamic adaptive threshold
Converting the input image into a binary image is one of the
most sensitive stages in localizing LPs due to spatial and
temporal variations encountered in the plate itself and the
environment around it resulting in several illumination
problems. Hence binarization of the image according to a
fixed global threshold is not suitable to overcome these
problems. In our system, a local adaptive method based on the
techniques described in [28] has been implemented to
determine the threshold at each pixel dynamically depending
on the average gray level in the neighborhood of the pixel. A
simple yet effective rule has been adopted to differentiate
between foreground and background pixels. If the pixel
intensity is higher than 90% of the local mean it is assigned to
the background; otherwise it is assigned to the foreground.
The 10% offset below the mean is chosen experimentally to
minimize the sensitivity to fluctuations in illumination. The
size of the window used to calculate the threshold for each
pixel is selected according to the image resolution and the
expected size of the license symbols. A 30x30 window has
been applied on the first set of image samples used in this
research, which resulted in a high accuracy rate in different
illumination conditions as will be presented in the results
section. Although some images can be binarized successfully
using Otsu's global threshold method [29] as shown in Fig.
3(a), others as that shown in Fig. 3(b) may produce incorrect
results as shown in Fig. 3(c). On the other hand, local adaptive
binarization will give satisfactory output as shown in Fig. 3(d)
for the same image in Fig. 3(b).
Fig. 3. (a) Converted binary image for image in Fig. 2, using Otsu's method,
(b) Car image with variable illumination, (c) output when using Otsu's
method for image in (b), (d) output when applying local adaptive threshold
method for the same image in (b).

3
C. Morphological operations
Morphological operations such as dilation and erosion are
important processes needed for most pattern recognition
systems to eliminate noisy objects and retain only objects
expected to represent the targeted patterns. In LP detection,
closing operation (dilation followed by erosion) is performed
to fill noisy holes inside candidate objects and to connect
broken symbols. On the other hand, opening (erosion followed
by dilation) is applied to remove objects that are thinner than
the LP symbols. In our system, closing is applied to fill
spaces that break the bodies of symbols using a 3-pixel-disk
element in the first experiment. This process is very important
especially for the recent Saudi LP layout where a light gray
watermark is used for authentication purposes. This
watermark becomes white after the binarization process and
breaks down most of the bodies of the LP symbols as shown
in Fig. 4(a). Fig. 4(b) shows the output after applying closing.
For non Saudi LP images, a disk radius of less length is used
to prevent filling the spacing between the LP symbols.
Removal of thin objects is performed in the size filtering
stage.
Fig. 4. Morphological closing using 3-pixel disk- element
(a)Input Binary Image, (b)Image after closing operation.
D. Connected Component Analysis (CCA) and objects
extraction
CCA is a well known technique in image processing that
scans an image and groups pixels in labeled components based
on pixel connectivity [30]. An 8-point CCA stage is performed
to locate all the objects inside the binary image produced from
the previous stage. The output of this stage is an array of N
objects. Fig. 5 shows an example of the input and output of
this stage.
Fig. 5. CCA example: (a) Input image, (b) output objects (N=2287).
E. Size filtering
The objects extracted from the CCA stage are filtered on
the basis of their widths Wobj and heights Hobj such that the
dimensions of the LP symbols lie between their respective
thresholds as follows:
Wmin ≤ Wobj ≤Wmax and Hmin ≤Hobj ≤ Hmax (2)
Where Hmin and Wmin are the values below which a symbol
cannot be recognized (8 pixels for example) and Wmax can be
set to the image width divided by the number of symbols in
the license number. Hmax is calculated as Wmax divided by the
aspect ratio of the used font. The ranges of these values can be
narrowed in the case of a mounted camera to speed up the
process of detection but for a moving camera, the ranges
depend on the required object to camera distance range. The
output of this stage is an array of M objects. An example for
the output of this stage is given in Fig. 6 after applying size
filtering on the objects shown in Fig. 5(b).
Fig. 6. M objects (64) output after size filtering of the N objects in Fig. 5(b).
IV. GA PHASE
In the following sections, the formulation of the GA phase
to resolve the 2D compound object detection problem will be
introduced in details, indicating the encoding method, initial
population setup, fitness function formulation, selection
method, mutation and crossover operator design and
parameters setting.
A. Chromosome encoding
Encoding of a compound object such as the LP is
accomplished based on the constituting objects inside it. Since
the next step after plate detection is to recognize the license
number, hence the main symbols identifying the plate number
should be included as a minimum. In the case of the recent
Saudi LP, for example, there are 4 Arabic digits and 3 English
letters. Other symbols in the LP can be added to extend the
representation for more layout discrimination if needed. In our
experiments, only the 7 symbols (4 digits and 3 letters) are
used to detect the LP number. Hence for the Saudi layout, as
an example, the chromosome will be composed of 7 genes as
shown in Fig. 7. An integer encoding scheme has been
selected where each gene i is assigned an integer j which
represents the index to one of the M objects output from the
size filtering stage. The information that will be used for each
object j is as follows:
 The upper left corner coordinates (X, Y) of the rectangle
bounding the object.
 The height (H) and width (W) of the rectangle
bounding the object.

4
W2W1
H2
H1
(X2,Y2)
(X1, Y1)
O1
O2
Fig. 7. A chromosome of 7 genes for the representation of the Saudi LP.
B. Defining the fitness function
The proposed fitness is selected as the inverse of the
calculated objective distance between the prototype
chromosome and the current chromosome. Before clarifying
how the objective distance is measured, we will show first
how the geometric relationships between the objects inside a
compound object are represented, followed by a discussion of
parameter adaption in case of various LP detection layouts.
Compound object representation
For any two objects, we will use two types of geometrical
relationships that can be defined as follows:
1. Position relationship:
The position relationship will be represented by the
relative distances between the bounding boxes of the
two objects in the X and Y directions.
2. Size relationship:
The size relationship will be represented as the
relative differences in their bounding boxes’ heights
and widths.
In the above relationships, relativity is achieved by
dividing on the height or width of the first object depending on
which is more stable for practical reasons although it is
logically to divide differences in heights on height and
differences in widths on width to compensate for scale
changes in the general case. For most LPs (see Appendix), the
heights of symbols are almost equal for both digits and letters
while some symbols have different widths than others. Hence,
normalized relationships between any two objects can be
based on the height of the first object.
Fig. 8. The layout of two objects O1and O2.
Considering the two objects O1 and O2 shown in Fig. 8,
the position relationship is defined in the two directions by the
following formulas:
– (3)
– (4)
The size relationship is defined by the following
formulas:
(5)
(6)
The above representation, although preserves the geometric
relationships between the rectangles bounding the objects, it
does not represent the objects’ shapes because they are
unknown in case of an unknown plate. Only the aspect ratio
for fixed width-fonts can be added for the first object as
follows:
AS1=W1/H1. (7)
To generalize the representation for any compound object
composed of L objects, only 4(L-1) relationships will be
required in addition to the aspect ratio for the first object.
For example, if we consider the Saudi LP shown in Fig. 9,
25 relationships will be required to represent the layout of the
seven lower license symbols (4 Arabic digits and 3 English
letters) (L=7).
Fig. 9. Saudi LP with rectangles bounding the represented symbols.
In general, for L-symbol LP we will need (4(L-1)+1) values
to represent the license number. Placing the values of the
different relationships in one matrix (excluding the aspect
ratio), produces what we called the geometric relationships
matrix GRM. Table I presents the values of the GRM for the
Saudi LP shown in Fig. 9; where the variable j denotes the
index of each symbol from left to right.
TABLE I: GRM VALUES FOR THE SAUDI LP.
j 1 2 3 4 5 6
RXj+1,j 0.615 0.615 0.615 0.77 1 1
RYj+1,j 0 0 0 0.2 0 0
RWj+1,j 0 0 0 -0.08 0 0
RHj+1,j 0 0 0 -0.5 0 0
To adapt the system for a different LP layout either in the
same country or in a different one, a different GRM matrix
can be simply defined.
Objective Distance (OD) and Fitness Formulation
Considering the distance between the prototype
chromosome p, corresponding to the input GRM, and any
chromosome k, five distance values can be defined as follows:
(8)
(9)
(10)
(11)
gene number (i) : 1 2 3 4 5 6 7
object
index ( j)
3 1 5 7 11 2 15

5
(12)
Clearly, these distance values will be considered as the
objective distance functions in our GA problem, which should
be minimized.
Combining the five objective distance functions into one
global objective distance function that represents the
distance between any chromosome k and the prototype
chromosome p, is performed through the following formula:
(13)
Where , , , and are weighting parameters
that should be given values according to the problem under
consideration.
Since, as stated in the literature, the fitness is a function that
should be maximized, hence the fitness of chromosome k
( ) can be related to the global objective distance function
as follows:
(14)
Adaption for the LP detection problem
The previous formulation can be used for the representation
of a compound object consisting of a group of smaller objects
and can be used to locate the compound object in an image
given that its GRM values are nearly fixed. This formulation
has an advantage of overcoming scaling effect. It can also
overcome orientation variability either by aligning the
compound objects to a certain direction line or by taking
projection parameters into account in the original formulation.
Although in our formulation, orientation independency is not
taken into consideration, detection of plates having different
orientations has been achieved as will be shown in the results
section due to the flexible range of accepted fitness (or
objective distance) values.
An important issue that should be considered when adapting
the previous model for LP detection is the variable nature of
the internal components (letters or digits) in an unknown plate.
In other words, the symbols inside an unknown plate can vary
in size and/or spacing. Hence representation of an unknown
plate license number will require some adaptation of the
general model according to the nearly stable features of the
symbols used in the plate and the relative spacing between
them. Referring to the Appendix which contains 25 LPs for 25
different countries in different continents, adaption can be
carried out as follows:
 Since, in some LP layouts especially in Arabic countries,
the symbols used can differ greatly in their widths, hence
the term corresponding to the relative width can be
neglected by setting the weighting parameter ww to zero.
 The aspect ratio of the starting letter is not fixed in case of
variable width fonts and also may be changed due to
perspective mapping. Hence, a range of accepted values
of the aspect ratio can be imposed in the size filtering
stage based on the characteristics of the used font. This
means that the parameter may also be set to zero.
 The values of the parameters , , can be selected
to reflect the importance of each term as follows.
-Since the license symbols are almost horizontally
aligned, then will be given the highest value (e.g. 4).
-For fixed height fonts, can be as high as (e.g. 4),
but for the general case its value can be selected lower
than (in case of Arabic letters) because horizontal
alignment is more important than height changes.
-The term corresponds to the horizontal
distance between LP symbols which can vary for
variable width fonts; hence its weighting parameter
will be given the lowest value (e.g. 1).
 Hence, the objective distance function for the detection
of the LP number for almost all LPs (Arabic digits and
English letters) shown in Appendix, can be put in the
following form:
(15)
It should be recorded here that the selection of the
weighting parameters affects and guides the genetic search
space during the production of new generations and hence
affects both speed and accuracy of the overall system.
Finally, by substituting from 15 into 14, the formula for the
fitness of a chromosome k ( ) is given as follows:
(16)
C. The selection method
In our system, the Stochastic Universal Sampling (SUS)
method has been adopted for the selection of offspring in the
new generation. In SUS method [31], each individual is
mapped to a continuous segment of a line equal in size to its
fitness as in roulette-wheel selection. Then, a number of
equally spaced pointers are placed over the line depending on
the percentage of individuals to be selected. In our system,
individuals of ninety percent of the population size (0.9 Z) are
selected to be exposed to mutation and crossover operators.
D. Mutation operators
Mutation is needed because successive removal of less fit
members in genetic iterations may eliminate some aspects of
genetic material forever. By performing mutation in the
chromosomes, GAs ensure that new parts of the search space
are reached to maintain the mating pool variety [32]. We have
implemented two types of interchangeably used mutation
operators; substitution operator and swap operator as follows:
Substitution operator
In this type of operators, a random position in the
chromosome is selected and the corresponding allele is
changed by a new random object from the M available objects.

6
The new object should be legitimate which means it does not
belong to the current mutated chromosome. Fig. 10(a) shows
an example of the application of this operator.
Swap operator
In this operator, we implemented the reciprocal exchange
mutation that selects two genes randomly and swaps them as
shown in Fig. 10(b). This operator has the advantage of
rearrangement of the mutated chromosome in a way that may
improve its fitness by reordering of the internal objects to
match the prototype’s order.
(a) Substitution operator. (b) Swap operator.
Fig. 10. Examples for the used mutation operators.
E. Crossover operator
There are many methods to implement the crossover
operator. For instance, single point crossover, two point
crossover, n-point crossover, uniform crossover, three parent
crossover and, alternating crossover [33], etc. These operators
are not suitable for our problem because the resultant children
will not be valid because of repeated genes that may be
produced in the generated chromosomes. Also, if we prevent
repetition, the resultant children‘s fitness will be enhanced
slowly because of the randomness of these mechanisms. An
alternative solution is to design a suitable crossover operator
that insures enhancement of the generated offspring. Since, in
case of LP detection problem, GA is used to search for a
sequence of objects having nearly the same y-position and
placed in order according to their x-positions, then the
problem can be gradually solved by dividing the recombined
chromosomes’ objects according to their y-positions into two
groups and then sorting each group (constituting a
chromosome) according to the x-positions. Following the
above discussion, we propose a new crossover method that
depends mainly on sorting as follows:
1.The two parent chromosomes are combined into one longer
array Carray that includes a number NC of non repeated
genes as shown in Fig. 11 (a, b, c). The underlined gene
number indicates its repetition and that only one copy of it
will be transferred to Carray.
2.The genes inside Carray are sorted in ascending order
according to the Y-coordinate of the object corresponding
to each gene as shown in Fig. 11(d).
3.Carray is scanned from left to right starting from index 1 to
L, to construct the first child giving it the first L genes as
shown in Fig. 11(e).
4.Carray is scanned from left to right starting from index
NC-L+1 to NC, to construct the second child giving it the
last L genes as shown in Fig. 11(f).
5.Each child is sorted in ascending order according to the X-
coordinate of each gene’s object to produce the final shape
of each child as shown in Fig. 11 (g, h) respectively.
Fig. 11. The proposed crossover operator steps.
Since this operator starts by combining objects without
repetition (Union operation) and then performs Sorting and
Partitioning followed by Sorting, we can call it USPS
crossover to differentiate it from the known sorting crossover.
Although sorting requires a significant time, the overall time is
a- Parent 1
1
1 2 3 4 5 6 7gene index
gene value
X, Y 12,18 9,4 3, 10 8, 11 2, 20 6, 5 14,13
5 6 4 9 7 8 11
b- Parent 2
1 2 3 4 5 6 7gene index
gene value
3 2 4 10 1 12 15
7, 12 4, 19 3, 10 2, 7 17, 3 19, 8 15, 16
c- Carray, after combination operation
gene index
gene value
1 2 3 4 5 6 7 8 9 10 11 12 13
5 6 4 9 7 8 11 3 2 10 1 12 15
e- Child 1, before sorting
gene value
1 6 8 10 12 4 9
f- Child 2, before sorting
gene value
9 3 11 15 5 2 7
g- Child 1, after sorting in ascending order according to X-
coordinate.
gene value 10 4 8 9 6 1 12
h- Child 2, after sorting in ascending order according to X-
coordinate.
gene value 7 2 3 9 5 11 15
d- Carray, with genes sorted in ascending order on Y
gene index
gene value
1 2 3 4 5 6 7 8 9 10 11 12 13
1 6 8 10 12 4 9 3 11 15 5 2 7
X, Y
4
1
1
1
2
6
6
6
3
9
5
4
3
2
6
61
19
5
17
9
23
2
1
4
21
11
10
45
24
1
29
6
8
15
18
3

7
reduced because the number of generations needed to reach to
the stopping criteria is sharply decreased as shown in Fig.
12(b) compared to the case of 2-point random crossover
shown in Fig. 12(a). The offspring of the random crossover
are repaired by replacing repeated genes by non repeated ones
randomly. Fig. 13 presents another 2 examples in which the
USPS crossover outperforms the 2-point random crossover.
Fig. 12. Objective Distance (OD) convergence graphs for the image
in Fig. 2, using 2-point random crossover in (a) and USPS crossover in (b).
Fig. 13. Objective Distance (OD) convergence graphs for the two images
shown in a-1 and a-2 with random crossover in b-1 and b-2 and USPS
crossover in c-1 and c-2 respectively.
Trying other types of crossover operators may give better
results than the tested one, but we are sure that our proposed
crossover will outperform them because it generates the final
solution almost in one step if the solution genes are included
in the original parents. Moreover, it gives excellent
intermediate solutions by clustering the objects based on their
y-positions and sorts them based on x-positions.
F. Replacement strategy
Many replacement strategies are used in case of replacing
only a portion of the population between generations. The
most common strategy is to probabilistically replace the less
fit individuals in the previous generation. In elitist strategy the
best fit individuals of the previous generation are appended to
the current population [34]. In our proposed system, the best
10% of the parents are selected and appended to the offspring
(90%) to produce the new generation (100%).
G. Stopping criteria
The GA stops if one of the following conditions is met:
1-The best chromosome’s objective distance (OD) is less
than 5. (This value is found by trial and error).
2-The average objective distance (AOD) is not improved for
6 successive generations. In this case, the chromosome
having minimum objective distance can be accepted if it
is less than 8. This maximum limit will affect the
allowable angle range for the detected license numbers as
shown in Table II.
TABLE II: OBJECTIVE DISTANCES (OD) FOR PLATES INCLINED AT DIFFERENT
ANGLES [A:-30◦
TO +45◦
].
A = - 3 0 ◦
,
OD=16.6
A = - 2 0 ◦
,
OD=11.2
A = 1 0 ◦
,
OD=6.4
A = 2 0 ◦
,
OD=11.2
A = 4 5 ◦
,
OD=19.2
3-The number of generations Ngen reaches to the maximum
number of generations MaxNgen (set to 20).
H. Parameters setting
The population size (Z) is selected dynamically according to
the formula developed in [34] as follows:
Z =1.65 × 2(0.21*blength)
(17)
Where blength is the length of the chromosome in case of
binary encoding. In case of integer encoding, for L genes and
M objects, we substitute for blength by:
blength =L (Log2(M)) (18)
Although formula (17) is driven in 1989, it speeds up the
convergence of the GA as shown in Fig. 14 (Z=845, Ngen=5)
compared to the graph in Fig. 12(b) with static population size
(Z=500, Ngen=12) for the same image in Fig. 2.
Fig. 14. OD convergence graph for dynamic population size (Z=845).

8
V. MINIMIZING CCAT PROBLEMS
The main drawback of all systems relying on CCAT is the
sensitivity to negative or positive noise that may cause some
symbols to be connected to other objects or broken into
smaller objects. Connectivity is also affected by many causes
such as bullets, dirt, aging, occlusion, shadows, or due to
image processing operations like dilation and erosion. The
effect of connected symbols either to their neighbors or to the
frame of the LP is the introduction of high objective distance
when matched with the corresponding GRM. This surely, will
cause the system to select another near symbol or to report
absence of LP in the current image. The same is said in the
case of broken bodies, where a large error is detected in the
relative width/height terms. The proposed solution for this
problem is to introduce a new argument in the genetic
algorithm which indicates the number of symbols to skip (NS)
during the evaluation of the objective distance. This number is
initialized to zero in the first run of the GA and according to
the optimum OD threshold value (ODT), a decision is made
either to accept the selected chromosome or to increase the NS
argument and execute a further run of the GA. Depending on
the number of LP symbols (L) under consideration, the
skipping number NS can reach to a maximum value given that
L-NS>=3. If NS is greater than zero then NS random numbers
having values between 1 and L are generated inside the OD
evaluation function, and the corresponding gene error
distances are skipped. Surely, these random numbers should
be stored in separate fields associated with each chromosome
as shown in Fig. 15.
NS=3
Fig. 15. An example for the locations of skipped genes stored in separate
fields associated with each chromosome.
Finally if the OD threshold (ODT) test is met, the location
and size of the skipped symbols are estimated based on the
GRM matrix and the non skipped symbols using some
geometrical rules. The flow chart illustrating the logic of these
steps is shown in Fig. 16. The proposed modification has
solved many problems happening due to static causes like dirt
and pullets or dynamic causes like shadows, lighting and even
blurring caused by camera movement because it gives another
dimension for solving the LP detection problem by expecting
locations of occluded or distorted symbols. Fig. 17 shows an
example of two images from two different layouts (Saudi and
Greece) containing touching symbols in two different ways
which have been correctly detected after GA modification.
Another simple yet very effective modification is done to
cluster objects based on their x-positions by introducing an
alternate USPS crossover which works interchangeably with
the described one but it sorts the unified parents based on x-
positions first instead of y and hence after the partitioning
step, children will be already sorted based on x-positions. This
modification speeds up the GA phase in case of large number
of objects output from CCAT by about 20% than without it.
Fig. 16. Flowchart for detecting LP’s symbols in case of touching and broken
symbols.
(a) (b)
Fig. 17. Examples for locating symbols touching the digit’s bounding box
(digit ‘1’ in (a)) and touching each other (last two digits in (b)).
VI. RESULTS AND DISCUSSION
The proposed system has been implemented using
MATLAB. Two experiments were carried out. The first
experiment was done on a Saudi LP dataset composed of 800
car images acquired at various camera-to-object relative
positions in different lighting conditions. To compare our
results, we consider the work published in [9], because it used
an annotated database which allows us to precisely experiment
our technique on the same dataset. Hence, the second
experiment is performed on the same 335 image samples used
by [9] that are available in [35] for Greece LPs (shown in the
Appendix table (column 2 row 2)). This database includes four
datasets: Day (color images large sample), Day (close view),
Day (with shadows), and Shadows in plate. Only one standard
Greece LP image is used to construct the GRM matrix to adapt
our system on this different layout as shown in Table III.
Noting that zero values in the 2nd
, 3rd
, 4th
rows mean same base
line, fixed width and fixed height respectively for all the
symbols used in the defined plate. In both experiments, the
third raw is neglected by setting the weighting parameter ww to
zero due to violation of the fixed width for the letter ‘I’ and
digit ‘1’. Results of our experiments are summarized in the
first three rows in Table IV; where false positive (FP) means
assigning incorrect locations to LP symbols. Surely the
remaining undetected cases are false negative (FN) cases
where each image includes an LP but the ODT test is not met
YES
YES
YES
NO
NO
NS=0
GA(NS)
OD<ODT
T
L-NS >= 3
Locate skipped
symbols
LP is not found
NS=NS+1
NS > 0
Send locations to next
stage
NO
gene number (i) : 1 2 3 4 5 6 7
object
index (j) 3 1 5 7 11 2 15
2 4 7

9
for all combinations of objects in the tested images. Results
for the two experiments done by [9] are shown in the 4th
and
5th
rows in Table IV. The first experiment done by [9] is
performed on the same 335 samples in [35]. The second
experiment is done after appending an enlarged version of the
largest 113 images in the first experiment to its 335 samples.
TABLE III: GRM FOR THE GREECE LP.
j 1 2 3 4 5 6
RXj+1,j 0.8 0.8 1.55 0.8 0.8 0.8
RYj+1,j 0 0 0 0 0 0
RWj+1,j 0 0 0 0 0 0
RHj+1,j 0 0 0 0 0 0
TABLE IV: RESULTS OF OUR EXPERIMENTS AND [9]’S EXPERIMENTS.
Exp #
Number
of test
samples
Detected False
negative
N#(rate)
False
positive
N#(rate)
Success
rate
without
skipping
with
skipping
1st
(ours) 800 738 52 8 (1%) 2 (0.25%) 98.75%
2nd
(ours) 335 297 30 6 (1.79%) 2 (0.59%) 97.61%
1&2(ours) 1135 1035 82 14 (1.23%) 4 (0.35%) 98.41%
1st
[9] 335 333 2 (0.6%) 45 ( 13.43%) 99.4%
2nd
[9] 448 441 7 (1.5%) 54 (12.05%) 98.4%
Although the detection rate in 1st
[9] is higher than in 2nd
(ours),
we have less FP rate (0.25%) compared to (13.4%). In
addition, the output of our system is at the second stage of LP
recognition systems (symbol segmentation stage), which
implies that the error percentage in [9] will increase after
performing the symbol segmentation stage. Moreover, the
method in [9] used 89 random samples from 335 samples to
adapt the system which can be considered as training samples
that are used again in the test phase. In our case, only one
sample (outside of test dataset) is required to construct the
GRM matrix, no training is needed. The second experiment
performed by the author in [9] was done to prove the ability of
his system to detect larger LPs, which resulted in extra 5
undetected samples (after adding 113 samples), which means
that the system accuracy is affected by scaling. In our case,
scaling, will not affect the results if done on the same dataset
as long as the candidate symbols lie within the specified
ranges in the size filtering stage. This can be verified from the
results of our first experiment where symbol heights ranges
from 16 to 300 pixels. Also, we can deduce that the system in
[9] is resolution dependent because all images were scaled to
640x480 before performing the experiments. However, the
following samples of images reveal the robustness and
distinction of our approach. Samples from the first set of
images in experiment 1 are shown in table V and samples
from the second set in experiment 2 are shown in table VI and
VII. The real number below each image indicates the objective
distance value according to the input prototype GRM for each
experiment. The maximum accepted value of the OD for both
experiments is 8 (increased to 15 to show cases –some of them
are outside the test samples- having physical or perspective
distortion or blurring due to movement of the capturing
device). Table VII includes samples categorized as in [35].
The shown images include the following difficulties:
 Different resolution, scaling, different perspective
distortion and different plate locations for all images in both
experiments.
 Different illumination conditions at day and night with
- Poor ambient lighting: Table V: E1, E2, E3, E4, and
Table VI: B1, D5.
- Blurred images: Table V: A2, B3, B4, C2, C3, C4, C5,
D5, D6 and Table VI: B4, E4, G4.
- Shadows: Table V: A2, A3, A4, A5, B4, D4, H1 and
Table VI: B2, B3, C2, C4, D4, F1.
- Low contrast: Table V: B2, D2 and Table VI: A1, C1,
C2, D3, F2.
- Strong headlight, back light or high flash influences:
Table V: F1, F2, F3, F4, F5 and Table VI: G2, G4.
 Plate distortion: Table V: A1, B1, C1, C2, C3, C4, C5,
C6, E5 and Table VI: A1, A2, A4, D3, E1, F1.
 Aging effects: Table V: A1, B1, E5, E6, H5, H6, G5, F6
and Table VI: A1, A2, C3, F3.
 Pictures cluttered with :
- Texture regions (inside or outside the vehicle): Table
V: A3, E6, H1, H2, H3, H5 and Table VI: A5, C4, D5,
E3, F1, F2, F3, G1, G3.
- Edge regions (inside or outside the vehicle): Table V:
A1, B4, E5, H4, H6, and Table VI: C1, D1, F4, G3.
- Different objects in the background: Table V: A3, C4,
D4, H2 and Table VI: A3, C1, C2, C4, E2, F3, F5, G5.
 Examples of images detected despite the presence of
other text similar to the license number’s text: Table V: D3,
G1, G2, G3, G4, G5 and Table VI: A5, G5.
 Both LP background and vehicle body have the same
color: Table V: B2, B5, C3, C4 and Table VI: B3.
 Plate background color is not white: Table VI: D1, D2.
 Touching symbols (detected after skipping modification):
Table V: A6, B6, C6, E6, H5 and Table VI: A4, A5, B3, B5,
C5, D3, D5, F1.
 Broken or missed symbols (detected after skipping
modification): Table V: A1, E5, F6, H6 and table VI: A1,
A2, C1, E5, F3.
Regarding the speed of our system, without code
optimization and working on a 2.6 GHZ PC with 2 GB RAM,
on average 0.12s is needed to locate the LP symbols for low
resolution images (640x480) and 0.34s for high resolution
images (2048x1536). This non linear relation between speed
and resolution is due to other factors that affect the speed of
different stages of the system such as the complexity of the
image which affects both image processing and GA stages.
The character recognition phase is expected to take not more
than 0.03s because all symbol images are now available, no
further segmentation is required. Hence, 2 to 6 images per
second can be fully recognized depending on the resolution
and capturing conditions. If we consider the detection speed
in [9], which ranges from 39ms to 49.8ms for 640x480
resolution, we will notice that our system is approximately
two times slower than [9] but many points should be
considered regarding this difference in speed. First, there is an
extra time needed to segment the LP into isolated symbols as
in our system. Second, the grayscale conversion and size
scaling times may not be considered in [9]. Third, the author
in [9] didn’t mention the programming environment which
may be visual C++ or MATLAB. Finally, we believe that
enhancing the speed of our system needs further code
optimization at many stages.

10
TABLE V: EXAMPLES OF IMAGES DETECTED IN EXPERIMENT 1 WITH OD VALUES UNDER EACH.
.

11
TABLE VI: EXAMPLES OF IMAGES DETECTED IN EXPERIMENT 2 WITH OD VALUES UNDER EACH IMAGE.

12
TABLE VII: EXAMPLES OF IMAGES DETECTED IN EXPERIMENT 2 AS CLASSIFIED IN [35].

13
Considering the genetic phase’s speed, great enhancement
has been achieved after using the USPS crossover operator.
Future research may consider clustering objects according to
their sizes and/or positions before being supplied to the
genetic phase to allow for the detection of multiple plates and
at the same time to increase the system speed. Currently, our
system can be used as it is in parking management systems,
and in the detection of LPs in pictures taken in emergent
circumstances that do not allow adjustment of the position and
orientation of the camera with respect to the vehicle. An
important point that should be recorded here is that through all
the experiments done, we have tried many types of local
adaptive thresholding methods, none of them gave 0% error
rate but after introducing the skipping part of the genetic phase
the error percentage due to binarization has been minimized as
shown in the final results. Local adaptive (or dynamic)
thresholding has been used a lot but integrating it with CCAT
and the skipping GA gives our technique distinction among
others. In spite of increasing the computation time of the
system, the skipping part in the genetic phase reduces human
intervention rate in case of system failure in the detection of
some LPs. In other words, more effort should be carried out in
the image processing phase to reduce the skipping time while
maintaining high accuracy rate of the system.
CONCLUSIONS
A new genetic based prototype system for localizing 2-D
compound objects inside plane images has been introduced
and tested in the localization of LP symbols. The results were
encouraging and a new approach for solving the LP detection
problem relying only on the geometrical layout of the LP
symbols has been experimentally proved. Also, a flexible
system has been introduced that can be simply adapted for any
LP layout by constructing its GRM matrix. The system proved
to be invariant to object distance (scaling), insensitive with
respect to perspective distortion within a reasonable angle
interval, and immutable to a large extent to the presence of
other types of images in the vehicle background. Due to the
independency on color and the adaptive threshold used for
binarization, the proposed system possessed high immunity to
changes in illumination either temporarily or spatially through
the plate area. Furthermore, our experiments proved that
although leaving some features in the compound object
representation due to the variable nature of the internal objects
such as the aspect ratios and the relative widths, a high
percentage success rate was achieved with the aid of the
adaptability aspect of the GAs. The ability of the system to
differentiate between LP text and normal text has been proved
experimentally. A very important achievement is overcoming
most of the problems arising in techniques based on CCAT by
allowing the GA to skip gradually and randomly one or more
symbols to reach to an acceptable value of the objective
distance. Moreover, an enhancement in the performance of the
developed GA has been achieved by applying the new USPS
crossover operators, which greatly improved the convergence
speed of the whole system. Finally, a new research dimension
for GAs has been opened to allow for the detection of multiple
plates and even multiple styles in the same image and to
increase the performance in terms of speed and memory and to
apply the same technique in other problem domains analogous
to the LP problem.
Appendix
A TABLE OF 25 LPS FOR 25 COUNTRIES IN THE 5 CONTINENTS.
ACKNOWLEDGMENT.
This paper was funded by the Deanship of Scientific
Research(DSR), King Abdulaziz University, Jeddah, under
grant No.(22-611- D1432). The authors, therefore,
acknowledge with thanks DSR technical and financial support.
REFERENCES
[1] Y. Qiu, M. Sun, and W. Zhou, “License Plate Extraction Based on
Vertical Edge Detection and Mathematical Morphology,”
International Conference on Computational Intelligence and
Software Engineering, pp. 1-5., 11-13 Dec. 2009.
[2] A. Ahmadyfard and V. Abolghasemi,“ Detecting License Plate
Using Texture and Color Information,” IST 2008, International
Symposium on Telecommunications, pp. 804–808, 2008.
[3] G. Li, R. Yuan, Z. Yang, and X. Huang, “A Yellow License Plate
Location Method Based on RGB Model of Color Image and
Texture of Plate,” Second Workshop on Digital Media and its
Application in Museum & Heritages, pp. 42–46, 2007.
[4] X. Shi, W. Zhao, Y. Shen, and O. Gervasi, "Automatic License
Plate Recognition System Based on Color Image Processing,"
Lecture Notes on Computer Science, Springer-Verlag, vol. 3483,
pp.1159 -1168, 2005.
[5] M. Deriche, “ GCC License Plates Detection and Recognition
Using Morphological Filtering and Neural Networks,” Int J. on
Comp. Sci. and Info Security, IJCSIS, vol. 8, No. 8, pp. 263-269,
Dec, 2010.
[6] O. Villegas, D. Balderrama, H. Domínguez and V. Sánchez, ”
License Plate Recognition Using a Novel Fuzzy Multilayer Neural
Network,” International Journal of Computers”, Issue 1,vol. 3,
2009.
[7] S.H. Mohades Kasaei, S.M. Mohades Kasaei and S.A. Monadjemi,
“A Novel Morphological Method for Detection and Recognition
of Vehicle License Plate,” American Journal of Applied Science,
vol.6 no.12, pp. 2066-2070, 2009.
[8] A. Theja, S. Jain, A. Aggarwal and V. Kandanvli, "License Plate
Extraction Using Adaptive Threshold and Line Grouping,"
International Conference on Signal Processing Systems (ICSPS),
vol.1, no., pp. 211-214, 5-7 July 2010.
[9] P. Tarabek, “Fast license plate detection based on edge density
and integral edge image,” International Conference on Applied
Machine Intelligence and Informatics (SAMI), pp. 37 – 40, 2012.
doi:10.1109/SAMI.2012.6208994
V i r g i n i a A r i z o n a C a l i f o r n i a V i c t o r i a Canada-Ontario
B o s n i a G r e e c e Great Britain D e n m a r k Germany-west
A r g e n t i n a B r a z i l C h i l e P a r a g u a y C o l o m b i a
E g y p t C a m e r o o n K e n i a M a u r i t a n i a S u d a n
A l b a h r a i n D u b a i J o r d a nO m a n Q a t a r

14
[10] V. Abolghasemi, and A. Ahmadyfard, ” A Fast Algorithm for
License Plate Detection,” International Conference on Visual
Information Systems, Springer, Vol. 4781, pp. 468-477, 2007.
[11] S. Roomi, M. Anitha and R. Bhargavi, ”Accurate License Plate
Localization,” International Conference on Computer,
Communication & Electrical Technology, 2011, pp. 92-99.
[12] W. Wang, Q. Jiang, X. Zhou, and W. Wan, ”Car License Plate
Detection Based on MSER,” In Proc. International Conference on
Consumer Electronics, Communications and Networks, pp. 3973-
3976, 2011.
[13] H. W. Lim and Y. H. Tay, “Detection of License Plate Characters
in Natural Scene with MSER and SIFT Unigram Classifier,” In
Proc. IEEE Conference on Sustainable Utilization and
Development in Engineering and Technology, pp. 95-98, 2010.
[14] H. Anoual, S. Fkihi, A. Jilbab, and D. Aboutajdine, “Vehicle
License Plate Detection in Images,” International Conference on
Multimedia Computing and Systems, pp. 1-5, 2011.
[15] X., Zhang, and S. Zhang, “A Robust License Plate Detection
Algorithm Based on Multi-Features,” International Conference
on Computer and Automation Engineering, vol. 5, pp. 598-602,
2010.
[16] D., Zheng, Y. Zhao and J. Wang, “An Efficient Method of License
Plate Location,” Pattern Recognition Letters, vol. 26 no.15, pp.
2431-2438, 2005.
[17] J. Xu, S. Li, and M. Yu, "Car License Plate Extraction Using Color
and Edge Information," Machine Learning and Cybernetics, 2004,
vol. 6, pp. 3904-3907.
[18] S. Z. Wang and H. M. Lee, "Detection and Recognition of License
Plate Characters With Different Appearances," Proc. Conf. Intell.
Transp. Syst., vol. 2, pp.979 -984, 2003.
[19] L. Carrera, M. Mora, J. Gonzalez and F. Aravena, “License Plate
Detection Using Neural Networks,” In Proceedings of IWANN
vol.2, pp. 1248-1255, 2009.
[20] M. I. Chacon and A. Zimmerman, "License Plate Location Based
on a Dynamic PCNN Scheme," Proc. Int. Joint Conf. Neural
Netw., vol. 2, pp.1195 -1200, 2003.
[21] S.-L. Chang, L.-S. Chen, Y.-C. Chung, and S.-W. Chen, "Automatic
License Plate Recognition," IEEE Trans. Intell. Transp. Syst., vol.
5, no. 1, pp. 42 -53, 2004.
[22] S. K. Kim , D. W. Kim and H. J. Kim "A Recognition of Vehicle
License Plate Using a Genetic Algorithm Based segmentation,"
Proc. Int. Conf. Image Process., vol. 1, pp. 661-664, 1996.
[23] J. Xiong, S. Du, D. Gao and Q. Shen, "Locating Car License Plate
Under Various Illumination Conditions Using Genetic
Algorithm," Proc. ICSP, vol.3, pp. 2502 -2505, 2004.
[24] Z. Ji-yin, Z. Rui-rui, L. Min and L. Yinin, ” License Plate
Recognition Based on Genetic Algorithm,” International
Conference on Computer Science and Software Engineering, vol.
1, pp. 965-968, Dec. 2008.
[25] V. P. de Araújo, R. D. Maia, M. F. S. V. D'Angelo and G. N. R.
D'Angelo, ”Automatic Plate Detection Using Genetic Algorithm,”
Proceedings of the 6th WSEAS International Conference on
Signal, Speech and Image Processing, pp. 43-48, Sep. 2006.
[26] K. I. Kim, K. Jung, J. H. Kim, S.-W. Lee and A. Verri, ”Color
Texture-Based Object Detection: An Application to License Plate
Localization,” Lecture Notes on Computer Science, Springer-
Verlag, vol. 2388, pp. 293-309, 2008.
[27] J. Cano and J. C. Perez-Cortes, ”Vehicle License Plate
Segmentation in Natural Images,” Lecture Notes on Computer
Science, Springer-Verlag, vol. 2652, pp.142-149, 2003.
[28] M. Sezgin and B. Sankur, “Survey Over Image Thresholding
Techniques and Quantitative Performance Evaluation,” Journal of
Electronic Imaging, vol. 13, pp. 146–165, 2004.
[29] N. Otsu, “A Threshold Selection Method From Gray Level
Histograms,” IEEE Trans. Syst., Man, Cybern., vol. 9, no. 1, pp.
62–66, Jan.1979.
[30] S. E. Umbaugh, Computer Vision and Image Processing, Prentice
Hall, New Jersey, 1998, pp. 133-138.
[31] J. E. Baker, “Reducing Bias and Inefficiency in the Selection
Algorithm,” International Conference on Genetic Algorithms and
their application, pp. 14-21, 1987.
[32] Z. H. Ahmed, “Genetic Algorithm for the Traveling Salesman
Problem Using Sequential Constructive Crossover,” International
Journal of Biometrics & Bioinformatics, vol. 3, pp. 96-105, 2010.
[33] W. Lenders, and C. Baier, “Genetic Algorithms for Variable
Ordering Problem of Binary Decision Diagrams,” Lecture Notes
in Computer Science, Springer-Verlag, New York, vol. 3469, pp.
1-20, 2005.
[34] G. E. Liepins and M. R. Hilliard, ”Genetic Algorithms:
Foundations and Applications,” Annals of Operations Research,
21 (1-4) ,pp. 31-58, 1989.
[35] Medialab Web site, (2012, DEC, 9). [Car Images]. Available:
http://www.medialab.ntua .gr/research/LPRdatabase.html.
G. Abo Samra received the B.S., M. S.,
and Ph.D degrees from the faculty of
Engineering in Cairo University, Egypt in
1983, 1988, and 1992, respectively, in
electronic and communication
engineering. He is currently an associate
professor at the faculty of Computing and
Information Technology, King Abdulaziz
University-Saudi Arabia. His research
interests include image processing, data
hiding, OCR, intelligent transportation systems, pattern
recognition, soft computing and computer vision. His research
has been supported by the Deanship of Scientific Research
(DSR), King Abdulaziz University and King Abdulaziz City
for Sciences and Technology.
F. Khalefah is a lecturer in the
Faculty of Computing and Information
Technology, King Abdulaziz
University-Saudi Arabia. His research
interests include image processing and
pattern recognition. His research has
been supported by the Deanship of
Scientific Research (DSR), King
Abdulaziz University.

Localization of license plate number using dynamic image processing techniques and genetic algorithms

More Related Content

What's hot (15)

Similar to Localization of license plate number using dynamic image processing techniques and genetic algorithms (20)

More from Chittipolu Praveen (7)

Recently uploaded (20)

Localization of license plate number using dynamic image processing techniques and genetic algorithms