Real Time Image Processing

A-PDF Merger DEMO : Purchase from www.A-PDF.com to remove the watermark

REAL TIME IMAGE PROCESSING
A SEMINAR REPORT

Submitted by

SONIYA KUMARI

in partial fulfillment for the award of the degree

of

BACHELOR OF TECHNOLGY
In

COMPUTER SCIENCES & ENGINEERING

SCHOOL OF ENGINEERING

COCHIN UNIVERSITY OF SCIENCE & TECHNOLOGY

KOCHI- 682022

MARCH 2010

Division of Computer Engineering
School of Engineering
Cochin University of Science & Technology
Kochi-682022
_________________________________________________________

CERTIFICATE

Certified that this is a bonafied record of the seminar titled


Done By
Soniya Kumari

of VII semester Computer Science & Engineering in the year 2010 in partial

fulfillment of the requirements for the award of Degree of Bachelor of Technology

in Computer Science & Engineering of Cochin University of Science & Technology

Dr.David Peter S Mrs Sheena S
Head of the Division Seminar Guide

ACKNOWLEDGEMENT

I am greatly indebted to Dr. David Peter, Head of Department, Division of Computer
Science, CUSAT for permitting me to undertake this work.

I express my heartfelt gratitude to my respected Seminar guide Mrs Sheena S for her
kind and inspiring advise which helped me to understand the subject and its semantic
significance.

I am extremely thankful to our seminar coordinator Mr. Sudheep Elayidom M for his
valuable suggestions for presentation issue and regarding the selection of topic.

I am also very thankful to my colleagues who helped and co-operated with me in conducting
the seminar by their active participation.

ABSTRACT


The multidisciplinary ﬁeld of real-time image processing has experienced a tremendous
growth over the past decade. The purpose of real time image processing is to improve
the video quality by eliminating the noise inside the sequence. It is intended to provide
guidelines and help bridge the gap between the theory and the practice of image and
video processing by providing a broad overview of proven algorithmic, hardware,
software tools and strategies.

RTIP involves many aspects of hardware and software in order to achieve high
resolution input,low latency capture ,high performance processing,efficient display. Mac
OS X offers all of the necessary features for the development of high performance RTIP
applications, although careful choice of peripherals and software techniques are required.
Regarding the potential of the parallel platform for image processing, in the near future
we will focus our attention on the improvement of the scheduling component, by using
processor units with different processing capacities and also other service policy for the
queue of jobs. Algorithm measuring basic queue parameters such as period of occurrence
between queues, the length and slope of occurrence have been discussed.

LIST OF FIGURES

FIGURE NAME PAGE NO

Fig 2.1 System Design 5

Fig 3.1 Test Image For Line Detection 13

Fig 5.1 Image Analysis System Structure 21

Fig 5.2 Queue Detection 27

Fig 5.3 Edge Detection 27

LIST OF SYMBOLS

RTIP – Real Time Image Processiung

ILP-Instruction Level Parallelism

DLP – Data Level Parallelism

TMJ – Traffic Movement at Junction

LIST OF TABLES

Table 2.1 Broadcast video standards 5

TABLE OF CONTENTS

CHAPTER NO TITLE PAGENO

ABSTRACT i
LIST OF FIGURES ii
LIST OF SYMBOLS iii
LIST OF TABLE iv
1. INTRODUCTION 1
1.1 PURPOSE 1
1.2 SCOPE 1
1.3 OVERALL DESCRIPTION 1
1.4 OBJECTIVE 2

2. HARDWARE PLATFORM TO RTIP 4
2.1 INTRODUCTION 4
2.2 REAL TIME IMAGE PROCESSING 4
2.3 SAMPLING RESOLUTION 5
2.4 LOW LATENCY VIDEO INPUT 6
2.5 LOW LATENCY OPERATING SYSTEM 6
SCHEDULING
2.6 HIGH PROCESSING PERFORMANCE 7
2.7 VIDEO CAPTURE HARDWARE 7
2.8 MAC OS X 8
3 REAL TIME IMAGE PROCESSING ON 9
DISTRIBUTED COMPUTER SYSTEM
3.1 INTRODUCTION 9
3.2 DEFINITIONS 9
3.2.1 REAL TIME IN PERPETUAL SENCE 9
3.2.2 REAL TIME IN SOFT SENSE 10

3.2.3 REAL TIME IN SIGNAL PROCESSING
SENSE 10
3.3 SOFT OPERATION INVOLVED IN RTIP 10
3.3.1 LEVELS OF OPERATIONS 10
3.4 PERFORMANCE
3.4.1 PARALLELISM 12
3.4.2 EXPERIMENTAL RESULT 13
3.4.3DISCUSSION 14
4. ALGORITHM SIMPLIFICATION STRATEGIES 15
4.1 INTRODUCTION 15
4.2 CORE SIMPLIFICATION STRATEGIES 15
4.2.1 REDUCTION IN NUMBER OF
OPERATIONS 16
4.2.2 REDUCTION IN AMOUNT OF DATA 17
4.2.3 SIMPLIFIED ALGORITHM 18

5. TRAFFIC QUEUE DETECTION ALGORITHM 20
5.1 INTRODUCTION 20
5.2 IMAGE PROCESSING APPLIED
TO TRAFFIC 20
5.3 QUEUE DETECTION ALGORITHM 23
6 CONCLUSION 28

REFRENCES 29


CHAPTER 1

INTRODUCTION

1.1 PURPOSE
The multidisciplinary ﬁeld of real-time image processing has experienced a tremendous
growth over the past decade. The purpose of real time image processing is to improve
the video quality by eliminating the noise inside the sequence. It present a generic
methodology for rapid prototyping of a real time algorithm and shows how this compact
hardware plateform is flexible enough to accurately support such complex applications
having real time requirements.It is intended to provide guidelines and help bridge the
gap between the theory and the practice of image and video processing by providing a
broad overview of proven algorithmic, hardware, software tools and strategies.

1.2 SCOPE

The performance requirements of image processing applications have continuously
increased the computing power of implementation platforms, especially when they are
executed under real time constraints. The real time applications may consist of different
image standards, or different algorithms used at different stages of the processing chain.
The computing paradigm using reconfigurable architectures promises an intermediate
trade-off between flexibility and performance.

1.3OVERALL DESCRIPTION

Real-time image processing differs from "ordinary" image processing in that the logical
correctness of the system requires not only correct but also timely outputs; that is
semantic validity entails not only functional correctness, but also deadline satisfaction.
Because of its nature, there are both supports for and obstacles to real-time image
processing. On the positive side, many imaging applications are well-suited for
DIVISION OF COMUTER SCIENCE
Page 1


parallelization and hence faster, parallel architectures. Furthermore, many imaging
applications can be constructed without using language construct that destroy determinism.
Moreover, special real-time imaging architectures are available or can theoretically be
constructed. A real-time system is one that must satisfy explicit bounded response time
constraints to avoid failure. Equivalently, a real-time system is one whose logical
correctness is based both on the correctness of the outputs and their timeliness. Notice
that response times of, for example, microseconds are not needed to characterize a
real-time system - it simply must have response times that are constrained and thus
predictable. In fact, the misconception that real-time systems must be "fast" is because in
most instances, the deadlines are on the order of microseconds. But the timeliness
constraints or deadlines are generally reflection of the underlying physical process being
controlled.
Some feel that real-time performance is easy to achieve. As we hope to show, that is
not always so, largely because most hardware and programming languages are not
suitable for real-time demands.
Problem concerning any practical image processing application to road traffic is the fact
that real world images are to be processed in real time. Mac OS X provides a capable
platform for realtime image processing (RTIP). It provides an overview of available
video capture hardware and presents development strategies to achieve high performance,
low latency image processing.

1.4 OBJECTIVES

A platform for real-time image processing has the aim to provide the following:
1. high resolution, high frame rate video input
2. low latency video input
3. low latency operating system scheduling
4. high processing performance
5. Many promising applications
a. video conferencing

Page 2


b. augmented reality
c. context aware computing
d. video-based interfaces for human-computer interaction

Page 3


CHAPTER 2

HARDWARE PLATFORM TO RTIP

2.1 INTRODUCTION

Real-time image processing (RTIP) promises to be at the heart of many developments in
computer technology: context aware computers, mobile robots, augmented reality and the
video-based interfaces for human computer interaction. These applications have significant
demands not only in terms of processing power: they must achieve real-time, low latency
response to their visual input.
Whilst most modern operating systems provide a wealth of multimedia features, these
features are usually oriented towards the playback or recording of media rather than
processing in real time. Different media representations and handling mechanisms are
often necessary for real-time processing. The operating system itself must also be capable
of efficient, low-latency response and processing. Mac OS X provides a robust operating
system with excellent latency performance and a rich multimedia framework that can be
applied, with some care, to RTIP applications. Suitable live image sources are also
required for RTIP. Once again, general purpose or recording/playback oriented devices
are not necessary suitable for this application domain.
As a relatively young platform, Mac OS X does have limited driver support for video
input hardware.
2.2 Real-Time Image Processing
A platform for real-time image processing must provide the following
• high resolution, high frame rate video input
• low latency video input
• low latency operating system scheduling
• high processing performance

Page 4


Fig 2.1:System Design

2.3 Sampling Resolution:
In the most general terms, image processing attempts to extract information from the
outside world through its visual appearance. Therefore adequate information must be
provided to the processing algorithm by the video input hardware. Precise requirements
will, of course, depend on the algorithm and application but usually both spatial and
temporal resolution are important. Broadcast video provides a practical reference point as
most cameras provide images in formats derived from broadcast standards regardless of
their computer interface(analog, USB etc).

STANDARD SPATIAL DIMENSION FRAME RATE
NTSC 720X480 30fps
PAL 768X576 25 fps

Table 2.1: Broadcast video standards

Page 5


2.4 Low latency video input:
All video input systems have intrinsic sources of latency in their hardware and
transmission schemes. Indeed, the relatively sparse temporal sampling (frame rate) typical
for video can itself be thought of as a source of latency equal to the frame duration.
Higher frame rate therefore allow for lower latency and more responsive RTIP systems.
Additional latency occurs in the transmission of video from the camera to the computer
interface. The sequential nature of almost all video frame transmission also imposes
latency equal to the frame transmission time (which is usually close to the frame
duration in order to minimise bandwidth requirements). This applies to digital
transmission schemes over USB or Firewire just as it does to analogue transmission.
Applications which use video as part of a feedback loop (through a human user or
electromechanical means) often have tight demands on the total latency in this feedback
loop.
For human interaction, common candidates for upper bounds on acceptable latency are:
• threshold of perceived causality (~50ms)
• threshold of perceived synchronicity (e.g. music ~10ms)
Given that the frame duration of a broadcast standard based video device will be at least
33ms (for NSTC 30fps) and we expect to have at least two frames of latency in the
video input device (camera and transmission system).

2.5 Low latency operating system scheduling:
Once the video signal arrives at the computer it will be processed and passed between a
number of software components. These components will depend on the type of video
capture hardware in use, but generally and in the minimum case there will be a driver
component and an application that performs the image processing. The driver is
responsible for receiving the transmission and presenting the video frame as a buffer of
pixels and is of course provided by the operating system vendor or hardware vendor.
This pixel buffer is then processed by the application which would then typically

Page 6


produce some output for the user or provide information to other application software
running on the system.
The ability of the operating system to respond to incoming video data and to schedule
each of these software components to run as soon as its data are available has a crucial
impact on system latency. If no input data is to be lost, buffering (and hence additional
latency) must be introduced to cover both lag and any variation in when data is
available and when it is passed to the next component. This lag and variation is related
to system interrupt latency and scheduling latency. Fortunately Mac OS X has excellent
low latency performance even under heavy system load.

2.6 High Processing performance:
Image processing algorithms are very bandwidth and processor speed intensive. High
bandwidth memory architecture, effective caching and high performance processors are
necessary for an RTIP platform. Altivec is an important factor in achieving good
performance, as image processing algorithms are usually highly parallel and therefore
well suited to SIMD optimisation. Recent changes in Macintosh hardware architecture are
also very promising for RTIP, in particular the emphasis on memory bandwidth in the
Power Mac G5 range.

2.7 Video Capture Hardware

Video capture hardware performs the vital role of handling the reception of the video
signal into the computer and presenting to the processor in a suitable form. Some
hardware integrates both camera and digitization functions together, such as t he DV
video cameras, and USB Webcams. Other systems perform only digitisation of an analog
video signal provided by an external camera. These devices are then connected to a
suitable system bus (PCI, Firewire or sometimes USB). Suitable devices for RTIP must
provide high resolution, high frame rate video at low latency. Making the video signal
available in an uncompressed format to the image processing software with low CPU
overhead is also important. These requirements unfortunately exclude many common
video input devices which provide only low quality input or introduce latency

Page 7


Through their compression or transmission schemes. The common classes of devices with
drivers available for Mac OS X can be considered for their suitability.

2.8 MAC OS X
Mac OS X offers all of the necessary features for the development of high performance
RTIP applications, although careful choice of peripherals and software techniques are
required.Using the recommended hardware and techniques outlined in this paper, low
latency high performance video capture and display is possible using the QuickTime
architecture and OpenGL. A general framework has been presented for the development
of RTIP applications on Mac OS X.

Page 8


CHAPTER 03

REAL TIME IMAGE PROCESSING ON DISTRIBUTED
COMPUTER SYSTEM

3.1 INTRODUCTION
Considering the need for real-time image processing and how this need can be met by
exploiting the inherent parallelism in an algorithm,it becomes important to discuss what
exactly is meant by the term “real-time,” an elusive term that is often used to describe
a wide variety of image processing systems and algorithms. From the literature, it can be
derived that there are three main interpretations of the concept of “real-time”, namely
real-time in the perceptual sense, real-time in the software engineering sense, and real-
time in the signal processing sense.

3.2 DEFINITIONS OF RTIP IN DIFFERENT SENSE

3.2.1 Real-time in the perceptual sense
It is used mainly to describe the interaction between a human and a computer device for
a near instantaneous response of the device to an input by a human user. For instance,
Bovik defines the concept of “real-time”in the context of video processing, describing
that “the result of processing appears effectively „instantaneously‟ (usually in a perceptual
sense) once the input becomes available”. Note that “real-time” imposes a maximum
tolerable delay based on human perception of delay, which is essentially some sort of
application-dependent bounded response time.

3.2.2 Real-time in the perceptual sense
It is used mainly to describe the interaction between a human and a computer device for
a near instantaneous response of the device to an input by a human user. For instance,
Bovik defines the concept of “real-time” in the context of video processing, describing
that “the result of processing appears effectively „instantaneously‟ (usually in a perceptual
Page 9


sense) once the input becomes available”. Note that “real-time” imposes a maximum
tolerable delay based on human perception of delay, which is essentially some sort of
application dependent bounded response time constraints to apply failures.

3.2.3 Real-time in the signal processing sense
It is based on the idea of completing processing in the time available between successive
input samples. An important item of note here is that one way to gauge the “real-time”
status of an algorithm is to determine some measure of the amount of time it takes for
the algorithm to complete all requisite transferring and processing of image data, and
then making sure that it is less than the allotted time for processing.

3.3Software operations involved in RTIP

3.3.1 Levels of image processing operations
The digital primary processing mainly consists of three stages: noise rejection, binary
representation, and edge extraction. Due to the fact that the noise can introduce errors in
other stages (like contour detection and feature extraction), the image noise rejection
must be the first stage in any digital image processing application. For these algorithms
it is recommend local operators which act in symmetrical neighborhoods of the
considered pixels. They have the advantage of simplicity and they can be implemented
easily implemented on dedicated hardware structures. This approach changes when
considering software processing. Digital images are essentially multidimensional signals
and are thus quite data intensive, requiring a significant amount of computation and
memory resources for their processing. The key to cope with this issue is the concept of
parallel processing who deals with computations on large data sets. In fact, much of
what goes into implementing an efficient image/video processing system centers on how
well the implementation, both hardware and software, exploits different forms of
parallelism in an algorithm, which can be data level parallelism - DLP or/and instruction
level parallelism – ILP . DLP manifests itself in the application of the same operation

Page 10


on different sets of data, while ILPmanifests itself in scheduling the simultaneous
execution of multiple independent operations in a pipeline fashion.
Image processing operations have been classified into three main levels,:-

Low level:- Low-level operators take an image as their input and produce an image as
their output, while intermediate-level operators take an image as their input and generate
image attributes as their output, and finally high-level operators take image attributes as
their inputs and interpret the attributes, usually producing some kind of knowledge-based
control at their output. One can hope that with an adequate task scheduling and a well
designed cluster of processors one can perform in real time low-level operations by
software parallelization. Low-level operations transform image data to image data. This
means that such operators deal directly with image matrix data at the pixel level.
Examples of such operations include color transformations, gamma correction, linear or
nonlinear filtering, noise reduction, sharpness enhancement, frequency domain
transformations, etc. The ultimate goal of such operations is to either enhance image data,
possibly to emphasize certain key features, preparing them for viewing by humans, or
extract features for processing at the intermediate-level. These operations can be further
classified into point, neighborhood (local), and global operations . Point operations are the
simplest of the low-level operations since a given input pixel is transformed into an
output pixel, where the transformation does not depend on any of the pixels surrounding
the input pixel. Such operations include arithmetic operations, logical operations, table
lookups, threshold operations, etc.

Intermediate-level:- Intermediate level operations transform image data to a slightly more
abstract form of information by extracting certain attributes or features of interest from
an image. This means that such operations also deal with the image at the pixel level,
but a key difference is that the transformations involved cause a reduction in the amount
of data from input to output. The goal by carrying out these operations (which include
segmenting an image into regions/objects of interest, extracting edges, lines, contours, or
other image attributes of interest such as statistical features) is to reduce the amount of

Page 11


data to form a set of features suitable for further high-level processing. Some
intermediate-level operations are also data intensive with a regular processing structure,
thus making them suitable candidates for exploiting DLP.

High level:- High-level operations interpret the abstract data from the intermediate-level,
performing high level knowledge-based scene analysis on a reduced amount of data.
These types of operations (for example recognition of objects) are usually
characterized by control or branch-intensive operations. Thus, they are less data intensive
and more inherently sequential rather than parallel.

3.4 Performing real time image processing on a distributed platform

3.4.1 Parallel platform model and scheduling principles
Our system model consists of P processor units. Each processor pi has capacity ci > 0,
i = 1,2,…, P. The capacity of a processor is defined as its speed relative to a reference
processor with unit-capacity. We assume for the general case that c1_c2 _… _cP..
A system is called homogeneous when c1=c2…=cP. The platform is conceived as a
distributed system [9]. Each machine is equipped with a single processor. In other words,
we do not consider interconnections of multiprocessors. The main difference with
multiprocessor systems is that in a distributed system, information about the system state
is spread across the different processors. In many cases, migrating a job from one
processor to another is very costly in terms of network bandwidth and service delay and
that the reason that we have considered for the beginning only the case of data
parallelism for a homogenous system. The intention was to test the general case of
image processing with both data and task parallelism, by developing a scheduling policy
with two components .The global scheduling policy decides to which processor an
arriving job must be sent, and when to migrate some jobs. At each processor, the local
scheduling policy decides when the processor serves which of the jobs present in its
queue.

Page 12


Jobs arrive at the system according to one or more interarrival-time processes. These
processes determine the time between the arrivals of two consecutive jobs. The arrival
time of job j is denoted by Aj. Once a job j is completed, it leaves
the system at its departure time Dj. The response time Rj of job j is defined as Rj =
Dj – Aj. The service time Sj of job j is its response time on a unit-capacity processor
serving no other jobs; by definition, the response time of a job with service time s on a
processor with capacity c’ is s/c’. We define the job set J(t) at time t as the set of jobs
present in the system at time t:For each job jÎJ(t), we define the remaining work Wr (t)
j at time t as the time it would take to serve the job to completion on a unit-capacity
processor. The service rate ) (t rjs of job j at time t (Aj_ t<Dj).

3.4.2Experimental results

A test image was processed first on a single processing unit, then on a test network
configured as a cluster with 2, 4 or 8 nodes, each node being a processor unit working
at 1 GHz with 128 MByte RAM. For each instruction utilized in the directional filtering
algorithm two measurements were executed, for images having 2002 or 10002 elements.

Fig 3.1: Test image for line detection

Page 13


3.4.3DISSCUSSION
The experiments show how to use parallelizable patterns, obtained for typical low level
image processing operations. In our study case the performance model is highly accurate
for parallel processing using convolution functions. Given the
results we are confident in that the proposed software architecture forms a powerful basis
for automatic parallelization and optimization of a wide range of image processing
applications.

Page 14


CHAPTER 04

ALGORITHM SIMPLIFICATION STRATEGIES

4.1 INTRODUCTION

An algorithm is simply a set of prescribed rules or procedures that are used to solve a
given problem [103, 130]. Although there may exist different possible algorithms for
solving an image/video processing problem, when transitioning to a real-time
implementation, having efficient algorithms takes higher precedence. Efficiency implies
low computational complexity as well as low memory and power requirements. Due to
the vast amounts of data associated with digital images and video, developing algorithms
that can deal with such amounts of data in a computational, memory, and power
efficient manner is a challenging task, especially when they are meant for real-time
deployment on resource constrained embedded platforms.

Thus, the very first step in transitioning an algorithm from a research environment to a
real-time environment involves applying simplification strategies to the algorithm. It is
more effective to perform these simplifications while still working in the research
development environment, which possesses a higher design flexibility over the
implementation environment.

4.2 CORE SIMPLIFICATION CONCEPTS

Examining the literature on real-time image and video processing reveals three major
concepts addressing algorithmic simplifications. These strategies include the following:

• reduction in number of operations;
• reduction in amount of data to be processed;
• utilization of simple or simplified algorithms.

Page 15


Reductions in operations and in amount of data to be processed are the most common
simpliﬁcation strategies for transitioning an algorithm to a real-time implementation.
While there are many different manifestations of the reduction strategies, the discussion
here provides a general classiﬁcation of the strategies that are scattered throughout the
literature. Each of these concepts will be expanded upon in the following subsections.

4.2.1 REDUCTION IN NUMBER OF OPERATIONS

Due to the enormous amount of data involved in image/video processing, every operation
counts, especially the time-consuming operations at the lower levels of the processing
hierarchy. Thus, reduction in the number of operations plays a major role in achieving
real-time performance

4.2.1.1 PURE OPERTAION REDUCTION

The strategy of pure operation reduction involves applying a transformation to reduce the
number of operations, which does not change the numerical outcome. If the numerical
outcome is changed, then the approach is referred to as either an approximation or a
suboptimal/alternative solution as discussed in the next subsection. Any operation that has
inherent symmetries or redundancies in its constituent computations is a candidate for
applying this strategy. Application of this strategy manifests itself in uncovering hidden
symmetries or redundancies within the computations, which can often be discovered by
expanding the computations by hand and carefully noting any mathematical identities or
properties .

4.2.1.2 OPERATION REDUCTION THROUGH APPROXIMATION

The strategy of approximations is similar to the strategy of reduction in computations in
that approximations involve applying transformations to reduce the number of operations,
but it differs from pure computational reduction due to the presence of approximation

Page 16


errors. The main objective of this strategy is to minimize errors as much as possible,
thus obtaining suboptimal results within an acceptable interval of time.

Consequently, in this strategy, one seeks a trade-off between the accuracy of the
outcome and the speed at which it is obtained. In fact, a real-time performance is often
obtained by trading off accuracy of processing with speed of processing, where an
“optimal” working point is found through experimentation. Examples of this strategy
include the use of lookup tables for computationally complex calculations, the use of a
suboptimal search technique over a time-consuming, exhaustive search technique, the
use of sums of absolute values over time-consuming squaring operations, and many
others.

4.2.1.3 ALTERNATE METHODS

Choosing an alternative method is a strategy that involves searching for a new algorithm
in order to attain faster processing while at the same time maintaining a required level
of accuracy. This strategy is similar to the strategy of approximations, except that it is
used primarily when the algorithm is too computationally complex for any
approximations or reductions to yield an acceptable real-time performance. Basically, this
strategy involves abandoning the computationally complex algorithm in favor of one that
is less computationally complex. A simpler algorithm is often developed by careful
consideration of data to be processed and properties of the objects being sought out.

4.2.2 REDUCTION IN AMOUNT OF DATA

In addition to the reduction of operations, the reduction of data to be processed also
plays a major role toward having real-time performance. This is simply because such
systems require the processing of large amounts of data in relatively short intervals of
time, and thus any reduction in the amount of data can lead to a reduction in the
processing time. This strategy involves applying a transformation to the underlying image

Page 17


data for the purpose of deriving a compact representation, which in turn speeds up
subsequent stages of processing. Reduction of data takes many forms in real-time
image/video processing systems including spatial or temporal down-sampling, spatial block
partitioning, region of interest or selective processing, formulating the algorithm in a
multi resolution or processing framework, appropriate feature extraction, etc. In all these
cases, a certain subset of pixels from an image frame is processed.

4.2.3 SIMPLIFIED ALGORITHM

In general, the fundamental idea behind real-time image/video processing systems is the
utilization of simple or simplified algorithms. A rule of thumb when transitioning to a
real-time implementation is to keep things as simple as possible, that is to say, look for
simple solutions using simple operations and computationally simple algorithms as
opposed to complex, computationally intensive algorithms, which may be optimal from a
mathematical viewpoint, but are not practical from a real-time point of view. With
embedded devices now being outfitted with vision capabilities, such as camera equipped
cell phones and digital still/video cameras, the deployment of those algorithms which are
not only computationally efficient but also memory efficient is expected to grow. Often,
algorithms are carefully analyzed in terms of their number of operations and
computational complexity, but equally important are their storage requirements. In
essence, simple algorithms provide a means of meeting real-time performance goals by
lowering computational burdens, memory requirements, and indirectly, power requirements
as well.

The increase in the practical applications of face detection and recognition has increased
the interest in their real-time implementations. Such implementations are possible via the
use of simple algorithms. For instance, in [92], a simple algorithm for face detection
based on the use of skin color features was discussed. In order to make the detection
algorithm robust to regions in the background with skin-like colors, a simplified method
was developed. First, motion detection via a simple frame differencing operation was
used to distinguish the human face from the background. Then, to reduce the noise in

Page 18


the difference image, a simple morphological opening operation was used. Finally, a
simple labeling operation was used to determine the region where the face was located.

Industrial inspection systems are known for their strict hard real-time deadlines, forcing
developers to devise algorithms with simple operations. For example, a simple algorithm
for the extraction of paper watermarks as part of a defect inspection system was
introduced. Due to the real-time constraints involved, a simple morphological white top-
hat operation was used to extract the watermarks. This simple operation was found to be
not only fast, but also robust to disturbances such as uneven illumination and low-
contrast watermarks. In this case, the morphological operations provided a means of
obtaining real-time performance. Similarly simple defect location and feature extraction
algorithms were employed to meet the hard real-time constraints in an industrial
inspection system. In order to speed up the location detection process, the image was
first binarized to find a rough estimate of the location of the defect. This rough location
was then propagated to the full-resolution image, and a region around this location
was selected for feature extraction. A simple gray-level difference method was used to
extract texture features from the narrowed down defect region. The use of relatively
simple operations enabled the real-time detection of defects and extraction of features for
the subsequent classification.

Page 19


CHAPTER 05

TRAFFIC QUEUE DETECTION ALGORITHM

5.1 INTRODUCTION

This chapter primarily aims at the new technique of video image processing used to
solve problems associated with the real-time road traffic control systems. There is a
growing demand for road traffic data of all kinds. Increasing congestion problems and
problems associated with existing detectors spawned an interest in such new vehicle
detection technologies. But the systems have difficulties with congestion, shadows and
lighting transitions.
Problem concerning any practical image processing application to road traffic is the fact
that real world images are to be processed in real time. Various algorithms, mainly
based on back ground techniques, have been developed for this purposes since back
ground based algorithms are very sensitive to ambient lighting conditions, they have not
yielded the expected results. So a real-time image tracking approach using edged
detection techniques was developed for detecting vehicles under these trouble-posing
conditions.

5.2 Image processing applied to traffic

Need for processing of traffic data: -
Traffic surveillance and control, traffic management, road safety and development of
transport policy.
Traffic parameters measurable: -
Traffic volumes, Speed, Headways, Inter-vehicle gaps, Vehicle classification, Origin and
destination of traffic, Junction turning.

Page 20


Fig 5.1 :image analysis system structure

Stages of image analysis: -
Image sensors used :
Improved vidicon cameras: automatic gain control, low SNR
ADC Conversion: - Analog video signal received from video camera is converted to
digital/binary form for processing
Pre-processing:
High SNR of the camera output reduces the quantity of processing enormous data
flow.

Page 21


To cope with this, two methods are proposed:
1. Analyze data in real time - uneconomical
2. Stores all data and analyses off-line at low speed.
Pipeline Preprocessing does this job

Stages in Pipeline Preprocessing :
(1) Spatial Averaging – contiguous pixels are averaged (convolution)
(2) Subtraction of background scene from incoming picture.
(3) Threshold – Large diff.s are true „1‟, small diff.s are false „0‟
(4) Data Compression – reduces resulting data.
(5) Block buffering – collects data into blocks.
(6) Tape Interface – blocks are loaded onto a digital cassette recorder

Two jobs to be done:
Green light on: - determine no. of vehicles moving along particular lanes and their
Classification by shape and size.
Red light on: - determine the backup length along with the possibility to track its
dynamics and classify vehicles in backup.

Methods of vehicle detection :
• Background frame differencing: -grey-value intensity reference image.

• Inter-frame differencing: -incoming frame itself becomes the background for the
following frame.
• Segmentation and classification: -Sub division of an image into its constituent parts
depending on the context.

Page 22


5.3 Queue Detection Algorithm
• approach described here is a spatial-domain technique to detect queue – implemented
in real-time using low-cost system.

• For this purpose two different algorithms have been used,
1.Motion detection operation ,
2.Vehicle detection operation.

• Motion detection is first – as in this case vehicle detection mostly gives positive
result, while in reality, there may not be any queue at all. Applying this scheme further
reduces computation time.

5.2.1 Motion detection operation: -
a) differencing two consecutive frames.
b) histogram of the key region parts of the frames is analyzed by comparing with the
threshold value.
c) key region should be at least 3-pixel-wide profile of the image along the road.
d) a median filtering operation is firstly applied to the key region (profile) of each
frame and one-pixel-wide profile is extracted.
e) difference of two profiles is compared to detect for motion.
f) when there is motion, the differences of the profiles are larger than the case when
there is no motion. The motion can be detected by selecting a threshold value.

Motion Detection Algorithm
The profile along the road is divided into a number of smaller profiles (sub-profiles)
The sizes of the sub-profiles are reduced by the distance from the front of the
camera.
Transformation causes the equal physical distances to unequal distances according to
the camera parameters.

Page 23


knowing coordinates of any 6 reference points of the real-world condition and the
coordinates of their corresponding images, the camera parameters (a11, a12…a33) are
calculated.
the operations are simplified for flat plane traffic scene - (Zo=0).

above equation is used to reduce the sizes of the sub-profiles - each sub profile
represents an equal physical distance.
no. of sub profiles depend on the resolution and accuracy required. The length of
subprofile
should be about length of the vehicle - both the detection algorithms work
accurately then.

Vehicle detection algorithm : -

o Following the application of the motion detection operation, a vehicle detection
operation is applied on the profile of the unprocessed image.
o To implement the algorithm in real time, two strategies are often applied: key region
processing and simple algorithms.
o Most of the vehicle detection algorithms developed so far are based on a
background differencing technique, which is sensitive to variations of ambient
lighting.
o The method used here is based on applying edge detector operators to a profile of the
image – Edges are less sensitive to the variation of ambient lighting and are used in
full frame applications (detection).
o Edge detectors consisting of separable medium filtering and morphological operators,
SMED (Separable Morphological Edge Detector) are applied to the key regions of the
image. ( The SMED approach is applied (f) to each sub-profile of the image and the
histogram of each sub-profile is processed by selecting Dynamic left-limit value and a
threshold value to detect vehicles.
o SMED has lower computational requirement while having comparable performance

Page 24


to other morphological operators
o SMED can detect edges at different angles, while other morphological operators are
unable to detect all kinds of edges.

Traffic movements at junctions (TMJ):

Measuring traffic movements of vehicles at junctions such as number of vehicles
turning in a different direction (left, right, straight) is very important for the analysis of
cross-section traffic conditions and adjusting traffic lights.
Previous research work for the TMJ parameter is based on a full-frame approach,
which requires more computing power and, thus, is not suitable for real-time
applications. We use a method based on counting vehicles at the key regions of the
junctions by using the vehicle-detection method.
The first step to measure the TMJ parameters using the key region method is to
cover
the boundary of the junction by a polygon in such a way that all the entry and exit
paths of the junction cross the polygon. However, the polygon should not cover the
pedestrian marked lines. This step is shown in the figure given below.
The second step of the algorithm is to define a minimum numbers of key regions
inside the boundary of the polygon, covering the junction.
These key regions are used for detecting vehicles entering and exiting the junction,
based on first vehicle –in first-vehicle-out logic.
Following the application of the vehicle detection on each profile, a status vector is
created for each window in each frame.
If a vehicle is detected in a window, a “one” is inserted on its corresponding status
vector, otherwise, a “Zero” is inserted.
Now by analyzing the status vector of each window, the TMJ parameters are
calculated for each path of the junction.

Page 25


Results and Discussions: -
The main queue parameters we were interested in identifying were the length of the
queue, the period of occurrence and the slope of the occurrence of the queue behind
the traffic lights.
To implement the algorithm in real-time, it was decided that the vehicle detection
operation should only be used in a sub-profile where we expect the queue will be
extended.

Page 26


Fig 5.2 : QUEUE DETECTION

Fig 5.3:-Edge Detection

Page 27


CHAPTER 06

CONCLUSION

RTIP involves many aspects of hardware and software in order to achieve high
resolution input,low latency capture ,high performance processing,efficient display. Mac
OS X offers all of the necessary features for the development of high performance RTIP
applications, although careful choice of peripherals and software techniques are required.
Regarding the potential of the parallel platform for image processing, in the near future
we will focus our attention on the improvement of the scheduling component, by using
processor units with different processing capacities and also other service policy for the
queue of jobs. Algorithm measuring basic queue parameters such as period of occurrence
between queues, the length and slope of occurrence have been discussed.

Page 28


REFERENCES

[1] http://www.scribd.com/doc/6738837/ Real-Time-Image-and-Video-Processing-From-
Research-to-Reality(First Edition 10 9 8 7 6 5 4 3 2 1 Printed in the United States of
America)

[2] http://www.apple.com/macosx/what-is-macosx

[3] http://www.101seminartopics.com/real-time-image-processing-applied-to-traffic-queue-
detection-algorithm/

[4] http://www.authorstream.com/Presentation/ pl_arun-360868-image-processing-ip-
arunpl-science -technology-ppt-powerpoint

[5] Rourke, A., and Bell, M.G.H.: ‘Queue detection and congestion monitoring using
mage processing’, Traffic Engg. and Control

[6] http://developer.apple.com/quicktime/icefloe/dispatch019.html

[7] A. Bovik, Introduction to Digital Image and Video Processing, in Handbook of Image &
VideoProcessing, A. C. Bovik, Ed., Elsevier AcademicPress, 2005.

[8]http://www.esiee.fr/~research/documents/Index/Actualites/Annexe_%20Distinctions/JR
TIP_CfP_Reconfiguarble%20Architecture_v1_070313.pdf

Page 29

Real Time Image Processing

More Related Content

Viewers also liked (20)

Similar to Real Time Image Processing (20)

Recently uploaded (20)

Real Time Image Processing