Exploiting the Potential of Augmented Reality

-
-
AR is a game changer that is boosting operational efficiency, transforming
customer engagements and creating new business opportunities
Exploiting the Potential of
Augmented Reality

2
Figure 1: AR offers a modified view of reality using computer-generated
sensory inputs
Augmented Reality (AR) is a promising
emerging technology that is capturing
the imagination of many businesses.
This paper aims to present an overview
of AR, its applications and underlying
technologies, and explore how AR can
improve productivity and efficiency and
potentially transform the nature of work
as we know it.
Augmented Reality is a field of computer
science where “a live direct or indirect view of a
physical, real-world environment whose
elements are augmented (or supplemented) by
computer-generated sensory input such as
sound, video, graphics or GPS data. It is related
to a more general concept called mediated
reality, in which a view of reality is modified
(possibly even diminished rather than
augmented) by a computer. As a result, the
technology functions by enhancing one’s
current perception of reality.”1
AR can be viewed as a natural consequence of
digital evolution. It’s an example of how
technology has the power to change the way we
perceive and interact with our environment. The
potential of AR is made possible by the mobile
revolution, which brought technology into the hands of
people in the forms of smartphones, tablets and now
wearable devices. Devices such as cameras are embedded
into these devices with programmable interfaces. With
touch as a primary input method, mobile devices are very
cogent in the experience they provide. In the
un-augmented real world, it is not possible to interact
physically with every object or parts of objects. This could
be due to obstacles in terms of size, location, safety or
physical characteristics, such as fragility.
Consider a car showroom. There are several models of
cars for display and the customer can be overwhelmed
when a salesperson explains the details of each model.
AR lets the customer learn about the various models on
their own terms, using an immersive experience that can
provide the information they need to make the buying
decision.
AR technology involves digitally recognizing an object
and augmenting it, for example by enhancing it with
relevant content including imagery, videos and
documents. It allows users to engage with real-world
objects by taking advantage of sophisticated visual
assimilation techniques. It removes obstacles and
delivers a deep level of interaction with the object and the
environment. Depending on the type of object, the
augmentation can create an immersive experience
leading to an improved cognition of the object. With
mobile devices, the experience is enhanced with various
input techniques such as touch and voice.
Consider again the application of AR in the car
showroom. When a customer enters a showroom, they
are handed a mobile device such as a tablet with
predefined software applications deployed. The customer
launches an app that internally launches the device
camera. When the customer approaches a particular car,
the app intelligently identifies the car and augments the
live frame on the device with relevant content. This might
include text about the key features of the model, a video
demonstrating the innovative engine technology, or an
animation of the interior.
AR can be applied in countless scenarios—factories,
offices, and commercial and public spaces—with the
objective of delivering rich, experiential information,
insight, and education unmatched by any other
The Case of AR
01
Reality
Augmentation
Content
Enhanced
Perception
1. Augmented Reality, Wikipedia, https://en.wikipedia.org/wiki/Augmented_reality

3
Figure 2: Information Augmentation for machines using AR helps employees in a
manufacturing facility
Business Use Cases
Both large and small businesses are at the
forefront when it comes to adopting and
applying AR. The technology is being used in a
variety of scenarios from employee training to
customer engagement. It has the potential to
improve operational efficiency and open new
avenues to innovate both products and services
that enhance customer engagement and loyalty,
and drive competitiveness. What follows are
four examples of how AR is being applied today
in the business world.
Manufacturing
Optimizing operations by increasing efficiency
is a constant focus in the manufacturing sector.
Many production tasks require quick and easy
access to the right data at the right time to
perform a specific task. For example, consider a
large shop floor with a variety of different
machines. The operation of each of these
machines includes monitoring them while in
use and scheduling regular maintenance. Given
the large amount of disparate information
available on each machine, it is usually not
possible to have instant access to information
on a specific machine’s operational settings or
maintenance and repair records.
Using AR, all of this information is accessible
instantly to the employee. Each machine can be
defined in the AR system and all the relevant
data associated with it can be filed. Using a
mobile device, the employee can learn about
the machine, access relevant data, and take
necessary actions to operate, maintain or repair
Warehouse Management
Large warehouses and distribution centers are
common in many industries, acting as
inventory hubs for manufacturers and retailers,
including e-commerce retailers. Some double
as sales centers accessible by customers. With
millions of SKUs in inventory, it can be
challenging for workers to find accurate
information about the products and manage
the picking and replenishment of inventory.
Warehouses that service customers directly face
the added challenge of accessing detailed
information about the products.
AR can be a very effective solution to help
warehouse workers/customers manage the
products on the shelves so that they can make
accurate selections. For example, the company
can provide workers with an app to use on their
mobile devices that augments the image of a
product with the relevant specifications when
viewed through the live camera frame. This can
increase productivity by decreasing the chance
of workers selecting the wrong product to be
shipped. In warehouses that allow customers
direct access, the app can significantly improve
their shopping experience by helping them find,
technology. AR can be a game changer for the
business world, given the need for companies to
boost productivity, increase the skills of their
employees, and create deeper engagements with
their suppliers and customers.
the machine. They can take advantage of
technologies such as virtual assistants along
with AR that collect and analyze data
continuously, providing insights and actionable
tasks that lead to simpler and more efficient
solutions that boost both capital and labor
productivity.

4
Figure 3: Warehouse workers can manage products on shelves
using AR
Figure 4: Pre-loaded AR on mobile devices enhances training by providing
interactive wizards
have to provide detailed instructions and
information to workers and educate them on
each individual machine and its components.
AR is very useful in such scenarios. Trainers are
given mobile devices preloaded with
AR-enabled training modules, where each
module acts as an interactive wizard to guide
the trainees through each step. It also expedites
the process of learning as the trainees can
repeat the wizard until they are proficient in the
module. The learning process can take place
independent of the trainer.
Training
Hands-on training is an essential part of
learning and development process at many
companies. The more complex the machines or
processes are, the more challenging the training
process and the longer it takes. For example,
consider a product assembly process in a
manufacturing plant that requires workers to
manage a variety of complex machines.
Trainers
Product Marketing
Product marketers are constantly trying to renew
and reinvent the product marketing process to
engage and excite the customer. Storytelling is
integral in the marketing process. Businesses
aim to continuously reorient themselves to adapt
evaluate and select the right products with
minimal involvement from staff. Also, the app
can provide accurate information about the kind
of products customers buy and the factors
involved in the decisions they make. This data
can be analyzed and used, for example, to
manage inventory levels and store popular
products closer to the customer service desk.

to changing market conditions and tell
compelling stories about their products to
existing and potential customers. Successful
storytelling is the starting point for increasing
sales and market share and driving customer
loyalty.
AR helps marketers create a “wow” factor that is
integrated into the storytelling through powerful
visuals and content. The AR solution enhances
the customer experience and can increase the
likelihood of a sale. Consider the car showroom
example described above. A customer is provided
with a mobile device that has a preloaded AR
module to use as they tour the showroom floor.
As they learn about the features and attributes of
the car models, they are also being marketed to
through the imagery and the messaging of the
content.
Figure 5: AR helps marketers provide immersive experiences for
their customers
The opportunity is similar in other industries,
such as retailing. AR solutions can help retail
marketers present their products in innovative
ways. For instance, a retail clothing store that
provides shoppers with AR-equipped mobile
devices might include the option of selecting the
product category the shopper is looking for and
directing them to that department. The visual
enhancements might include information and
visuals about items on sale and the quality of the
products they are shopping for, as well as direct
them to articles of clothing that are
complementary.

6
Figure 7: Object definition in an AR solution
Solutions
02
Solution Model
There are a variety of models included in the
range of AR solutions, from those that are
device-only to others that include a client and
server. Regardless of the type, the typical
solution comprises the following four
categories:
Object Definition
Object Classification
Object Detection and Tracking
Object Augmentation
Object Classification
Once the objects of interest are identified and
defined in the system, they are classified into
specific groups based on either predefined or
user-defined parameters. The goal of the
classification is to enable best possible
detection of the object when it is detected by
the device. The object classification is done by
extracting features from the visuals—still
images and videos—that were collected during
the definition of the object, either using an
automated or user-triggered process. These
features can be shape, contour, color,
histogram, texture or other characteristics.
Further, content creators use algorithms to
extract features from digital images, including
the size/rotation/illumination invariant. SURF
(Speeded Up Robust Features), SIFT (Size
Invariant Feature Transform), GLOH (Gradient
Location and Orientation Histogram), and ORB
(Oriented FAST and Rotated BRIEF) are some
of the algorithms widely used to create image
descriptors and feature extractions.
Object Definition
Object Definition is a key step in developing an
AR solution. Content creators first define the
object and collect and save visual
representations of the object that may include
still images and videos. The visuals are
captured in as many dimensions as possible
with the best possible resolution and quality
and are used in subsequent process steps to
classify and detect the object. Content creators
also define tags or areas of interest in the object
that identify specific parts of the object and
associated content. This content is used to
augment the live scene that is captured on the
device. This content can include plain text,
01 —
02 —
03 —
04 —
documents, videos, images, animations, and
other visual constructs.
Figure 6: The four elements of a typical AR solution
Object Definition
Object
Augmentation
Object
Classification
Object Detection
and Tracking
Visual
Elements
Attribute 1
Attribute 2
Attribute N
.
.
.
Object
Content
Aa

Figure 8: Object classification in an AR solution
Figure 9: Object detection and tracking in an AR solution
Figure 10: Object Augmentation in an AR solution
7
Object Detection and Tracking
Detection of objects is an end-user triggered
process. A device with a camera is established
to detect and augment the objects of interest
that have been predefined and classified. The
camera captures the current live frame and
provides it as an input to the detection
algorithms that are associated with the
classification being performed. Several
combinations of features and image descriptors
on the live frame are used along with
feature-matching algorithms such as FLANN
matcher/brute force to detect the object.
Once the object is detected with the movement
of the device in the user’s hand and the change
in the user’s position, the object is tracked in
the live frame.
Object Augmentation
Augmentation of the detected object on a live
camera frame involves enhancing the object in
the live camera frame with visual elements that
are predefined or generated based on the
properties defined for the object. The
associated tags and content that are defined
along with the object can be used to construct
these augmented visual elements. Animations
and graphics can also be used together with
standard elements—such as text, images and
video—to enhance the live scene.
Objects
Shape based Classification
Classifier
7
Objects
Shape based Classification
Classifier Objects
Color based Classification
Classifier

Figure 11: A device-only AR solution
Figure 12: A client-server based AR solution
8
Solution Architecture
AR solutions involve understanding reality and
converting it into a digital form. The most
efficient and widely available transducer to do
the job is the camera. A live feed from a digital
camera is used to capture and convert the
image of an object into an understandable
format that AR uses to detect and recognize the
same live object when the camera is focused on
it again. AR solutions can be implemented
differently based on factors such as how and
where the solution is being applied and the
scale of use. The type of solutions can be
classified as following:
Device Only
Device and Server
Device Only
A device-only solution will have all the
components associated with the process steps
of AR described in the solution model above
but will reside on a single device. A simple, yet
robust AR solution can be implemented with a
camera that has an acceptable minimum
resolution, and good processing and storage
capability. The following diagram illustrates a
device-only solution:
Device and Server
A device-and-server solution has components
associated with the four process steps of the AR
solution but they are distributed across a device
and a server. This typically is a client/server
model in the application world. The following
diagram illustrates the device-and-server
solution:
The main disadvantage of the device-only
solution is the limited capacity—both processing
and storage—of the device. The solution can only
scale as much as the device capacity can support.
For example, on a device with 64 GB storage, the
size of the image, video and classifier data will
be limited to a maximum of around 56
GB—excluding the operating system—and the
processing speeds will be limited by the number
of cores on that device.
01 —
02 —
Figure 11: A device-only AR solution
This solution has the advantage of utilizing the
power of a server or cluster of servers with higher
processing and storage capabilities than the device
for feature extraction and training. This enables
the solution to scale. The detection process can be
initiated either on the server or on the local device
depending on the capability of the network
available to the device.
As with any client/server application, network
latency is a disadvantage for the AR
device-and-server solution. When the process of
detection is executed on the device, it leads to a
solution closer to the device-only solution with the
exception that the image classification and
training is performed on the server and all the
defined objects and their associated attributes are
stored on the server. The required data for
detection and augmentation can be synchronized
with the device, thereby exploiting the capabilities
of both the device and the server(s).
Device-only solutions are useful when they need
to be executed independently of a network. For
example, if a small, packaged product is to be
marketed to customers across different
locations by augmenting information about the
product and creating a compelling AR customer
experience, the marketer can use a device-only
solution and create this experience without
dependency on the network.

Benefits
9
While physical paper catalogs and manuals are
useful, it would be difficult for anyone to
remember every detail of an object that they deal
with, based only on what they have read. AR
solutions help enhance the assimilation of
information by utilizing visual images, sound and
touch. These solutions render complex sensory
information about objects that enhance learning.
03
Enhanced Assimilation
In a world where digital systems have redefined
the way we capture and store information about
various objects, being able to render such
information as required plays a significant role in
improving productivity. Conventional interfaces
such as web-based and mobile interfaces that
communicate digital content can enhance
learning and focus attention. However, an
interactive experience has the capability to more
fully capture and hold the user’s attention and
keep them engaged. Interactive experiences are
the core of AR solutions and contribute to
increasing productivity.
Increased Productivity
Information about complex objects and systems
is difficult to visualize and communicate to
others. This could be due in part to the sheer
volume of information about, and the structural
complexity of the object or system. AR solutions
can help manage this complexity by providing
content for augmentation for different part and
Managing Complexity
levels of the object. This layered approach helps
users learn the details of each part and
component while progressively building an
understanding of the overall object or system.
For example, consider a complex office all-in-one
printer that provides capabilities such as image
scanning, messaging and photocopying. The
printer has several components such as paper
trays and toners. Understanding these
components is necessary for maintenance. AR
can help visualizing the attributes of each of the
components of the machine cascading down from
the most complex part to the least complex part.
Users of digital systems in general prefer to
collaborate with other users—family, friends and
co-workers—to shape opinions and make
decisions about the products they purchase. AR
solutions can accommodate collaboration by
including live interaction between multiple users
who share their views and opinions during the
AR experience.
Collaborative experiences are particularly useful
for remote training. For example, consider a
scenario where a trainer is explaining the
functionality of a sophisticated machine to a
group of people in a different location. When the
trainer focuses a live camera device on the
machine and its key components, AR can help
trainees view augmented content such as a video.
With an appropriate solution, the trainees can
also point to an area of interest in the video and
ask the trainer for more information through
augmentation.
Collaborative Experiences

One of the key challenges in developing AR
solutions is creating object-detection algorithms
that can identify objects in all orientations, sizes
and lighting conditions. However, there is no
one-stop, sure-fire algorithm that can
consistently deliver a 100% match during the
object detection. Hence, the aim in detecting
objects is to maximize the probability of correctly
detecting the object. This uncertainty requires
solution designers to make assumptions about
certain scenarios, which may limit how well the
solution can work beyond the scenario they are
designing for.
04
10
Challenges
Accurate Detection and Recognition
of Objects
Another challenge with object detection is the
issue of “false positives.” A false positive is a
match to an object that does not exist in the
template/definition within the solution. That is,
while defined objects are correctly detected as
expected, the objects that aren’t valid or not
defined are incorrectly detected to be valid
objects. This issue can be resolved by providing a
larger sample of images and videos to train the
system and by applying customized algorithms
that can filter with greater granularity during
detection.
For example, consider two objects such as an IP
phone and a notebook computer that are the
same color. If there are not enough images for
either of these objects, it is possible that the IP
phone could be detected as a notebook and vice
versa.
False Positives
Mobile devices come in various screen sizes and
dimensions. The amount of information that can be
augmented for any given object on a device depends
on the screen’s available real estate. For example,
when there is a defined amount of information that
can be used to augment the object, it becomes more
challenging to manage the augmentation the smaller
the device’s screen size. A trade off needs to be made
between how generic and pared down the solution
can be made versus how many tailored solutions can
be made for different devices.
For instance, consider an iPad and an iPhone. The
screen sizes of these two devices are quite different.
For the same object, the amount of information that
can be augmented on an iPad will be greater that on
an iPhone. If the same amount of information is
attempted to be shown on the iPhone as the iPad, it
may lead to information clutter that will distort the
view and hamper the user experience.
Size of Mobile Device
Accurate tracking of an object in a live frame as the
user moves the mobile device around the object in
the contextual space is dependent on the quality of
the sensors and GPS in the device. These factors
can limit the deployment of the AR solution on a
limited set of devices based on their hardware
capabilities.
Accurate Object Tracking
Augmented Reality solutions rely on visual
input—images and video—of objects captured and
trained to eventually be detected and recognized. It
is important to ensure the sources of these inputs
are authorized to collect and store the information.
While this is relatively low risk for in-house worker
training and education solutions, it can be an issue
for public-facing solutions.
Security

11
The recent release of the Pokémon Go app is an
evidence of how geo-location is being married
with AR to provide an immersive user experi-
ence.
All the object recognition challenges for AR
solutions can be reduced to some extent by using
the user’s location as one of the dimension for
overlaying the data and identifying the object.
For example, if the user is in front of a building
that is a point of interest (POI) and the AR
application is able to display the building’s floor
plan, then the AR app need not rely on image
recognition. Instead it can use the location of the
user as the primary image recognition input to
identify the building.
05 Location and AR In certain cases, AR can utilize the camera’s depth
sensor, a new component that is starting to be
introduced into tablets. As devices get more sophisti-
cated, and accessories such as glasses and wearables
mature, usage of AR solutions will increase.
One area of location-based AR of interest to develop-
ers is the automotive market. Auto use cases will use
location-based input to stitch the live stream from
multiple cameras in the car with POI data to display
augmented images on the windshield.

Augmented Reality on various visual interfaces is
a promising paradigm providing an opportunity
to create immersive experiences for users. Given
the pervasiveness of mobile devices such as
phones and tablets, creating these experiences
The Way Forward
06
12
can help companies grab a larger mind share of
users and increase brand loyalty. As the
Internet-of-Things wave spawns new devices and
enhances existing ones, AR is poised to play a
potentially pivotal role in creating the next
generation of user experiences. With the
technology evolving rapidly, now is the time to
invest resources to develop AR solutions that will
define the future of the user experience.
Aricent is a global design and engineering services provider working with market leaders to
develop best-in-class products and services in the areas of communication infrastructure,
software and internet services, and embedded systems. Aricent has developed a software
framework that provides application programming interfaces (APIs) for developing integrated
Augmented Reality applications and solutions on mobile platforms for different domains.
The key features of Aricent’s Augmented Reality Framework (ARF) include:
Image Processing Engine
Feature Extraction
Multi-Class Object Detection
Gesture Recognition
Proximity Sensing, Ambient Location, Absolute Location (for both indoor and outdoor
locations)
Content/Template Definition
Floor Map Integration
Activity Mapping (Location, Image/Text, Content, Gesture Events, Point of Interest)
Alerts and Notification
Barcode/QR Code/OCR-Bbased Detection of Objects

About Aricent
Aricent is a global design and engineering company innovating for the digital
era. With more than 12,000 design and engineering talent and over 25 years of
experience, we help the world’s leading companies solve their most important
business and technology innovation challenges - from Customer to Chip.
© 2016 Aricent. All rights reserved.
All Aricent brand and product names are service marks, trademarks, or registered marks of Aricent in
the United States and other countries.
13
Contacts
Raghu Kishore Vempati, Principal Systems Engineer, Innovation
Email: raghu.vempati@aricent.com

Exploiting the Potential of Augmented Reality

More Related Content

What's hot (18)

Similar to Exploiting the Potential of Augmented Reality (20)

More from Aricent (10)

Recently uploaded (20)

Exploiting the Potential of Augmented Reality