Pratik Desai Ph.D dissertation defense

Department of
Electrical Engineering
A Semantic SituationAwareness Framework
for Indoor Cyber-Physical Systems
Dissertation Committee
Director Dr. Kuldip Rattan
Co-Director Dr. Amit Sheth
Dr. Marian Kazimierczuk
Dr. Frank Zhang
Dr. Guru Subramanyam
Ph.D. in Engineering Dissertation Defense
Pratikkumar Desai
Monday, 4/29/2013

Department of
Embedded systems
e.g. thermostat
Networked embedded
system
e.g. wireless
sensor networks
Cyber-physical
system
e.g. Intelligent traffic
management
systems

Department of
Cyber-Physical Systems
http://www.cs.binghamton.edu/~tzhu/
Cyber : Computation, communication, and control that are discrete, logical, and
switched.
Physical : Natural and human-made systems governed by the laws of physics
and operating in continuous time.
Cyber-Physical Systems (CPS) : Systems in which the cyber and physical
systems are tightly integrated at all scales and levels

Department of
CPS Examples
Disaster
Management
Smart
Grid
Smart
Home
Remote Patient
Monitoring
Traffic
Management
Air Traffic
Control
Military Drones

Department of

Department of
Motivation
Mobile sensing platform
Situation: Actual fire
at chair

Department of
Motivation
Event : Fire
from temperature and CO2
data
Event : Fire
from temperature and CO2
data

Department of
Motivation
Uncertainty: Sensor data
e.g. Due to resoultion, calibration
or robustness of sensors

Department of
Motivation
Incomplete domain knowledge
e.g. Unknown sources in the environment

Department of
Context
Environmental
context
Location
e.g. temperature,
CO2, heart rate
e.g. coordinates
Contextual
situation
awareness
Location
awareness
Contextual situation awareness:
“is a process of comprehending meaning
of environmental context in terms of
events or entities”
Location awareness:
“is a process of identifying objects from raw
spatial information and their relationship
with the ongoing events”
Context
“is a physical phenomenon, measured
using sensors, and product of an event”

Department of
Contextual situation awareness + Location awareness
Raw
environmental
sensor data
Raw spatial
information
Entities (High level
abstractions)
Object-Entity
relationships
Situation

Department of
IntellegO
0
100
C
temperature
CO2
0
1000
ppm
Fire
DryIce
RoomHeater
NormalCondition
Quality-type EntityQuality
LowTemp
HighTemp
LowCO2
HighCO2
• Abductive reasoning
• Crisp abstractions
http://wiki.knoesis.org/index.php/Intellego
Domain Knowledge
Base

Department of
Temperature: 500 C HighTemp
CO2: 1010 ppm HighCO2
Temperature: 500 C HighTemp
CO2: 999 ppm LowCO2
DryIce
RoomHeater

Department of
Motivation
Uncertainty: Sensor data
e.g. Due to limitation, calibration
or robustness of sensors
Incomplete domain knowledge
e.g. Unknown sources in the environment

Department of
Fuzzy abstractions
0
1 LowCO2
ppm800
HighCO2
1200
1160 ppm
0.9
0.1
µ
a

Department of
Fuzzy abductive reasoning
500 C
1160 ppm
0
80
120
C
temperature
CO2
0
800
1200
ppm
LowTemp Fire
DryIce
RoomHeater
NormalCondition
Quality-type EntityQuality
HighTemp
LowCO2
HighCO2

Department of
Evaluation – Contextual Situation Awareness
Reasoning approach Accuracy Precision Recall
Crisp abductive reasoning 86 % 78.57 % 73.33 %
Fuzzy abductive reasoning 94 % 92.85 % 86.66 %

Department of
Semantic Web
• Semantic web:
• Formally define the meaning of information on web.
• Provide expressive representation, formal analysis of resources.
• Ontology
• Formally represents knowledge as a set of concepts within
a domain and the relationships between pairs of concepts.
• RDF (Resource Description Framework)
• Graph-based language for modeling of information.
• Allows linking of data through named properties.
http://www-ksl.stanford.edu/kst/what-is-an-ontology.html
Subject Object
Predicate
HighTemp
InheresIn
Fire

Department of
Raw Sensor
Data
SSN
annotated
Observations
Low-level
fuzzy
abstractions
(qualities)
Fuzzy
reasoning
High-level
Abstractions
(entity)
SSN
Ontology
Domain Ontology Fuzzy Inference
rules ontology
Fuzzification
Rules
Observation process Perception process
Contextual situation awareness (Semantic modeling)

Department of
Traditional Indoor Localization Techniques
• Active Badge and Active Bat system.
• RADAR: An In-building RF-based user location
and tracking system.
• RFID radar
• Object tracking with multiple cameras
• Computer vision based localization
• Wireless Sensor Network
RF
Camera
TDoA

Department of
TDoA (Time Difference of Arrival)
Listener
Beacon
Stage 1: Beacon transmits RF and
Us signals together
Stage 2: Listener receives
RF signal first at Trf and
starts the clock
Stage 3: At Tus time US signals is received
by Listener
Stage 3: Distance is calculated using ∆T
and speed of signals

Department of
Trilateration
Number of nodes = 3.
Outlier rejection and Multilateration

Department of
The Proposed Algorithm
• Utilizes fusion of RSS (received signal strength) of RF
signal and TDoA data for accurate distance estimation.
• The algorithm stages:-
• RSSI data training
• Distance estimation
• Localization
• Uses TDoA as a primary distance estimation technique.
• RSSI data is trained and converted into appropriate
distance measurements.
• The proposed algorithm can be used in absence of one or
many TDoA links.

Department of
Initial Conditions
• Distances between all beacons are known and fixed

Department of
0 ? ? ?
? 0 ? ?
? ? 0 ?
? ? ? 0
L
B4
B2
B3
B1
0 ? ? ?
? 0 ? ?
? ? 0 ?
R14 ? ? 0
0 ? ? ?
R12 0 ? ?
? ? 0 ?
? ? ? 0
0 ? ? ?
? 0 ? ?
R13 ? 0 ?
? ? ? 0
0 ? ? ? R1L T1L
? 0 ? ? ? ?
? ? 0 ? ? ?
? ? ? 0 ? ?
RSSI Link
TDoA Link
Beacon B1 Transmit Data

Department of
0 R21 ? ?
R12 0 ? ?
? ? 0 ?
? ? ? 0
L
B4
B2
B3
B1
0 ? ? ?
R12 0 ? ?
? ? 0 ?
R14 R24 ? 0
0 ? ? ?
R12 0 ? ?
? ? 0 ?
? ? ? 0
0 ? ? ?
R12 0 ? ?
R13 R23 0 ?
? ? ? 0
0 ? ? ? R1L T1L
R12 0 ? ? R2L T2L
? ? 0 ? ? ?
? ? ? 0 ? ?
RSSI Link
TDoA Link

Department of
0 R21 R31 ?
R12 0 ? ?
R13 R23 0 ?
? ? ? 0
L
B4
B2
B3
B1
0 ? ? ?
R12 0 ? ?
R13 R23 0 ?
R14 R24 R34 0
0 ? ? ?
R12 0 R32 ?
R13 R23 0 ?
? ? ? 0
0 ? ? ?
R12 0 ? ?
R13 R23 0 ?
? ? ? 0
0 ? ? ? R1L T1L
R12 0 ? ? R2L T2L
R13 R23 0 ? R3L T3L
? ? ? 0 ? ?
RSSI Link
TDoA Link

Department of
0 R21 R31 R41
R12 0 ? ?
R13 R23 0 ?
R14 R24 R34 0
L
B4
B2
B3
B1
0 ? ? ?
R12 0 ? ?
R13 R23 0 ?
R14 R24 R34 0
0 ? ? ?
R12 0 R32 R42
R13 R23 0 ?
R14 R24 R34 0
0 ? ? ?
R12 0 ? ?
R13 R23 0 R43
R14 R24 R34 0
0 ? ? ? R1L T1L
R12 0 ? ? R2L T2L
R13 R23 0 ? R3L T3L
R14 R24 R34 0 R4L T4L
RSSI Link
TDoA Link

Department of
Evaluation– Proposed Algorithm

Department of
Bed-1
Chair-2
Chair-1
Sofa-1
Fireplace-1
Treadmill-1
Desk-1
Stove-1
Bedroom-1 Bedroom-2
Gym-1
Drawingroom-1
Kitchen-1
Plant-1
Indoor
Environment

Department of
Hierarchical mapping of
the indoor environment

Department of
Chair-1
Drawingroom-
1
xsd:float
xsd:float
xsd:float
xsd:float
xsd:float
xsd:float
xsd:string
ChairPOI
POI
Drawingroom
is-a
has individual
is-a
StructuralComponent
has individual
DefiningCoveragespace(Chair-1)

Department of
Raw location : ( x, y) = (190 cm, 570 cm)

Department of
Object-entity relationship
Structural Components Point of Interests Entities
hasIndividual
hasApplicableEntity

Department of
Evaluation – Location Awareness
Chair-1
Sofa-1
Fireplace-1
Drawingroom-1
Plant-1
(140,560)
(140,640) (430,640)
(430,560)
(60,480) (200,480)
(60,140) (200,140)
(180,110)
(180,10)
(110,340)
(10,340)
(610,360) (760,360)
(610,240) (760,240)
( )
( )
Mobile-robot route

Department of
Reasoning
approach
Precision Recall
Crisp abductive
reasoning
87.5 % 43.75 %
Fuzzy abductive
reasoning
100 % 50 %
Location aided
fuzzy abductive
reasoning
100 % 88.89 %
Location aided
reasoning
Location independent
reasoning
Fireplace

Department of
EntitiesQualities
Indoor
Positioning
System
Environment
Sensors
Situation
Fuzzy
Abductive
Reasoning
Fuzzy
Abstraction
Rules
Contextual Situation Awareness
Semantic
Object
Identifier
Spatial
Reasoning
LocationAwareness
Domain
Knowledge
Comprehensive Framework
(System level)

Department of
Raw
Physical
Context
Data
SSN
annotated
Observations
Low-level
Fuzzy
Abstractions
(Qualities)
High-level
Abstractions
(Entities)
Optimized
Situation
Raw
Location
Data
Semantic
Location
Identifier
SSN
Ontology
Domain Ontology Fuzzy Abductive
Reasoning Rules
Indoor Location
Ontology
Comprehensive Framework
(Semantic modeling)

Department of
Temp: 150 C
CO2:1120 ppm
Temp: 180 C
CO2:1160 ppm
Location (b):
(400,150,30)
Location (a):
(200,250,20)
Object coverage area
Mobile robot path

Department of
Fire: 0.80
Heater: 0.20
Fire: 0.90
Heater: 0.10
Location (b):
(400,150,30)
Location (a):
(200,250,20)
Mobile robot path

Department of
Fire: 0.80
Heater: 0.20
Fire: 0.90
Heater: 0.10
Location (b): Chair
Location (a): Fireplace
Mobile robot path

Department of
Fire: 0
Heater: 0
Fire: 0.90
Heater: 0.10
Location (b): Chair
Location (a): Fireplace
Mobile robot path

Department of
Key Contributions
• Developed a fusion based indoor localization algorithm to
achieve accurate spatial information of the sensing
platform.
• Accurate indoor localization algorithm.
• Surveillance and tracking of mobile robots in indoor environments.
• Integration of indoor positioning results with virtual world environment.
Related papers:
• P. Desai, N. Baine, and K. S. Rattan, “Fusion of RSSI and TDoA Measurements from Wireless Sensor Network
for Robust and Accurate Indoor Localization,” in International Technical Meeting of The Institute of Navigation,
2011, pp. 223–230.
• P. Desai, N. Baine, and K. S. Rattan, “Indoor localization for global information service using acoustic wireless
sensor network,” in Proceedings of SPIE, 2011, vol. 8053, no. 1, pp. 805304–805304–10.
• P. Desai and K. S. Rattan, “System Level Approach for Surveillance Using Wireless Sensor Networks and PTZ
Camera,” in 2008 IEEE National Aerospace and Electronics Conference, 2008, pp. 353–357.
• P. Desai and K. S. Rattan, “Indoor localization and surveillance using wireless sensor network and Pan/Tilt
camera,” in Proceedings of the IEEE 2009 National Aerospace Electronics Conference NAECON, 2009, pp. 1–6.
• An invited journal paper in preparation.

Department of
Key Contributions
• Introduced fuzzy abstraction and inference technique to
comprehend events via handling the uncertainty in the
context information & the ambiguity in the domain
knowledge.
• P. Desai, C. Henson, P. Anatharam, and A. Sheth, “SECURE: Semantics Empowered
resCUe Environment (Demonstration Paper),” in 4th International Workshop on Semantic
Sensor Networks (SSN 2011), 2011, pp. 110–113.
• A journal paper in preparation.
• Developed semantic mapping technique for indoor objects
to aid the situational context awareness results via further
discriminating not applicable events.
• Developed and deployed a comprehensive situation
awareness framework for cyber-physical system.
• A journal paper in preparation.

Department of
Future work
• Richer spatio-temporal relation modeling between indoor
objects and entities
• Efficient coverage space for the indoor objects
• Accurate indoor localization via smartphones

Department of
Acknowledgements

Department of
Questions?

Pratik Desai Ph.D dissertation defense

More Related Content

Similar to Pratik Desai Ph.D dissertation defense (20)

More from Pratik Desai, PhD (6)

Recently uploaded (20)

Pratik Desai Ph.D dissertation defense

Editor's Notes