Efficient mission planning in communication constrained environment

1 / 70
EFFICIENT MISSION PLANNING FOR ROBOT
NETWORKS IN COMMUNICATION CONSTRAINED
ENVIRONMENTS
PhD Candidate: Md Mahbubur Rahman
Major Professor: Dr. Leonardo Bobadilla
Committee Members: Dr. Bogdan Carbunar
Dr. Ning Xie
Dr. Wei Zeng
Dr. Ali Mostafavi
SCIS, Florida International University
June 6, 2017

Introduction
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Thrust-4: Multi-Optimal
Motion Planning
2 / 70

Robot Networks
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
3 / 70
A robot network is a system that,
Deploys a set of robots that are interconnected.
Collectively achieves a set of activities.
Performs safe movements avoiding obstacles and other robots.
Co-exists with humans in a hybrid mission.
Optimizes mission cost as deﬁned by the operator.
Military Network Robot Swarm Remote Operation

Remote Controlled Networked Systems
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
4 / 70
(a) (b) (c)
(d) (e) (f)
(g) (h) (i)
Figure 1: (a) IHMC humanoid robot; (b) US Military drone (UAV); (c) Sandia Lab’s mining drone;
(d) Sandia’s robot swarm; (e) da Vinci surgical system developed by Intuitive Surgical; (f) K5 security
robot; (g) Google’s waymo self-driving car; (h) SAM-100 mason robot; (i) BoniRob agricultural robot
from Bosch

Communication Challenged Environment
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
5 / 70
Connected robots are becoming common in many unmanned
systems (e.g, telepresence, construction, underwater vehicles).
Continuous connectivity is difﬁcult to maintain due to geographical,
natural (terrain, atmospheric, electromagnetic) phenomena.
Communication modalities may be restricted due to risk of adversarial
interference, intentional jamming. Commonly found in military
applications (surveillance, guarding).
Also providing communication to remote areas (e.g, drone) is
challenging.
Relay Communication Military Network

Proposed Research Areas
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
6 / 70
We investigate four main thrusts:
Communication Mapping: Methods to perform a remote operation
with the help of one or more intermediate relay robots.
LoS Based Resilient Robot Network: Ensures that a number of
robots are in the direct Line of Sight of each other.
Communication Aware Safe Planning: Quantify safety score for a
fully autonomous construction project.
Multi-Optimal Motion Planning: Develop sampling based motion
planning algorithm that optimizes multiple objectives.
Material Storage Fabrication Area
Office
Dump Truck crane
start
goal
goal
Trajectory
Building # 1
Building # 2
160ft
180 ft
Communication Relay Direct Visibility Construction Automation

Thrust-1: Communication Relay
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
7 / 70

Motivation
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
8 / 70
Communication relay has vital importance in military, mining,
surveillance and rescue missions, where robots are remotely
controlled by an operator (e.g, drone) who stays in a safe location.
Problem: Wireless signal over distance degrades. Obstacles, terrain,
weather condition further attenuates the signal.
Solution: Relay robots are placed in between the operator and
remotely placed robotic units.
Unmanned Ground Vehicle Intermediate relays

Two Research Problems
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
9 / 70
1. Single Remote Unit: Chain of intermediate relays.
2. Multiple Remote Units: A spanning tree of relays and units.
** Where to place the given number of relays to achieve maximum signal
quality?
Relay Chain Relay Tree

Literature Review
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
10 / 70
Burdakov’10: Discretizes the environment into a grid and uses k-Hop
Bellman Ford algorithm to ﬁnd the shortest sequence of grid points.
No reusable data structure is developed like us.
Tekdas and Volkan ’10: A dynamic programming solution is proposed
and the running time of the algorithm which is O(nm), compared to
our polynomial time algorithm with a running time of O(mn2)
(n=number of points on a grid and m=number of available relays).
Dixon and Frew’12: Do not consider obstacles and use aircraft as
relays that are not static and orbit around a control point. Therefore,
the solution cannot guarantee uninterrupted service for large
geographical area.
Common Drawbacks:
***Additionally, no reusability of solution is possible for the above works.
***Existing literature do not consider multiple remote units.

Problem Formulation
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
11 / 70
m relay vehicles, A1, A2, . . . , Am and p mobile units
B1, B2, . . . , Bp and one static operator K.
K, Ai and Bj all are represented with conﬁgurations (x, y) ∈ E.
Free space communication cost, fF between ρ1 and ρ2,
fF (ρ1, ρ2) =
γd2(ρ1, ρ2) if d(ρ1, ρ2) < dth
∞ otherwise
(1)
The signal loss in the presence of obstacles, fO(ρ1, ρ2, O), includes the
costs resulting from diffraction (fDF ), fading (fFA):
fO(ρ1, ρ2, O) =
0 if ρ1ρ2 has LoS
fDF (O) + fFA(O) otherwise
(2)
Finally, the total communication cost fC is deﬁned as:
fC(ρ1, ρ2) = fF (ρ1, ρ2) + fO(ρ1, ρ2, O) (3)

Complexity and Solution to Relay Chain Problem
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
12 / 70
A bi-criteria shortest path decision problem was proven to be
NP-Complete (arkin’91) when we need to decide if a path with m + 1
links (for m relays) is the shortest.
The optimization version of calculating the shortest m + 1 link path is
NP-Hard.
A geometric Discretization process is applied to decompose the
plane.
A common method is to decompose the environment into a grid of n
points.
Uniform Grid Adaptive Grid Triangular Mesh

Steps of Computing the Relay Chain: Communication Graph
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
13 / 70
A graph G is computed where all the decomposed cells are
considered as the vertices.
An edge between two vertices represents the communication and the
weight is measured by the communication cost fC:
fC(ρ1, ρ2) = fF (ρ1, ρ2) + fO(ρ1, ρ2, O)
0
2
6
1
3
4
9
7
5
8
Figure 2: (a) A uniform grid; (b) Connected communication graph G with the
weights in fC;

Relay Chain: Layered Graph
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
14 / 70
Create a (m + 2)-layered graph if m relays are available.
Layer l0 contains a single node corresponding to the operator’s
position.
Each other layer li has n nodes corresponding to n grid points.
Edges between subsequent layers li and li+1 will follow G.E.
0
2
6
1
3
4
9
7
5
8
Figure 3: (b) Connected communication graph G with the weights in fC; (c)
Directed layered graph G generated from G.

Relay Chain: Modiﬁed BFS
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
15 / 70
Apply a modiﬁed BFS Algorithm with a hashmap that assigns the parents.
The running time is O(V + E) as every node and edge is visited once.
There are total (m + 1)(n − 1) + 1 nodes for m + 2 layers and the total number of edges is
at most |E| = (number of edges in m + 1 layers) + (number of edges in layer
l0) = m(n − 1)(n − 2) + n − 1 = O(mn2).
Algorithm 1 multiRelaySingleUnit(G(V, E))
1: G(V, E) = calculateGraph(G)
2: vs.cost = 0, and v.parent = NULL; ∀v ∈ V
3: Enqueue(Q, vs)
4: h[v.id] = ∞; ∀v ∈ G.V
5: while Q = ∅ do
6: u = Dequeue(Q)
7: for v ∈ u.Neighbors do
8: if u.cost + fC (u, v) < h[v.id] then
9: v.parent = u
10: v.cost = u.cost + fC (u, v)
11: h[v.id] = v.cost
12: Enqueue(Q, v)
13: end if
14: end for
15: end while
Directed layered graph G
Communication map M0
c

Extraction of a Relay Chain
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
16 / 70
Check unit position in layer lm+1: If found, backtrack all the way to
layer l0, otherwise check in a lower layer.
Resolution Completeness: Returns success if we ﬁnd the unit node
in between layers l1 and lm+1. Otherwise return failure (no solution in
the current grid resolution).
At location 5 At location 8 At location 9

Relay Chain: Using Communication Map
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
17 / 70
Relay Increase: Two, three and four. Unit at cell 5 and operator at cell 0.
m=2 m=3 m=4

Second Problem: Connecting Multiple Units
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
18 / 70
Connect the p units through m available intermediate relays to a
static base operator.
Formation will be a tree instead of a chain. Has similarity to the
(p, m)-Steiner tree problems discussed in [Watel ’13].
This tree spans over p ﬁxed terminal nodes and m variable nodes.
However, We must limit the unit nodes from branching.
Here is an instance of 2-units, 2-relays and 1-operator problem. The units
are at positions 4 and 9.

Relay Tree: Steps of Computing a Tree
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
19 / 70
Compute a min-cost tree, with exactly m relays, p units, and one operator.
p + 1 ﬁxed nodes be VT = {vs} ∪ VB, where vs is the operator node
and VB ⊂ V is the set of nodes corresponding to the remote units.
Select m relay locations from the remaining n − p − 1 nodes. This
means
n−p−1
m sets of nodes are selected.
Create
n−p−1
m possible directed graphs. Unit nodes in a graph are
not allowed to have outgoing edges.
Compute min-Arborescence tree for all the
n−p−1
m graphs.
Select the best one in terms of communication cost.
0
2
1
4
9
0 2 3
9
4
Communication Graph Two Candidate Graphs
VT = {0, 9, 4}

Relay Tree: Complete Algorithm and Example
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
20 / 70
Algorithm 2 multiRelayMultiUnit(G(V, E))
1: VT = {vs} ∪ VB
2: ϑm = {ν ∈ P(V VT ) : |ϑ| = m}
3: for νi ∈ ϑm do
4: Vi = νi ∪ VT
5: Gi = computeDiGraph(Vi)
6: if Gi.connected() then
7: Ti = minArborescence(Gi)
8: T .add(Ti)
9: end if
10: end for
11: return failure if T = Null
12: return argmin
Ti∈T
[f
T
C (Ti)]
The loop of line 3 runs
n−p−1
m
times, which can
be simpliﬁed as O(nm
).
Tarjan’s algorithms runs in O(E + V log V ) =
O(m2
+ mp + p).
The worst case running time is O(nm
(m2
+
mp + p)).
0
2
1
4
9
0 1 2
4
9
(a) (b)
0 2 3
9
4
0 2
3
9
4
(c) (d)
Figure 4: (a) A sub-graph G1 con-
structed with ν1 = {v1, v2}; (b) Re-
sulting min-arborescence tree T1 of G1;
(c) Another candidate sub-graph G2 with
ν1 = {v2, v3}; and (d) Candidate tree
T2

Chain Formation on Line of Sight Based Systems
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
21 / 70
(a) (b) (c)
(d) (e) (f)
Figure 5: Multi relay chain simulation: (a) Four relays forming a chain; (b) and (c) Number of relays
are reduced to three and two, respectively; (d) Shadow region Φ1 for one relay; (e) and (f) Adaptive
grid decomposition for three relays and one relay.

Relay Chain Formation (Radio Network)
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
22 / 70
(a) (b) (c)
(d) (e) (f)
Figure 6: Communication can now be established through the obstacles: (a) Four relays, (b)
three relays and (c) one relay connecting the unit to the operator. (d), (e) and (f) are the adaptive grid
decomposition with four, three and two available relays, respectively.

Multi-Unit Multi-Relay Tree Formation
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
23 / 70
(a) (b) (c)
(d) (e) (f)
Figure 7: Multi-Relay Multi-Unit simulations. (a) and (b) show min-arborescence tree for two relays
serving four and six units, respectively; (c) shows three relays serving three units and (d) is a case of
three relays connecting ﬁve units; (e) and (f) are min-arborescence tree for four relays connecting the
units to the operator.

Hardware Experiment on Miniature World
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
24 / 70
(a) A* path planning (b) A chain formation
(c) Multi-Unit system (d) A tree formation
Video Link:https://youtu.be/r44K-HVONc4.

Experiment in ECS Corridor
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
25 / 70
(a) (b)
(c) (d) (e)
Figure 8: Large area deployment: (a) A corridor map; (b) Placements of relays as a chain formation;
(c) Maximum signal loss has been measured at 23 dB (decibel); (d) The maximum communication cost
was measured at 37 dB; (e) Three relays are deployed and the maximum link cost has been reduced
to 25 dB from 37 dB.

Comparison of Running Time with Best Known Solution
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
26 / 70
Our layered graph, G, computation takes slightly more time as it has redundant nodes.
Computation of reusable map Ms
c is faster for smaller environments as we use a modified BFS
algorithm on G, compared to a modified Bellman-Ford algorithm used by Brdakov’10.
We achieve significant improvements in the subsequent computations than Burdakov’10, as we
only need to extract a chain of relays from Ms
c instead of recomputing the entire data structure.
Table 1: Analysis of Running Time (in seconds)
Nodes Our Method Burdakov et el.
Building G+
Ms
c computation
Subsequent
Runs
Building G +
k-hop BF
Subsequent
Runs
361 6.70+0.438 0.0052 2.39+1.23 1.05
400 8.15+0.58 0.0067 2.66+1.62 1.50
625 23.82+2.41 0.0081 6.54+4.12 4.06
729 35.57+3.70 0.0079 11.39+7.78 7.23
900 49.86+8.01 0.0095 13.52+8.93 8.85
1089 85.01+14.07 0.012 23.03+14.51 14.87

Summary of Thrust-1
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
27 / 70
Solution for a relay chain system using a m + 2-layered graph.
Developed a polynomial time algorithm through modifying a breadth
ﬁrst search algorithm.
Estimate the optimal placements using min-Steiner tree algorithms
to serve multiple units.
Develop a hardware test-bed to perform real world experiments.

Thrust-2: LoS Based Resilient Robot
Networks
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
28 / 70

LoS Based Robot Network
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
29 / 70
Other form of communication (satellite, radio, wi-fi) are not feasible.
Significant use in military mission, patrolling, monitoring.
LoS based communication is more reliable, secure and efficient.
B1
B2
B3
A1 A2
B5
B4
(a) Military mission (b) 2D representation

Research Questions
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
30 / 70
What will be the optimal locations to place the vehicles?
How to know when the network gets disconnected?
How to relocate the vehicles to recover the units that went out of
sight?
What is the minimum number of vehicle to form a LoS based
connected relay network?
B1
B2
B3
A1 A2
B5
B4
Military mission 2D representation
B1 disconnected Mutual Disconnection

State of the Art
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
31 / 70
Obermeyer’12: Proposed a scheme to visit all the visibility polygons
using a single vehicle. Leads to redundancy when polygons intersect.
No consideration of formation of a constrained relay network among
multiple robots.
ORourke’87 and Erickson’11: Discuss Art Gallery (ORourke’87) and
landmark placement (Erickson’11) problems. However, only ﬁxed
positioning was considered and vehicles motion policies were not
considered during relocation.
Bhadauria’11 and Zavlanos’09: A mobile unit uploading,
downloading, and distributing data to static nodes are common in data
muling. However, our communication is based on line-of-sight instead
of the proximity of sensor nodes.
Bhattacharya’07 and Muppirala’05: Our work has similarity to path
planning that maintains visibility with a single static landmark.

Problem Formulation
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
32 / 70
n vehicles, A1, A2, . . . , An equipped with high-performance
computing devices that serve m units B1, B2, . . . , Bm.
qi = (x, y, θ) ∈ Ci is the conﬁguration for vehicle Ai and
rj = (x, y) ∈ Bj is the conﬁguration of unit Bj.
V (qi) and V (rj) are the visibility polygons of vehicle Ai and unit
Bj.
The state space, X = C1 × C2 × · · · × Cn × B1 × B2 × · · · × Bm.
B1
B2
B3
A1 A2
B5
B4
Sample Mission 2D representation

Problem Deﬁnition
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
33 / 70
A subset Xcomm ⊆ X is considered the set of communication-valid
states if and only if each unit is visible by at least one servicing
vehicle and all the servicing vehicles form a connected graph.
We must satisfy the following two conditions in order to have a
communication-valid state:
∀j, ∃i s.t. rj ∈ V (qi) for 1 ≤ j ≤ m and 1 ≤ i ≤ n (4)
{(qj, qk)|qk ∈ V (qj) for 1 ≤ j, k ≤ n, k = j} ≡ CC(x) (5)
where CC(x) is a connected component formed by all the
vehicle-vehicle connections.
B1
B2
B3
A1 A2
B5
B4

Problem Deﬁnition (cont’d)
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
34 / 70
Problem 1: Communication State Validation
Given the workspace W, a set of obstacles O, a set of conﬁgurations
q1, q2, . . . , qn for robot vehicles, and r1, r2, . . . , rm for mobile units,
determine whether a state x ∈ X is communication-valid or not.
Problem 2: Communication Validity Restoration
Given W and O, the current state space x ∈ X, and a set of
disconnected units D, select several vehicles to relocate and compute
their new goal region, XG, that will reconnect all the units in D.

Checking Vehicle-Vehicle Connectivity
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
35 / 70
Form a vehicle relay graph GA(VA, EA) where
VA = {A1, A2, . . . , An} and EA = {eij|qi ∈ V (qj)}.
Compute n × n Laplacian matrix, L(GA) where,
i) lij =
−1 if an edge exists between i and j
0 otherwise
ii) lii = − n
k=1,k=i lik.
Check whether the second-smallest eigenvalue λ2(L(GA)) > 0.
B1
B2
B3
A1 A2
B5
B4
A1 A2 1 −1
−1 1
(a) (b) GA (c) L(GA); λ2(L(GA)) = 2.

Checking Communication Validity
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
36 / 70
Form a unit graph GB(VB, EB) where
VB = {A1, A2, . . . , An, B1, B2, . . . , Bm} and
EB = {eij|rj ∈ V (qi) where n < j ≤ n + m and 1 < i ≤ n}
Form a graph G(V, E) = GA ∪ GB.
Compute (m + n) × (m + n) Laplacian matrix, L(G) and check
whether the second-smallest eigenvalue λ2(L(G)) > 0.
A1 A2 A1
B1
B2
B5
B4
A2
B3
A1
B1
B2
B5
B4
A2
B3
GA GB G = GA ∪ GB.
L(G) =









1 0 0 0 0 −1 0
0 1 0 0 0 −1 0
0 0 2 0 0 −1 −1
0 0 0 1 0 0 −1
0 0 0 0 1 0 −1
−1 −1 −1 0 0 4 −1
0 0 −1 −1 −1 −1 4









and
Real Eigenvalues : {0; 0.5505; 1.5857; 4.4142; 5.4494}

Communication Validity Algorithm
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
37 / 70
Algorithm 3 communicationCheck(x, O)
1: GA = gA(x)
2: if λ2(L(GA)) ≤ 0 then
3: return false
4: end if
5: GB = gB(x)
6: G = GA ∪ GB
7: if λ2(L(G)) ≤ 0 then
8: return false
9: else
10: return true
11: end if
** Visibility polygon computation takes O(n) (Gindy ’81) and the n
vehicles takes O(n2). Computing the eigenvalues generally takes O(n3)
in the worst case. Therefore the running time of the above Algorithm is
O(n3).

Communication Validity Experiment
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
38 / 70
(a) Communication Invalid (b) λ2(GA) = 2; λ2(G) = 0.
(c) λ2(GA) = 0. (d) λ2(GA) = 1; λ2(G) = 0.5024.

Solution of Disconnection: Single Vehicle Relocation
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
39 / 70
The set of disconnected units D who have no covering vehicle.
Set of candidate vehicles is, C ∈ P(A) that doesn’t break the
existing connectivity of graph GA, if relocated.
C = A Ai s.t. λ2(L(GA(VA Ai, EA ei))) ≤ 0. (6)
Deﬁne Hi as the set of hard constrained units Ai ∈ C which are only
visible from Ai.
Goal polygon Xi
G of a candidate vehicle Ai ∈ C must be inside the
visibility polygons of 1) the disconnected unit Bj, 2) at least one other
vehicle and 3) inside the polygon V (Hi) of hard constrained units.

Goal Locations to Relocate a Vehicles
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
40 / 70
We may get multiple goal locations for a vehicle and select the largest
one:
Xi
G =



max
Ak=Ai,1≤k≤n
V (rj) ∩ V (qk); if Hi = ∅
max
Ak=Ai,1≤k≤n
V (rj) ∩ V (qk) ∩ V (Hi); otherwise
(7)
We select the optimal vehicle Ai in terms of the relocation cost of a
vehicle from its current position xi
s to the computed goal region Xi
G
using the RRT* motion planner.
D = {E} X2
G X4
G RRT* Path

BonnMotion Experiment
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
41 / 70
Random Waypoint:
(a) time = 4 (b) time = 5 (c) time = 7 (d) time = 15
Nomadic:
(a) (b) (c) (d)

General Positioning Problem
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
42 / 70
What are the minimum number of vehicles to cover all the units?
What are the optimal locations to place them?
Solution: A set cover approximation solution works.
How to plan if we only have a single vehicle?
Solution: It is a NP-Hard problem. Can be reduced from Traveling
Salesman with Neighbor (TSPN), as patrolling is the only solution.
converted to
visibility polygonspolygons to visit
units added
Visibility Polygons TSP with Neighbors

Environment Decomposition
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
43 / 70
Decompose the environment into a set of polygons F = {P1, P2, . . . , Pρ},
based on visibility polygon intersection.
Assign a label yP to each polygon:
yP = {Bj, Bk, . . . , Bl}; ∀c ∈ {j, k, . . . , l} V (Bc) ∩ Pi = Pi. (8)
Assign a score ˆs(P) to each polygon:
ˆs(P) = γ · area(P) +
Bk∈yp
α − β · d(P, Bk) (9)
(a) (b)

Greedy Set Cover Algorithm
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
44 / 70
Algorithm 4 multiRobotPlacement (B, O)
1: V = {V (B1), V (B2), . . . , V (Bm)}
2: F = decompose(V)
3: ∀P ∈ F, yP = assignLabel(V)
4: ∀P ∈ F, ˆs(ti) = assignScore(P, yP )
5: Γ = ∅; U = {B1, B2, . . . , Bm}
6: while U = ∅ do
7: Select P ∈ F that maximizes |yP ∩ U|
8: U = U − yP
9: Γ = Γ ∪ {P}
10: end while
11: return Γ
The running time of this set cover approximation is O(|U||F| min(|U|, |F|)).
B1
B2
B3
B4
B5
B6
(a) (b)

Vehicle Assignment
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
45 / 70
Case |Γ| = 1: Trivial case. Deploy a vehicle in the sole polygon.
Case |Γ| < n: Enough vehicles for all the polygons. May need
patrolling if we have multiple connected components.
Case |Γ| ≥ n: One vehicle per polygon. The last vehicle may do the
patrolling if we have multiple connected components.
A1
P2
P5
P3
P1
P6
P4
A4
A3
A5
A2
A1
A3
A4
A2
A5
C2C1
(a)|Γ| = 6 and n = 6 (b) GA
Figure 10: (a) A set of six polygons, Γ = {P1, P2, P3, P4, P5, P6} computed by approximate
set cover (Algorithm 4) that are to be covered by n = 6 available vehicles; (b) Two connected compo-
nents C1 and C2 are computed from vehicle graph GA;

Patrolling Policy and Route Computation
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
46 / 70
Compute the vehicle-graph GA(VA, EA)
Apply the connected component algorithm to get a set of components C1, C2, . . . , Cκ
Merge the visibility polygons of the vehicles under a component Ci to make a single big
polygon, PCi
:
PCi
=
Aj ∈Ci
V (Aj) (10)
Find uncovered polygon set, ΓU = Γ {P1, . . . , P|VA|}.
A1
P2
P5
P3
P1
P6
P4
A4
A3
A5
A2
A1
A3
A4
A2
A5
C2C1
(a)|Γ| = 6 and n = 6 (b) GA
(c) GCC
A (d) TSP tour

Patrolling Route Computation
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
47 / 70
Create a directed connected-component graph GCC
A (VCC
A , ECC
A ), where the vertices are
composed of the component polygons and uncovered polygons,
VCC
A = {PC1
, PC2
. . . , PCκ } ∪ ΓU
(11)
The weighted directed edge eCC
ij ∈ ECC
A between two vertices PCC
i , PCC
j ∈ VCC
A is
computed using a motion planning algorithm such as RRT*, A* or combinatorial planning.
Apply the approximate Geometric TSP algorithm to compute the sequence of polygons to visit.
A1
P2
P5
P3
P1
P6
P4
A4
A3
A5
A2
A1
A3
A4
A2
A5
C2C1
(a)|Γ| = 6 and n = 6 (b) GA
(c) GCC
A (d) TSP tour

Simulation Examples on Random Environments
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
48 / 70
B1B2
B3
B4
B5
B6
B1
B2
B3
B4
B5
B6
(a) Trivial case (b) Two vehicles are required
(c) Two static one dynamic vehicles (d) Three static vehicles are enough

ROS and Gazebo Computer Simulation
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
49 / 70
Virtual World Visibility Polygon Decomposition
One Vehicle Scenario Three Vehicles Scenario

Hardware Experiment
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
50 / 70
A Ground Robot An Unit
Computer Vision Output Raspberry Pi+Camera
Video Link:https://youtu.be/j ey2ok27Q.

Summary of Thrust-2
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
51 / 70
Developed algorithms to check the communication valid state.
Compute the minimum number of robots to keep visibility to all the
units.
Estimate the optimal locations of the robots.
Generate a patrolling policy in case we don’t have enough robots.

Thrust-3: Communication Aware Safe
Planning
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
52 / 70

Motivation
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
53 / 70
Construction zones are ideal test-bed because:
Construction zones are a source of accidents with signiﬁcant loss of
lives.
Approximately 75% of struck-by fatalities in construction projects
involve moving equipment such as trucks, excavators, or cranes.
The dynamic and continuously changing nature of construction
jobsites make risk assessment challenging.
There are a lot of continuous motions equivalent to the robotic motion
planning.
Humans and heavy equipment can be modeled as different types of
robot.

Alternative Plans Scheduling
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
54 / 70
All possible topological sorting on activity graph.
P1 = [EX1, CP1, EX2, CP2]
P2 = [EX1, EX2, CP1, CP2]
P3 = [EX2, EX1, CP1, CP2].
Try to schedule activities in parallel when possible.
Simulation stops when all the current activities are ﬁnished.
Example: P1 = [EX1, CP1, EX2, CP2]
Parallel execution of CP1 and EX2 is possible. EX1 and CP1 is not
possible.
Material Storage Fabrication Area
Office
Dump Truck crane
start
goal
goal
Trajectory
Building # 1
Building # 2
160ft
180 ft
S F
CP1EX1
CP2EX2
(a) (b)

Discrete Event Simulation
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
55 / 70
Discrete event system speciﬁcation was used to model the simulation.
ES = {E, Z, EL, fη, fz, zI}, for each activity.
An example excavation events are, EEX = {L, H, D, R}.
Event transition function, fEX
η (L, z) = H.
State transition function, fEX
z (L, z).
zI is the initial system state.
L D
R
H
D
L
(b)(a)
RO

Trajectory for Human
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
56 / 70
Maximum clearance roadmap was used to generate trajectories for
the workers.
Achieved by Generalized Voronoi Diagram.
(a) (b)

Safe Policy Generation
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
57 / 70
Following is an example s-t space graph for the shortest path. The
worker faces only one collision out of the two possible collisions.
Approximately 10 seconds delay for the worker as he lets the moving
body to pass.

Risk Heatmap Generation
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
58 / 70
Environment is decomposed into grid and colored based on Euclidean
distance metric.
R(gi, t) =
|Qt|
j=0
|Bj |
k=0
α
d(gi, Bk(t)) + β
and ragg(gi) =
tf
t=ti
R(gi, t)
tf − ti
.
(12)

Managerial Implication
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
59 / 70
Graphs are generated to visualize the comparison among different
plans.
Plan3Plan2Plan1
0 2 4 6 8 10 12 14
EX1 CP1EX2 CP2
EX1EX2 CP1CP2
EX2 EX1 CP1 CP2
Project Timelines for Different Plans
Duration (Days)
1 2 3 4 5 6 7 8 9 11 13
12001600
Safety Scores Throughout Project Duration
Days
SafetyScore
plan1
plan2
plan3
1 3 5 7 9 11 13 15 17 19
16003600560076009600
Safety Scores vs Resource Increase
No of Resources
SafetyScore
CP1EX2
EX1EX2
CP1CP2
5 15 25 35 45 55 65 75 85 95
1200280044006000
Safety Scores vs Speed Increase
Speed (MPH)
SafetyScore
Plan1
Plan2
Plan3

Summary of Thrust-3
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
60 / 70
Generate several alternative project plans.
Schedule them using Discrete Event Simulation (DEVS).
Calculate safe trajectories for robots and humans.
Estimate the safety score and generate risk heatmaps.
Select the optimal plan considering safety and project duration.

Thrust-4: Multi-Optimal Motion
Planning
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
61 / 70

Trajectory Generation for Robots
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
62 / 70
Motion Planning: We want to calculate a feasible trajectory from start
to goal obeying differential constraint (e.g. cannot move sidewise) and
avoiding all obstacles.
Multiple Objectives: Optimize path length, travel time, fuel
consumption, maximize safety and avoid adversaries.

Choose Parent and Rewire RRT* Tree
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
63 / 70
Start a tree from the start state xI.
Expands by sampling random points in free space.
Connecting the new node to the nearest one.
Attempt to correct neighboring connection.
Calculate a path that optimizes multiple cost vector L.

Cooperative Tree Expansion
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
64 / 70
Two cooperative trees Tu and Tv expand in parallel while affecting each other.
lc : X × X → {0, 1} checks whether the two newly sampled vertices xu, xv from
the two trees, Tu and Tv cooperate.
We assign one cost vector L = {l1, l2, . . . , ln} to each state xu ∈ X and xv ∈ X
of the random trees.
Reward: If they cooperate, ωk : X × X → R≥0
and
lk(xu) = lk(xu) − ωk(xu, xv)
lk(xv) = lk(xv) − ωk(xu, xv).
Penalty: If they don’t cooperate, ρk : X × X → R≥0
and
lk(xu) = lk(xu) + ρk(xu, xv)
lk(xv) = lk(xv) + ρk(xu, xv).

Sample Multiobjective Path Generation
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
65 / 70
RRT* Multi-RRT* Multi-RRT* (two landmarks)
Multi-RRT* (5000 ierations) Multiple Vehicles Avoiding Adversaries

Future Research Direction
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
66 / 70
Communication Relay: a) 3D Relay placement can be computed
with minimal modiﬁcation. b) Use other vehicles in an outdoor setup to
test the impact of different motion dynamics on signal strength.
LoS Robot Network: a) Solution in a partial known map is feasible
using gap navigation and shadow exploration techniques. b) A
feedback based planner can be used to collect the information about
the units that intend to go out of sight.
Construction Project: a) Stochastic nature of the workplace instead
of our deterministic model. b) The proposed system can also be used
for employee training in other complex workplaces such as
manufacturing and product assembly line.
Muli-Objective Path Planning: a) Our ideas can be adapted to other
sampling-based motion planners such as RRT, PRM and PRM*. b)
Gradient descent method can be considered that will try to bias the
trajectories towards the speciﬁc goal functions.

Publications
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
67 / 70
Journal Paper:
[ 1 ] An Automated Methodology for Worker Path Generation and Safety Assessment in Construction
Projects. Rahman, M., Carmenate, T., Bobadilla, L., Zanlongo, S. and Mostafavi, A. IEEE Transaction
on Automation Science (T-ASE), 2016.
Conference Papers:
[ 2 ] Sampling-Based Planning Algorithms for Multi-Objective Missions, Rahman, M., Bobadilla, L.,
and Rapp, B., IEEE CASE, 2016, Fort Worth, TX, USA.
[ 3 ] Establishing Line-of-Sight Communication Via Autonomous Relay Vehicles, Rahman, M.,
Bobadilla, L., and Rapp, B. , 2016 IEEE MILCOM, Baltimore, MD, USA.
[ 4 ] A Coupled Discrete-Event and Motion Planning Methodology for Automated Safety Assessment
in Construction, Rahman, M., Carmenate, T., Bobadilla, L., Zanlongo, S. and Mostafavi, A. 2015. IEEE
ICRA, 2015, Seattle, WA, USA
[ 5 ] Ex-Ante Assessment of Struck-by Safety Hazards in Construction Projects: A Motion Planning
Approach, Rahman, M., Carmenate, T., Bobadilla, L., and Mostafavi, A., 2014 IEEE CASE, Taipei,
Taiwan
[ 6 ] Modeling and Analyzing Occupant Behaviors in Building Energy Analysis Using an Information
Space Approach. T Carmenate, M Rahman, D Lenate, L Bobadilla, A Mostafavi, IEEE CASE 2015.
[ 7 ] Multi-Robot Planning for Non-Overlapping Operator Attention Allocation. S Zanlongo, M M
Rahman, F Abodo and L Bobadilla. IEEE Robotic Computing, Taiwan, 2017.

Patent, Award and Other Publications
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
68 / 70
US Patent:
[ 1 ] US Patent No. 15/033,778 for Context based algorithmic framework for identifying and classifying
embedded images of follicle units, M Rahman, S S Iyengar, W Zeng.
Award:
[ 1 ] Florida International University’s Dissertation Year Fellowship award.
Under Review:
[ 1 ] Optimal Placement and Patrolling of Autonomous Vehicles in Visibility-Based Robot Networks.
Rahman, M., Bobadilla, L., Abodo, F. and Rapp, B. IEEE Transaction on Robotics (T-RO).
[ 2 ] Relay Vehicle Formations for Optimizing Communication Quality in Robot Networks, Rahman, M.,
Abodo, F., Bobadilla, L., and Rapp, B., IEEE R-AL, 2017.
[ 3 ] Multi-Vehicle Patrolling with Limited Visibility and Communication Constraints, Alam, T., Rahman,
M., Bobadilla, L., and Shell, D., IEEE MILCOM, 2017.
Other Accepted Paper and Poster:
[ 1 ] Context based algorithmic framework for identifying and classifying embedded images of follicle
units, M Rahman, S S Iyengar, W Zeng, SPIE Medical Imaging, 2014, San Diego, USA.
[ 2 ] Poster: Hybrid Motion Planning/Discrete-Event Simulation Approach Construction Safety
Planning, 5th Workshop on Formal Methods for Robotics and Automation, RSS 2014, San Francisco.

Acknowledgment
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
69 / 70
Dr. Leonardo Bobadilla: Continuous support and great advice.
Dr. Bogdan, Dr. Ali, Dr. Ning and Dr. Zeng: For their time, valuable
feedback and recommendation letter.
Sebastian, Tauhid, and Greg: Many research ideas and they
constructive feedback.
Triana, Franklin: Experiments and simulation works.
Carlos and Olga: For administrative assistance and advice.
⇒ Florida International University’s Graduate School: Support with the
Dissertation Year Fellowship award.
⇒ US Army Research Lab: Funding a number of my research projects.

Thank you!
Introduction
Thrust-1:
Communication Relay
Thrust-2: LoS Based
Resilient Robot
Networks
Thrust-3:
Communication Aware
Safe Planning
Motion Planning
70 / 70
Thank you very much!

Efficient mission planning in communication constrained environment

More Related Content

What's hot (18)

Similar to Efficient mission planning in communication constrained environment (20)

Recently uploaded (20)

Efficient mission planning in communication constrained environment