SlideShare a Scribd company logo
Received July 29, 2016, accepted August 26, 2016, date of publication September 7, 2016, date of current version September 28, 2016.
Digital Object Identifier 10.1109/ACCESS.2016.2606501
Towards Low Cost Prototyping of Mobile
Opportunistic Disconnection Tolerant
Networks and Systems
MILENA RADENKOVIC1, JON CROWCROFT2, AND MUBASHIR HUSAIN REHMANI3
1School of Computer Science, The University of Nottingham, Nottingham, NG7 2RD, U.K.
2Computer Laboratory, University of Cambridge, Cambridge, CB2 1TN, U.K.
3COMSATS Institute of Information Technology, Islamabad 45550, Pakistan
Corresponding author: M. Radenkovic (mvr@cs.nott.ac.uk)
This work was supported in part by the Project Health Monitoring and Life-Long Capability Management for SELf-SUStaining
Manufacturing Systems through the Commission of the European Communities under the 7th Framework Programme, under Grant 609382.
ABSTRACT Fast emerging mobile edge computing, mobile clouds, Internet of Things, and cyber physical
systems require many novel realistic real-time multi-layer algorithms for a wide range of domains, such
as intelligent content provision and processing, smart transport, smart manufacturing systems, and mobile
end-user applications. This paper proposes a low-cost open source platform, MODiToNeS, which uses
commodity hardware to support prototyping and testing of fully distributed multi-layer complex algorithms
over real-world (or pseudoreal) traces. MODiToNeS platform is generic and comprises multiple interfaces
that allow real-time topology and mobility control, deployment and analysis of different self-organized
and self-adaptive routing algorithms, real-time content processing, and real-time environment sensing
with predictive analytics. Our platform also allows rich interactivity with the user. We show deployment
and analysis of two vastly different complex networking systems: a fault and disconnection-aware smart
manufacturing sensor network and cognitive privacy for personal clouds. We show that our platform design
can integrate both contexts transparently and organically and allows a wide range of analysis.
INDEX TERMS Disruption tolerant networking, mobile ad hoc networks, prototypes, wireless
communication, wireless sensor networks.
I. INTRODUCTION
Over the recent years there has been a growing interest in
designing and testing novel mobile wireless and opportunistic
network communication protocols and systems for a wide
range of vastly different application scenarios, such as smart
manufacturing, mobile social networks and smart wellbeing
domains. Researchers increasingly aim to test their novel net-
work architectures and protocols under realistic constraints
after initially optimising theoretical models. Newly emerging
services and applications for Internet of Things (IoTs) and
Cyber Physical Systems (CPSs) require development of new
intelligent communication, storage and processing architec-
tures. We propose a novel platform where IoT ubiquitous
devices can host services and communicate in peer-to-peer
manner via adaptive mobile delay/disconnection tolerant
opportunistic networks. This paper describes a novel multi-
layer intelligent Mobile Opportunistic and Disconnection
Tolerant Networking (MODiToNeS) platform that supports
various mobility and connectivity patterns, adaptive
communication protocols, and a wide range of smart algo-
rithms for intelligent content processing. We argue that our
platform can be used to help research community test highly
complex self-organised cognitive distributed protocols and
architectures as well as serve as an educational resource
which uses open source software, low cost hardware, simple
control interfaces and modelling structures. We believe that
MODiToNeS will help advance the research and educational
opportunities available to the cognitive DTN and oppor-
tunistic network communication communities by providing
a ‘‘real-world’’ fully distributed platform where researchers
and students can develop and test their cognitive protocols
and applications while being able to observe how they behave
in a real world hybrid (wireless and wired, mobile and
static) environments. We argue that it is very important to
allow researchers to validate their core assumptions and
hypothesis made when proposing new complex algorithms
and systems as early as possible to avoid building inaccurate
and unusable protocols. MODiToNeS allows us to tackle
VOLUME 4, 2016
2169-3536 
 2016 IEEE. Translations and content mining are permitted for academic research only.
Personal use is also permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
5309
M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems
exactly that through incremental or evolutionary prototyping.
Using simple open source interfaces, researchers are able
to rapidly develop distributed algorithms and deploy them
in a real environment saving time in developing component
evaluation and simulation techniques while providing them
with valid feedback about how their components interacted
with different kinds of dynamic environments. Additional
advantages of using MODiToNeS are multifold and include:
first, low cost which refers to the total cost of the platform
ownership being less than the cost of a mid-high end laptop
(around £2000), and second the platform being fundamental
to performing early feasibility tests and quick prototyping
before embarking in complex simulations (e.g. NS-3 [29] or
Mininet [30]).
In this paper, we describe two different example scenar-
ios prototyped in MODiToNeS: first, smart manufacturing
Fault Aware and Disconnection aware communication smart
sensors protocol (FDASS) [10]); and second, privacy aware
mobile personal clouds (CogPriv) [4]. More specifically, we
show how our open-source distributed platform allows build-
ing, deploying and testing of 1) fault aware and disconnection
aware framework prototyping and testing for smart manufac-
turing and 2) adaptive mobile privacy aware personal clouds
prototyping and testing when sharing various kinds of data via
different routing protocols via networks with different levels
of privacy leakage.
The paper is organised as follows. Section 2 reviews related
work on state of the art testbeds for smart data communi-
cation in mobile and wireless networks. Section 3 proposes
the multi-layer architecture of our MODiToNeS platform,
introduces its key control planes and describes support for
cross layer data communication that can on-the-fly adapt to
dynamic link properties and changing requirements of the
users. Section 4 describes smart manufacturing opportunis-
tic and disconnection tolerant sensor network architecture
scenario prototyping with smart adaptive routing protocols
such as FDASS [10]. Section 5 describes peer to peer mobile
clouds scenario which deploys smart protocols [4] for data
sharing, monitoring and interaction. We show that both
CogPriv and FDASS outperform other competitive and
benchmark protocols across a range of metrics in line with
previously done simulation based work in [4] and [10].
In addition, we evaluate realistic resource costs for FDASS
and CogPriv across a range of resource metrics in real time.
Section 6 gives summary and future work directions.
II. RELATED WORK
As mobile edge computing and cognitive networks are still
emerging filed, there are limited simulation and testbed
environments which allow prototyping and testing of new
emerging applications, protocols and services. We review a
range of state of the art testbeds for wireless networks and
applications, and identify how our proposal defers from each
one of them. Similarly, the majority of the current simulator
implementations have limited number of control features as
they are based on the basic wireless sensor networks and
communication protocols. In this paper, we propose a novel
multilayer platform that uses low cost smart devices (e.g.
such as Raspberry PIs) to prototype rich set of intelligent
and interactive complex communication algorithms and dis-
tributed network architectures.
Radenkovic and Milic-Frayling [6] propose the design and
architecture for a low cost light weight testbed for a Personal
Cloud based on Raspberry Pi with a range of sensors (RasPiP-
Cloud). RasPiPCloud supports multiple on demand virtual
containers to host different services and applications that
can collect, store and share data with varying different lev-
els of privacy. RasPiPCloud utilizes opportunistic networks
communication to communicate with the heterogeneous
sensors and other devices. RasPiPCloud can have multi-
ple containers [5] such as: Healthcare, Finance, and Social
Network with additional container template ready for rapid
on demand deployment. Each container gets installed and
runs its purpose specific applications to ensure secure data
fencing and protection. Radenkovic and Milic-Frayling [6]
do not describe the support for multi-user communication.
This paper focuses on multi clouds communication support
in MODiToNeS.
Romero et al. [13] propose cognitive testbed for wireless
sensor networks as an emerging technology with a potential
to avoid traditional wireless problems such as reliability,
interferences and spectrum scarcity in wireless sensor net-
works. Romero et al. [13] argue that cognitive wireless sensor
networks testbeds are an important tool for future develop-
ments, protocol strategy testing and algorithm optimization
in real scenarios. This paper focuses on sparse and potentially
disconnected topologies in addition to large dense topologies.
State of the art work in [16] proposes Haystack system which
aims to allow unobtrusive and comprehensive monitoring of
network communications on mobile phones entirely from
user space. Haystack correlates disparate contextual informa-
tion to illuminate mobile phone app performance, privacy and
security. While Haystack runs locally on a user’s phone and
can provide highly useful real world data traces that we can
use in our platform, our platform is fully distributed and can
run different applications and contexts.
TKN Wireless Indoor Sensor network Test-
bed (TWIST) [22], developed by the TKN at the TU Berlin,
is one of the largest academic testbeds for experimenting
with WSN applications at indoor deployment scenarios.
It provides basic services like node configuration, network-
wide programming, out-of-band extraction of debug data
and gathering of application data. It also presents several
novel features such as active control of the power supply
of the nodes. The testbed in [11] uses setup which con-
sists of 102 TmoteSky nodes operating at 2.4 GHz and
102 eyesIFX nodes at 868 MHz resulting in a fairly regu-
lar grid deployment pattern with an inter-node distance of
3 m. The Virginia Tech COgnitive Radio NEtwork Testbed
(VT-CORNET) [21] is a collection of cognitive nodes
deployed in a building on the Virginia Tech campus. The
testbed consists of a total of 48 static SDR nodes based on
5310 VOLUME 4, 2016
M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems
USRP210, located at the ceiling. In addition to the static
nodes, low-power mobile nodes are also available in order
to provide an environment that accommodates a wide variety
of research topics. All devices used in this testbed are based
on SDR and are not suitable for WSNs because of their high
power consumption. Despite their possibilities for frequency
mobility, the solution implemented by this test-bed does not
support CWSN implementation. Both [21] and [22] focus
on the network layer communications and do not consider
other IoT (middleware) layer and different application and
data types which MODiToNeS includes.
While there has been extensive research and standard-
isation work being done in the areas of verification and
validation for product lifecycle for different application
areas [34], [35], in our paper we follow general guidelines
for the physical prototyping which has been identified as an
open research problem for the intelligent mobile opportunis-
tic research community and complex network protocols and
systems they aim to propose [8], [34]. Physical testing is
still an expected industry practice, frequently linked to prod-
uct certification. Moreover, as we target complex systems
modelled with complex temporal networks assuming likely
loss of connectivity and data, physical prototyping generates
valuable knowledge and data that can be utilised to enhance
the design of future products or variants.
III. MOBILE OPPORTUNISTIC DELAY/DISCONNECTION
TOLERANT NETWORKS AND SYSTEMS
PLATFORM (MODiToNeS)
A. OVERVIEW OF THE MODiToNeS PLATFORM
We propose a novel platform, MODiTOnNeS, which is highly
suitable for fast prototyping of applications that are dis-
tributed, cognitive (context-aware), intelligent (able to use
various on demand self-organised and adaptive routing and
machine learning algorithms), interactive and driven by real
world connectivity and application traces. MODiToNsS plat-
form contains five programmable layers for dynamic and
on demand control including: 1) cognitive/smart hardware
devices which are able to store and process real time data and
are equipped with different heterogeneous sensors and differ-
ent types of types of communication interfaces; 2) topology
control plane to enable rich diversity of mobile and fixed
network topologies; 3) control plane for enabling different
intelligent routing protocols; 4) control plane to enable dif-
ferent real time analytics and machine learning protocols
which are suitable for different applications and 5) interac-
tive real time user dashboard to allow user interaction and
notifications for different application types. This architecture
is shown in Figure 1. We argue that it is important to enable
different control interfaces for different layers in order to
enable a more complete and useful platform that promotes
opportunistic disconnection tolerant networking and mobile
edge computing research which is fundamental for pervasive
computing, IoTs and CPSs research and services.
While MODiToNeS draws inspiration from the work such
as Castalia [2], it focuses on different set of properties as
FIGURE 1. Overview of the layered architecture of MODiToNeS.
we target cross layer design, opportunistic smart communi-
cation protocols in challenged networks and enabling high
level analysis and visualisation accessible to the user or at
the edges. We provide a modular and simple open source
implementation (inspired by ONE [18]) for topology con-
trol and monitoring, resources monitoring and analysis and
data monitoring and visualisation. Each of our smart nodes
has multiple communication interfaces which can be pro-
grammed dynamically with different parameters to start or
stop as well as to control and monitor different wired or
wireless active channels which communicate different types
of sensor and user application data. MODiToNeS provides the
developer with simple open source functionality to change the
default interface used to send data on demand as well as to
the change the active channel on the fly. In addition, our plat-
form allows running of different real world and connectivity
traces in real time which can be mobile, static or hybrid net-
works. We assume that connectivity traces are in accordance
with the syntax of the StandardEventsReader format used
in ONE. MODiToNeS also allows programmable topologies
where the user can program the dynamic connectivity among
the nodes. Our approach allows testing of various protocols
against multiple real world conditions by allowing real world
topology information retrieval which is fed to the platform
in order to mirror any real world topologies. We enable
VOLUME 4, 2016 5311
M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems
automated reconfiguration of the platform such as experi-
ment repetitions with controlled parameter changing so that
our approaches can be tested against multiple real-life con-
ditions. We also allow a user to in a simple way deploy
different routing algorithms on the distributed MODiToNeS
nodes to support various routing behaviour and topologies.
Overview of the layered architecture of the platform is shown
in Figure 1.
Each MODiToNeS node is a low cost single-board com-
puter (Raspberry Pi model B) which provides good process-
ing power, flexible storage, and has good software support.
It can be integrated with a number of wired and wireless sen-
sors using GPIO, I2C, RF modules, 802.11 Wifi, Bluetooth
and USB. We currently have over 80 Raspberry Pi nodes.
Figure 2 shows one Raspberry PI node which uses on-board
Ethernet port, an 802.11n Wi-Fi dongle for wireless network
connectivity, and uses IBR-DTN [3], [26] to provide P2P
DTN capabilities. Figure 3 shows one hierarchical deploy-
ment of over 20 Raspberry PIs with different low sensors such
as wireless temperature sensor, pressure sensor, magnetome-
ter and 3-axis accelerometer.
FIGURE 2. DTNPi with 801.11 adapter and XRF receiver for wireless
sensors.
FIGURE 3. A hierarchical architecture Raspberry PI sensor network
example.
B. MULTI-LAYER CONTROL PLANES
There is a growing need for generic network platforms that
can combine real-world delay and other challenging network
conditions with the flexibility of simulators to support a
range of different application domains. MODiToNeS enables
significantly lower time between the early prototype and pro-
duction system deployment. Using Raspberry PIs (or similar
hardware) allows having large number of hardware nodes in
a relatively small physical space at a low cost. For example,
influence of mobility on systems performance is complex
and is usually evaluated in simulator environments. Contrary
to this, our MODiToNeS platform enables the integration of
many different distributed mobility patterns.
We have had several diverse projects which benefited from
the design and deployment of a generic MODiToNeS plat-
form that satisfies the following requirements:
• Allowing the coexistence of multiple independent
projects (e.g. nodes 1-17 for Project A, nodes 20-50 for
Project B)
• Allowing experimental orchestration in terms of deter-
mining how many run to have, selecting senders and
receivers, determining messages rates and sizes, deter-
mining which protocols to run.
• Allowing for change of network environment within the
scope of a single project to simulated different network
topologies for different simulation runs (e.g. nodes 1-5
communicate with nodes 6-10, even ID nodes commu-
nicate with odd ID nodes)
• Allowing for real-time change of network topology
emulating DTN [1], [27] or MANET where nodes can
come in and out of contact with each other within the
span of a few seconds.
We address these requirements by proposing real-time
programmable interface for network topology configura-
tion. This control plane is used for fast automated and pro-
grammable configuration of the network plane. The network
layer consists of the head node and variable number of worker
nodes directly connected e.g. via a wired Ethernet switch
(see Figure 3, Figure 4 and Figure 5). The existence of
the head node is fundamental for allowing integrated and
holistic view of the design in order to be able to validate
in an integrated manner. We have designed, developed and
deployed a set of tools on the head node that allow real time
configuration changes, command executions, as well as run-
ning and deploying distributed smart protocol modules (such
as FDASS [10], MWCC [11], CogPriv [4], CafeREP [17],
etc) to all nodes. All tools are based on PERL, C/C++,
PYTHON and IBR-DTN suitable for our low cost hard-
ware (Raspberry PIs). We use open source PNP4Nagios that
visualise RRD files generated by sensor readings and other
time series data. Through combination of deploying differ-
ent routing and content dissemination protocols as well as
dynamic/programmable firewall configurations to all nodes,
the network topology can be changed and certain nodes can
be included or excluded from it. For example configuring a
node firewall to drop all incoming and outgoing packets will
make this node ‘‘invisible’’ to all other nodes. By changing
firewall rules we can achieve any traditional network topolo-
gies (such as tree, star or full mash) or any complex temporal
networks with our platform (by reading connectivity trace
files from http://uk.crawdad.org which is a shared wireless
5312 VOLUME 4, 2016
M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems
FIGURE 4. (a) Algorithm for the control planes in the Master node.
(b) Algorithm for a Worker node.
network data resources for the research community or gen-
erating and capturing similarly formatted connectivity files
from other real work devices). We can also exclude a node
from the sensor topology while a particular experiment is
in mid-run for cases when we want to test a node fault and
disconnection. Similarly, we enable real time configuration
of different topologies across different experiments for vari-
ous prototypes. For static topologies, the configured network
topology remains the same for the duration of the individual
trail. We use a simple format to design and describe the
desired network topology. It describes the tuples connectivity
that comprises the network topology as well as contains the
connection characteristics between the connected tuples as
shown below:
TimeStamp:Address1:Address2:<UP|DOWN>
[,RATE,DELAY,LOSS]
An example connectivity line describing only connectivity
is given:
0:10.0.10.2:10.0.10.17:UP
FIGURE 5. MODiTONeS topology used in the performance tests for
manufacturing scenario.
This executes the below firewall and traffic rate configura-
tion commands:
For Node 10.0.10.2
iptables -A INPUT -s 10.0.10.17 -j ACCEPT
iptables -A OUTPUT -d 10.0.10.17 -j ACCEPT
For Node 10.0.10.17:
iptables -A INPUT -s 10.0.10.2 -j ACCEPT
iptables -A OUTPUT -d 10.0.10.2 -j ACCEPT
An example connectivity line describing connectivity with
network characteristics is given below:
0:10.0.10.2:10.0.10.17:UP:rate 2Mbit,delay
150ms,loss 7%
For Node 10.0.10.2:
iptables -A INPUT -s 10.0.10.17 -j ACCEPT
iptables -A OUTPUT -d 10.0.10.17 -j ACCEPT
iptables -t mangle -A POSTROUTING -d
10.0.10.2 -j CLASSIFY -set-class 1:100
tc class add dev eth0 parent 1: classid
1:100 htb rate 2Mbit
tc qdisc add dev eth0 parent 1:100 handle
100: netem delay 150ms loss 3%
For Node 10.0.10.17:
iptables -A INPUT -s 10.0.10.2 -j ACCEPT
iptables -A OUTPUT -d 10.0.10.2 -j ACCEPT
iptables -t mangle -A POSTROUTING -d
10.0.10.2 -j CLASSIFY -set-class 1:100
tc class add dev eth0 parent 1: classid
1:100 htb rate 10Mbit
tc qdisc add dev eth0 parent 1:100 handle
100: netem delay 150ms loss 3%
In addition to static topologies, MODiToNEs also
supports dynamic topology configuration when the con-
figured network topology is expected to change within
the duration of the individual trail. We can use this to
emulate DTN [1] and MANET network environments
VOLUME 4, 2016 5313
M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems
and run prototype experiments with real-world connectivity
and data dissemination traces like Infocom [19], Rollernet
[7], [9], [28] etc. We provide a mechanism that allows us to
update the network configuration by using the above men-
tioned connectivity/network topology format where entries
reflect changes in chronological order:
TimeStamp:Address1:Address2:
<UP|DOWN> [,RATE,DELAY,LOSS]
7:10.0.10.2:10.0.10.17:UP
128:10.0.10.2:10.0.10.17:DOWN
or
14:10.0.10.2:10.0.10.17:UP:rate 2Mbit,
delay 150ms,loss 7%
58:10.0.10.2:10.0.10.17:DOWN
Figures 4a and 4b give an overview of the pseudo code
of the master MODiToNeS node and distributed working
MODiToNeS nodes respectively.
IV. SMART MANUFACTURING SCENARIOS
A. PROTOTYPING FAULT AND DISCONNECTION AWARE
SMART MANUFACTURING
We describe the hardware, software and algorithms we use
to prototype smart manufacturing sensor network that we
have used in smart manufacturing project EU Selsus [12] and
facilitate real world deployment of complex architectures and
communication protocols.
We describe the design and implementation for a sensor
network prototype in MODiToNEs that can reproduce a pro-
duction floor sensor network environment and emulate vari-
ous sensor network topologies and communication patterns
that we then integrate within the EU SelSus project [12].
We assume that we have smart sensing nodes which oper-
ate as MODiToNeS nodes and provide sensing, compu-
tation, storage and communication together with allowing
self-configuration, fault tolerance and self-monitoring. The
MODiToNeS platform allows development and evaluation
of different novel and benchmark protocols FDASS [10],
Prophet [24] and Epidemic/Flooding protocols [23] intended
for use with smart sensors.
A common and widely used production environment
monitoring sensor network topology is a tree topology of
depth 2 – the lowest layer of nodes including heteroge-
neous sensor nodes, the middle layer including gateways/
aggregators/processors and the top layer referring to the
head/cloud/master node [30]–[34]. We build and demon-
strate a logical depth 2 tree sensor network topology in
MODiToNes comprising of four sensor nodes, two aggrega-
tor nodes and one central cloud node (shown in Figure 5).
The MODiToNeS sensor nodes are equipped with a range
of sensors including directly attached camera sensor, MEMS
Sensor Evaluation Board with Low power 3D magnetometer,
3-Axis digital accelerometer, temperature / high precision
pressure sensor, and a remote ANT+ protocols compatible
sensor. The MODiToNeS sensor nodes are unable to detect
each other’s’ presence in the network and are only able to
communicate with the two MODiToNeS aggregator nodes.
Each MODiToNeS aggregator node is able to communi-
cate with the all sensors nodes and the central MODiToNeS
cloud node. The MODiToNeS aggregators are also unable
to detect each other’s presence on the network. The cen-
tral MODiToNeS node is able to communicate with any of
the MODiToNeS aggregators. We assume that, during nor-
mal operation of the MODiToNeS sensor node, it captures
their corresponding sensors’ readings as well its real time
local resources utilisation (including CPU load, memory,
disk usage, I/O) at configured time intervals. Each node is
able to store the measurements locally to be available for
localised queries and also generates simple format messages
with sensor measurements which are sent to the central cloud
node. The only neighbours that any MODiToNeS sensor node
detects are the two MODiToNeS aggregators. The individual
MODiToNeS sensor nodes can transmit their sensor mea-
surements messages to any of the two MODiToNeS aggre-
gators but in normal operation they ‘‘prefer’’ their local
MODiToNeS aggregator. When a MODiToNeS aggregator
receives a MODiToNeS sensor reading message, it stores
the sensor measurements locally to be available for localised
query and also forwards the messages directly onto the
central MODiToNeS cloud node. The MODiToNeS aggre-
gator nodes also forward resource measurements to the
MODiToNeS cloud node in the same way the MODiToNeS
sensor nodes do. Each MODiToNeS aggregator stores the
sensor measurements of its belonging sensor nodes and can
provide them if queried locally or remotely. In this way, the
central MODiToNeS cloud node receives measurements from
all MODiToNeS sensors within the sensor network including
resource utilisation readings (Fig 6 and Fig 7). All sensor
readings are being stored in RRD format as this format is well
suited for time-series data like network bandwidth, tempera-
tures, CPU load, etc. The data are stored in a circular buffer
based database, thus the system storage footprint remains
constant over time. Note that this is distinct from the tradi-
tional concept of round-robin scheduling. Readings for sep-
arate sensors get allocated their individual RRD databases.
Each Raspberry Pi node is running a web service that allows
real-time queries of sensor states and reading as well as
historical information and visualisation via PNP4Nagios.
Figure 6 and Figure 7 show long term file system utilisation
and long term memory utilisation for a MODiToNeS node.
We observe that memory utilisation is firmly below the full
FIGURE 6. Long term file system utilisation for a MODiToNeS node.
5314 VOLUME 4, 2016
M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems
FIGURE 7. Long term memory utilisation for a MODiToNeS node.
utilisation and that the file system is not over-utilised as it gets
monthly archiving of experiment log to external long term
storage.
B. ROUTING PROTOCOL EVALUATION
In order to better understand simulated performance of
fault aware disconnection tolerant smart sensor network
protocol (FDASS) [10], we prototype and test FDASS against
Flooding and Prophet protocols in MODiToNeS. We have
developed an intelligent framework that aims to improve reli-
ability of the manufacturing plant in the face of varying net-
work connectivity and non-uniform distribution of different
types of faults in the network. Fault and Disconnection Aware
Smart Sensing framework (FDASS) [10] is able to detect and
identify misperforming nodes in a fully distributed fashion
in order to isolate them, reroute the traffic away from them
and notify the sinks about the type, location and time of the
failures. FDASS builds on and extends multi-path transport
approaches to combine fault analytics layer with the complex
network topology and resources analytics layer into a com-
plex heterogeneous network for manufacturing environments
as shown below. As all MODiToNeS nodes run IBR-DTN
on Raspberry PIs, we implemented the FDASS protocol as
a IBR-DTN routing component written in C++. IBR-DTN
also includes other benchmark protocols (such as Epidemic
and Prophet). This allows direct performance comparison
between different protocols in a real world environment with
MODiToNeS.
Each MODiToNeS node was augmented with a sensor
simulator capable of generating varying numbers of pseudo
realistic sensor readings on demand. The simulated sensor
readings were chosen over real sensors to increase the diver-
sity of sensing ranges and frequencies [31]–[33] compared to
the available low cost sensor types we have. The simulated
sensor readings were padded to 100 bytes to ensure each
reading had a consistent size. Each sensor reading was also
timestamped as it was taken with millisecond precision. The
head node also timestamped the bundles, with millisecond
precision, as they were received. These timestamps were used
to calculate the time taken for the bundle to propagate through
the network.
Each experimental prototype run lasted 60 minutes with
each sensor being polled once every second. Between each
run the number of sensors per sensor node was increased by
one until ten sensors per node was reached. All nodes were
FIGURE 8. (a) Delivery success with increasing numbers of sensors.
(b) Delivery delays with increasing numbers of sensors.
rebooted between each run to ensure the nodes were in a
known state. Each experiment was repeated three times and
the averages of these three runs were recorded.
Figure 8a shows the recorded bundle delivery success rate
achieved and Figure 8b shows the bundle delivery delay
observed as the number of sensors per sensor node increases
from one to ten. Figures 8a and 8b show that FDASS outper-
forms both the Flooding and Prophet protocols.
Figure 9 and 10 demonstrate FDASS robust functionality
in MODiToNeS by emulating faults of the MODiToNeS
aggregators. Consider MODiToNeS Aggregator 2 fails first
by losing network connectivity. We observe in the Figure 9
VOLUME 4, 2016 5315
M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems
FIGURE 9. View of two MODiToNeS aggregators messages during failure
disconnections and recovery.
FIGURE 10. Centralized view of messages hops, delays, and delivery
success.
that all sensor messages get redirected via MODiToNeS
Aggregator 1. After a few minutes, MODiToNeS Aggre-
gator 1 loses network connectivity as well. At this stage
there is no route between the MODiToNeS sensor nodes and
the central MODiToNeS node. During this time, all sensor
readings are being stored by the sensor nodes where they
are generated. After another few minutes both MODiToNeS
Aggregators recover their connectivity and we observe the
peak in traffic generated due to the instantaneous delivery of
all stored sensor readings.
We aim to deploy MODiToNeS Raspberry PI nodes with
FDASS in the real world manufacturing shop floors to enable
further validation with real users and improve reliability of
diverse factory communications.
V. MOBILE CLOUDS SCENARIO
A. PROTOTYPING MOBILE CLOUDS OVERVIEW
As an example of a different architecture that can be built and
tested in MODiToNeS platform, we describe the design of
predictive mobile clouds where mobile sensing and real time
predictive analytics algorithms are incorporated in dynamic
mobile clusters of MODiToNeS nodes. Each smart
MODiToNeS platform node allows intelligent real time deci-
sion making that can predict (and change) the behaviour
of the network communication of itself and other nodes’.
Even though machine learning and analytics techniques have
been widely recognised as important for context prediction
in mobile computing and many theoretical and simulation
based works exist, real world implementations are still scarce
and remain interesting future research challenge [8]. The
mobile cloud (MC) prototype over MODiToNeS example
we describe here supports new paradigm shift that combines
anticipatory systems [8] and adaptive collaborative propos-
als [e.g. 17,20] where computer devices base their actions
on the predictive models of themselves, the environment
and the other nodes. MODiToNeS support consideration of
multiple criteria including different complex temporal graphs
centrality predictions as well as resource, movement and
behaviour predictions.
We view mobile clouds (MC) as new approach that bridges
the gap between the device(s), environments and the user.
In MODiToNeS platform, the prototype of each MC is
equipped with a range of sensors (accelerometer, gyroscope,
temperature, pressure, heart rate sensor) that can sense the
environment and monitor the context, as well as run real time
predictive analytics (or other machine learning) algorithms
to develop models that predict occurrences of various events.
Our MODiToNeS MC also allows rich real time interaction
with the users as well as sharing among MCs over different
intelligent protocols, different applications and data types.
Additionally, each MODiToNeS MC is able to interact with
the environment and can adaptively change its behaviour in
different situations.
Of particular interest in this platform is to investigate the
performance characteristics of our MC smart data commu-
nication algorithms in the face of different users’ require-
ments for privacy in different contexts. In [11], we describe a
Mobile Wellbeing Cloud Companion (MWCC) testbed pro-
totype which is able to continuously process accelerometer
and gyroscope from the physical environment and process
the readings using various machine learning algorithms to
identify several user activity features. These are analysed in
real time and correlated with the heart rate signals to identify
if the heart rate is normal or not for the current user activity.
In [4], we proposed CogPriv that explored through simula-
tions how different levels of privacy can be supported via
adaptively changing network connectivity in both sparse and
dense topologies. In this paper, we build CogPriv prototype
in MODiToNeS and test it both in terms of quality of the
experience metrics (such as achieved end-to-end privacy and
delays) and the quality of service metrics (such as memory,
I/O, CPU with resource limited devices.).
CogPriv considers users who may be running a social
network that allows them to stay in contact with their friends
at the same time as regularly monitoring their long term
medical condition and being in contact with the hospital.
5316 VOLUME 4, 2016
M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems
These two types of applications have different privacy
requirements and need their data to be stored and shared
in different ways in order to adapt to each required privacy
requirements dynamically. Figure 11 shows example sensor
integration for mobile personal cloud (MPC) prototype in
MODiToNeS on a Raspberry Pi with an Xtrinsic sensor board
with temperature, pressure, and acceleration sensors.
Figure 12 shows MODiToNeS Raspberry PI device that cap-
tures, stores and processes a range of user and environment
data such as heart rate and pedometer.
FIGURE 11. MODiToNeS Raspbery Pi B with a Xtrinsic sensor board and a
WiPi wireless adapter.
FIGURE 12. MODiToNeS Raspberry Pi with Suunto and WiPi USB module,
Garmin heartrate sensor and a smaprtphone displaying readings.
CogPriv in MODiToNeS extends the bundle protocol based
on RFC 5050 [25], [26] that provides API for DTN applica-
tions to exchange and route bundles among distributed nodes
in an intelligent P2P manner. CogPriv P2P DTN (IBR-DTN)
module in MODiToNeS provides multi flow real time bundle
forwarding based on a range of criteria such as source ID,
Virtual Machine (VM) ID, application privacy requirements,
destination ID so that different incoming bundles can be
matched to the appropriate network interface in real time.
At its core, CogPriv comprises multiple stages: it probes
local cellular network to identify the likelihood of any middle
boxes that may compromise user traffic, requests the remote
destination nodes to provide their estimations of the cellu-
lar network privacy levels, and collaborates and cooperates
with the local network nodes to determine the best local
next hop. CogPriv routing protocol can range dynamically
and adaptively from providing fully cellular single hop end
to end communication to fully localised multi hop mobile
opportunistic communication. Through collaborations and
cooperation in the local neighbourhoods, each node can
understand its environment and neighbours better. More
specifically, each CogPriv MODiToNeS node exchanges
their own cellular network privacy statistics and predictions
to negotiate feasibility of using cellular network for the par-
ticular application, analytics of their own resource predic-
tions and social connectivity analytics. Note that both social
connectivity traces and middle boxes information are fed to
the MODiToNeS master node from external real world traces
(e.g. utilising http://uk.crawdad.org/). In this paper, we show
measured achieved end-to-end privacy, end-to-end delays,
end-to-end number of hops and transitions, I/O, memory and
CPU costs. Each CogPriv MODiToNeS node privacy level is
important to consider as it is the core criteria for forwarding
the data and deciding on the next hop and via which interface.
More detailed description of CogPriv Decision Algorithm is
described in [4].
B. COGNITIVE PRIVACY EXPERIMENT SCENARIO
We carry out evaluation of CogPriv in MODiToNeS against
fully cellular communication and fully local social oppor-
tunistic networks across a range of different network condi-
tions and user traffic types using a range of metrics. We show
how data can be shared with different levels of privacy in
light of untrusted infrastructure. We use findings identified
in [14] and [15] that show widespread use of transparent
middle boxes such as HTTP and DNS proxies in the cellular
infrastructure which are able to analyse and actively modify
user traffic and thus compromise user privacy and security.
In [4] we provided rich set of simulation based experiments
with real world traces of middle boxes [14], connectivity [7],
interests [7] and friendships [7]. This paper addresses these
scenarios and proposes a way of integrating different layers
within our MODiToNeS platform and exploring how differ-
ent intelligent routing can exploit maximally trusted routes
based on the real time probes and collaboration with the
MODiToNeS nodes that may be infrastructure nodes or fully
ad hoc local nodes based on the local context sensing.
We base our deployment on the real-world data traces of
different probes for mobile networks across 112 countries
and over 200 mobile providers obtained by netalyzr
in [14] and [15]. We select traces from Germany as its number
of mobile networks providers best suits our real world user
communication trace [7]. For every mobile node we obtain
the probability for the network spying on the web traffic
by calculating the percentage of tests returning positive vs
the total number of tests performed. For every mobile
VOLUME 4, 2016 5317
M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems
network, we obtain the probability of it spying on web traffic
by averaging the values obtained by all individual mobile
nodes on this particular network. Based on the real cellular
networks in Germany, we average privacy levels into five
evenly distributed privacy threat levels .e.g. minimum (0%)
such as ALICE and NETZCLUB, low (25%) such as M-NET,
medium (50%) such as BASE, MEDION , high (75%) such
as CONGSTAR, maximum (100%) such as FYVE.
While in our previous work, we developed extensions to
the ONE simulator [18] that utilise this data in order to return
middle boxes presence probability discovered when perform-
ing probing of different cellular networks, in this paper we
feed this data to MODiToNeS to drive different testbed nodes’
behaviour (to act as middle boxes or not). To enable dynamic
real world physical connectivity (and disconnections) among
MODiToNeS platform nodes, we drive the MODiToNeS fire-
wall configuration for each MODiToNeS testbed node with
the of real world Facebook connectivity traces [7] during
the whole time of the experiments. We range the privacy
levels of the data being transmitted starting form maximum
to minimum privacy requirements with three intermediary
levels. We run 5 randomly selected combinations of sources
and receivers for each cellular network privacy level.
FIGURE 13. End-to-end privacy.
1) RESULTS
Figure 13 shows that end to end privacy levels remain higher
for MODiToNeS CogPriv approach than for cellular only and
mobile social ad hoc communication independently of the
level of presence of middle boxes in the cellular infrastructure
i.e. ranging from no middle boxes to wide range of middle
boxes, the performance of cognitive privacy drops from 100%
privacy level to 80%. This is in contrast with the cellular
network which drops end to end privacy linearly with the
amount of the middle boxes in the cellular network.
MODiToNeS CogPriv approach also outperforms fully local
social ad hoc approach because the delays that are associated
with the bundles time out and invoke the nodes to utilise
cellular infrastructure that may have privacy leaks.
FIGURE 14. End-to-end number of hops.
Figure 14 shows statistical analyses of MODiToNeS
CogPriv number of hops with increased number of middle
boxes in the cellular architecture. We observe that the num-
bers range between 1 and 4 across all levels of middle boxes
presence.
Figure 15 shows that MODiToNeS CogPriv delays
increase slowly until the infrastructure is fully compromised
at which point the delays become the same as the local ad
hoc approach. The cellular network approach has the lowest
FIGURE 15. End-to-end delays.
5318 VOLUME 4, 2016
M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems
delays but this is due to privacy being compromised and the
traffic taking single hop (direct) cellular link between the end
nodes.
Figure 15 shows delay distributions for highly private
traffic bundles when the cellular infrastructure contains dra-
matically different amount of middle boxes. We observe
that the delays are the lowest when the infrastructure is not
compromised as the MODiToNeS CogPriv approach takes
cellular single hope router to the destination. As MODiToNeS
CogPriv discovers increasing number of middle boxes in
the cellular networks, the delays will increase but still be
significantly lower than the local ad hoc approach. Even
though there are some bundles that may take up to 27 minutes
until 60% of surveillance of the cellular network over
MODiToNeS, the average still remains low and below
11 minutes. For the cellular network where there is 80%
to 100% of middle box presence, the delays range from
45 minutes to 79 minutes. These sorts of delays are appro-
priate for non-emergency applications where the users value
their privacy and can tolerate delays such as regular daily
checks for users with long-term medical conditions.
FIGURE 16. End-to-end number of transitions.
In Figure 16 we show the number of transitions between
i MODiToNeS nfrastructure and MODiToNeS local ad hoc
when the security of the cellular network decreases. It is
interesting to see that while the number of hops is relatively
low (reaching 4 for highly compromised cellular networks),
up to 50% of these hops are transitions between the infras-
tructure and local communication. This shows that supporting
adaptive transitioning between infrastructure and local com-
munication is highly beneficial.
The previous figures have shown that delays and hop by
hop counts increase as MODiToNeS CogPriv moves adap-
tively from fully cellular mode to the fully opportunistic mode
while managing very high levels of end to end privacy. More
specifically, we show that the MODiToNeS CogPriv achieves
privacy of end to end connections which is almost constant
FIGURE 17. Short and long term node resource utilisation visualisation.
while neither the delays nor the hop count is significantly
increased.
Figure 17 shows short term and long term CPU load,
memory usage and IO usage for MODiToNeS CogPriv nodes.
We observe that, despite complex algorithm and low
VOLUME 4, 2016 5319
M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems
resources devices, MODiToNeS CogPriv memory usage
remains firmly under the full usage. CPU load is in the lower
half of the total CPU utilisation for the majority of time
while IO at the critical level for the majority of time (note
that this critical level has been administratively assigned to
be 2 K/sec).
VI. CONCLUSIONS AND FUTURE WORK
We proposed a novel platform MODiToNeS that supports real
time multi-layer and multi-dimensional communication and
analysis distributed architectures which can combine various
aspects of smart mobile social, transport and other CPS sys-
tems with the particular focus on testing real world novel
reliable and intelligent communications among potentially
low resourced devices.
We envisage increasing need for complex systems of
devices including vehicles, humans and infrastructure. Within
such systems, various communication paradigms need to be
supported including the following: ad hoc communication
among people, among vehicles (vehicle to vehicle), com-
munication between vehicles and infrastructure (vehicles to
road side units and vice versa), human and the vehicle (vehi-
cle notifying and guiding the driver as well as the driver
providing on the fly information that can potentially dif-
fer from the vehicles information) and human and com-
pany/home/hospital (human sharing information about their
trip/health and getting information or instructions back).
In this context, MODiToNeS platform can support the con-
cept of Internet of Things joined with the concept of Internet
of vehicles or mobile social networks representing future
trends of smart transportation and mobility applications. Cur-
rent research and services typically allow central remote real
time monitoring of various information while MODiToNeS
allows users to interact in real time with the prototypes where,
query and add additional information on any unexpected
events. MODiToNeS builds on and extends existing research
to develop a prototype distributed system which allows rich
interactivity with the end user and real time localised ana-
lytics and predictions as well as remote data communication
for non real time analysis. Capturing diverse collection of
information locally (which can include any environment and
context data), providing real time data analysis and predic-
tion which is visualised and fed back to the users is key
for increasing reliability and efficiency of communication
in such environments. We envisage that MODiToNeS will
play an important role when integrating and testing human
behaviour in the design and development of Cyber Physical
Systems in mobile social, mobile health care and vehicular
networks for critical safety applications.
REFERENCES
[1] K. Scott and S. Burleigh, Bundle Protocol Specification,
document RFC 5050, Nov. 2007.
[2] Castalia Wireless Sensor Network Simulator, accessed on Jul. 25 2016.
[Online]. Available: http://castalia.research.nicta.com.au
[3] IBR-DTN, accessed on Jul. 25 2016. [Online]. Available:
https://trac.ibr.cs.tu-bs.de/project-cm-2012-ibrdtn
[4] M. Radenkovic, ‘‘Cognitive privacy for personal clouds,’’ Mobile Inf.
Syst., vol. 2016, Apr. 2016, p. 17, Art. no. 7107103, doi: 10.1155/
2016/7107103.
[5] LXC—Linux Containers, accessed on Jul. 25 2016. [Online]. Available:
https://linuxcontaingers.org
[6] M. Radenkovic and N. Milic-Frayling, ‘‘Demo: RasPiPCloud: A light-
weight mobile personal cloud,’’ in Proc. 10th ACM MobiCom Workshop
Challenged Netw. (CHANTS), New York, NY, USA, 2015, pp. 57–58.
[Online]. Available: http://dx.doi.org/10.1145/2799371.2799373
[7] A. Socievole, F. De Rango, and A. Caputo, ‘‘Wireless contacts, Facebook
friendships and interests: Analysis of a multi-layer social network in an
academic environment,’’ in Proc. IFIP Wireless Days (WD), Nov. 2014,
pp. 1–7.
[8] V. Pejovic and M. Musolesi, ‘‘Anticipatory mobile computing: A survey of
the state of the art and research challenges,’’ ACM Comput. Surv., vol. 47,
no. 3, 2015, Art. no. 47.
[9] J. Leguay and F. Benbadis. (Feb. 2009). CRAWDAD Data Set
Upmc/Rollernet (v.2009-02-02). [Online]. Available: http://crawdad.
cs.dartmouth.edu/upmc/rollernet
[10] M. Radenkovic, I. Kostadinov, and B. Wietrzyk, ‘‘Increasing communica-
tion reliability in manufacturing environments,’’ in Proc. IEEE IWCMC,
Dubrovnik, Croatia, Aug. 2015, pp. 1377–1383.
[11] M. Radenkovic and S. Ha, ‘‘Low-cost mobile personal clouds,’’ in Proc.
IEEE IWCMC, 2016.
[12] EU FP7. SelSus: Health Monitoring and Life-Long Capability Manage-
ment for Self-Sustaining, accessed on Jul. 25 2016. [Online]. Available:
http://www.selsus.eu/
[13] E. Romero, J. Blesa, A. Tena, G. Jara, J. Domingo, and A. Araujo,
‘‘Cognitive test-bed for wireless sensor networks,’’ in Proc. IEEE DYSPAN,
Apr. 2014, pp. 346–349.
[14] N. Vallina-Rodriguez, S. Sundaresan, C. Kreibich, N. Weaver, and
V. Paxson, ‘‘Beyond the radio: Illuminating the higher layers of
mobile networks,’’ in Proc. 13th Annu. Int. Conf. Mobile Syst., Appl.,
Services (MobiSys), New York, NY, USA, 2015, pp. 375–387.
[15] N. Vallina-Rodriguez, S. Sundaresan, C. Kreibich, and V. Paxson, ‘‘Header
enrichment or ISP enrichment?: Emerging privacy threats in mobile net-
works,’’ in Proc. ACM SIGCOMM HotMiddlebox, London, U.K., 2015,
pp. 25–30.
[16] A. Razaghpanah et al. (2015). ‘‘Haystack: A multi-purpose mobile vantage
point in user space.’’ [Online]. Available: https://arxiv.org/abs/1510.01419
[17] M. Radenkovic, ‘‘Andre Grundy: Efficient and adaptive congestion control
for heterogeneous delay-tolerant networks,’’ Ad Hoc Netw., vol. 10, no. 7,
pp. 1322–1345, 2012.
[18] A. Keränen, J. Ott, and T. Kärkkäinen, ‘‘The ONE simulator for
DTN protocol evaluation,’’ in Proc. 2nd Int. Conf. Simulation Tools
Techn. (Simutools), Brussels, Belgium,, 2009, Art. no. 55. [Online]. Avail-
able: http://dx.doi.org/10.4108/ICST.SIMUTOOLS2009.5674
[19] J. Scott, R. Gass, J. Crowcroft, P. Hui, C. Diot, and A. Chaintreau.
(May 2009). CRAWDAD Dataset Cambridge/Haggle (V.2009-05-29).
[Online]. Available: http://crawdad.org/cambridge/haggle/20090529, doi:
10.15783/C70011
[20] M. Radenkovic, A. Benslimane, and D. McAuley, ‘‘Reputation
aware obfuscation for mobile opportunistic networks,’’ IEEE Trans.
Parallel Distrib. Syst., vol. 26, no. 1, pp. 230–240, Jan. 2015,
doi: 10.1109/TPDS.2013.265.
[21] T. Newman, A. He, J. Gaeddert, B. Hilburn, T. Bose, and J. Reed, ‘‘Virginia
tech cognitive radio network testbed and open source cognitive radio
framework,’’ in Proc. 5th Int. Conf. Testbeds Res. Infrastruct. Develop.
Netw. Commun. Workshops (TridentCom), 2009, pp. 1–3.
[22] V. Handziski, A. K¨opke, A. Willig, and A. Wolisz, ‘‘Twist: A scalable
and reconfigurable testbed for wireless indoor experiments with sensor
networks,’’ in Proc. 2nd Int. Workshop Multi-Hop Ad Hoc Netw., Theory
to Reality (REALMAN), 2006, pp. 63–70.
[23] A. Vahdat and D. Becker, ‘‘Epidemic routing for partially connected ad hoc
networks,’’ Citeseer, Tech. Rep.
[24] A. Lindgren, A. Doria, and O. Schelén, ‘‘Probabilistic routing in inter-
mittently connected networks,’’ in Service Assurance With Partial and
Intermittent Resources (Lecture Notes in Computer Science). 2004,
pp. 239–254.
[25] S. Schildt, T. Lorentzen, J. Morgenroth, W.-B. Pöttner, and L. Wolf,
‘‘Free-riding the BitTorrent DHT to improve DTN connectivity,’’ in Proc.
7th ACM Int. Workshop Challenged Netw. (CHANTS), 2012, pp. 9–16.
[Online]. Available: http://dx.doi.org/10.1145/2348616.2348619
5320 VOLUME 4, 2016
M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems
[26] M. Doering, S. Lahde, J. Morgenroth, and L. Wolf, ‘‘IBR-DTN: An effi-
cient implementation for embedded systems,’’ in Proc. 3rd ACM Workshop
Challenged Netw., 2008, pp. 117–120.
[27] K. Fall, ‘‘A delay-tolerant network architecture for challenged
internets,’’ in Proc. Conf. Appl., Technol., Archit., Protocols Comput.
Commun. (SIGCOMM), 2003, pp. 27–34. [Online]. Available:
http://dx.doi.org/10.1145/863955.863960
[28] F. Benbadis, J. Leguay. (Feb. 2009). CRAWDAD Dataset Upmc/Rollernet
(V.2009-02-02), accessed on Jul. 25 2016. [Online]. Available:
http://crawdad.org/upmc/rollernet/20090202, doi: 10.15783/C7ZK53
[29] NS3, accessed on Jul. 25 2016. [Online]. Available:
https://www.nsnam.org/
[30] B. Lantz, B. Heller, and N. McKeown, ‘‘A network in a laptop: Rapid
prototyping for software-defined networks,’’ in Proc. 9th ACM SIGCOMM
Workshop Hot Topics Netw. (HotNets), 2010, Art. no. 19.
[31] A. Bleakie and D. Djurdjanovic, ‘‘Analytical approach to similarity-based
prediction of manufacturing system performance,’’ Comput. Ind., vol. 64,
no. 6, pp. 625–633, Aug. 2013.
[32] M. Hermann, ‘‘Artificial intelligence in monitoring and the mechanics of
machining,’’ Comput. Ind., vol. 14, nos. 1–3, pp. 131–135, 1990.
[33] W. Hu, A. Starr, and A. Leung, ‘‘A multisensor-based system for manu-
facturing process monitoring,’’ Proc. Inst. Mech. Eng. B, J. Eng. Manuf.,
vol. 215, no. 9, pp. 1165–1175, 2001.
[34] P. G. Maropoulos and D. Ceglarek, ‘‘Design verification and
validation in product lifecycle,’’ CIRP Ann.-Manuf. Technol., vol. 59,
no. 2, pp. 740–759, 2010. [Online]. Available: http://dx.doi.org/
10.1016/j.cirp.2010.05.005
[35] (2015). ISO/IE IS 9000:2015—Quality Management. [Online].
Available: http://www.iso.org/iso/home/standards/management-standards/
iso_9000.htm
MILENA RADENKOVIC received the Ph.D.
degree from The University of Nottingham, U.K.
and the Dipl.-Ing. degree from the University
of Nis, Serbia. Her research spans the areas of
mobile, delay and disconnection tolerant networks
and services, intelligent P2P multimedia systems,
mobile clouds, intelligent security and privacy, and
their applications to different application domains.
She has been the Principle Investigator of several
EPSRC and EU grants. She currently works on
a number of open research questions such as improving reliability and
efficiency of mobile social and vehicular networks, designing new intelligent
data routing protocols, improving energy efficiency of continuous large scale
remote monitoring, and new adaptive security and privacy techniques for
data transmission in such environments. She has organized and chaired
multiple ACM and IEEE conferences, served on many ACM and IEEE
conference program committees. She has been an Editor and a Guest Editor
of many premium journals and published scientific papers in many premium
venues including the IEEE TRANSACTIONS on VEHICULAR TECHNOLOGY, the
IEEE TRANSACTIONS on PARALLEL and DISTRIBUTED COMPUTING, the Elsevier
Ad Hoc Networks, the ACM CHANTS/Mobicom, the IEEE Multimedia, the
MIT Press PRESENCE, and the ACM Multimedia. She has authored one
international patent on real time saleable signal processing in the Internet and
acted as a Scientific Expert for EU European Commission and Engineering
Physics Scientific Research Council U.K. for over ten years.
JON CROWCROFT (F’04) received the degree in
physics from Trinity College, University of Cam-
bridge in 1979, and the M.Sc. degree in computing,
and the Ph.D. degree from UCL, in 1981 and 1993,
respectively. He has been the Marconi Professor
of Communications Systems with the Computer
Laboratory since 2001. He has worked in the area
of Internet support for multimedia communica-
tions for over 30 years. Three main topics of inter-
est have been scalable multicast routing, practical
approaches to traffic management, and the design of deployable end-to-end
protocols. Current active research areas are opportunistic communications,
social networks, and techniques and algorithms to scale infrastructure-free
mobile systems. He is a fellow the Royal Society, a fellow of the ACM, a
fellow of the British Computer Society, and a fellow of the IET and the Royal
Academy of Engineering.
MUBASHIR HUSAIN REHMANI (M’15–
SM’16) received the B.Eng. degree in computer
systems engineering from the Mehran Univer-
sity of Engineering and Technology, Jamshoro,
Pakistan, the M.S. degree from the University
of Paris XI, Paris, France, and the Ph.D. degree
from the University Pierre and Marie Curie, Paris,
France, in 2004, 2008, and 2011, respectively.
He is currently an Assistant Professor with the
COMSATS Institute of Information Technology,
Wah Cantonment, Pakistan. He was a Post-Doctoral Fellow with the Uni-
versity of Paris Est, France, in 2012. His research interests include cognitive
radio ad hoc networks, smart grid, wireless sensor networks, and mobile
ad hoc networks. He served in the TPC for the IEEE ICC 2015, the IEEE
WoWMoM 2014, the IEEE ICC 2014, the ACM CoNEXT Student Workshop
2013, the IEEE ICC 2013, and the IEEE IWCMC 2013 conferences. He is
currently an Editor of the IEEE COMMUNICATIONS SURVEYS and TUTORIALS
and an Associate Editor of the IEEE Communications Magazine, the IEEE
ACCESS, the Computers and Electrical Engineering (Elsevier), the Journal of
Network and Computer Applications (Elsevier), the Ad Hoc Sensor Wireless
Networks, the Wireless Networks (Springer) Journal, and the Journal of
Communications and Networks. He is also serving as a Guest Editor of
the Ad Hoc Networks (Elsevier), the Future Generation Computer Systems
(Elsevier), the IEEE ACCESS, the Pervasive and Mobile Computing (Elsevier),
and the Computers and Electrical Engineering (Elsevier). He is the Founding
Member of IEEE Special Interest Group on Green and Sustainable Network-
ing and Computing with Cognition and Cooperation. He received the cer-
tificate of appreciation, an "Exemplary Editor of the IEEE COMMUNICATIONS
SURVEYS and TUTORIALS for the year 2015" from IEEE Communications
Society.
VOLUME 4, 2016 5321

More Related Content

PDF
Proposal of a Transparent Relay System with vNIC for Encrypted Overlay Networks
PDF
Manet mobile ad hoc network – challenges, security and protocols-2
PDF
Evaluating content-centric communication over power line communication infras...
PDF
I05225564
PDF
CONTEXT INFORMATION AGGREGATION MECHANISM BASED ON BLOOM FILTERS (CIA-BF) FOR...
PDF
An extensible, programmable, commercial-grade platform for internet service a...
PDF
AN EFFICIENT SECURE CRYPTOGRAPHY SCHEME FOR NEW ML-BASED RPL ROUTING PROTOCOL...
PDF
International Journal on AdHoc Networking Systems (IJANS)
Proposal of a Transparent Relay System with vNIC for Encrypted Overlay Networks
Manet mobile ad hoc network – challenges, security and protocols-2
Evaluating content-centric communication over power line communication infras...
I05225564
CONTEXT INFORMATION AGGREGATION MECHANISM BASED ON BLOOM FILTERS (CIA-BF) FOR...
An extensible, programmable, commercial-grade platform for internet service a...
AN EFFICIENT SECURE CRYPTOGRAPHY SCHEME FOR NEW ML-BASED RPL ROUTING PROTOCOL...
International Journal on AdHoc Networking Systems (IJANS)

What's hot (18)

PDF
Wireless Mesh Networks Based on MBPSO Algorithm to Improvement Throughput
PDF
IJCER (www.ijceronline.com) International Journal of computational Engineerin...
PPTX
Hsing cuhk talk wireless 5_g_why-how-what_mar 31 2014
PDF
MOBILE CROWD SENSING RPL-BASED ROUTING PROTOCOL FOR SMART CITY
PDF
PDF
Security-aware fair transmission scheme for 802.11 based cognitive IoT
PDF
An efficient transport protocol for delivery of multimedia content in wireles...
PDF
Review on security issues of AODV routing protocol for MANETs
PDF
A Trust-Based Predictive Model for Mobile Ad Hoc Networks
PDF
ICCT2017: A user mode implementation of filtering rule management plane using...
PDF
Six Degrees of Separation to Improve Routing in Opportunistic Networks
PDF
Ijsrp p10274
PDF
European quantum computing roadmap uploaded by Skip Sanzeri
PDF
Iaetsd key reconstruction and clustering opponent nodes in minimum cost
PDF
Techniques of lattice based
PDF
TRUST BASED ROUTING METRIC FOR RPL ROUTING PROTOCOL IN THE INTERNET OF THINGS
PDF
M.Phil Computer Science Network Security Projects
Wireless Mesh Networks Based on MBPSO Algorithm to Improvement Throughput
IJCER (www.ijceronline.com) International Journal of computational Engineerin...
Hsing cuhk talk wireless 5_g_why-how-what_mar 31 2014
MOBILE CROWD SENSING RPL-BASED ROUTING PROTOCOL FOR SMART CITY
Security-aware fair transmission scheme for 802.11 based cognitive IoT
An efficient transport protocol for delivery of multimedia content in wireles...
Review on security issues of AODV routing protocol for MANETs
A Trust-Based Predictive Model for Mobile Ad Hoc Networks
ICCT2017: A user mode implementation of filtering rule management plane using...
Six Degrees of Separation to Improve Routing in Opportunistic Networks
Ijsrp p10274
European quantum computing roadmap uploaded by Skip Sanzeri
Iaetsd key reconstruction and clustering opponent nodes in minimum cost
Techniques of lattice based
TRUST BASED ROUTING METRIC FOR RPL ROUTING PROTOCOL IN THE INTERNET OF THINGS
M.Phil Computer Science Network Security Projects
Ad

Similar to Fault tolerance on cloud computing (20)

PPT
Smart manufacturing through cloud based-r-nabati--dr abdulbaghi ghaderzadeh
PDF
Toward a real time framework in cloudlet-based architecture
DOCX
chapter 3.docx
PDF
chapter 3.pdf
PDF
Heterogeneous Networks of Remote Monitoring with High Availability and Resili...
DOCX
abstract.docx
PDF
abstract.pdf
PDF
Keynote Talk on Recent Advances in Mobile Grid and Cloud Computing
DOCX
AF-2599-P.docx
PDF
Reconfigurable data intensive service for low latency cyber-physical systems ...
PDF
A Context and User Aware Smart Notification System
PDF
Opportunistic Routing in Delay Tolerant Network with Different Routing Algorithm
PDF
2013 2014 ieee projects titles with abstract
PDF
Medium access in cloud-based for the internet of things based on mobile vehic...
DOCX
WSN-Proposal
PDF
Service oriented cloud architecture for improved
PDF
Service oriented cloud architecture for improved performance of smart grid ap...
PDF
Analysis and assessment software for multi-user collaborative cognitive radi...
PDF
Towards Interoperable, Cognitive and Autonomic IoT Systems: an Agent-based Ap...
PPTX
Integrating Wireless Sensor Network into Cloud Services for Real-time Data Co...
Smart manufacturing through cloud based-r-nabati--dr abdulbaghi ghaderzadeh
Toward a real time framework in cloudlet-based architecture
chapter 3.docx
chapter 3.pdf
Heterogeneous Networks of Remote Monitoring with High Availability and Resili...
abstract.docx
abstract.pdf
Keynote Talk on Recent Advances in Mobile Grid and Cloud Computing
AF-2599-P.docx
Reconfigurable data intensive service for low latency cyber-physical systems ...
A Context and User Aware Smart Notification System
Opportunistic Routing in Delay Tolerant Network with Different Routing Algorithm
2013 2014 ieee projects titles with abstract
Medium access in cloud-based for the internet of things based on mobile vehic...
WSN-Proposal
Service oriented cloud architecture for improved
Service oriented cloud architecture for improved performance of smart grid ap...
Analysis and assessment software for multi-user collaborative cognitive radi...
Towards Interoperable, Cognitive and Autonomic IoT Systems: an Agent-based Ap...
Integrating Wireless Sensor Network into Cloud Services for Real-time Data Co...
Ad

More from www.pixelsolutionbd.com (20)

PDF
Adaptive fault tolerance_in_real_time_cloud_computing
PPT
Software rejuvenation based fault tolerance
PPT
Privacy preserving secure data exchange in mobile p2 p
PPT
Adaptive fault tolerance in cloud survey
PPT
Adaptive fault tolerance in real time cloud_computing
PPT
Protecting from transient failures in cloud deployments
PPT
Protecting from transient failures in cloud microsoft azure deployments
PDF
Fault tolerance on cloud computing
PDF
Fault tolerance on cloud computing
PDF
Fault tolerance on cloud computing
PDF
Fault tolerance on cloud computing
PDF
Fault tolerance on cloud computing
PDF
Cyber Physical System
PDF
Fault tolerance on cloud computing
PDF
Real time service oriented cloud computing
PDF
Performance, fault tolerance and scalability analysis of virtual infrastructu...
PDF
Comprehensive analysis of performance, fault tolerance and scalability in gri...
PDF
A task based fault-tolerance mechanism to hierarchical master worker with div...
PPT
Privacy preserving secure data exchange in mobile P2P
Adaptive fault tolerance_in_real_time_cloud_computing
Software rejuvenation based fault tolerance
Privacy preserving secure data exchange in mobile p2 p
Adaptive fault tolerance in cloud survey
Adaptive fault tolerance in real time cloud_computing
Protecting from transient failures in cloud deployments
Protecting from transient failures in cloud microsoft azure deployments
Fault tolerance on cloud computing
Fault tolerance on cloud computing
Fault tolerance on cloud computing
Fault tolerance on cloud computing
Fault tolerance on cloud computing
Cyber Physical System
Fault tolerance on cloud computing
Real time service oriented cloud computing
Performance, fault tolerance and scalability analysis of virtual infrastructu...
Comprehensive analysis of performance, fault tolerance and scalability in gri...
A task based fault-tolerance mechanism to hierarchical master worker with div...
Privacy preserving secure data exchange in mobile P2P

Recently uploaded (20)

PPTX
Foundation to blockchain - A guide to Blockchain Tech
PPTX
IOT PPTs Week 10 Lecture Material.pptx of NPTEL Smart Cities contd
PDF
PPT on Performance Review to get promotions
PPTX
UNIT-1 - COAL BASED THERMAL POWER PLANTS
PPTX
Lecture Notes Electrical Wiring System Components
PDF
Mitigating Risks through Effective Management for Enhancing Organizational Pe...
PPTX
OOP with Java - Java Introduction (Basics)
PPTX
MET 305 2019 SCHEME MODULE 2 COMPLETE.pptx
PPTX
KTU 2019 -S7-MCN 401 MODULE 2-VINAY.pptx
PPTX
additive manufacturing of ss316l using mig welding
PDF
Automation-in-Manufacturing-Chapter-Introduction.pdf
PPTX
Engineering Ethics, Safety and Environment [Autosaved] (1).pptx
PDF
TFEC-4-2020-Design-Guide-for-Timber-Roof-Trusses.pdf
PDF
Mohammad Mahdi Farshadian CV - Prospective PhD Student 2026
PPTX
CH1 Production IntroductoryConcepts.pptx
PDF
Operating System & Kernel Study Guide-1 - converted.pdf
PPTX
Geodesy 1.pptx...............................................
PDF
The CXO Playbook 2025 – Future-Ready Strategies for C-Suite Leaders Cerebrai...
PDF
Enhancing Cyber Defense Against Zero-Day Attacks using Ensemble Neural Networks
PPTX
bas. eng. economics group 4 presentation 1.pptx
Foundation to blockchain - A guide to Blockchain Tech
IOT PPTs Week 10 Lecture Material.pptx of NPTEL Smart Cities contd
PPT on Performance Review to get promotions
UNIT-1 - COAL BASED THERMAL POWER PLANTS
Lecture Notes Electrical Wiring System Components
Mitigating Risks through Effective Management for Enhancing Organizational Pe...
OOP with Java - Java Introduction (Basics)
MET 305 2019 SCHEME MODULE 2 COMPLETE.pptx
KTU 2019 -S7-MCN 401 MODULE 2-VINAY.pptx
additive manufacturing of ss316l using mig welding
Automation-in-Manufacturing-Chapter-Introduction.pdf
Engineering Ethics, Safety and Environment [Autosaved] (1).pptx
TFEC-4-2020-Design-Guide-for-Timber-Roof-Trusses.pdf
Mohammad Mahdi Farshadian CV - Prospective PhD Student 2026
CH1 Production IntroductoryConcepts.pptx
Operating System & Kernel Study Guide-1 - converted.pdf
Geodesy 1.pptx...............................................
The CXO Playbook 2025 – Future-Ready Strategies for C-Suite Leaders Cerebrai...
Enhancing Cyber Defense Against Zero-Day Attacks using Ensemble Neural Networks
bas. eng. economics group 4 presentation 1.pptx

Fault tolerance on cloud computing

  • 1. Received July 29, 2016, accepted August 26, 2016, date of publication September 7, 2016, date of current version September 28, 2016. Digital Object Identifier 10.1109/ACCESS.2016.2606501 Towards Low Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems MILENA RADENKOVIC1, JON CROWCROFT2, AND MUBASHIR HUSAIN REHMANI3 1School of Computer Science, The University of Nottingham, Nottingham, NG7 2RD, U.K. 2Computer Laboratory, University of Cambridge, Cambridge, CB2 1TN, U.K. 3COMSATS Institute of Information Technology, Islamabad 45550, Pakistan Corresponding author: M. Radenkovic (mvr@cs.nott.ac.uk) This work was supported in part by the Project Health Monitoring and Life-Long Capability Management for SELf-SUStaining Manufacturing Systems through the Commission of the European Communities under the 7th Framework Programme, under Grant 609382. ABSTRACT Fast emerging mobile edge computing, mobile clouds, Internet of Things, and cyber physical systems require many novel realistic real-time multi-layer algorithms for a wide range of domains, such as intelligent content provision and processing, smart transport, smart manufacturing systems, and mobile end-user applications. This paper proposes a low-cost open source platform, MODiToNeS, which uses commodity hardware to support prototyping and testing of fully distributed multi-layer complex algorithms over real-world (or pseudoreal) traces. MODiToNeS platform is generic and comprises multiple interfaces that allow real-time topology and mobility control, deployment and analysis of different self-organized and self-adaptive routing algorithms, real-time content processing, and real-time environment sensing with predictive analytics. Our platform also allows rich interactivity with the user. We show deployment and analysis of two vastly different complex networking systems: a fault and disconnection-aware smart manufacturing sensor network and cognitive privacy for personal clouds. We show that our platform design can integrate both contexts transparently and organically and allows a wide range of analysis. INDEX TERMS Disruption tolerant networking, mobile ad hoc networks, prototypes, wireless communication, wireless sensor networks. I. INTRODUCTION Over the recent years there has been a growing interest in designing and testing novel mobile wireless and opportunistic network communication protocols and systems for a wide range of vastly different application scenarios, such as smart manufacturing, mobile social networks and smart wellbeing domains. Researchers increasingly aim to test their novel net- work architectures and protocols under realistic constraints after initially optimising theoretical models. Newly emerging services and applications for Internet of Things (IoTs) and Cyber Physical Systems (CPSs) require development of new intelligent communication, storage and processing architec- tures. We propose a novel platform where IoT ubiquitous devices can host services and communicate in peer-to-peer manner via adaptive mobile delay/disconnection tolerant opportunistic networks. This paper describes a novel multi- layer intelligent Mobile Opportunistic and Disconnection Tolerant Networking (MODiToNeS) platform that supports various mobility and connectivity patterns, adaptive communication protocols, and a wide range of smart algo- rithms for intelligent content processing. We argue that our platform can be used to help research community test highly complex self-organised cognitive distributed protocols and architectures as well as serve as an educational resource which uses open source software, low cost hardware, simple control interfaces and modelling structures. We believe that MODiToNeS will help advance the research and educational opportunities available to the cognitive DTN and oppor- tunistic network communication communities by providing a ‘‘real-world’’ fully distributed platform where researchers and students can develop and test their cognitive protocols and applications while being able to observe how they behave in a real world hybrid (wireless and wired, mobile and static) environments. We argue that it is very important to allow researchers to validate their core assumptions and hypothesis made when proposing new complex algorithms and systems as early as possible to avoid building inaccurate and unusable protocols. MODiToNeS allows us to tackle VOLUME 4, 2016 2169-3536 2016 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. 5309
  • 2. M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems exactly that through incremental or evolutionary prototyping. Using simple open source interfaces, researchers are able to rapidly develop distributed algorithms and deploy them in a real environment saving time in developing component evaluation and simulation techniques while providing them with valid feedback about how their components interacted with different kinds of dynamic environments. Additional advantages of using MODiToNeS are multifold and include: first, low cost which refers to the total cost of the platform ownership being less than the cost of a mid-high end laptop (around £2000), and second the platform being fundamental to performing early feasibility tests and quick prototyping before embarking in complex simulations (e.g. NS-3 [29] or Mininet [30]). In this paper, we describe two different example scenar- ios prototyped in MODiToNeS: first, smart manufacturing Fault Aware and Disconnection aware communication smart sensors protocol (FDASS) [10]); and second, privacy aware mobile personal clouds (CogPriv) [4]. More specifically, we show how our open-source distributed platform allows build- ing, deploying and testing of 1) fault aware and disconnection aware framework prototyping and testing for smart manufac- turing and 2) adaptive mobile privacy aware personal clouds prototyping and testing when sharing various kinds of data via different routing protocols via networks with different levels of privacy leakage. The paper is organised as follows. Section 2 reviews related work on state of the art testbeds for smart data communi- cation in mobile and wireless networks. Section 3 proposes the multi-layer architecture of our MODiToNeS platform, introduces its key control planes and describes support for cross layer data communication that can on-the-fly adapt to dynamic link properties and changing requirements of the users. Section 4 describes smart manufacturing opportunis- tic and disconnection tolerant sensor network architecture scenario prototyping with smart adaptive routing protocols such as FDASS [10]. Section 5 describes peer to peer mobile clouds scenario which deploys smart protocols [4] for data sharing, monitoring and interaction. We show that both CogPriv and FDASS outperform other competitive and benchmark protocols across a range of metrics in line with previously done simulation based work in [4] and [10]. In addition, we evaluate realistic resource costs for FDASS and CogPriv across a range of resource metrics in real time. Section 6 gives summary and future work directions. II. RELATED WORK As mobile edge computing and cognitive networks are still emerging filed, there are limited simulation and testbed environments which allow prototyping and testing of new emerging applications, protocols and services. We review a range of state of the art testbeds for wireless networks and applications, and identify how our proposal defers from each one of them. Similarly, the majority of the current simulator implementations have limited number of control features as they are based on the basic wireless sensor networks and communication protocols. In this paper, we propose a novel multilayer platform that uses low cost smart devices (e.g. such as Raspberry PIs) to prototype rich set of intelligent and interactive complex communication algorithms and dis- tributed network architectures. Radenkovic and Milic-Frayling [6] propose the design and architecture for a low cost light weight testbed for a Personal Cloud based on Raspberry Pi with a range of sensors (RasPiP- Cloud). RasPiPCloud supports multiple on demand virtual containers to host different services and applications that can collect, store and share data with varying different lev- els of privacy. RasPiPCloud utilizes opportunistic networks communication to communicate with the heterogeneous sensors and other devices. RasPiPCloud can have multi- ple containers [5] such as: Healthcare, Finance, and Social Network with additional container template ready for rapid on demand deployment. Each container gets installed and runs its purpose specific applications to ensure secure data fencing and protection. Radenkovic and Milic-Frayling [6] do not describe the support for multi-user communication. This paper focuses on multi clouds communication support in MODiToNeS. Romero et al. [13] propose cognitive testbed for wireless sensor networks as an emerging technology with a potential to avoid traditional wireless problems such as reliability, interferences and spectrum scarcity in wireless sensor net- works. Romero et al. [13] argue that cognitive wireless sensor networks testbeds are an important tool for future develop- ments, protocol strategy testing and algorithm optimization in real scenarios. This paper focuses on sparse and potentially disconnected topologies in addition to large dense topologies. State of the art work in [16] proposes Haystack system which aims to allow unobtrusive and comprehensive monitoring of network communications on mobile phones entirely from user space. Haystack correlates disparate contextual informa- tion to illuminate mobile phone app performance, privacy and security. While Haystack runs locally on a user’s phone and can provide highly useful real world data traces that we can use in our platform, our platform is fully distributed and can run different applications and contexts. TKN Wireless Indoor Sensor network Test- bed (TWIST) [22], developed by the TKN at the TU Berlin, is one of the largest academic testbeds for experimenting with WSN applications at indoor deployment scenarios. It provides basic services like node configuration, network- wide programming, out-of-band extraction of debug data and gathering of application data. It also presents several novel features such as active control of the power supply of the nodes. The testbed in [11] uses setup which con- sists of 102 TmoteSky nodes operating at 2.4 GHz and 102 eyesIFX nodes at 868 MHz resulting in a fairly regu- lar grid deployment pattern with an inter-node distance of 3 m. The Virginia Tech COgnitive Radio NEtwork Testbed (VT-CORNET) [21] is a collection of cognitive nodes deployed in a building on the Virginia Tech campus. The testbed consists of a total of 48 static SDR nodes based on 5310 VOLUME 4, 2016
  • 3. M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems USRP210, located at the ceiling. In addition to the static nodes, low-power mobile nodes are also available in order to provide an environment that accommodates a wide variety of research topics. All devices used in this testbed are based on SDR and are not suitable for WSNs because of their high power consumption. Despite their possibilities for frequency mobility, the solution implemented by this test-bed does not support CWSN implementation. Both [21] and [22] focus on the network layer communications and do not consider other IoT (middleware) layer and different application and data types which MODiToNeS includes. While there has been extensive research and standard- isation work being done in the areas of verification and validation for product lifecycle for different application areas [34], [35], in our paper we follow general guidelines for the physical prototyping which has been identified as an open research problem for the intelligent mobile opportunis- tic research community and complex network protocols and systems they aim to propose [8], [34]. Physical testing is still an expected industry practice, frequently linked to prod- uct certification. Moreover, as we target complex systems modelled with complex temporal networks assuming likely loss of connectivity and data, physical prototyping generates valuable knowledge and data that can be utilised to enhance the design of future products or variants. III. MOBILE OPPORTUNISTIC DELAY/DISCONNECTION TOLERANT NETWORKS AND SYSTEMS PLATFORM (MODiToNeS) A. OVERVIEW OF THE MODiToNeS PLATFORM We propose a novel platform, MODiTOnNeS, which is highly suitable for fast prototyping of applications that are dis- tributed, cognitive (context-aware), intelligent (able to use various on demand self-organised and adaptive routing and machine learning algorithms), interactive and driven by real world connectivity and application traces. MODiToNsS plat- form contains five programmable layers for dynamic and on demand control including: 1) cognitive/smart hardware devices which are able to store and process real time data and are equipped with different heterogeneous sensors and differ- ent types of types of communication interfaces; 2) topology control plane to enable rich diversity of mobile and fixed network topologies; 3) control plane for enabling different intelligent routing protocols; 4) control plane to enable dif- ferent real time analytics and machine learning protocols which are suitable for different applications and 5) interac- tive real time user dashboard to allow user interaction and notifications for different application types. This architecture is shown in Figure 1. We argue that it is important to enable different control interfaces for different layers in order to enable a more complete and useful platform that promotes opportunistic disconnection tolerant networking and mobile edge computing research which is fundamental for pervasive computing, IoTs and CPSs research and services. While MODiToNeS draws inspiration from the work such as Castalia [2], it focuses on different set of properties as FIGURE 1. Overview of the layered architecture of MODiToNeS. we target cross layer design, opportunistic smart communi- cation protocols in challenged networks and enabling high level analysis and visualisation accessible to the user or at the edges. We provide a modular and simple open source implementation (inspired by ONE [18]) for topology con- trol and monitoring, resources monitoring and analysis and data monitoring and visualisation. Each of our smart nodes has multiple communication interfaces which can be pro- grammed dynamically with different parameters to start or stop as well as to control and monitor different wired or wireless active channels which communicate different types of sensor and user application data. MODiToNeS provides the developer with simple open source functionality to change the default interface used to send data on demand as well as to the change the active channel on the fly. In addition, our plat- form allows running of different real world and connectivity traces in real time which can be mobile, static or hybrid net- works. We assume that connectivity traces are in accordance with the syntax of the StandardEventsReader format used in ONE. MODiToNeS also allows programmable topologies where the user can program the dynamic connectivity among the nodes. Our approach allows testing of various protocols against multiple real world conditions by allowing real world topology information retrieval which is fed to the platform in order to mirror any real world topologies. We enable VOLUME 4, 2016 5311
  • 4. M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems automated reconfiguration of the platform such as experi- ment repetitions with controlled parameter changing so that our approaches can be tested against multiple real-life con- ditions. We also allow a user to in a simple way deploy different routing algorithms on the distributed MODiToNeS nodes to support various routing behaviour and topologies. Overview of the layered architecture of the platform is shown in Figure 1. Each MODiToNeS node is a low cost single-board com- puter (Raspberry Pi model B) which provides good process- ing power, flexible storage, and has good software support. It can be integrated with a number of wired and wireless sen- sors using GPIO, I2C, RF modules, 802.11 Wifi, Bluetooth and USB. We currently have over 80 Raspberry Pi nodes. Figure 2 shows one Raspberry PI node which uses on-board Ethernet port, an 802.11n Wi-Fi dongle for wireless network connectivity, and uses IBR-DTN [3], [26] to provide P2P DTN capabilities. Figure 3 shows one hierarchical deploy- ment of over 20 Raspberry PIs with different low sensors such as wireless temperature sensor, pressure sensor, magnetome- ter and 3-axis accelerometer. FIGURE 2. DTNPi with 801.11 adapter and XRF receiver for wireless sensors. FIGURE 3. A hierarchical architecture Raspberry PI sensor network example. B. MULTI-LAYER CONTROL PLANES There is a growing need for generic network platforms that can combine real-world delay and other challenging network conditions with the flexibility of simulators to support a range of different application domains. MODiToNeS enables significantly lower time between the early prototype and pro- duction system deployment. Using Raspberry PIs (or similar hardware) allows having large number of hardware nodes in a relatively small physical space at a low cost. For example, influence of mobility on systems performance is complex and is usually evaluated in simulator environments. Contrary to this, our MODiToNeS platform enables the integration of many different distributed mobility patterns. We have had several diverse projects which benefited from the design and deployment of a generic MODiToNeS plat- form that satisfies the following requirements: • Allowing the coexistence of multiple independent projects (e.g. nodes 1-17 for Project A, nodes 20-50 for Project B) • Allowing experimental orchestration in terms of deter- mining how many run to have, selecting senders and receivers, determining messages rates and sizes, deter- mining which protocols to run. • Allowing for change of network environment within the scope of a single project to simulated different network topologies for different simulation runs (e.g. nodes 1-5 communicate with nodes 6-10, even ID nodes commu- nicate with odd ID nodes) • Allowing for real-time change of network topology emulating DTN [1], [27] or MANET where nodes can come in and out of contact with each other within the span of a few seconds. We address these requirements by proposing real-time programmable interface for network topology configura- tion. This control plane is used for fast automated and pro- grammable configuration of the network plane. The network layer consists of the head node and variable number of worker nodes directly connected e.g. via a wired Ethernet switch (see Figure 3, Figure 4 and Figure 5). The existence of the head node is fundamental for allowing integrated and holistic view of the design in order to be able to validate in an integrated manner. We have designed, developed and deployed a set of tools on the head node that allow real time configuration changes, command executions, as well as run- ning and deploying distributed smart protocol modules (such as FDASS [10], MWCC [11], CogPriv [4], CafeREP [17], etc) to all nodes. All tools are based on PERL, C/C++, PYTHON and IBR-DTN suitable for our low cost hard- ware (Raspberry PIs). We use open source PNP4Nagios that visualise RRD files generated by sensor readings and other time series data. Through combination of deploying differ- ent routing and content dissemination protocols as well as dynamic/programmable firewall configurations to all nodes, the network topology can be changed and certain nodes can be included or excluded from it. For example configuring a node firewall to drop all incoming and outgoing packets will make this node ‘‘invisible’’ to all other nodes. By changing firewall rules we can achieve any traditional network topolo- gies (such as tree, star or full mash) or any complex temporal networks with our platform (by reading connectivity trace files from http://uk.crawdad.org which is a shared wireless 5312 VOLUME 4, 2016
  • 5. M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems FIGURE 4. (a) Algorithm for the control planes in the Master node. (b) Algorithm for a Worker node. network data resources for the research community or gen- erating and capturing similarly formatted connectivity files from other real work devices). We can also exclude a node from the sensor topology while a particular experiment is in mid-run for cases when we want to test a node fault and disconnection. Similarly, we enable real time configuration of different topologies across different experiments for vari- ous prototypes. For static topologies, the configured network topology remains the same for the duration of the individual trail. We use a simple format to design and describe the desired network topology. It describes the tuples connectivity that comprises the network topology as well as contains the connection characteristics between the connected tuples as shown below: TimeStamp:Address1:Address2:<UP|DOWN> [,RATE,DELAY,LOSS] An example connectivity line describing only connectivity is given: 0:10.0.10.2:10.0.10.17:UP FIGURE 5. MODiTONeS topology used in the performance tests for manufacturing scenario. This executes the below firewall and traffic rate configura- tion commands: For Node 10.0.10.2 iptables -A INPUT -s 10.0.10.17 -j ACCEPT iptables -A OUTPUT -d 10.0.10.17 -j ACCEPT For Node 10.0.10.17: iptables -A INPUT -s 10.0.10.2 -j ACCEPT iptables -A OUTPUT -d 10.0.10.2 -j ACCEPT An example connectivity line describing connectivity with network characteristics is given below: 0:10.0.10.2:10.0.10.17:UP:rate 2Mbit,delay 150ms,loss 7% For Node 10.0.10.2: iptables -A INPUT -s 10.0.10.17 -j ACCEPT iptables -A OUTPUT -d 10.0.10.17 -j ACCEPT iptables -t mangle -A POSTROUTING -d 10.0.10.2 -j CLASSIFY -set-class 1:100 tc class add dev eth0 parent 1: classid 1:100 htb rate 2Mbit tc qdisc add dev eth0 parent 1:100 handle 100: netem delay 150ms loss 3% For Node 10.0.10.17: iptables -A INPUT -s 10.0.10.2 -j ACCEPT iptables -A OUTPUT -d 10.0.10.2 -j ACCEPT iptables -t mangle -A POSTROUTING -d 10.0.10.2 -j CLASSIFY -set-class 1:100 tc class add dev eth0 parent 1: classid 1:100 htb rate 10Mbit tc qdisc add dev eth0 parent 1:100 handle 100: netem delay 150ms loss 3% In addition to static topologies, MODiToNEs also supports dynamic topology configuration when the con- figured network topology is expected to change within the duration of the individual trail. We can use this to emulate DTN [1] and MANET network environments VOLUME 4, 2016 5313
  • 6. M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems and run prototype experiments with real-world connectivity and data dissemination traces like Infocom [19], Rollernet [7], [9], [28] etc. We provide a mechanism that allows us to update the network configuration by using the above men- tioned connectivity/network topology format where entries reflect changes in chronological order: TimeStamp:Address1:Address2: <UP|DOWN> [,RATE,DELAY,LOSS] 7:10.0.10.2:10.0.10.17:UP 128:10.0.10.2:10.0.10.17:DOWN or 14:10.0.10.2:10.0.10.17:UP:rate 2Mbit, delay 150ms,loss 7% 58:10.0.10.2:10.0.10.17:DOWN Figures 4a and 4b give an overview of the pseudo code of the master MODiToNeS node and distributed working MODiToNeS nodes respectively. IV. SMART MANUFACTURING SCENARIOS A. PROTOTYPING FAULT AND DISCONNECTION AWARE SMART MANUFACTURING We describe the hardware, software and algorithms we use to prototype smart manufacturing sensor network that we have used in smart manufacturing project EU Selsus [12] and facilitate real world deployment of complex architectures and communication protocols. We describe the design and implementation for a sensor network prototype in MODiToNEs that can reproduce a pro- duction floor sensor network environment and emulate vari- ous sensor network topologies and communication patterns that we then integrate within the EU SelSus project [12]. We assume that we have smart sensing nodes which oper- ate as MODiToNeS nodes and provide sensing, compu- tation, storage and communication together with allowing self-configuration, fault tolerance and self-monitoring. The MODiToNeS platform allows development and evaluation of different novel and benchmark protocols FDASS [10], Prophet [24] and Epidemic/Flooding protocols [23] intended for use with smart sensors. A common and widely used production environment monitoring sensor network topology is a tree topology of depth 2 – the lowest layer of nodes including heteroge- neous sensor nodes, the middle layer including gateways/ aggregators/processors and the top layer referring to the head/cloud/master node [30]–[34]. We build and demon- strate a logical depth 2 tree sensor network topology in MODiToNes comprising of four sensor nodes, two aggrega- tor nodes and one central cloud node (shown in Figure 5). The MODiToNeS sensor nodes are equipped with a range of sensors including directly attached camera sensor, MEMS Sensor Evaluation Board with Low power 3D magnetometer, 3-Axis digital accelerometer, temperature / high precision pressure sensor, and a remote ANT+ protocols compatible sensor. The MODiToNeS sensor nodes are unable to detect each other’s’ presence in the network and are only able to communicate with the two MODiToNeS aggregator nodes. Each MODiToNeS aggregator node is able to communi- cate with the all sensors nodes and the central MODiToNeS cloud node. The MODiToNeS aggregators are also unable to detect each other’s presence on the network. The cen- tral MODiToNeS node is able to communicate with any of the MODiToNeS aggregators. We assume that, during nor- mal operation of the MODiToNeS sensor node, it captures their corresponding sensors’ readings as well its real time local resources utilisation (including CPU load, memory, disk usage, I/O) at configured time intervals. Each node is able to store the measurements locally to be available for localised queries and also generates simple format messages with sensor measurements which are sent to the central cloud node. The only neighbours that any MODiToNeS sensor node detects are the two MODiToNeS aggregators. The individual MODiToNeS sensor nodes can transmit their sensor mea- surements messages to any of the two MODiToNeS aggre- gators but in normal operation they ‘‘prefer’’ their local MODiToNeS aggregator. When a MODiToNeS aggregator receives a MODiToNeS sensor reading message, it stores the sensor measurements locally to be available for localised query and also forwards the messages directly onto the central MODiToNeS cloud node. The MODiToNeS aggre- gator nodes also forward resource measurements to the MODiToNeS cloud node in the same way the MODiToNeS sensor nodes do. Each MODiToNeS aggregator stores the sensor measurements of its belonging sensor nodes and can provide them if queried locally or remotely. In this way, the central MODiToNeS cloud node receives measurements from all MODiToNeS sensors within the sensor network including resource utilisation readings (Fig 6 and Fig 7). All sensor readings are being stored in RRD format as this format is well suited for time-series data like network bandwidth, tempera- tures, CPU load, etc. The data are stored in a circular buffer based database, thus the system storage footprint remains constant over time. Note that this is distinct from the tradi- tional concept of round-robin scheduling. Readings for sep- arate sensors get allocated their individual RRD databases. Each Raspberry Pi node is running a web service that allows real-time queries of sensor states and reading as well as historical information and visualisation via PNP4Nagios. Figure 6 and Figure 7 show long term file system utilisation and long term memory utilisation for a MODiToNeS node. We observe that memory utilisation is firmly below the full FIGURE 6. Long term file system utilisation for a MODiToNeS node. 5314 VOLUME 4, 2016
  • 7. M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems FIGURE 7. Long term memory utilisation for a MODiToNeS node. utilisation and that the file system is not over-utilised as it gets monthly archiving of experiment log to external long term storage. B. ROUTING PROTOCOL EVALUATION In order to better understand simulated performance of fault aware disconnection tolerant smart sensor network protocol (FDASS) [10], we prototype and test FDASS against Flooding and Prophet protocols in MODiToNeS. We have developed an intelligent framework that aims to improve reli- ability of the manufacturing plant in the face of varying net- work connectivity and non-uniform distribution of different types of faults in the network. Fault and Disconnection Aware Smart Sensing framework (FDASS) [10] is able to detect and identify misperforming nodes in a fully distributed fashion in order to isolate them, reroute the traffic away from them and notify the sinks about the type, location and time of the failures. FDASS builds on and extends multi-path transport approaches to combine fault analytics layer with the complex network topology and resources analytics layer into a com- plex heterogeneous network for manufacturing environments as shown below. As all MODiToNeS nodes run IBR-DTN on Raspberry PIs, we implemented the FDASS protocol as a IBR-DTN routing component written in C++. IBR-DTN also includes other benchmark protocols (such as Epidemic and Prophet). This allows direct performance comparison between different protocols in a real world environment with MODiToNeS. Each MODiToNeS node was augmented with a sensor simulator capable of generating varying numbers of pseudo realistic sensor readings on demand. The simulated sensor readings were chosen over real sensors to increase the diver- sity of sensing ranges and frequencies [31]–[33] compared to the available low cost sensor types we have. The simulated sensor readings were padded to 100 bytes to ensure each reading had a consistent size. Each sensor reading was also timestamped as it was taken with millisecond precision. The head node also timestamped the bundles, with millisecond precision, as they were received. These timestamps were used to calculate the time taken for the bundle to propagate through the network. Each experimental prototype run lasted 60 minutes with each sensor being polled once every second. Between each run the number of sensors per sensor node was increased by one until ten sensors per node was reached. All nodes were FIGURE 8. (a) Delivery success with increasing numbers of sensors. (b) Delivery delays with increasing numbers of sensors. rebooted between each run to ensure the nodes were in a known state. Each experiment was repeated three times and the averages of these three runs were recorded. Figure 8a shows the recorded bundle delivery success rate achieved and Figure 8b shows the bundle delivery delay observed as the number of sensors per sensor node increases from one to ten. Figures 8a and 8b show that FDASS outper- forms both the Flooding and Prophet protocols. Figure 9 and 10 demonstrate FDASS robust functionality in MODiToNeS by emulating faults of the MODiToNeS aggregators. Consider MODiToNeS Aggregator 2 fails first by losing network connectivity. We observe in the Figure 9 VOLUME 4, 2016 5315
  • 8. M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems FIGURE 9. View of two MODiToNeS aggregators messages during failure disconnections and recovery. FIGURE 10. Centralized view of messages hops, delays, and delivery success. that all sensor messages get redirected via MODiToNeS Aggregator 1. After a few minutes, MODiToNeS Aggre- gator 1 loses network connectivity as well. At this stage there is no route between the MODiToNeS sensor nodes and the central MODiToNeS node. During this time, all sensor readings are being stored by the sensor nodes where they are generated. After another few minutes both MODiToNeS Aggregators recover their connectivity and we observe the peak in traffic generated due to the instantaneous delivery of all stored sensor readings. We aim to deploy MODiToNeS Raspberry PI nodes with FDASS in the real world manufacturing shop floors to enable further validation with real users and improve reliability of diverse factory communications. V. MOBILE CLOUDS SCENARIO A. PROTOTYPING MOBILE CLOUDS OVERVIEW As an example of a different architecture that can be built and tested in MODiToNeS platform, we describe the design of predictive mobile clouds where mobile sensing and real time predictive analytics algorithms are incorporated in dynamic mobile clusters of MODiToNeS nodes. Each smart MODiToNeS platform node allows intelligent real time deci- sion making that can predict (and change) the behaviour of the network communication of itself and other nodes’. Even though machine learning and analytics techniques have been widely recognised as important for context prediction in mobile computing and many theoretical and simulation based works exist, real world implementations are still scarce and remain interesting future research challenge [8]. The mobile cloud (MC) prototype over MODiToNeS example we describe here supports new paradigm shift that combines anticipatory systems [8] and adaptive collaborative propos- als [e.g. 17,20] where computer devices base their actions on the predictive models of themselves, the environment and the other nodes. MODiToNeS support consideration of multiple criteria including different complex temporal graphs centrality predictions as well as resource, movement and behaviour predictions. We view mobile clouds (MC) as new approach that bridges the gap between the device(s), environments and the user. In MODiToNeS platform, the prototype of each MC is equipped with a range of sensors (accelerometer, gyroscope, temperature, pressure, heart rate sensor) that can sense the environment and monitor the context, as well as run real time predictive analytics (or other machine learning) algorithms to develop models that predict occurrences of various events. Our MODiToNeS MC also allows rich real time interaction with the users as well as sharing among MCs over different intelligent protocols, different applications and data types. Additionally, each MODiToNeS MC is able to interact with the environment and can adaptively change its behaviour in different situations. Of particular interest in this platform is to investigate the performance characteristics of our MC smart data commu- nication algorithms in the face of different users’ require- ments for privacy in different contexts. In [11], we describe a Mobile Wellbeing Cloud Companion (MWCC) testbed pro- totype which is able to continuously process accelerometer and gyroscope from the physical environment and process the readings using various machine learning algorithms to identify several user activity features. These are analysed in real time and correlated with the heart rate signals to identify if the heart rate is normal or not for the current user activity. In [4], we proposed CogPriv that explored through simula- tions how different levels of privacy can be supported via adaptively changing network connectivity in both sparse and dense topologies. In this paper, we build CogPriv prototype in MODiToNeS and test it both in terms of quality of the experience metrics (such as achieved end-to-end privacy and delays) and the quality of service metrics (such as memory, I/O, CPU with resource limited devices.). CogPriv considers users who may be running a social network that allows them to stay in contact with their friends at the same time as regularly monitoring their long term medical condition and being in contact with the hospital. 5316 VOLUME 4, 2016
  • 9. M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems These two types of applications have different privacy requirements and need their data to be stored and shared in different ways in order to adapt to each required privacy requirements dynamically. Figure 11 shows example sensor integration for mobile personal cloud (MPC) prototype in MODiToNeS on a Raspberry Pi with an Xtrinsic sensor board with temperature, pressure, and acceleration sensors. Figure 12 shows MODiToNeS Raspberry PI device that cap- tures, stores and processes a range of user and environment data such as heart rate and pedometer. FIGURE 11. MODiToNeS Raspbery Pi B with a Xtrinsic sensor board and a WiPi wireless adapter. FIGURE 12. MODiToNeS Raspberry Pi with Suunto and WiPi USB module, Garmin heartrate sensor and a smaprtphone displaying readings. CogPriv in MODiToNeS extends the bundle protocol based on RFC 5050 [25], [26] that provides API for DTN applica- tions to exchange and route bundles among distributed nodes in an intelligent P2P manner. CogPriv P2P DTN (IBR-DTN) module in MODiToNeS provides multi flow real time bundle forwarding based on a range of criteria such as source ID, Virtual Machine (VM) ID, application privacy requirements, destination ID so that different incoming bundles can be matched to the appropriate network interface in real time. At its core, CogPriv comprises multiple stages: it probes local cellular network to identify the likelihood of any middle boxes that may compromise user traffic, requests the remote destination nodes to provide their estimations of the cellu- lar network privacy levels, and collaborates and cooperates with the local network nodes to determine the best local next hop. CogPriv routing protocol can range dynamically and adaptively from providing fully cellular single hop end to end communication to fully localised multi hop mobile opportunistic communication. Through collaborations and cooperation in the local neighbourhoods, each node can understand its environment and neighbours better. More specifically, each CogPriv MODiToNeS node exchanges their own cellular network privacy statistics and predictions to negotiate feasibility of using cellular network for the par- ticular application, analytics of their own resource predic- tions and social connectivity analytics. Note that both social connectivity traces and middle boxes information are fed to the MODiToNeS master node from external real world traces (e.g. utilising http://uk.crawdad.org/). In this paper, we show measured achieved end-to-end privacy, end-to-end delays, end-to-end number of hops and transitions, I/O, memory and CPU costs. Each CogPriv MODiToNeS node privacy level is important to consider as it is the core criteria for forwarding the data and deciding on the next hop and via which interface. More detailed description of CogPriv Decision Algorithm is described in [4]. B. COGNITIVE PRIVACY EXPERIMENT SCENARIO We carry out evaluation of CogPriv in MODiToNeS against fully cellular communication and fully local social oppor- tunistic networks across a range of different network condi- tions and user traffic types using a range of metrics. We show how data can be shared with different levels of privacy in light of untrusted infrastructure. We use findings identified in [14] and [15] that show widespread use of transparent middle boxes such as HTTP and DNS proxies in the cellular infrastructure which are able to analyse and actively modify user traffic and thus compromise user privacy and security. In [4] we provided rich set of simulation based experiments with real world traces of middle boxes [14], connectivity [7], interests [7] and friendships [7]. This paper addresses these scenarios and proposes a way of integrating different layers within our MODiToNeS platform and exploring how differ- ent intelligent routing can exploit maximally trusted routes based on the real time probes and collaboration with the MODiToNeS nodes that may be infrastructure nodes or fully ad hoc local nodes based on the local context sensing. We base our deployment on the real-world data traces of different probes for mobile networks across 112 countries and over 200 mobile providers obtained by netalyzr in [14] and [15]. We select traces from Germany as its number of mobile networks providers best suits our real world user communication trace [7]. For every mobile node we obtain the probability for the network spying on the web traffic by calculating the percentage of tests returning positive vs the total number of tests performed. For every mobile VOLUME 4, 2016 5317
  • 10. M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems network, we obtain the probability of it spying on web traffic by averaging the values obtained by all individual mobile nodes on this particular network. Based on the real cellular networks in Germany, we average privacy levels into five evenly distributed privacy threat levels .e.g. minimum (0%) such as ALICE and NETZCLUB, low (25%) such as M-NET, medium (50%) such as BASE, MEDION , high (75%) such as CONGSTAR, maximum (100%) such as FYVE. While in our previous work, we developed extensions to the ONE simulator [18] that utilise this data in order to return middle boxes presence probability discovered when perform- ing probing of different cellular networks, in this paper we feed this data to MODiToNeS to drive different testbed nodes’ behaviour (to act as middle boxes or not). To enable dynamic real world physical connectivity (and disconnections) among MODiToNeS platform nodes, we drive the MODiToNeS fire- wall configuration for each MODiToNeS testbed node with the of real world Facebook connectivity traces [7] during the whole time of the experiments. We range the privacy levels of the data being transmitted starting form maximum to minimum privacy requirements with three intermediary levels. We run 5 randomly selected combinations of sources and receivers for each cellular network privacy level. FIGURE 13. End-to-end privacy. 1) RESULTS Figure 13 shows that end to end privacy levels remain higher for MODiToNeS CogPriv approach than for cellular only and mobile social ad hoc communication independently of the level of presence of middle boxes in the cellular infrastructure i.e. ranging from no middle boxes to wide range of middle boxes, the performance of cognitive privacy drops from 100% privacy level to 80%. This is in contrast with the cellular network which drops end to end privacy linearly with the amount of the middle boxes in the cellular network. MODiToNeS CogPriv approach also outperforms fully local social ad hoc approach because the delays that are associated with the bundles time out and invoke the nodes to utilise cellular infrastructure that may have privacy leaks. FIGURE 14. End-to-end number of hops. Figure 14 shows statistical analyses of MODiToNeS CogPriv number of hops with increased number of middle boxes in the cellular architecture. We observe that the num- bers range between 1 and 4 across all levels of middle boxes presence. Figure 15 shows that MODiToNeS CogPriv delays increase slowly until the infrastructure is fully compromised at which point the delays become the same as the local ad hoc approach. The cellular network approach has the lowest FIGURE 15. End-to-end delays. 5318 VOLUME 4, 2016
  • 11. M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems delays but this is due to privacy being compromised and the traffic taking single hop (direct) cellular link between the end nodes. Figure 15 shows delay distributions for highly private traffic bundles when the cellular infrastructure contains dra- matically different amount of middle boxes. We observe that the delays are the lowest when the infrastructure is not compromised as the MODiToNeS CogPriv approach takes cellular single hope router to the destination. As MODiToNeS CogPriv discovers increasing number of middle boxes in the cellular networks, the delays will increase but still be significantly lower than the local ad hoc approach. Even though there are some bundles that may take up to 27 minutes until 60% of surveillance of the cellular network over MODiToNeS, the average still remains low and below 11 minutes. For the cellular network where there is 80% to 100% of middle box presence, the delays range from 45 minutes to 79 minutes. These sorts of delays are appro- priate for non-emergency applications where the users value their privacy and can tolerate delays such as regular daily checks for users with long-term medical conditions. FIGURE 16. End-to-end number of transitions. In Figure 16 we show the number of transitions between i MODiToNeS nfrastructure and MODiToNeS local ad hoc when the security of the cellular network decreases. It is interesting to see that while the number of hops is relatively low (reaching 4 for highly compromised cellular networks), up to 50% of these hops are transitions between the infras- tructure and local communication. This shows that supporting adaptive transitioning between infrastructure and local com- munication is highly beneficial. The previous figures have shown that delays and hop by hop counts increase as MODiToNeS CogPriv moves adap- tively from fully cellular mode to the fully opportunistic mode while managing very high levels of end to end privacy. More specifically, we show that the MODiToNeS CogPriv achieves privacy of end to end connections which is almost constant FIGURE 17. Short and long term node resource utilisation visualisation. while neither the delays nor the hop count is significantly increased. Figure 17 shows short term and long term CPU load, memory usage and IO usage for MODiToNeS CogPriv nodes. We observe that, despite complex algorithm and low VOLUME 4, 2016 5319
  • 12. M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems resources devices, MODiToNeS CogPriv memory usage remains firmly under the full usage. CPU load is in the lower half of the total CPU utilisation for the majority of time while IO at the critical level for the majority of time (note that this critical level has been administratively assigned to be 2 K/sec). VI. CONCLUSIONS AND FUTURE WORK We proposed a novel platform MODiToNeS that supports real time multi-layer and multi-dimensional communication and analysis distributed architectures which can combine various aspects of smart mobile social, transport and other CPS sys- tems with the particular focus on testing real world novel reliable and intelligent communications among potentially low resourced devices. We envisage increasing need for complex systems of devices including vehicles, humans and infrastructure. Within such systems, various communication paradigms need to be supported including the following: ad hoc communication among people, among vehicles (vehicle to vehicle), com- munication between vehicles and infrastructure (vehicles to road side units and vice versa), human and the vehicle (vehi- cle notifying and guiding the driver as well as the driver providing on the fly information that can potentially dif- fer from the vehicles information) and human and com- pany/home/hospital (human sharing information about their trip/health and getting information or instructions back). In this context, MODiToNeS platform can support the con- cept of Internet of Things joined with the concept of Internet of vehicles or mobile social networks representing future trends of smart transportation and mobility applications. Cur- rent research and services typically allow central remote real time monitoring of various information while MODiToNeS allows users to interact in real time with the prototypes where, query and add additional information on any unexpected events. MODiToNeS builds on and extends existing research to develop a prototype distributed system which allows rich interactivity with the end user and real time localised ana- lytics and predictions as well as remote data communication for non real time analysis. Capturing diverse collection of information locally (which can include any environment and context data), providing real time data analysis and predic- tion which is visualised and fed back to the users is key for increasing reliability and efficiency of communication in such environments. We envisage that MODiToNeS will play an important role when integrating and testing human behaviour in the design and development of Cyber Physical Systems in mobile social, mobile health care and vehicular networks for critical safety applications. REFERENCES [1] K. Scott and S. Burleigh, Bundle Protocol Specification, document RFC 5050, Nov. 2007. [2] Castalia Wireless Sensor Network Simulator, accessed on Jul. 25 2016. [Online]. Available: http://castalia.research.nicta.com.au [3] IBR-DTN, accessed on Jul. 25 2016. [Online]. Available: https://trac.ibr.cs.tu-bs.de/project-cm-2012-ibrdtn [4] M. Radenkovic, ‘‘Cognitive privacy for personal clouds,’’ Mobile Inf. Syst., vol. 2016, Apr. 2016, p. 17, Art. no. 7107103, doi: 10.1155/ 2016/7107103. [5] LXC—Linux Containers, accessed on Jul. 25 2016. [Online]. Available: https://linuxcontaingers.org [6] M. Radenkovic and N. Milic-Frayling, ‘‘Demo: RasPiPCloud: A light- weight mobile personal cloud,’’ in Proc. 10th ACM MobiCom Workshop Challenged Netw. (CHANTS), New York, NY, USA, 2015, pp. 57–58. [Online]. Available: http://dx.doi.org/10.1145/2799371.2799373 [7] A. Socievole, F. De Rango, and A. Caputo, ‘‘Wireless contacts, Facebook friendships and interests: Analysis of a multi-layer social network in an academic environment,’’ in Proc. IFIP Wireless Days (WD), Nov. 2014, pp. 1–7. [8] V. Pejovic and M. Musolesi, ‘‘Anticipatory mobile computing: A survey of the state of the art and research challenges,’’ ACM Comput. Surv., vol. 47, no. 3, 2015, Art. no. 47. [9] J. Leguay and F. Benbadis. (Feb. 2009). CRAWDAD Data Set Upmc/Rollernet (v.2009-02-02). [Online]. Available: http://crawdad. cs.dartmouth.edu/upmc/rollernet [10] M. Radenkovic, I. Kostadinov, and B. Wietrzyk, ‘‘Increasing communica- tion reliability in manufacturing environments,’’ in Proc. IEEE IWCMC, Dubrovnik, Croatia, Aug. 2015, pp. 1377–1383. [11] M. Radenkovic and S. Ha, ‘‘Low-cost mobile personal clouds,’’ in Proc. IEEE IWCMC, 2016. [12] EU FP7. SelSus: Health Monitoring and Life-Long Capability Manage- ment for Self-Sustaining, accessed on Jul. 25 2016. [Online]. Available: http://www.selsus.eu/ [13] E. Romero, J. Blesa, A. Tena, G. Jara, J. Domingo, and A. Araujo, ‘‘Cognitive test-bed for wireless sensor networks,’’ in Proc. IEEE DYSPAN, Apr. 2014, pp. 346–349. [14] N. Vallina-Rodriguez, S. Sundaresan, C. Kreibich, N. Weaver, and V. Paxson, ‘‘Beyond the radio: Illuminating the higher layers of mobile networks,’’ in Proc. 13th Annu. Int. Conf. Mobile Syst., Appl., Services (MobiSys), New York, NY, USA, 2015, pp. 375–387. [15] N. Vallina-Rodriguez, S. Sundaresan, C. Kreibich, and V. Paxson, ‘‘Header enrichment or ISP enrichment?: Emerging privacy threats in mobile net- works,’’ in Proc. ACM SIGCOMM HotMiddlebox, London, U.K., 2015, pp. 25–30. [16] A. Razaghpanah et al. (2015). ‘‘Haystack: A multi-purpose mobile vantage point in user space.’’ [Online]. Available: https://arxiv.org/abs/1510.01419 [17] M. Radenkovic, ‘‘Andre Grundy: Efficient and adaptive congestion control for heterogeneous delay-tolerant networks,’’ Ad Hoc Netw., vol. 10, no. 7, pp. 1322–1345, 2012. [18] A. Keränen, J. Ott, and T. Kärkkäinen, ‘‘The ONE simulator for DTN protocol evaluation,’’ in Proc. 2nd Int. Conf. Simulation Tools Techn. (Simutools), Brussels, Belgium,, 2009, Art. no. 55. [Online]. Avail- able: http://dx.doi.org/10.4108/ICST.SIMUTOOLS2009.5674 [19] J. Scott, R. Gass, J. Crowcroft, P. Hui, C. Diot, and A. Chaintreau. (May 2009). CRAWDAD Dataset Cambridge/Haggle (V.2009-05-29). [Online]. Available: http://crawdad.org/cambridge/haggle/20090529, doi: 10.15783/C70011 [20] M. Radenkovic, A. Benslimane, and D. McAuley, ‘‘Reputation aware obfuscation for mobile opportunistic networks,’’ IEEE Trans. Parallel Distrib. Syst., vol. 26, no. 1, pp. 230–240, Jan. 2015, doi: 10.1109/TPDS.2013.265. [21] T. Newman, A. He, J. Gaeddert, B. Hilburn, T. Bose, and J. Reed, ‘‘Virginia tech cognitive radio network testbed and open source cognitive radio framework,’’ in Proc. 5th Int. Conf. Testbeds Res. Infrastruct. Develop. Netw. Commun. Workshops (TridentCom), 2009, pp. 1–3. [22] V. Handziski, A. K¨opke, A. Willig, and A. Wolisz, ‘‘Twist: A scalable and reconfigurable testbed for wireless indoor experiments with sensor networks,’’ in Proc. 2nd Int. Workshop Multi-Hop Ad Hoc Netw., Theory to Reality (REALMAN), 2006, pp. 63–70. [23] A. Vahdat and D. Becker, ‘‘Epidemic routing for partially connected ad hoc networks,’’ Citeseer, Tech. Rep. [24] A. Lindgren, A. Doria, and O. Schelén, ‘‘Probabilistic routing in inter- mittently connected networks,’’ in Service Assurance With Partial and Intermittent Resources (Lecture Notes in Computer Science). 2004, pp. 239–254. [25] S. Schildt, T. Lorentzen, J. Morgenroth, W.-B. Pöttner, and L. Wolf, ‘‘Free-riding the BitTorrent DHT to improve DTN connectivity,’’ in Proc. 7th ACM Int. Workshop Challenged Netw. (CHANTS), 2012, pp. 9–16. [Online]. Available: http://dx.doi.org/10.1145/2348616.2348619 5320 VOLUME 4, 2016
  • 13. M. Radenkovic et al.: Toward Low-Cost Prototyping of Mobile Opportunistic Disconnection Tolerant Networks and Systems [26] M. Doering, S. Lahde, J. Morgenroth, and L. Wolf, ‘‘IBR-DTN: An effi- cient implementation for embedded systems,’’ in Proc. 3rd ACM Workshop Challenged Netw., 2008, pp. 117–120. [27] K. Fall, ‘‘A delay-tolerant network architecture for challenged internets,’’ in Proc. Conf. Appl., Technol., Archit., Protocols Comput. Commun. (SIGCOMM), 2003, pp. 27–34. [Online]. Available: http://dx.doi.org/10.1145/863955.863960 [28] F. Benbadis, J. Leguay. (Feb. 2009). CRAWDAD Dataset Upmc/Rollernet (V.2009-02-02), accessed on Jul. 25 2016. [Online]. Available: http://crawdad.org/upmc/rollernet/20090202, doi: 10.15783/C7ZK53 [29] NS3, accessed on Jul. 25 2016. [Online]. Available: https://www.nsnam.org/ [30] B. Lantz, B. Heller, and N. McKeown, ‘‘A network in a laptop: Rapid prototyping for software-defined networks,’’ in Proc. 9th ACM SIGCOMM Workshop Hot Topics Netw. (HotNets), 2010, Art. no. 19. [31] A. Bleakie and D. Djurdjanovic, ‘‘Analytical approach to similarity-based prediction of manufacturing system performance,’’ Comput. Ind., vol. 64, no. 6, pp. 625–633, Aug. 2013. [32] M. Hermann, ‘‘Artificial intelligence in monitoring and the mechanics of machining,’’ Comput. Ind., vol. 14, nos. 1–3, pp. 131–135, 1990. [33] W. Hu, A. Starr, and A. Leung, ‘‘A multisensor-based system for manu- facturing process monitoring,’’ Proc. Inst. Mech. Eng. B, J. Eng. Manuf., vol. 215, no. 9, pp. 1165–1175, 2001. [34] P. G. Maropoulos and D. Ceglarek, ‘‘Design verification and validation in product lifecycle,’’ CIRP Ann.-Manuf. Technol., vol. 59, no. 2, pp. 740–759, 2010. [Online]. Available: http://dx.doi.org/ 10.1016/j.cirp.2010.05.005 [35] (2015). ISO/IE IS 9000:2015—Quality Management. [Online]. Available: http://www.iso.org/iso/home/standards/management-standards/ iso_9000.htm MILENA RADENKOVIC received the Ph.D. degree from The University of Nottingham, U.K. and the Dipl.-Ing. degree from the University of Nis, Serbia. Her research spans the areas of mobile, delay and disconnection tolerant networks and services, intelligent P2P multimedia systems, mobile clouds, intelligent security and privacy, and their applications to different application domains. She has been the Principle Investigator of several EPSRC and EU grants. She currently works on a number of open research questions such as improving reliability and efficiency of mobile social and vehicular networks, designing new intelligent data routing protocols, improving energy efficiency of continuous large scale remote monitoring, and new adaptive security and privacy techniques for data transmission in such environments. She has organized and chaired multiple ACM and IEEE conferences, served on many ACM and IEEE conference program committees. She has been an Editor and a Guest Editor of many premium journals and published scientific papers in many premium venues including the IEEE TRANSACTIONS on VEHICULAR TECHNOLOGY, the IEEE TRANSACTIONS on PARALLEL and DISTRIBUTED COMPUTING, the Elsevier Ad Hoc Networks, the ACM CHANTS/Mobicom, the IEEE Multimedia, the MIT Press PRESENCE, and the ACM Multimedia. She has authored one international patent on real time saleable signal processing in the Internet and acted as a Scientific Expert for EU European Commission and Engineering Physics Scientific Research Council U.K. for over ten years. JON CROWCROFT (F’04) received the degree in physics from Trinity College, University of Cam- bridge in 1979, and the M.Sc. degree in computing, and the Ph.D. degree from UCL, in 1981 and 1993, respectively. He has been the Marconi Professor of Communications Systems with the Computer Laboratory since 2001. He has worked in the area of Internet support for multimedia communica- tions for over 30 years. Three main topics of inter- est have been scalable multicast routing, practical approaches to traffic management, and the design of deployable end-to-end protocols. Current active research areas are opportunistic communications, social networks, and techniques and algorithms to scale infrastructure-free mobile systems. He is a fellow the Royal Society, a fellow of the ACM, a fellow of the British Computer Society, and a fellow of the IET and the Royal Academy of Engineering. MUBASHIR HUSAIN REHMANI (M’15– SM’16) received the B.Eng. degree in computer systems engineering from the Mehran Univer- sity of Engineering and Technology, Jamshoro, Pakistan, the M.S. degree from the University of Paris XI, Paris, France, and the Ph.D. degree from the University Pierre and Marie Curie, Paris, France, in 2004, 2008, and 2011, respectively. He is currently an Assistant Professor with the COMSATS Institute of Information Technology, Wah Cantonment, Pakistan. He was a Post-Doctoral Fellow with the Uni- versity of Paris Est, France, in 2012. His research interests include cognitive radio ad hoc networks, smart grid, wireless sensor networks, and mobile ad hoc networks. He served in the TPC for the IEEE ICC 2015, the IEEE WoWMoM 2014, the IEEE ICC 2014, the ACM CoNEXT Student Workshop 2013, the IEEE ICC 2013, and the IEEE IWCMC 2013 conferences. He is currently an Editor of the IEEE COMMUNICATIONS SURVEYS and TUTORIALS and an Associate Editor of the IEEE Communications Magazine, the IEEE ACCESS, the Computers and Electrical Engineering (Elsevier), the Journal of Network and Computer Applications (Elsevier), the Ad Hoc Sensor Wireless Networks, the Wireless Networks (Springer) Journal, and the Journal of Communications and Networks. He is also serving as a Guest Editor of the Ad Hoc Networks (Elsevier), the Future Generation Computer Systems (Elsevier), the IEEE ACCESS, the Pervasive and Mobile Computing (Elsevier), and the Computers and Electrical Engineering (Elsevier). He is the Founding Member of IEEE Special Interest Group on Green and Sustainable Network- ing and Computing with Cognition and Cooperation. He received the cer- tificate of appreciation, an "Exemplary Editor of the IEEE COMMUNICATIONS SURVEYS and TUTORIALS for the year 2015" from IEEE Communications Society. VOLUME 4, 2016 5321