LibreSocial - P2P Framework for Social Networks - Overview

A P2P-Framework
for Social Networks
Jun.-Prof. Dr.-Ing.
Kalman Graffi
www.p2pframework.com
Jun.-Prof. Dr.-Ing. Kalman Graffi, P2P-Framework for Social Networks, www.p2pframework.com 1

A P2P-Framework for Social Networks
Overview – A quick 101

LifeSocial – A Secure P2P-based OSN Platform
History
 Developed since 2007
• > 55 students worked on it
• See you at CeBIT 2013
 Aiming at applicable results in p2p research
Goal
 Facebook-like user experience
• Basis functionality extendible through plugins
• Data-centric (profiles) and user-to-user (chat,video) interaction
• BUT: security guarantees
 Operator view
• Completely p2p-based
• BUT: with quality of service control and guarantees
 Research
• New application leads to new requirements
• New requirements to new results

Main Ideas in LifeSocial
General p2p platform
 Combining a wide set of useful modules
• Storage, messaging, security, caching, app-hosting, multicast, pub/sub …
• Distributed data structures, monitoring, automated self-control
 Social network on top of platform
• Build through “plugins” (apps), using platform and each other
• Extendable, configurable GUI supports app growth
Security goals
 Access controlled secure storage
 Secure (encrypted, authenticated, integer) communication
 No trust assumptions  anybody may be bad
Functional goals
 For the users: Facebook-like
 Also thinking of providers: quality monitoring and control plane

User View: Rich Functionality
Wide set of functionality
 GUI-Framework like in Eclipse
 Fast and user-friendly performance
 Plugin-based application:
• Profile
• Login
• Friends
• Groups
• Mails
• Photos
• Chat
• …
K. Graffi et al., “LifeSocial.KOM: A P2P-based Platform for Secure Online Social Networks”, In: IEEE P2P’10

User View

Management View

P2P Framework for Social Networks
Graphical User Interface Framework
Plugin GUI Plugin GUI Plugin GUI
Commands Interface
Optional Plugins
OpOtipotnioanl aPll uPgluingsins
Mandatory Plugins
Mandatory Plugins
Mandatory Plugins
Remote Operations:
Distributed Data
Structures: Prefix
tree, List, Set, Groups
Access Control
Identity Management
Structured P2P Overlay
Internet
Storage and
Replication
Testing
Communication
Channels:
Monitoring
1-to-1, Pub/Sub,
Aggregation Tree
User & Group
Management
P2P Framework Plugins and Apps GUI
App - Market

P2P Framework - Overlay
Identity Management
Structured p2p overlay
 FreePastry
 For security reasons
• Buckets as routing entries
• Redundant routing
• Iterative routing
Identity management
 To identify users uniquely:
 Create private & public key
 Use public key as node ID
• Integrates public key infrastructure
• Enables signing & encryption of
communication
User & group management
 Group keys
 Group member management
 Forming of nested groups (subsets)
and group key inheritance
Internet
User & Group
Management

P2P Framework - Storage
Access Control
Storage and
Replication User & Group
Identity Management
PAST
 Storage and replication
 Verification of authorship
 Modified to
• allow in place updates of data
• serve data from previous
downloaders (load balancing)
Access control
 Sign and encrypt content
 Access control based on
• User lists
• Groups
Internet
Management

P2P Framework – Communication and Data Structures
Distributed data structures
Remote Operations:
Distributed Data
Structures: Prefix
Access Control
Identity Management
 Normal approach:
• Get data, change locally, upload
 Remote operations:
• Send only information to change
• E.g. insert “alice” in list
 Supported structures:
• Set, list, groups, prefix tree
Communication channels
 1-to-1 (based on ID)
• E.g. to copy/stream file
 Topic-based pub/sub
• Using Scribe
 Streaming
• Using Splitstream
 Multi-criterion indexing and search
• E.g. Person name, age, location …
 Aggregation tree
• Sum, min, max, avg
• For individual topics
Internet
Storage and
Replication
Communication
Channels:
1-to-1, Pub/Sub,
Aggregation Tree
User & Group
Management
P2P Framework

P2P Framework - Services
Remote Operations:
Distributed Data
Structures: Prefix
Access Control
Identity Management
Monitoring
 Captures events from plugins
and in framework
 Uses aggregation tree to gather
statistics on system
Testing
 Enables to initiate remote
commands for testing
 E.g. Initiates in a 50 nodes network
defined behavior set (based on
measurements)
App – Market
 All Plugins are OSGi – bundles
 Can be loaded, installed at runtime
 Define dependencies
 App market:
• Host, search and install plugins
• Determine and get dependencies
Internet
Storage and
Replication
Communication
Channels:
1-to-1, Pub/Sub,
Aggregation Tree
User & Group
Management
Monitoring
Testing
P2P Framework
App - Market

LifeSocial: Plugins implementing a Social Network
Commands Interface
Optional Plugins
Remote Operations:
Distributed Data
Structures: Prefix
Access Control
Storage and
Replication
Identity Management
Plugins:
 Using the P2P Framework
 Written in Java, OSGi
• Open Services Gateway initiative
• Supports component model
(bundle/service)
 Functionality of social networks
 Easy Plugin-to-Plugin
communication
• Over shared storage
– E.g. photos
• Over Plugin ID based messaging
– E.g. Chat-app to chat-app
Mandatory Plugins
 Login, Profile, Friends, Wall,
Groups, Photos, Chat, Messaging,
Filetransfer (1-to-1), App-Market
Optional Plugins
 Voting, Multi-Chat, One-Click-
Filehosting, Forum Internet
Communication
Channels:
1-to-1, Pub/Sub,
Aggregation Tree
User & Group
Management
Monitoring
Testing
Mandatory Plugins
Mandatory Plugins
Mandatory Plugins
P2P Framework Plugins and Apps
App - Market

LifeSocial: Graphical User Interface
Graphical User Interface
 GUI Framework able to host
individual Plugin Views
 Allows to arrange views
• Save / load arrangements
Commands Interface
Optional Plugins
Remote Operations:
Distributed Data
Structures: Prefix
Access Control
Identity Management
Views
 Each Plugin comes with 1+
views
 Some views use several
Plugins
 E.g. friends selector
 E.g. message views
• inbox, outbox, compose
Internet
Storage and
Replication
Communication
Channels:
1-to-1, Pub/Sub,
Aggregation Tree
User & Group
Management
Monitoring
Testing
Mandatory Plugins
Mandatory Plugins
Mandatory Plugins
App - Market

A P2P-Framework for Social
Networks
P2P Overlay – Pastry
 Root of Trust
 Modified Routing – Buckets
 Iterative Routing
 Parallel Routing
 Handling Weak Nodes

P2P Framework - Overlay
Identity Management
Structured p2p overlay
 FreePastry
 For security reasons
• Buckets as routing entries
• Redundant routing
• Iterative routing
Identity management
 To identify users uniquely:
 Create private & public key
 Use public key as node ID
• Integrates public key infrastructure
• Enables signing & encryption of
communication
User & group management
 Group keys
 Group member management
 Forming of nested groups (subsets)
and group key inheritance
Internet
User & Group
Management

Pastry / FreePastry – Introduction
Pastry: P2P overlay
 Providing DHT functionality and Key-based Routing interface
Two metrics
 ID distance
 Physical distance (“Proximity”)
128-bit-IDs, arranged in a circle
 Variable b defines the size of the routing steps = 2^b; usual value = 4
 Tradeoff between routing table size and maximum number of hops
 Lookup hops scale with 푂(푙표푔2푏 푁 )
FreePastry: prototypical implementation of Pastry
 Current version 2.1: released on 13.3.2009
 Java based, Sun JDK version 1.5.0
 NodeID: 160 bits, 20 byte: 10 hexadecimal number
Custom modification (2012)
 Java package is now part of p2p framework code
 Allows modification

State Information in FreePastry
ID Space: [0 , 2^160[
 Randomly assigned while joining
 Base b (2 in example)
 b=4  hexadecimal in FreePastry
Routing table
 Used for prefix-based routing
 Typical size:
• log_(2^b) (N) rows
• 2^b – 1 entries per row
 Row nr. i contains only nodeIDs
sharing a prefix of length i with current
node
Leaf set
 |L| closest node IDs
 Typical size: L = 2^b or 2x2^b
Neighborhood set
 M entries (typically M = 2x2^b)
 Contains the nodeIDs and IP
addresses of locally closest nodes
Routing state of node 10233102, base 4

Pastry – Original Routing Procedure
In each routing step:
 Prefix-based forwarding:
 A node forwards a message to another node
• whose ID shares with the target key a prefix
• that is at least one digit (= b bits) longer
• than the prefix that is shared with the current node’s ID
If no such node is found:
 Numerical distance based forwarding:
 the message is forwarded to a node
• with the same shared prefix length which is numerically closer

Routing Protocol
Message for key K arrives at node X
 Let X= 10233102, b=2
1. Check if K in scope of Leaf Set
 E.g. K = 10233030
 Direct forwarding to 10233033
2. If not (1) use Routing Table
 Let l:= prefix length of K and X
 E.g. K = 10320102, l=2
 Check level 3, prefix 103  10-3-
23302
3. If not (1) and no routing table
entry
 E.g. K = 10233300
 Pick closest peer from routing table:
10233-2-32, as closer than 10233102
4. If X is closest to K than any node
in Leaf Set (and Routing Table)
 X is responsible for K, routing ends
Routing state of node 10233102, base 2

A
B
…
T
Z
X
Pastry – Node join
New node X wants to join
 A is assumed to be physically close to X
 Z is assumed to be responsible for the key “X”
Join protocol
 X asks existing node A to route JOIN message to key X
 JOIN message will be routed to node Z which is closest to key X
 A, Z and all nodes on the route send their state tables to X
X uses following sets as basis for its routing sets
 A’s neighborhood set
 Z’s leaf set
The n-th row of the routing table is copied from the n-th node encountered
during the JOIN message routing process
 n = 0: A’s row 0
 n = 1: B’s row 1…
Finally, X sends a copy of its state tables to all nodes contained in them
so that those nodes can update their state tables

Modification of Pastry’s Routing Table: Buckets
Introduction of Buckets
 Each routing table entry contains k
many contacts
 Allows for
• Parallel routing
• Node replacement
upon node failure
Coping with node failures
 Nodes leave unexpectedly (fail)
 For detection:
• Periodic checks of table entries
• Keep-alive messages
 If node does not answer: failed
• Failure in Leaf Set:
– Update entry with leaf set of
furthest node
• Failure in Routing Table:
– Ask nodes in same row as failed
node
– If all in row failed: as nodes in
higher row
Contact 1
Contact 2
Contact 3
…
Contact k

Pastry – Modified Routing Procedure
Prefix-based, parallel and iterative routing
 Loop until no closer node to target ID is revealed
• Lookup initiator sends out alpha parallel lookups
– To the alpha nodes sharing the longest prefix in his routing table with target ID
• Contacted nodes answer alpha contacts closest to the target ID
 Closest found node is responsible
Protocol details
 Alpha answers arrive randomly
• Process their results only
if their sent contacts are better
than previous contacts
 Joining protocol requires
adaptation

Modification of Pastry’s Routing Table: Weak Nodes
Handling of weak nodes
 Weak nodes should not
participate in
• Routing, routing maintenance
• Storage
 Simply marking weak nodes
• Odd port: strong node
• Even port: weak node
• Port information part of contact
info
• Other marking solution requires
additional marking information
(+data structures in code) or
signaling protocols
Routing to/from weak nodes
 Weak nodes are inserted only in
the leaf set (of close nodes)
 Their routing table also contains
only the leaf set entries

Root of Trust: Authenticated Node IDs
Approach
 Immutable UserID
• Identifying the user throughout the
system/application
• To be used by the plugins: in
friendlists, groups …
 Mutable NodeID = PublicKey
• PublicKey uses 160 bit Elliptic Curve
• Username and Password are
hashed, used as basis for the Private
Key
• Private Key is used for calculation of
Public Key
 Mapping from the NodeID to the
UserID stored at NodeID
• At IDs hash(UserID_i) (with i=1,…)
store link to NodeID
• Initial link at new user registration
– Requires no proof on user
– Is signed with new NodeID/PubKey
• Further changes of UserIDNodeID
link
– Are only allowed for previous signer
– Single Items only to be modified by
original author (signature check)
Goals
 Users should be authenticated
 Messages confidentially, integer
and authenticated sent/received
 Login from any device possible,
credentials in the network

Root of Trust: Effects
Integrated Publiy Key Infrastructure
 If UserID known, easy to obtain PublicKey
 PublicKey can be stored
Secure communication
 Confidential, integer, authenticated
 Sending messages
• Message is signed with Public Key (=PubKey) of sender
• Message is encrypted with PubKey of addressed peer
 Receiving messages
• Decrypt message using Private Key (=PrivKey)  auth., confidentiality
• Receiver checks signature of sender  integrity
 For ease:
• User-User specific symmetric shared key might be exchanged securely before
– E.g. using Diffie-Hellman
Secure Storage
 Items are signed when stored, can only be changed by original author

Properties of Pastry / FreePastry
Advantages
Well documented, clear APIs
 Modular, extendable software
 Large user base, still
maintained
Basic functionality
 Routing, DHT (key-value
mapping)
 Distributed storage
Disadvantages  solved
Now: support for heterogeneity
 All nodes are treated equally
 Strong, long-living peers should
do more
 See at storage part
Now: built-in security
mechanisms
 Identity attacks, routing attacks
 Sensitive to malicious nodes
Limited API  P2P Framework
 “Only DHT”
 Also requires sufficient
replication, additional services

Networks
Storage and Replication
 Replication
 Access Control
 Load-balancing

P2P Framework - Storage
Access Control
Storage and
Replication User & Group
Identity Management
PAST
 Storage and replication
 Verification of authorship
 Modified to
• allow in place updates of data
• serve data from previous
downloaders (load balancing)
Access control
 Sign and encrypt content
 Access control based on
• User lists
• Groups
Keys
 160 bit elliptic curve keys
 128 bit AES symmetric keys
Internet
Management

Positioning in the Network
User albums
object key =
„user name“+“album“
List of user albums:
1. object key a
2. object key b
3. object key c
4. object key d
User album A
Image x
Image y
...
object key a
List of images:
1. object key x
2. object key y
3. object key v
4. object key r
...
object key x
image
object key y
image
Distributed Storage
 Object ID based Routing
User album D
object key d
List of images:
1. object key n
2. object key m
3. object key k
4. object key l
...
P2P Overlay
 Peer ID based Routing
Internet
 IP based Routing

Document Types, Obvious Storage Keys
Image x
Image y
High granularity of stored data objects
Better load balancing of the resources
Used for
 Atomic data: profiles, login info, “emails”
 Linked lists: friend lists, groups, multicast
Allows for complex data structures
User album A
Image n
Image m
User Albums
storage key =
„user name“+“album“
List of user albums:
1. storage key a
2. storage key b
3. storage key c
4. storage key d
...
storage key a
List of images:
1. storage key x
2. storage key y
3. storage key v
4. storage key r
...
storage key x
image
storage key y
image
User album D
storage key d
List of images:
1. storage key n
2. storage key m
3. storage key k
4. storage key l
...
storage key n
image
storage key m
image
Profile
storage key p =
“User_Kalman_Graffi”
Name: Kalman
Age: 30
University:
Universität
Düsseldorf

Atomic Single Item Operations
Basis for security must be
 Easy to deploy, fast and reliable
Requirements for access control and item security
 All sensible information is stored in DHT (no trust assumed)
 Integrity / authentication
• Author is authenticated through signatures
• Signature allow to detect modifications
• Only one author for one data item
• Content integrity can be checked, false authorship can be rejected
 Confidentiality: only authorized users should be able to read content
• The storing node is typically not allowed
• Data items can/should be encrypted
Data object must be atomic
 Must contain all relevant security information
 Replication
• Multiple storage of the same content
• Find 1 copy to read, all copies to write consistently
• Replication independent of security
Support for inplace modification
Support for group-based rights
H(„my data“ )
= 3107
2207
2906
1008 1622 2011
PeerID = PubKey
3485
709
611
?

Replication Extension of Pastry: PAST
Idea
 Store objects also on k further nodes in leaf set
 Remember Key-based Routing (KBR):
• nodehandle [] replicaSet (key  k, int  max rank)
– Returns an ordered set of peers of magnitude (max rank)
on which replicas of the object with key k can be stored
– The nodes which become roots for the key k when the local node fails
• update(nodehandle n, bool joined)
– Upcall: informs that node n has either joined or left the local neighbor set
 If responsible peer fails
• New responsible node (is in replicaSet)
– Is informed of peer leave by update upcall
– Answers to lookups for new objects it is responsible for
– Calls new replicaSet and deployes replicas (if needed)

PAST Evaluation
Good
 ID related replication: 1 lookup sufficient to find object
 Replication ratio flexible (might depend on object / peer properties)
 Failed replica nodes are detected by overlay: easy to react
Drawback
 Replication not peer heterogeneity aware
• Weak nodes might be overloaded by replication task
• See next slide how to handle that in the P2P Framework
 Security
• Replicas all in one ID area: easier to attack
• Still open

Modification: Heterogeneity Aware Storage
Best strategy to match nodes
having and wanting a file?
Object i
Requester
 Considering:
• Memory, storage and bandwidth
 Leading to
• Balanced load regarding the
load provision in the system
Approach
 Responsible node remembers
downloading nodes (providers)
• Maintains a provider list
• Periodically checks their
bandwidth
– Through message size and
transmission time
• Forwards download query
(1 hop) to them based on load
• High capacity  high probability
to be chosen
P3
P4
P5
P1
P2
P6
P7
Object i
Requester
Object i
Provider
P3
P4
P5
Object i
Provider
P1
P2
P6
P7
10
Responsible
for object 3
Peer Qual./Load
P1
P2
P6
ok
good
weak
P7 ok
Get object 3
from peer 2
Provider to
use: peer 2

Discussion: Heterogeneity Aware Storage
Heterogeneity aware storage
 In addition to replication
 Local solution
• If responsible node wants to share load, he may, but does not have to
• Source of file irrelevant for requester
– As files are atomically authenticated
Periodic capacity checks
 Required some overhead
 Are propagated back, so that nodes are aware of their capacities
 Can be used for further purpose

Evaluation: Heterogeneity Aware Storage
Setup:
 10 Machines, 3x TestCaseLoadPhoto in 12 secodns
Evaluation:
 Red: responsible node, blue: a chosen provider

Simple Distributed Access Control for Single Objects
Goals:
 Authentification of hosts (original author)
• Allows „Write“ access control
 „Read“ access control on individual data items
Idea:
 Remember established public key infrastructure
 Write access control
• Initial write is free, data item is signed, authors Public Key is added
• Overwrite on existing Object ID only allowed to original author
• Protection against replay attacks:
– Random nonce is stored with item; nonce must also be signed with new item
• Storing nodes verifies signature
• ToDo: Replication nodes must also verify signature and verify correct
behavior of storing node
 Read access control:
• Encrypt all stored data with unique symmetric key: Key_S
• Encrypt the symmetric key for all privileged reader: (Key_S)enc(PubKey_i)
• Attach the encrypted symmetric key to the encrypted data

Example of Distributed Access Control
SharedItem
objectID Header
Privileged users
Payload
Signed CryptedItem
objectID Key list
userID A – key A
userID B – key B
userID C – key C
…
3
Byte array
containing
encrypted
SharedItem
Symmetric Key
Pub
User A
Encrpyted
with
Symmetric Key
Pub
User B
Encrpyted
with
Pub
User A
Pub
User B
[userID A] =
[userID B] =
1
extract
Serialized and encrypted with
symmetic key
2
userIDs
are public
keys
wrap symmetric key
with public key
4
5
Signature

Group-based Access Control
 Groups have own keys  Treated as single user
• Single data objects encrypted/signed with group keys
• Only group members can read / write
 Group key is stored in single item by group creator
• Read access for group members
Group provides for its members
 A symmetric key S_G and an asymmetric key pair (Public: e_G, Private: d_G)

Challenge: Hierarchical structures of groups
 Share content with combination of groups:
• G1 OR G2: Encrypt with 2 keys, provide both
• G1 AND G2: Encrypt data with first key,
encrypt result with second key
 Hierarchical groups: G1 subgroup of G2
Groups may contain
 User (keys)
 Group (keys)
Encrypted item has a reference
to the group used for encryption

Networks
Peer-to-Peer Framework
 Storage
• Distributed Data Structures
– Sets, Lists, Prefix-Hash Trees
• Remote Operations
• Access Control for Distributed Data Structures
 Information Cache and Callbacks
 Communication Channels
• 1-to-1, Multicast, Publish / Subscribe
• Search
 App-Store

P2P Framework – Communication and Data Structures
Distributed data structures
Remote Operations:
Distributed Data
Structures: Prefix
Access Control
Identity Management
 Normal approach:
• Get data, change locally, upload
 Remote operations:
• Send only information to change
• E.g. insert “alice” in list
 Supported structures:
• Set, list, groups, prefix tree
Communication channels
 1-to-1 (based on ID)
• E.g. to copy/stream file
 Topic-based pub/sub
• Using Scribe
 Streaming
• Using Splitstream
 Multi-criterion indexing and search
• E.g. Person name, age, location …
 Aggregation tree
• Sum, min, max, avg
• For individual topics
Internet
Storage and
Replication
Communication
Channels:
1-to-1, Pub/Sub,
Aggregation Tree
User & Group
Management
P2P Framework

Distributed Data Structures (DDS)
Motivation:
 Put / Get of single data items does not match social network
 Social network data structures:
• (Unorderd) Set: Friends, Group members, …
• (Ordered) List: Wall entries, Email posts, Forum entries
• Prefix Hash Trees: App categories, User categories …
Distributed Data Structures
 Concept: Single data items with
• Characteristic meta data
• Payload
• Pointers to next elements in the
„distributed“ data structure
On right: functions of the list
 Location of i-th element
– Key hash(„listname“ + i/bucketsize)

Basics of Distributed Data Structures
Interfaces
 Common access to a DDS instance
for application
 DDS in framework split in several
buckets
 Buckets handled individually
Requirements:
 Only owner allowed to edit/ delete a
list entry
 Everyone allowed to add a list entry
 Support groups: only group users
allowed to read or add a list entry
Buckets contain:
 Individual entries
• May belong to different users
• Number of entries per bucket
configurable
 Pointer to next bucket
Application using the Distributed Linked List

Distributed Data Structures: Write and Read Access
Add entry to the list
 Free to write:
• Sign element with private key
of the owner
 Write only for group members
• Sign element with private key
of the owner
• Encrypt each element of the
bucket with symmetric group
key
• Sign the bucket with private
group key
Read entry
 Find and retrieve entry
 Verify entry and bucket
signature
Edit an existing element
 Create and sign new entry
 Sign bucket (all entries of it)
 Storing node recognizes list
buckets and verifies signatures
of bucket and elements

Distributed Data Structures: Deletion of Entries
Deleting an existing element
 Deletion = overwrite with empty “Delete”-Entry (deletedItem)
 Create and sign new “Delete”-entry
 Sign bucket (all entries of it)
 Storing node recognizes list buckets and verifies signatures of bucket
and elements
Problem
 deletedItems stay in list
 With time: list might contain almost only deleted Items
Solution:
 Algorithm consolidates two buckets
• deletedItems shifted one bucket to the right
 Called after each write operation if list contains deletedItem
• Only users with write rights can sign consolidated buckets

Consolidate Protocol
Before
 BK contains a deletedItem
 Consolidate(BK,BK+1) is called
After
 BK  BK‘, BK+1  BK+1‘
 DeletedItem now on most right
position in BK+1‘
 All other elements shifted to left
by one position

Remote Operations
DistributedList use the idea of remote operations
 User wants to change a stored item
 User sends request for change to storing node
 Storing node performs change and stores the file
Advantage:
 User need not transfer whole file but just the changes
Problem:
 Write protection needs to be considered
 Storing node performs changes and stores the modified item
 But: storing node is usually not owner of the item
• Cannot compute valid signature
 User sends delta information, how to sign whole new data item?

Protocol for Remote Operation with Write Access
Node C wants to modify an object stored at S:
 If C does not have the current object
• It requests the current version of the file that should be changed from S.
 C has the current object
• If C already has a version of the file, it sends a hash of it to S.
 S compares the received hash with the hash of its current object version
• If the hashes differ: S replies with the current version
• If the hashes match: it acknowledges briefly.
 C performs its changes locally and afterwards computes the signature of
the modified file and the hash of the file that should be overwritten
 It sends a request containing the desired changes, the signature and the
hash of the file on which C locally performed the changes to S.
 S checks whether its current version and the version on which C has
performed its changes are the same by comparing the hashes
• If they are not equal, step 3 is applied.
• If the hashes are equal: S performs the desired changes and then inserts the
file into the network using the signature received from C.

Evaluation of the Remote Operation and Deletion Consolidation
Setup: 1000 operations
 Setting A presented
Traffic savings:
 Through remote operations
and buckets with 10 entries
 Consolidation further saves
traffic
Action distribution:
Consolidation
 Reduces number of
deletedItems in list drastically

Accessing the Data: Information Cache and Callbacks
Pull approach: Information
Cache
 Interface: getData(key)
 Immediate response:
Plugins Plugins
Remote Operations:
Distributed Data
Structures: Prefix
Access Control
Identity Management
• Data
– Lifetime x min, then new
lookup
• Pending
– While pending: data is
retrieved
• Not available
– Negative lookup
 Allows data reuse by several
plugins – information container
Internet
Storage and
Replication
Communication
Channels:
1-to-1, Pub/Sub,
Aggregation Tree
User & Group
Management
P2P Framework
Information Cache

Accessing the Data: Information Cache and Callbacks
Push Approach: Callbacks /
Continuations
 Plugins register for data(key)
 If lookup for it is successfull
• Data is deliverd to waiting plugin
Framework supports different
situations:
 Continuation - Asynchronous
operations
• e.g. storing and continue after it
 Receiver - Register for receiving
content
• e.g. message receiving functions
 Listener - Listen to ongoing
operations
 e.g. status reports for file
transfers

Communication Structures
1-to-1 (based on ID)
 E.g. to copy/stream file
 For files also 1-click-hosting
• Store encrypted (Key K) file
• Under specific ID
• Retrieve by (ID+Key) (“link“)
Scribe: topic-based pub/sub
 Using Scribe
 Create channel, e.g. for
hash(“p2phhu”)
 Send invitation via 1-to-n to
selected users  they join
p2phhu channel
 Messages sent to channel
p2phhu are received by
subscribed users
1-to-n
 Send message 1-to-1 to n users
Multi-criterion indexing and
search
 File has metadata: attributes
• Attributes: E.g. Person name,
age, location …
 Attributes have values
 Search for file with attr_i = x_i
Aggregation tree
 Sum, min, max, avg, std dev
 For individual topics 
• One channel per metric

Main Communication Classes
Communication Classes
 1-1, 1-N: MessageChannel
• Identified by unique name
• Both comm. partners need to create
this channel
• 1-to-N: Sender defines list of
receivers
 N-to-m: TopicChannel
• Identified by unique name
• Participants subscribe at unique
name
• All messages sent to list are
forwarded to subscribers
 Aggregate
• Attribute<T> holds for a single
measurement:
– name, unit, aggregation strategy
and the sensor it was measured
• Aggregation strategies
– Sum, count, average, min, max…
• Statistics on all nodes in AggChannel

Multi-Criterion Search
Query Language:
 Keyword: describes a document
through a string
 Attribute/Value: attribute
attached with numeric value
Query:
 collection of keywords or
attribute/value pairs
 (Keyword | (Attribute = Value))*
void indexItemWithTags ( Serializable
identifier,Collection <String > keywords );
void removeItemIndexWithTags ( Serializable
identifier, Collection <String > keywords );
Collection <Serializable>
searchAllItemsWithTags
(Collection <String > keywords );
Serializable searchItemWithTags (Collection
<String > keywords);
Distributed Query Engine:
Indexing
 Takes any object and a collection of
keywords
 Afterwards the object can be retrieved
throughout the p2p network
Querying
 Takes a query (collection of keywords)
 Returns the collection of objects,
where the query-keywords subset of
index-keywords of the objects

Implemented Search Engines, Common Indexing
LocalJoin
NetworkJoin
ScribeJoin
BloomNetJoin

Testing the Query Engines
Observation:
 Churn disrupts system performance (even at
5%)
 ScribeEngine has best query latency
• but long reconnect reconnect and indexing
latencies
 BloomJoin does not signicantly reduce
network costs
 ScribeEngine handles queries with many
keywords good
 The others handle queries with very popular
keywords good

Networks
Peer-to-Peer Framework - Continued
 Monitoring
• Tree-based Monitoring
 Testing
• Test-Plugin

P2P Framework - Services
Remote Operations:
Distributed Data
Structures: Prefix
Access Control
Identity Management
Testing
 Enables to initiate remote
commands for testing
 E.g. Initiates in a 50 nodes network
defined behavior set (based on
measurements)
Monitoring
 Captures events from plugins
and in framework
 Uses aggregation tree to gather
statistics on system
App – Market
 All Plugins are OSGi – bundles
 Can be loaded, installed at runtime
 Define dependencies
 App market:
• Host, search and install plugins
• Determine and get dependencies
Internet
Storage and
Replication
Communication
Channels:
1-to-1, Pub/Sub,
Aggregation Tree
User & Group
Management
Monitoring
Testing
P2P Framework
App - Market

Motivation for Testing and Monitoring
Test Plugin for LifeSocial
 Coordinated automated
execution & delegation of tasks
 Easy test setup
 Generation of content
 Expandable / Support for new
LifeSocial plugins
Wall
23 %
Initial
71 %
Messages
16 %
Profile and
Friends
71 %
Search
9 %
11 %
64 %
22 %
10 %
16 %
69 %
9 % 5 %
Photos Communities
67 %
86 %
82 %
Monitoring:
 Information on system status can be used
for optimized decisions
• E.g. peer count defines size of time-to-live
• E.g. churn pattern defines stabilization
frequency
 Necessary to identify (bad) quality of
mechanisms
• Too much overhead
• Too slow routing
• Efficiency leaks
 Helps in designing better mechanisms

Test Plugin I – Easy Setup & Generation of Content
Automated Setup using Java RMI
 “First” node creates P2P network and serves as bootstrap node
(Master node)
 Other nodes join P2P network and await orders (Slave nodes)
User content is generated without need for user interaction
 Resource folder contains photos and files
 Values for messages, group names, city names, country names, user
interests, usernames, …
• Example: login.txt file contains more than 4000 possible usernames

Test Plugin II – Automated Execution of Activities
Activity: Execution of a social network function
 Has parameters and preconditions
 Example
• Activity: Send a chat message to a friend
• Precondition: Logged in and at least one friendship to another user
• Parameter: Actual message content and name of the friend
• If user has no friendship to another user, Test Plugin automatically sends
a friend request to a random (online) user
Preconditions are fulfilled if necessary
If activity parameters are missing
 they are chosen randomly from
the resource files
Wall
23 %
Initial
71 %
Messages
16 %
Profile and
Friends
71 %
Search
9 %
11 %
64 %
22 %
10 %
16 %
69 %
9 % 5 %
Photos Communities
67 %
86 %
82 %

Test Plugin II – Automated Execution of Activities
View Group
Send Friend
Request
View Wall
Bootstrap Server Running
Existing User
Logged in User
Register
Login
Relogin Logout
Change Profile
Picture / Information
View
Profile
Friendship to other User
Send Chat Message
Active Chat Conversation
Reply to
Chat Message
Send Offline
Message
Active Conversation
Reply to
Offline Message
Create Group
View Friends
Groups
Group exists
Join Group
Member of Group
Send Group
Message
Send Wall Post
Wall Post Exists
Comment Wall Post
Search
User / Group
Incoming Friend Request
Accept Friend
Request
Existing Photo
View Friends
Photo
Create Photo Album
Existing Photo Album
Upload Photo
View Chat History
View
Offline Message

Test Plugin III – Delegation of Tasks
Every test participant can delegate tasks (social network
activities) to other test participants
 Using 1-m messaging communication
Queue with remaining tasks to execute
Timed test plans
Realisitic churn model

Structured
Overlay: DHT
Monitoring
Is everything running fine?
How to debug and to gain insight?
How to improve the running
system?
Underlay:
The Internet
H(„my data“ )
= 3107
2207
7.31.10.25
12.5.7.31
peer-to-peer.info
95.7.6.10
berkeley.edu planet-lab.org
86.8.10.18
2906
3485
2011
1008 1622
709
611
89.11.20.15
?

Design decisions in SkyEye.KOM
New layer (vs. integrated)
 New layer allows wider applicability
 Set on top of KBR-compatible structured p2p overlays
Proactive (vs. reactive)
 System state information is continuously interesting for all users
Monitoring topology: tree (vs. bus, ring, star, mesh)
 Fixed out and in degree
Position assignment: dynamic and deterministic
 Deterministic IDs used in topology, dynamically resolved with DHT
For all structured P2P overlays
 Covered by DHT-function: route(msg, key), lookup(key)

Reliable structured p2p overlay
 “Key-based Routing” – operations
• boolean isMyKey(Key K)
• void route(key K, Message M, node hint)
Building a tree topology
 Introduce new overlay layer
• With own ID space ([0,1[)
 Create tree topology in new overlay
• Using routing of p2p structured overlay
Concept of new layer
 Decouples from specific p2p overlay
 Unified ID space [0,1]
Assumptions

Tree-based Monitoring Mechanism
Idea:
 Create (additional) tree topology
 Protocol:
• Periodically
– Calculate aggregate of own local view and received from child nodes
– Send aggregate to parent node
• Root calculates global view
– And passes global view to all peers
Used in the p2p framework: SkyEye.KOM
 Assumes structured p2p overlay
 Aims at high precision with low overhead

SkyEye.KOM: Tree Topology
Tree of information domains
Domain Domain ID
0.09 0.2 0.31 0,4 0.5 0.6 0.75 0.9
0 1
 Domain: ID interval
• E.g. [0, 0.5[ or [0.75, 0.875[
• Largest domain, level 0: [0,1[
 Domain ID: “middle value” in interval
 Domain size split in β parts per level
Domain IDs build tree topology
 Node degree: β child nodes
 Tree topology of domains does not change
over time!
 Assignment of peers to domains dynamic
Peers to Domain ID assignment
 Peers calculate Domains in which they are
located
 For those domains, they calculate the
Domain IDs (퐾푝푙
)
 If peer is responsible: position defined
1
10
50
20
30
40
45
15 P2P Overlay
Internet
0.25
0.375
0,3125
0.75
0.125 0.625 0.875
0.3125
0.5

SkyEye.KOM: Communication
Tree-overlay  p2p overlay
 Reconvert 푡푟푒푒퐼퐷 ∈ [0,1[
to 푝푒푒푟퐼퐷 ∈ {0, … , 2160 − 1}
 Coordinator:
• Responsible for Domain ID
• Check via DHT function
– isMyKey(Key K)
 For communication in tree
• Use route-function of overlay
• route(Msg M,Key K,Node next)
Example tree
 Tree degree (β) = 2
• Results in logarithmic tree size
 Balanced, if ID space balanced
 Not always β children
• Peers may be Coordinators at
various levels

SkyEye.KOM: Communication Protocol
Gathering global view
 All peers measure local status 푥푖
 Periodically sent to parent peer
•  Update Interval (UI)
1. Independent updates
in UI intervals per node
2a
Aggregated
view
1b
2. ACKs with view of parent
peer for every update
1β
2b
… 1a
β child nodes
2β
Aggregation
 Direct: count, sum, minimum,
maximum, sum of squares
 Derived: mean, variance, std.
deviation
Dissemination of global view
 Global view in root
 Every update message is
acknowledged
 Contains global view from level
above
Global view
Local measures, (synchronized signal in simulations)

Aggregation Functions

Activity Amount of Repetitions Planned Duration (in
Minutes)
Login 1 15
Change Profile Picture 1 3
Send Friend Request 10 3
Create Photo Album 10 2
Upload Photo 200 15
View Friend’s Photo 100 10
Join Group 30 10
Send Group Message 100 10
View Group Messages 10 5
Send Wall Post 100 10
Comment Wall Post 100 50
View Friend’s Wall 5 5
Random Activities - 30
Logout 1 50
Example: Test Plan

Example: Test 3 – Results
Amount of Nodes Available and Used Storage
60000
50000
40000
30000
20000
10000
Storage Dispatcher Size
Used Storage
Available Storage
125
100
75
50
25
0
Nodes
Global network nodes
0 60 120 180 240
Nodes
Time [m]
0
0 60 120 180 240
Data [MB]
Time [m]

Load – Messages Load – StorageItems
Social Network Load - Messages
25000
20000
15000
10000
5000
Social Network Load - StorageItems
Groups created
Photo albums created
Photos uploaded
Login items created
Profiles created
14000
12000
10000
8000
6000
4000
2000
0
Group Messages sent
Wall Posts sent
Wall Comments sent
Friend Requests sent
0 60 120 180 240
Units
Time [m]
0
0 60 120 180 240
Units
Time [m]

Photos Plugin Wall Plugin
12000
10000
8000
6000
4000
2000
Wall Plugin
Wall posts send
Wall posts commented
Walls requested
90000
80000
70000
60000
50000
40000
30000
20000
10000
0
Photos Plugin
0 60 120 180 240
Units
Time [m]
Albums created
Photos uploaded
Albums stored
Photos stored
0
0 60 120 180 240
Units
Time [m]

Groups Plugin Login Plugin
700
600
500
400
300
200
100
Login Plugin
Login Items created
Login Items stored
14000
12000
10000
8000
6000
4000
2000
0
Groups Plugin
Groups Items created
Group Items stored
Group Messages sent
0 60 120 180 240
Units
Time [m]
0
0 60 120 180 240
Units
Time [m]

Storage per Node Replication Count
Storage Used per Node
Maximum Storage Used per Node
Mean Storage Used per Node
Minimum Storage Used per Node
1e+06
100000
10000
1000
100
10
Replication Count
Overall Replications
Maximum Replications per Node
Mean Replications per Node
Minimum Replications per Node
600
500
400
300
200
100
0
0 60 120 180 240
Data [MB]
Time [m]
1
0 60 120 180 240
Units
Time [m]

Bandwidth – Messages Traffic
Bandwidth - Messaging
60
50
40
30
20
10
Bandwidth - Storage
Bandwidth - Data sent
2500
2000
1500
1000
500
0
Bandwidth - Messages sent
0 60 120 180 240
messages/sec
Time [m]
0
0 60 120 180 240
KB/sec
Time [m]

Amount of Errors Memory Usage
FreePastry Errors
900
800
700
600
500
400
300
200
100
Memory Usage
Maximum memory used by an instance
Mean memory used by an instance
Minimum memory used by an instance
6000
5000
4000
3000
2000
1000
0
0 60 120 180 240
Errors
Time [m]
Pastry errors
Decryption errors
PAST errors
0
0 60 120 180 240
Memory [MB]
Time [m]

Test Results – Summary I
Synthetic Behavior of P2P Framework tested with 25, 50 and 100 nodes
 Amount of actual executed activities measured (uploaded photos, commented
wall posts, …)
 Measured values match the expected values (according to the test plans)
 LifeSocial behaves as expected during synthetic tests
Performance
 All activities finished in scheduled time or earlier (except for commenting wall
posts)
• > 19.000 photos in 13 minutes, 9600 group messages in 8 minutes, 9600 wall posts sent
in 7 minutes
Bandwidth usage per node
 Always below 30 Kb/s
 In average: around 3 Kb / s
Data Storage
 Each Item replicated at least 2 times (matches defined replication factor)
 An average node contributed around 200 Mb storage space with a max-min load
divergence of 350Mb:50Mb

Test Results – Summary II
Monitoring and Test Plugin work reliable
 Small variances on monitoring results, still precise
 Test Plugin is a powerful tool to coordinate large-scale distributed tests
But: Prior tests revealed errors in LifeSocial
 Memory leak in FreePastry (has been fixed)
 Failure in MessageDispatcher
• Deadlock leads to unresponsive application
• Chat Plugin has been disabled for remaining tests
 Error-prone usage of LoginItem to obtain node ID
• If the application crashes, the system assumes the user is still online
Failures in Search & Wall Plugin occur in unstable test
environment (with churn)
 Failure in Search Plugin
• Leads to endless search operations
 Failure in Wall Plugin
• ClassCastException leads to application crash

Appstore - Requirements
Appstore: Offer and retrieve new plugins
Some Requirements:
 Two roles: plugin provider and plugin user
 Plugin provider can publish new plugins, release new versions, edit or
delete them
 Plugin user can search for plugins, download, install them and
manage installed plugins
 Installed plugins: start/stop/update/uninstall
Related GUI for the above mentioned functionalities:
 Publisher View
 Search View
 Installed Plugins View

Appstore - Details
Plugin – Model
 Plugin object
 Metadata: Plugin meta, version meta
Private repository (with unique user-related name)
 Contains the information of:
• Published plugins
– Offered plugins
• Installed plugins
– In order to maintain app-list in the network
– Allows to install all apps at other device
Publisher actions: publish, remove, release new version, edit metadata
 Storage: Plugin object and metadata  network
 Meta-information of the published plugins  private repository
Search actions: precise search by name, download and install
 Exact plugin name is required
 Output: list of avaliable versions
Plugin user actions: start, stop, update, uninstall

Networks
Plugins and Graphical User Interface
 Mandatory Plugins
 Optional Plugins
 GUI Framework
 Perspectives
 Individual Views

LifeSocial: Plugins implementing a Social Network
Commands Interface
Optional Plugins
Remote Operations:
Distributed Data
Structures: Prefix
Access Control
Storage and
Replication
Identity Management
Plugins:
 Using the P2P Framework
 Written in Java, OSGi
• Open Services Gateway initiative
• Supports component model
(bundle/service)
 Functionality of social networks
 Easy Plugin-to-Plugin
communication
• Over shared storage
– E.g. photos
• Over Plugin ID based messaging
– E.g. Chat-app to chat-app
Mandatory Plugins
 Login, Profile, Friends, Wall,
Groups, Photos, Chat, Messaging,
Filetransfer (1-to-1), App-Market
Optional Plugins
 Voting, Multi-Chat, One-Click-
Filehosting, Forum Internet
Communication
Channels:
1-to-1, Pub/Sub,
Aggregation Tree
User & Group
Management
Monitoring
Testing
Mandatory Plugins
Mandatory Plugins
Mandatory Plugins
P2P Framework Plugins and Apps
App - Market

LifeSocial: Graphical User Interface
 GUI Framework able to host
individual Plugin Views
 Allows to arrange views
• Save / load arrangements
Commands Interface
Optional Plugins
Remote Operations:
Distributed Data
Structures: Prefix
Access Control
Identity Management
Views
 Each Plugin comes with 1+
views
 Some views use several
Plugins
 E.g. friends selector
 E.g. message views
• inbox, outbox, compose
Internet
Storage and
Replication
Communication
Channels:
1-to-1, Pub/Sub,
Aggregation Tree
User & Group
Management
Monitoring
Testing
Mandatory Plugins
Mandatory Plugins
Mandatory Plugins
App - Market

Current Plugins
Login
 Create and login to user account
Profile
 Create and share personal information
Friends
 Link to friends
Mails
 Send and receive messages
Photos
 Upload, share and watch photos
Wall
 Write public messages, post own
status, comment messages
Chat
 Chat with a friend
Multi-Chat
 Create / Join a topic channel and chat
Browser
 Extended views to display formated
entries or websites
Groups
 Join groups and use dedicated working
environments
Forum (Group element)
 Create threads and topics
 Discuss on topics
Voting
 Create polls, define answers, invite
participants for public/private votes
File Storage
 One-click file hosting: upload, get a link,
share the link and allow other users to
download files
App Store
 Create and share new plugins
 Maintain own plugin list independent of used
device
Monitoring
 Monitor usage of the network
 Existing graphical user interface
Test plugin
 Remotely control network peers for
automatic test plan execution
 Content generation and real user
environment simulation

(Outdated) Plugin Architecture Overview

Plugins to GUI Relation

User View

Management View

Graphical User Interface Details
Task of views
 Combine functionality from framework to applications
 Often uses several plugins
Plugins might come with 0-n views
 Email has 2+ views
• Compose, inbox/outbox
 Some plugins have no views
• Search plugin only as helper
Perspectives
 View combinations and positions can be stored as perspectives
 Allows for customized allocations
• E.g. Working perspective, communincation perspective …

Networks
Outlook – Future Work

Current Steps in LifeSocial
Ongoing project group (16 students)
 “A Peer-to-Peer Framework for Social Networks”
 Goal
• Finalize p2p framework: secure, quality-controlled, …
• Add further essential functionality to framework
– Live multimedia streams (chatting, streaming)
• Add more novel applications
2014 presentation at CeBIT
 Like 2009, 2010, 2012, 2013
Contributions for the community
 New applications induce new research challenges
 Publish p2p framework as open source
 Initiate a community for further developments

Several Open Topics and Application Areas
E-Learning Environment
 Live lectures, tests
 Data repository
Realtime Communication
 Voice / video
P2P Wiki
 Versioning
P2P Normsetting Environment
 Proposals, Votes
 Decisions
General Peer-to-Peer Framework
Service and Ressource Access Networking and Storage Monitoring and Control Security and Privacy

Publications
P2P Framework – LifeSocial Overview
 K. Graffi, C. Groß, D. Stingl, D. Hartung, A. Kovacevic, R.
Steinmetz „LifeSocial.KOM: A Secure and P2P-based Solution
for Online Social Networks“. In: Proc. of the IEEE Consumer
Communications and Networking Conference, (IEEE
CCNC’11), Januar 2011
 K. Graffi, C.Groß, P. Mukherjee, A. Kovacevic, R. Steinmetz:
„LifeSocial.KOM: A P2P-based Platform for Secure Online
Social Networks“. In: Proc. of the 10th IEEE International
Conference on Peer-to-Peer Computing, (IEEE P2P’10),
August 2010.
 K. Graffi, S. Podrajanski, P. Mukherjee, A. Kovacevic, R.
Steinmetz: „A Distributed Platform for Multimedia
Communities“. In: Proc. of the IEEE International Symposium
on Multimedia (IEEE ISM ’08), December 2008.
Security
 K. Graffi, P. Mukherjee, B.Menges, D. Hartung, A. Kovacevic,
R. Steinmetz: „Practical Security for P2P-based Social
Networks“. In: Proc. of the 9th IEEE International Conference
on Local Computing Networks, (IEEE LCN’09), August 2009.
Monitoring
 K. Graffi, D.Stingl, J. Rueckert, A. Kovacevic, R. Steinmetz:
„Monitoring and Management of Structured P2P Systems“. In:
Proc. of the 9th IEEE International Conference on Peer-to-Peer
Computing, (IEEE P2P’09), September 2009.
 K. Graffi, A. Kovacevic, S. Xiao, R. Steinmetz: „SkyEye.KOM:
An Information Management Over-Overlay for Getting the
Oracle View on Structured P2P Systems“. In: Proc. of the 14th
IEEE International Conference on Parallel and Distributed
Systems (IEEE ICPADS’08), December 2008.
 K.Graffi, „Monitoring and Management of Peer-to-Peer
Systems“, Technische Universität Darmstadt, 2010.
Heterogeneity handling
 K.Graffi, S. Kaune, K.Pussep, A.Kovacevic, R.Steinmetz:
“Load Balancing for Multimedia Streaming in Heterogeneous
Peer-to-Peer Systems“. In: Proc. of the 8th ACM SIGMM Int.
Workshop on Network and Operating Systems Support for
Digital Audio and Video (ACM NOSSDAV ’08), May 2008.
Related evaluation tools:
 M. Feldotto, K. Graffi: “Comparative Evaluation
of Peer-to-Peer Systems using
PeerfactSim.KOM “. In Proc. of the IEEE
International Conference on High Performance
Computing and Simulation (IEEE HPCS ’13),
2013
 K. Graffi: “PeerfactSim.KOM: A P2P System
Simulator – Experiences and Lessons
Learned“. In: Proc. of the IEEE International
Conference on Peer-to-Peer Computing, (IEEE
P2P’11), August 2011
Related load-balancing / heterogeneity
awareness
 L. Bremer, K. Graffi: “Symbiotic Coupling of
P2P and Cloud Systems: The Wikipedia Case“.
In Proc. of the IEEE International Conference
on Communications (IEEE ICC ’13), 2013
 P. Wette, K. Graffi: “Adding Capacity
Awareness and Load Balancing to
Homogeneous Distributed Hash Tables“. In
Proc. of the IEEE International Conference on
Networked Systems (IEEE NetSys ’13), 2013
 K. Graffi, C. Groß, D. Stingl, H.Nguyen, A.
Kovacevic, R. Steinmetz: „Towards a P2P
Cloud: Reliable Resource Reservations in
Unreliable P2P Systems“.In: Proc. of the 16th
IEEE International Conference on Parallel and
Distributed Systems, (IEEE ICPADS’10),
December 2010

A Peer-to-Peer Framework for Social Networks
More information: www.p2pframework.com

LibreSocial - P2P Framework for Social Networks - Overview

More Related Content

Viewers also liked (20)

Similar to LibreSocial - P2P Framework for Social Networks - Overview (20)

More from Kalman Graffi (19)

Recently uploaded (20)

LibreSocial - P2P Framework for Social Networks - Overview