Big data and machine learning for network research problems
2 likes231 views
The document discusses how big data and machine learning techniques can be applied to network research problems. It describes using a random neural network (RNN) reinforced by reward and punishment to select optimal paths between network nodes based on metrics like delay. The RNN algorithm chooses probe paths, measures delays, and sends packets along minimum delay paths, rewarding neurons for good paths and punishing neurons for bad paths to guide future path selection. The approach was evaluated in a self-healing overlay network that monitors internet path quality between nodes and reroutes traffic based on performance metrics, showing the potential for machine learning to optimize network routing using collected data.
![Random Neural Network (RNN)
• At each time interval, the algorithm chooses a subset of paths to probe, and measures the sum of
edge delays in the probed paths.
• The algorithm then sends its packet over the minimum latency path among those it has probed.
• Each neuron in RNN represents a path from source to destination.
• RNN helps to select K paths to destination out of N paths.
• State of n connected neurons are represented by a vector k(t) = [k1(t), k2(t), . . . , kn(t)], where ki(t)
is a non-negative integer valued random variable representing the “potential” of the i-th neuron
being excited at time t.
16](https://image.slidesharecdn.com/bigdataandmachinelearningfornetworkresearchproblemssabidur2016-161223073129/85/Big-data-and-machine-learning-for-network-research-problems-16-320.jpg)
Big data and machine learning for network research problems
- 1. Big data and machine learning for network research problems Sabidur Rahman Friday Group Meeting December 22, 2016 krahman@ucdavis.edu http://www.linkedin.com/in/kmsabidurrahman/
- 2. Agenda • Big data and machine learning • How these two fit into networking world? • Random Neural Network • Reinforcement learning • Paper review: O. Brun, L. Wang, and E. Gelenbe, “Big data for Autonomic Intercontinental Overlays”, IEEE Journal on Selected Areas in Communications, vol. 34, no. 3, March 2016. 2
- 4. Machine learning • Network today is producing large amount of data. • What to do with the collected data? • Network performance enhancement • Smarter routing decision • Network management and planning • Network automation • Analytics and automation of this huge amount of data depends largely on data analytics and machine learning techniques. 4
- 5. Random Neural Network (RNN) • When a biophysical neuron is excited, it transmits a train of signals, called spikes (firing of signals), along its axon to either excite or inhibit the receiving neurons. • The combined effect of excitatory (positive) and inhibitory (negative) inputs change the potential level of the receiving neuron and determine whether it will become excited. • Neurons in RNN interact by exchanging positive and negative spiking signals with certain probability. • RNN model is used in pattern recognition, classification, image processing, combinatorial optimization and communication systems. 5 Timotheou S. The random neural network: a survey. The computer journal. 2010 Mar 1;53(3):251-67. Excitatory = causing, or constituting excitation Inhibitory = suppress or restrain an impulse
- 6. Reinforcement learning The basic reinforcement learning model consists of: • Set of environment and agent states S • Set of agent actions A • Policies of transitioning from states to actions • Rules that determine reward (or punishment) of a transition (G and R) 6 Sutton RS, Barto AG. Reinforcement learning: An introduction. Cambridge: MIT press; 1998 Mar 1.
- 7. Paper review O. Brun, L. Wang, and E. Gelenbe, “Big data for Autonomic Intercontinental Overlays”, IEEE Journal on Selected Areas in Communications, vol. 34, no. 3, March 2016. 7
- 8. Introduction • Multihop intercontinental network that uses IP to communicate between nodes. • IP routing often results in sub-optimal paths with respect to metrics such as end-to-end round trip delay. • QoS of such routes can be optimized using the collected network statistics. • Objective: Select route to provide better QoS than IP. • Machine learning based scheme to exploit large scale data collected from communicating node pairs. 8
- 9. (Big) Data collection • ICMP-based ping to collects round-trip delay data at the overlay nodes, every two minutes • 24 hours with 20 overlay nodes: collect up to some 2.7 × 10^5 data points per day. 9
- 10. 10
- 12. Routing overlay • Routing overlay is a method to improve performance without changing underlying network. • Overlay network is formed by software routers deployed over the Internet. • Operates by monitoring the quality of Internet paths (latency, bandwidth, loss rate) between overlay nodes. • When primary path becomes unavailable or suffers from congestion, re-routes packets along an alternate path. • Routing overlays can be used to quickly recover from path outages, and also improve QoS of data flows. • This approach makes it possible to override the routes determined by Internet protocols and to route traffic based on metrics directly related to the performance needs of the application. 12
- 13. Self-healing and self-optimising routing overlay • SMART overlay network is formed by software agents that are deployed at VMs. • On each VM, a Transmission (TA) and Reception Agent(RA) run together with various Applications or tasks. • Each VM’s software router is the Proxy that monitors the quality of the overlay paths towards other destinations, selects the best paths, and forwards the packets over these paths. • TA receives the packets that are being sent to other Applications at other sites. • RA receives packets from the local Proxy, decapsulate and delivers them to the appropriate Application in the VM. • TA, RA and Proxy enables the control of path of packets through the network, without the applications being aware that their data flows are routed by the overlay. 13
- 14. Proxy: details • Monitoring Agent: monitors the quality of the Internet paths between the local cloud and the other clouds. • Routing Agent: drives the monitoring agent and uses the data it collects to discover an optimal path. • Forwarding Agent: forwards each incoming packet to its destination on the path it was instructed to use by Routing agent. • Source routing: Routing table of source proxy describes the complete path between overlay proxies to be followed by a packet to reach its destination, while the path between proxies is determined by the conventional IP protocol. 14
- 15. Packet forwarding process • SMART header contains the sequence of intermediate proxies. 15
- 16. Random Neural Network (RNN) • At each time interval, the algorithm chooses a subset of paths to probe, and measures the sum of edge delays in the probed paths. • The algorithm then sends its packet over the minimum latency path among those it has probed. • Each neuron in RNN represents a path from source to destination. • RNN helps to select K paths to destination out of N paths. • State of n connected neurons are represented by a vector k(t) = [k1(t), k2(t), . . . , kn(t)], where ki(t) is a non-negative integer valued random variable representing the “potential” of the i-th neuron being excited at time t. 16
- 17. Random Neural Network • A neuron i of the RNN is said to be excited whenever ki(t) > 0, in which case it can fire and send signals at an average rate ri 17 Timotheou S. The random neural network: a survey. The computer journal. 2010 Mar 1;53(3):251-67. Qi <- Ki(t) ri From out of network To out of network From other neurons To other neurons Positive signal received = higher probability of firing Neg. signal received = higher probability of not firing
- 18. Reinforcement learning • “Guide” RNN to choose better paths by: • “Reward” neurons representing good paths. • “Punish” neurons representing bad paths. 18 1. Increasing probability of it’s “firing” 2. Reducing probability of others “firing” 1. Decrease probability of it’s “firing” 2. Increasing probability of others “firing” G = minimum delay Higher delay = lower reward
- 19. Algorithm 19 2. Select neurons with highest probability at time tl 3. Calculate rewards. where G = minimize delay 4. p* is the best path 7. Good path: reward the neuron! 10.Increase positive weights going into the neuron 11. Increase negative weights leading to other neurons
- 20. Algorithm(continued..) 20 13. Bad path: punish the neuron! 16. Increase negative weights going into the neuron 17. Increase positive weights leading to other neurons (give other paths a higher chance of being selected) 22. Re-normalize w to avoid ever increasing weights 26. Update decision threshold T. β ∈ (0, 1) is used to introduce forgetfulness. Higher β will give more importance lesson learned from previous events.
- 21. Results(1) 21
- 22. Results(2) 22
- 23. Results(3) 23
- 24. Summary • Increasing amount of data collected from networks can help to improve performance. • Machine learning tools are useful to analyze and learn from these huge data. • Usefulness of other interesting machine/deep learning tools are yet to be explored in Network research. 24