(Powerpoint slides)
- 1. Traffic Dynamics at a Commercial Backbone POP Nina Taft Sprint ATL Co-authors: Supratik Bhattacharyya, Jorjeta Jetcheva, Christophe Diot
- 2. Outline Part 1: what are the traffic demands between pairs of POPs? How stable is this demand? Part 2: what are the paths taken by those demands? Are link utilizations levels similar throughout the backbone? Part 3: is there a better way to spread the traffic across paths? Can we divert some traffic to lightly loaded paths?
- 3. The Sprint IPMon Project Passive monitoring Capture header (44 bytes) from every packet full TCP/IP headers, no http information Use GPS time stamping - allows accurate correlating of packets on different links Day long traces Simultaneously monitor multiple links and sites. Collect routing information along with packet traces. Traces archived for future use
- 4. IP Backbone : POP-to-POP view POP fanout: one row of POP-to-POP traffic matrix OC-48 OC-12 OC-192
- 5. POP-to-POP Traffic Matrix Measure traffic over different timescales Divide traffic per destination prefix, protocol, etc. For every ingress POP : Identify total traffic to each egress POP Further analyze this traffic City A City B City C City A City B City C
- 6. The Mapping Problem What is the egress POP for a packet entering a given ingress POP?
- 7. Monitored links at a single POP Core Core Core Publicpeer 2 web host ISP Public peer 1 Date : Aug 9, 2000 Access Access Access Access
- 8. Traffic Fanout: POP level granularity
- 9. Fanout: web host links
- 10. Time-of-Day for POP level granularity
- 11. Day-Night Variation : Webhost #1 % reduction at night between 20-50% depending upon access link
- 12. Summary so far ... Wide disparity in “traffic demands” among egress POPs POPs can be roughly categorized as : small, medium, large; and they maintain their rank during the day. Traffic is heterogeneous in space yet stable in time. 20-50% reduction at night
- 13. Outline Part 1: what are the traffic demands between pairs of POPs? How stable is this demand? Part 2: what are the paths taken by those demands? Are link utilizations levels similar throughout the backbone? Part 3: is there a better way to spread the traffic across paths? Can we divert some traffic to lightly loaded paths?
- 14. Paths used by traffic demands Our Observations (summary) routing policies concentrate traffic on a few paths: between two POPs, all the traffic uses either the same route, or 1 or 2 routes the ISIS weights are changed very infrequently (once a month), so routing is fairly static there are many underutilized routes
- 15. Is backbone traffic balanced?
- 16. Part 3 : Can we divert some traffic to lightly loaded paths? Approach: to improve load balancing by rerouting only a few flows scalable Which flows? Heavy hitters. How identify heavy hitters: Consider: destination prefix-based flows at fixed prefix lengths: 8 and 16 BGP table entries (variable prefix length)
- 17. Streams based on destination prefix Stream : all packets in a group with same /8 destination address prefix Traffic grouped by egress POPs Ingress : Webhost Link Similar results for /16 and bgp table prefixes
- 18. Stability of prefix-based streams R i (n) = the rank of flow i at time slot n i,n,k = | R i (n) - R i (n+k) | each time slot Stability of prefix rank
- 19. Conclusions We have used our data to build components of traffic matrices for traffic engineering Heterogeneous traffic fanout from POP Current routing practices lead to many underutilized links and paths thus, there is a lot of room for improved load balancing techniques. Load-balancing using flows selected via destination-prefixes is a simple and promising criterion
- 20. Ongoing Work Intra-domain Routing : Choosing ISIS link weights Multi-path routing Flow Characterization at the network prefix level Inference techniques for building POP-to-POP traffic matices
Editor's Notes
- #4: . Because its passive monitoring it is transparent to the network. Our GPS timestamps are accurate within 5 microseconds
- #5: Link speeds are included just to show diversity of backbone. The fanout from a particular POP represents one row of a POP-to-POP traffic matrix
- #6: There are many things one could put in the traffic matrix. Usually it represents an average volume, but that volume has an associated time scale (measurement interval) and a traffic granularity. What goes in the matrix depends upon what you want to do with it. Our measurements are averaged over one day, and represent average bandwidth (total bandwidth between cities - i.e. no subdivision into prefixes, etc.)
- #7: This is a multistep process. The BGP table gives you a next hop node outside the Sprint network. Use recursive lookups in the BGP table to find last Sprint node. The need to map Sprint node to a POP. Needed to use a combination of methods (name parsing, BGP_community_ID, etc) The goal is to convert the BGP table to one which gives the egress POP (rather than a next hop node) for a given prefix.
- #8: The arrows indicate the links monitored. We used day long traces this shows the structure of a POP: *peers connect into core routers, clients connect to access routers, inner core surrounded by access routers, etc.
- #9: Shows total demand from ingress to each egress POP – egress POPs can be categorized roughly as large (>35), medium(10-15) and small (<5). Matches intuition: (1) big pops in east and west coasts because that’s where international trunks terminate & that’s where population concentrated. (2) variation high depending upon # and type, of customers and servers. Thus expect inter-POP flows to be highly variable. However, what if we want to build prediction techniques – if we add a new link, does the type of link matter? Is POP granularity too coarse?
- #10: Looking at this we see the fanout is similar except for 2 POPs. Natural to ask what the distribution of the fanout is, so can compare. This also shows some load balancing behavior.
- #11: Large POP most volatile, medium POP slow steady decline, small POP stable. They maintain their ordering (large,medium,small) throughout the day. At night this is blurred. Verified for other combinations of POPs. Hard to find busy hour. There is increase at night sometime. Different POPs behave differently. Can’t have one single model for a POP.
- #12: With the exception of 1 or 2 egress POPs, the percentage reduction at night is quite similar among POPs. This plot is for one access link. Averaging across all egress POPs, get about 40% overall reduction at night. The other links show between 20-50% reduction at night. This is less than we expected. It is important to know that there is less and less traffic reduction at night time.
- #13: Time of day behavior for a POP should be separated into different components, one for each type of access link.
- #16: The combination of everything we have seen up to now: routing on a few paths, existance of underutilized paths, low utilization levels on large numbers of links, diverse traffic demands - ALL this means that one can do a better job load balancing (i.e., there is much room for improvement in today’s systems).
- #18: Rank streams in decreasing order of traffic volume has implications for traffic engineering. Load balance the elephants during the day. Helps scalability if only have to engineer the network tightly for elephants and can do loose control for mice. Can’t use this info for load balancing, unless ordering remains stable throughout the day. (lead to next slide) these types of results have been seen before (the type of plot, and this data for network prefixes). What’s new here is showing that this property exists for specific fixed prefix lengths too.
- #19: This graph means that 70% of the time a flow will not change its rank by more than 5 (for example). 70% of streams in the top 10%, stay in the top 10%. 70% of those in the bottom half, stay in the bottom half. The point here is to show that elephants tend to stay elephants. This data is probably from the new york pop (#12)