Managing Global Distributed Network
0 likes125 views
This document discusses Cloudflare's global distributed network and how it manages over 120 points of presence around the world. It provides key metrics on traffic handled and describes techniques used such as anycast routing and an "orange cloud" to direct traffic to the nearest entry point quickly. The challenges of managing such a vast network are also outlined, along with approaches to monitoring, automation, and standardization that help ensure network resilience.
![Some daily usage examples
● salt [target_device] net.cli "junos_show_command"
● salt-run net.find [target_device]
● salt-run bgp.neighbors [bgp_asn]
● salt [target_device] [anycast.disable | anycast.enable]
● salt [target_device] [transit.disable | transit.enable]
[transit_name]](https://image.slidesharecdn.com/sgnog5-201022025202/85/Managing-Global-Distributed-Network-17-320.jpg)
![Global configuration push
● salt [target_device] net.load_template [config_template]
○ Configuration template with Jinja2 format
○ Dynamic configuration loader
■ Flexible
○ Allow dry-run
○ Example: salt -N junos-cdn-edges net.load_template test-lar](https://image.slidesharecdn.com/sgnog5-201022025202/85/Managing-Global-Distributed-Network-23-320.jpg)
Managing Global Distributed Network
- 1. Jimmy Lim SGNOG 5 jhalim@cloudflare.com Singapore, 5 September 2017 Managing Global Distributed Network
- 2. Cloudflare in a glance
- 3. This is the new style. ● 4+ million zones/domains ● 43+ billion DNS queries/day ● How? ○ Orange cloud ○ Global distributed network in 120+ locations Still growing fast! ○ Anycast routing Protect and accelerate any website online
- 4. Benefit of orange cloud ● Direct visitors to the nearest entry point ○ Fast! ■ Lesser hops ■ Reduced latency ● Save bandwidth! ○ Lesser requests to origin ○ Mitigate DDoS ● Resiliency ○ 120+ locations!
- 5. Orange cloud vs grey cloud
- 7. 1 new PoP per week!
- 8. Challenges ● 120+ locations ● 500+ transit/exchange ports ● 500+ network equipments ● Countless alerts!
- 10. Building Resilience Network ● Stable hardware and software ● Automatic configuration template/peer review ● Solid monitoring system ● Global network engineering ● Network automation
- 11. Hardware and Software ● Proper evaluation and testing ○ Fits requirement ○ Bugs free ○ Scalable ● Global standardization ○ Same models of hardware ○ Same software codes ● No mass software upgrade! ○ Small PoP first ○ Deploy in batches
- 12. Solid Monitoring System ● Reduced unwanted alerts ○ Only gets relevant alerts ○ Silence PoP/ports during maintenances ● Monitor the performance of transit providers ○ Detects packet loss on their backbone ○ Provides automatic related traceroutes ○ Actions based on severity ■ Disabling the PoP automatically ■ Disabling traffic on related transit provider automatically ■ Suggests on actions to do
- 13. Alerts Channel and Dashboard
- 14. Alerts Channel and Dashboard
- 15. Global Network Engineering ● Follow the sun approach ○ San Francisco -> Singapore -> London -> San Francisco ● Doing all stuffs ○ Technical operations ○ Network engineering ○ Network expansion projects ○ New PoPs deployment ○ Peering stuffs ● Very fast response to network issues and escalation
- 16. Network Automation ● Open source recipe: napalm-salt Apricot'2017 Network Automation by Salt+Napalm by Mircea Ulinic
- 17. Some daily usage examples ● salt [target_device] net.cli "junos_show_command" ● salt-run net.find [target_device] ● salt-run bgp.neighbors [bgp_asn] ● salt [target_device] [anycast.disable | anycast.enable] ● salt [target_device] [transit.disable | transit.enable] [transit_name]
- 18. Net finder salt runner example
- 19. BGP salt runner example
- 20. Get advertised routes to particular BGP peer
- 21. PeeringDB lookup on IXPs info
- 22. BGP peering configuration push
- 23. Global configuration push ● salt [target_device] net.load_template [config_template] ○ Configuration template with Jinja2 format ○ Dynamic configuration loader ■ Flexible ○ Allow dry-run ○ Example: salt -N junos-cdn-edges net.load_template test-lar
- 25. Global configuration push using template
- 26. Peering Update in Asia
- 27. In August 2015 ● AKL-IX (Auckland) ● APE (Auckland) ● BBIX (Tokyo, Osaka, Singapore) ● Equinix (Hong Kong, Osaka, Singapore, Sydney, Tokyo) ● HKIX (Hong Kong) ● JPIX (Tokyo, Osaka) ● JPNAP (Tokyo, Osaka) ● Megaport (Auckland, Singapore, Sydney) ● PIPE (Sydney)
- 28. Addition of Peering Exchange as for now ● AMS-IX (Hong Kong) ● KINX (Incheon) ● MYIX (Kuala Lumpur) ● NSW-IX (Sydney) ● SGIX (Singapore) ● VIC-IX (Melbourne) ● WAIX (Perth)
- 29. Globally ● Registered in 150 public peering exchange points ● About 120 of them are up and running! ● More details: https://www.peeringdb.com/asn/13335 ● On-going 100G deployment ● Peering Contact: peering@cloudflare.com
- 30. Q&A