IPv6 & Containers
0 likes348 views
The document discusses the integration of IPv6 within container technologies such as Docker and Kubernetes, highlighting the lightweight nature and efficiency of containers compared to virtual machines. It covers various networking options, configurations, and benefits of using IPv6, including improved task management and reduced complexity in networking. Additionally, it addresses the current limitations and future prospects of IPv6 support in cloud environments and emphasizes the scalability of IPv6 address usage for containers.
IPv6 & Containers
- 2. Agenda • Containers? • IPv6 for Docker • IPv6 for Kubernetes
- 3. Linux Containers • A Linux container lets you run a Linux system within another Linux system. • A container is a group of processes on a Linux machine. • Those processes form an isolated environment. • Inside the container, it (almost) looks like a VM. • Outside the container, it looks like normal processes running on the machine. • It looks like a VM, but it is more efficient: Containers = Lightweight Virtualization
- 4. Containers and Virtual Machines App A Bins/ Libs Hypervisor (Type 2) Host OS Server Host OS Server App A’ Bins/ Libs Guest OS App B Bins/ Libs Guest OS Bins/Libs Bins/Libs Container Control App A App A’ App B App B’ App C’ App C’ VM Container Containers are isolated but share OS and where appropriate bins/libraries Guest OS
- 5. Containers are almost like Virtual Machines • Containers have their own network interface (and IP address) • Can be bridged, routed... just like with KVM, VMware etc. • Containers have their own filesystem • For example a Debian host can run Fedora container (and vice-versa) • Security: Containers are isolated from each other • Two containers can't harm (or even see) each other • Resource Control: Containers are isolated and can have dedicated resources • Soft & hard quotas for RAM, CPU, I/O... • Though… • Apps in Containers share the kernel of the host OS (i.e. Linux guests only) • Containers are light-weight, fast to start, allow for >10x density compared to VMs
- 8. Docker bridge with IPv6 https://docs.docker.com/v17.09/engine/userguide/networking/default_network/ipv6 • No more NAT, all ports are exposed • Docker assigns IPv6 addresses sequentially • Default GW needs to be in same subnet • Set accept_ra to 2
- 9. Reaching the internet with NDP Proxy • sudo sysctl net.ipv6.conf.default.proxy_ndp=1 • sudo sysctl net.ipv6.conf.all.proxy_ndp=1 • docker network create testv6council --ipv6 --subnet 2a02:2789:724:eb8:1::/80 • sudo ip -6 neigh add proxy 2a02:2789:724:eb8:1:ff:ff:ff dev br-e25957a13b1f • sudo ip -6 neigh add proxy 2a02:2789:724:eb8:1::2 dev ens33
- 12. Docker Networking • Bridge network driver (--driver=bridge) • IPv6 can be enabled (--ipv6) • None network driver (--driver=none) • Host network driver (--driver=host) • Overlay network driver (--driver=overlay) – Multi-Host using VXLAN • MACVLAN network driver (--driver=macvlan) • IPv6 can be enabled • Remote drivers – compatible with CNM (Container Network Model) • Contiv, Weave, Calico…
- 13. Kubernetes • Container orchestrator • Runs and manages containers • Supports multiple cloud and bare-metal environments • Inspired and informed by Google's experiences and internal systems • 100% Open source, written in Go • Manage applications, not machines • Rich ecosystem of plug-ins for scheduling, storage, networking
- 14. Nodes, Pods, Containers • Node: • A server • Cluster: • Collection of nodes • Pod: • Collection of containers; • Nodes can run multiple Pods
- 15. Services overview • “Pods can come and go, services stay” • Define a single IP/Port combination that provides access to a pool of pods • By default a service connects the client to a Pod in a round- robin fashion • This solves the dilemma of having to keep up with every transient IP address assigned by Docker
- 16. Container Network Interface (CNI) • Proposed by CoreOS as part of appc specification • Common interface between container run time and network plugin • Gives driver freedom to manipulate network namespace • Network described by JSON config • Plugins support two commands: - Add Container to Network - Remove Container from Network • Many CNI plugins available: • Calico, Flannel, Weave, Contiv… Container Network namespace Driver plumbing Kubernetes, Rocket… Container Network Interface Plugins
- 17. • IPv4 Parity, no API Changes • CNI 0.6.0 Bridge & Host-Local IPAM • ip6tables & ipvs • Kube-DNS & CoreDNS • kubeadm Rel 1.9 (Alpha) • Dual-Stack, parallel IPv4/IPv6 • Multiple IPs per pod • Multiple IPs per service IPv6 in Kubernetes Rel 1.11 (Beta) Rel 1.12 (targeting) • SRv6 • Istio IPv6 • Multiprefix Routing… Planning and Preparing Source: SRv6LB @ Kubecon https://www.youtube.com/watch?v=RRKUeyFaqEA
- 18. IPv6 Containers @ Facebook (!k8s) • Every server gets a /64 • Unique IPv6 Address per task • Each task gets its own IPv6 /128 • Each task gets the entire port space • No more port collisions (!!!) • Simpler scheduling and accounting • /54 per Rack • /44 per Cluster (/48 in edge) • /37 DC Fabric • No NATs!
- 19. What about the public cloud? • GCE/GKE does not have IPv6 support • VPC networks only support IPv4 unicast traffic. They do not support broadcast, multicast, or IPv6 traffic within the network. • Can use IPv6 with load-balancing: • https://cloud.google.com/compute/docs/load-balancing/ipv6 • Azure, no IPv6 on AKS • IPv6 load-balancer: • https://docs.microsoft.com/en-us/azure/load-balancer/load-balancer-ipv6-overview • Long list of limitations: • A single IPv6 address can be assigned to a single network interface in each VM. • The load balancer routes the IPv6 packets to the private IPv6 addresses of the VMs using network address translation (NAT). • Azure VMs cannot connect over IPv6 to other VMs, other Azure services, or on-premises devices. They can only communicate with the Azure load balancer over IPv6. However, they can communicate with these other resources using IPv4. • Amazon • Should work with EC2 instances • Each VPC is given a unique /56 address prefix from within Amazon’s GUA (Global Unicast Address); you can assign a /64 address prefix to each subnet in your VPC • Maximum amount of IPv6 addresses per interface: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI
- 20. Why IPv6 in containers? • Future-ready, IPv6 is coming anyway… • Less configuration (no port forwarding) • Less state (no remembering which port for which service) • Less moving parts (easier diagnosis of faults) • Less variation between deployments • Forces you to do proper security
- 21. The scale of IPv6 for containers • Every docker host a routed /64 • Never re-use IPv6 address again • How long would it take to burn through that /64? • How about 10,000,000 per second ? • A standard /64 prefix in IPv6 is 18,446,744,073,709,600,000 addresses. • 18,446,744,073,709,600,000 IPv6 addresses / (10,000,000 IPv6 addresses/second * 60 sec/min * 60 min/hr * 24 hr/day * 365 days/yr) = 58,494 years • A single /48 contains 65536 /64s • 58,494 years * 65536 = 3,833,478,626 (3.8 billion years) Ed Horley (VP engineering Groupware) http://www.howfunky.com/2015/06/ipv6-docker-and-building-for-scale.html
- 22. References • IPv6 and containers – a horror story • Matt Palmer (Linux bearded guy) • https://blog.apnic.net/2018/03/22/ipv6-and-containers-a-horror-story/ • SRv6LB @ Kubecon • Pierre Pfister (Cisco SE) & Mark Townsley (Cisco Fellow) • https://www.youtube.com/watch?v=RRKUeyFaqEA • BRKSDN-2115 • Frank Brockners (Cisco Distinguished Engineer) • https://www.ciscolive.com/global/on-demand-library/?search=BRKSDN-2115#/session/BRKSDN-2115 • Containers, virtualisation and IPv6 • Steve Youell (JP Morgan) • http://www.ipv6.org.uk/2016/08/31/ipv6-council-meeting-october-2016/ • IPv6 in cloud deployments • Shannon McFarland (Cisco Distinguished Engineer) • http://www.rmv6tf.org/wp-content/uploads/2017/04/04-IPv6-Cloud-Deployment-RMv6tf-submit-min-1.pdf • IPv6, Docker and building for scale • Ed Horley (Groupware) • http://www.howfunky.com/2015/06/ipv6-docker-and-building-for-scale.html
- 23. Thanks!