Container Network Interface: Network Plugins for Kubernetes and beyond
13 likes14,165 views
The document provides an overview of container networking in Kubernetes, highlighting the Kubernetes networking model which assigns IPs per pod and discusses various networking plugins like macvlan and ipvlan. It outlines the process of configuring Container Network Interfaces (CNI) using JSON, detailing steps involved in adding and removing containers from a network. The document also emphasizes the integration of CNI plugins with Kubernetes and the potential for future enhancements in networking capabilities.
Container Network Interface: Network Plugins for Kubernetes and beyond
- 1. Container Network Interface: Network plugins for Kubernetes and beyond Eugene Yakubovich @eyakubovich
- 2. Kubernetes networking model - IP per pod - Pods in the cluster can be addressed by their IP
- 3. How to network containers together? - Cloud provider integration - AWS - GCE
- 4. How to network containers together? linux-bridge macvlan ipvlan Open vSwitch Weave Project Calico flannel
- 5. How to allocate IP addresses? - From a fixed block on a host - DHCP - IPAM system backed by SQL database - SDN assigned: e.g. Weave
- 6. How do you mix and match? (macvlan | ipvlan) + (DHCP | host-local)
- 7. Order matters! - macvlan + DHCP ○ Create macvlan device ○ Use the device to DHCP ○ Configure device with allocated IP - Routed + IPAM ○ Ask IPAM for an IP ○ Create veth and routes on host and/or fabric ○ Configure device with allocated IP
- 8. Container Runtime (e.g. k8s) veth macvlan ipvlan OVS Container Networking Interface (CNI)
- 9. CNI - Container can join multiple networks - Network described by JSON config - Plugin supports two commands - Add container to the network - Remove container from the network
- 10. User configures a network $ cat /etc/cni/net.d/10-mynet.conf { "name": "mynet", "type": "bridge", "ipam": { "type": "host-local", "subnet": "10.10.0.0/16" } }
- 11. CNI: Step 1 Container runtime creates network namespace and gives it a named handle $ cd /var/lib/cni $ touch myns $ unshare -n mount --bind /proc/self/ns/net myns
- 12. CNI: Step 2 Container runtime invokes the CNI plugin $ export CNI_COMMAND=ADD $ export CNI_NETNS=/var/lib/cni/myns $ export CNI_CONTAINERID=5248e9f8-3c91-11e5-... $ export CNI_IFNAME=eth0 $ $CNI_PATH/bridge </etc/cni/net.d/10-mynet.conf
- 13. CNI: Step 3 Inside the bridge plugin (1): $ brctl addbr mynet $ ip link add veth123 type veth peer name $CNI_IFNAME $ brctl addif mynet veth123 $ ip link set $CNI_IFNAME netns $CNI_IFNAME $ ip link set veth123 up
- 14. CNI: Step 3 Inside the bridge plugin (2): $ IPAM_PLUGIN=host-local # from network conf $ echo $IPAM_PLUGIN { "ip4": { "ip": "10.10.5.9/16", "gateway": "10.10.0.1" } }
- 15. CNI: Step 3 Inside the bridge plugin (3): # switch to container namespace $ ip addr add 10.0.5.9/16 dev $CNI_IFNAME # Finally, print IPAM result JSON to stdout
- 16. Kubernetes + CNI + Docker - Kubernetes has its own network plugins - CNI "driver" is a k8s network plugin - Future: make CNI native plugin system
- 17. Kubernetes + CNI + Docker - k8s starts "pause" container to create netns - k8s invokes its plugin (CNI driver) - k8s CNI driver executes a CNI plugin - CNI plugin joins "pause" container to network - Pod containers use "pause" container netns
- 18. Kubernetes + rkt - rkt natively supports CNI - Kubernetes delegates to rkt to invoke CNI plugins
- 20. Want to work on upstream Kubernetes or distributed systems infrastructure? CoreOS San Francisco is hiring. Work at CoreOS coreos.com/careers