Kuryr kubernetes: the seamless path to adding pods to your datacenter networking
5 likes1,504 views
The kuryr-kubernetes project aims to unify networking for virtual machines (VMs), bare metal, and Kubernetes pods, facilitating a smoother transition to cloud-native applications and microservices. It leverages OpenStack's Neutron for networking, enabling both Layer 2 and Layer 3 connectivity between VMs and pods while supporting bare metal and existing services in VMs. The architecture includes a centralized controller that maps Kubernetes resources to Neutron resources, enhancing resource management and connectivity within the cluster.
![Kuryr CNI Driver
● Kuryr CNI driver only
communicates with
Kubernetes API
○ Kubelet already has
connection to K8s API
○ Performs local binding
of the neutron port
● Supports CNI versioned
output (0.3.0)
● Watches Pod resources
for Controller-driven vif
annotations
"annotations": {
"openstack.org/kuryr-vif": {
"active": true,
"address": "fa:16:3e:6c:1f:ff",
"bridge_name": "br-int",
"has_traffic_filtering": true,
"id": "ba8f8d4b-1dfb-4aaf-8ab2-80c25711da3f",
"network": {
"bridge": "br-int",
"id": "a10c5bf4-99b2-4b0d-82b1-2a2639dda4de",
"label": "private",
"mtu": 1450,
"multi_host": false,
"should_provide_bridge": false,
"should_provide_vlan": false,
"subnets": {[{
"cidr": "10.0.0.0/26",
"dns": [],
"gateway": "10.0.0.1",
"ips": [{
"address": "10.0.0.8"}],
"routes": []
}]}
},
"plugin": "ovs",
"port_profile": {
"interface_id": "ba8f8d4b-1dfb-4aaf-8ab2-80c25711da3f"
},
"preserve_on_delete": false,
"vif_name": "tapba8f8d4b-1d"
}
}](https://image.slidesharecdn.com/kuryr-kubernetes-theseamlesspathtoaddingpodstoyourdatacenternetworking1-170512161158/85/Kuryr-kubernetes-the-seamless-path-to-adding-pods-to-your-datacenter-networking-13-320.jpg)
![Join us
● Project launchpad
○ https://launchpad.net/kuryr-kubernetes
● Repository
○ https://github.com/openstack/kuryr-kubernetes
● IRC
○ Weekly meeting #openstack-meeting-4 Mondays 14:00 UTC
○ #openstack-kuryr at Freenode
● Mailing list
○ [kuryr] in openstack-dev@lists.openstack.org](https://image.slidesharecdn.com/kuryr-kubernetes-theseamlesspathtoaddingpodstoyourdatacenternetworking1-170512161158/85/Kuryr-kubernetes-the-seamless-path-to-adding-pods-to-your-datacenter-networking-23-320.jpg)
Kuryr kubernetes: the seamless path to adding pods to your datacenter networking
- 1. Kuryr-Kubernetes Adding Pods to your Datacenter Networking Irena Berezovsky @irenab Antoni Segura Puimedon @celebdor
- 2. Kuryr-Kubernetes Project motivation ● Hard to connect VMs, bare metal and nested containers ○ No unified networking infrastructure ● Overlay 2 for Pods running in VMs ○ Performance, latency, SLA, management penalties ● Need for a smooth transition to the Cloud Native Applications ○ Ability to transition workloads to microservices at your own pace ○ VMs and Pods sharing networking infrastructure ● Bare Metal OpenStack Storage support with Fuxi-Kubernetes
- 3. Kuryr-Kubernetes Project Mission ● Neutron, unified, community sourced networking for Pods & VMs ● OpenStack vendor support experience in the Container space ● Get Neutron users faster into container workloads ○ VMs and Pods on the same Neutron network ○ Enable both L2 and L3 connectivity between OS VMs and K8s Pods ● Easier transition to microservices ○ Connect to VM layer in the same infrastructure
- 4. Bare Metal Use Case ● Centralized Kuryr Controller ● Kuryr Controller maps ○ K8s Pods into Neutron ports ○ K8s Services into Neutron Load Balancers ● Kuryr CNI on each Worker node performs Pod binding
- 5. Pod in VM Use Case ● Security ● Easier node allocation ● Single overlay ● VM and Pods as targetable network resources ● Can use either Neutron trunk ports or macvlan based VM port allocation
- 6. Mixed Use Case ● Connect to existing services in VMs ● Legacy applications alongside microservices ● VM NFVs ● Use existing cloud for Kubernetes workloads
- 7. Supported functionality ● Pods networking ○ Kubernetes native networking ○ Pods as Neutron ports on the cluster Neutron network ○ Single tenant ○ Full connectivity enabled by default ● Kubernetes ClusterIP Services ○ Implemented by Neutron LBaaSv2 ● Bare Metal and Pod in VM support
- 9. Kubernetes Services ● Cluster IP translates to Neutron VIP ● Service endpoints translate to Pool Members ● Uses Neutron Lbaas v2 ● Planned addition of Octavia driver
- 11. Kuryr Controller ● Secure connection to the Neutron API Server ○ Keystone as Authorization service ● Watches Kubernetes API resources with a service account ● Stevedore Plugin based Network resources translation ○ Handlers: Receive Kubernetes resource events and patch them ○ Drivers: Used by handlers to allocate Neutron resources, allowing multiple implementations and vendors. ● Os-vif for interface plugging
- 12. Kuryr Controller ServiceAccount --- apiVersion: v1 kind: ServiceAccount metadata: name: kuryrctl namespace: kube-system --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: kuryrctl-global subjects: - kind: User name: kuryrctl apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: kuryrctl apiGroup: rbac.authorization.k8s.io --- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: kuryrctl rules: - apiGroups: - "" verbs: - get - list - watch resources: - deployments - endpoints - ingress - nodes - pods - policies - services - apiGroups: - "" verbs: - update - patch resources: - endpoints - ingress - pods - policies - nodes - services - services/status
- 13. Kuryr CNI Driver ● Kuryr CNI driver only communicates with Kubernetes API ○ Kubelet already has connection to K8s API ○ Performs local binding of the neutron port ● Supports CNI versioned output (0.3.0) ● Watches Pod resources for Controller-driven vif annotations "annotations": { "openstack.org/kuryr-vif": { "active": true, "address": "fa:16:3e:6c:1f:ff", "bridge_name": "br-int", "has_traffic_filtering": true, "id": "ba8f8d4b-1dfb-4aaf-8ab2-80c25711da3f", "network": { "bridge": "br-int", "id": "a10c5bf4-99b2-4b0d-82b1-2a2639dda4de", "label": "private", "mtu": 1450, "multi_host": false, "should_provide_bridge": false, "should_provide_vlan": false, "subnets": {[{ "cidr": "10.0.0.0/26", "dns": [], "gateway": "10.0.0.1", "ips": [{ "address": "10.0.0.8"}], "routes": [] }]} }, "plugin": "ovs", "port_profile": { "interface_id": "ba8f8d4b-1dfb-4aaf-8ab2-80c25711da3f" }, "preserve_on_delete": false, "vif_name": "tapba8f8d4b-1d" } }
- 14. Kuryr CNI ServiceAccount --- apiVersion: v1 kind: ServiceAccount metadata: name: kuryrcni namespace: kube-system --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: kuryrcni-global subjects: - kind: User name: kuryrcni apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: kuryrcni apiGroup: rbac.authorization.k8s.io --- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: # "namespace" omitted since ClusterRoles are not namespaced name: kuryrcni rules: - apiGroups: - "" verbs: - get - list - watch resources: - pods
- 15. Controller - CNI baremetal pod creation
- 16. Trunk ports - Neutron extension
- 17. Trunk ports - Neutron extension
- 18. Controller - CNI pod-in-VM creation
- 19. Cluster service creation flow
- 20. Scaling Kuryr ● Generic resource Pooling ○ VIF ○ Load Balancers* ● Stevedore pluggability to choose pooling behavior ● Pre-allocates Neutron resources in batch operations ● Burst tolerant
- 21. Demo: Guestbook ● 2-tier ● 3 services ● PHP frontend, Redis backend
- 22. ● Features ○ LoadBalancer Kubernetes Service Type ○ Resource Management ○ Ingress support ○ Policy support ○ Multi-Tenancy, Multiple Networks support ○ Management CLI ● Improvements ○ CNI split into exec and daemon ○ Handler/Driver Profiles ○ Active-Passive HA What’s Next
- 23. Join us ● Project launchpad ○ https://launchpad.net/kuryr-kubernetes ● Repository ○ https://github.com/openstack/kuryr-kubernetes ● IRC ○ Weekly meeting #openstack-meeting-4 Mondays 14:00 UTC ○ #openstack-kuryr at Freenode ● Mailing list ○ [kuryr] in openstack-dev@lists.openstack.org
- 24. Resources ● Documentation ○ https://docs.openstack.org/developer/kuryr-kubernetes ● Getting started ○ https://ltomasbo.wordpress.com/2017/01/29/side-by-side-and-nested-kubernetes-and-op enstack-deployment-with-kuryr/