Challenges of Kubernetes On-premise Deployment
Download as PPTX, PDF0 likes542 views
The document discusses challenges of deploying Kubernetes on-premise, including how load balancers are provisioned without cloud providers, using Nginx and Haproxy for load balancing on bare metal. It also covers how persistent volumes are provisioned with CSI drivers like Ember CSI to interface with storage backends, and tools for deploying and managing on-premise Kubernetes clusters like RKE.
Challenges of Kubernetes On-premise Deployment
- 1. Challenges of Kubernetes On- premise Deployment Sa Pham - RnD Department BizFly Cloud
- 2. Challenges ● Expose service type Load Balancer ● Persistent Volume ● Deployment tools
- 3. Common Apps deployment patterns on Kubernetes With stateless application, we use kind Deployment to create multiple pods and Service to expose the application We have 3 types of Service ● NodePort ● ClusterIP ● Load Balancer
- 4. Service type Load Balancer? On Cloud Environment, With Load Balancer Service type, Kubernetes will create a new Load Balancer for that service. For example: Open port 443 on Load Balancer and forward to port 30000 on worker nodes. Kube- proxy will forward request from port 30000 to correct pod.
- 5. How Load balancers are provisioned? Before, cloud providers have to contribute to kubernetes source code to work with their cloud -> In-tree Providers
- 6. How Load balancers are provisioned? Later, Cloud Controller Manager (CCM) is decoupled from kubernetes source code. When service Load Balancer is created, CCM call to Cloud to create Load Balancer and Pool, Listener of Load Balancer.
- 7. How Load balancers are provisioned? On bare-metal environment ?
- 8. Load Balancer on Bare Metal Environment Fortunately, most of our applications using HTTP to communicate from outside. We use Nginx-Ingress for Layer 7 Load Balancer and Haproxy For Layer 4 Load Balancer.
- 9. Our architecture - Use host-based routing and path based routing at ingress
- 10. Common Apps deployment patterns on Kubernetes Some applications (database, cache, ...) require Persistent Volume, then we use kind StatefulSet for these applications. Requirements: Persistent Volume with Storage Backend (Storage class)
- 11. How Persistent Volumes are provisioned? On Cloud Environment, Cloud providers have Block Storage Service and File System Service. We can use them to provide Persistent Volume for Kubernetes Container Storage Interface (CSI) was born.
- 12. How Persistent Volumes are provisioned?What is Container Storage Interface?
- 13. How Persistent Volumes are provisioned? CSI on Cloud Environment and Bare metal Environment
- 14. How Persistent Volumes are provisioned? CSI Driver List: Drivers - Kubernetes CSI Developer Documentation
- 15. CSI Drivers What will we do if there is no existing CSI driver for our Storage? - Use Ember CSI - embercsi/ember-csi: Multi-vendor CSI plugin supporting over 80 storage drivers - Write a new one. - How to write a Container Storage Interface (CSI) plugin · Fatih Arslan Ember CSI using Cinder drivers to work with storage backend A lot of storage backends are supported by Cinder community
- 16. CSI Drivers We are using Ember CSI as CSI Driver for our SAN Storage. Cons: - Multiple pools, multiple deployments
- 17. Storage backend for Kubernetes on-premise Some applications with old architectures, - Store data on local disk - Use NFS for shared storage How did we run on kubernetes ?
- 18. Storage backend for Kubernetes on-premise Cephfs performance on our cluster We used tocdo.net script to test on a kubernetes pod.
- 19. Deployment tools and manage kubernetes With managed kubernetes service (such as GKE, AKS, EKS, ..), we don't have to manage kubernetes master node. And easy to add a new node.
- 20. On Premise We need a tool which can - quick deploy a new cluster - easy to control the cluster - easy to add a new node to cluster - Infrastructure as code (IaC) - Separate etcd cluster
- 21. Deployment tools and manage kubernetes Too many deployment tools for on-premise kubernetes. - Kubeadm - RKE - Kubespray - Kubernetes the hard way - ...