Amazon Elastic Kubernetes Service (EKS) From Zero to Day 1
0 likes377 views
The document provides an overview of EKS (Amazon Elastic Kubernetes Service) and its management, including container orchestration, infrastructure as code (IaC) with Terraform, and Kubernetes add-ons. Key topics include EKS setup, IAM roles for service accounts (IRSA), network access configurations, and GitOps tools for continuous deployment. It also discusses best practices for security, scaling, and cost optimization in Kubernetes environments.
Amazon Elastic Kubernetes Service (EKS) From Zero to Day 1
- 1. THAILAND
- 2. EKS From Zero to Day 1 V-ris Jaijongrak | 25 Jan 2025
- 3. About Me Senior Platform Engineer at Pi Securities PCL All AWS Certified (12 x) CNCF Certified Kubestronaut (5 x) Main Responsibilities: - CI/CD (ArgoCD, Github Actions) - EKS - Cloud Platform Engineer (AWS) https://www.linkedin.com/in/v-ris-jaijongrak/
- 4. Key Takeaways - Kubernetes as container orchestration and EKS - IaC for EKS with Terraform - EKS setup with addons - EKS Tune up suggestions (for Gitops, Karpenter, Auto mode)
- 5. Introduction
- 6. Containers & Orchestration A solution of build once deploy anywhere popularized by Docker Build instruction stored in `Dockerfile` Inside docker image, consists of layers constructed by command in Dockerfile Deploying containers also becomes a problem especially with microservices 1. Networking 2. Resource Management 3. Security policies
- 7. EKS Kubernetes High-level perspective: - Control Plane (Kubernetes API server + etcd) - Worker Nodes (kube-proxy, kubelet, deployments, …) EKS is a managed Kubernetes Service (Amazon Elastic Kubernetes Service) Like ECS, you can run EKS with EC2 worker nodes, Fargate or Outposts
- 8. IaC (Terraform/CDK8s) & AWS IA EKS Blueprints IaC (Infrastructure as Code) is an approach to simplify the control of the Infrastructure via code instead of static config and deploying every piece by different tools. Terraform module for EKS: (registry.terraform.io) AWS IA: github.com/aws-ia/terraform-aws-eks-blueprints-addons
- 9. EKS
- 10. Barebone EKS terraform example github.com/guxkung/eks-terraform It is my personal github, I will try to get more educational sandbox example to update with (soon)
- 11. Architecture
- 12. EKS Add-ons To extend EKS capabilities and management, EKS introduced Add-ons as plugin: AWS-Managed: CloudWatch Observability, GuardDuty, SageMaker, etc. Marketplace add-ons: allow integration with EKS from 3rd party vendors
- 13. EKS Cluster Access AWS introduces a new approach to access/update the EKS Cluster via Kubernetes API “EKS Cluster Access” Previously, the authentication can be done via specialized ConfigMap embedded into EKS cluster to create an allow-list Instead of embedding that in the ConfigMap, the meta-data itself can now be referenced via EKS API directly and allow the entry to be more specific. (and more AWS IAM statement alike)
- 14. EKS Cluster Endpoint AWS recommends the Cluster to be only privately accessible for security reason. (then Networking Security for Private Networking applies, i.e., Security Group, NACL) The second best would be allowing only IP from CIDR to access from public endpoint. (the DNS entry can be resolved from public internet but accessing from other addresses will be blocked) This settings can be modified anytime, but be aware of losing connections from application side/tools
- 15. IRSA (IAM Role to Service Account) IAM Role: S3BusinessBucketAccess (allows S3 access to business bucket) Service Account: business:billing-sa (billing application in business namespace can assume the role and perform S3 actions on the bucket)
- 16. IRSA Step-by-step From our example, this would be the steps to setup IRSA 1. Create the IAM role (permission policy + trusted policy) 2. Create ServiceAccount and Associate with IAM role in EKS 3. Utilize the Service Account in the K8s applications/pods This approach is more efficient because the token can be stored at agent instead of directly connect to OIDC provider of the EKS cluster
- 17. EKS Pod Identity Association To let Kubernetes applications connect with AWS services, it is now recommended to use Pod Identity associations (IAM role -> K8s Service Account) The steps are now: 1. Install EKS Pod Identity AddOn to the EKS cluster → pod-identity-agent 2. Create the IAM role 3. Create association with Console/EKS API for IAM Role to ServiceAccount 4. Utilize the Service Account in the K8s applications/pods The connection to OIDC provider of the EKS cluster is more efficient this way
- 18. Appendices
- 19. Custom Networking Customizing the VPC-CNI add-ons to support network capabilities such as: SNAT configuration (allows pods to reside in different subnet than nodes), the NAT can be done at node level or subnet level via route table Pod IP Pool (IPAM) secondary ENI will utilize the pre-allocated IP address range and assigned to the pods, IPAM guarantees that the IP address requested is available Max pods configuration on nodes after consolidating the settings, max pods number will be calculated and put on definition of nodegroup
- 20. Custom CNI For advanced CNI, can be installed in chain-mode or migrate to stand-alone
- 21. GitOps Alternatively, if you really start from zero with a centralized management cluster/account KubeFirst is also viable (Community edition will be restricted with publicly accessible only) Tools for GitOps you may consider: More projects can be found at https://landscape.cncf.io/
- 22. Auto Mode & Karpenter Cost Optimization, operation excellence are the main concerns for this Karpenter/Auto mode Auto Mode leaves the optimization of the workload on the nodepool to be fully managed service (i.e., AWS will manage the network, nodes implicitly) with some additional cost To use all the custom plugin/deployments manually deploy Karpenter is recommended
- 23. Summary - We learned about foundation of Kubernetes and EKS (Control Plane + Data Plane (Worker Node) - Introduction to EKS and EKS Addons (coredns, kube-proxy, VPC-CNI, etc.) - A Demo of Terraform to construct EKS cluster (and auto-mode example to try-out) - Gitops concept and the Karpenter (nodepool and node scaling)
- 24. Q&A