Kubernetes on AWS => EKS || CNCF Meetup Zurich, Feb 2019
Download as PPTX, PDF1 like219 views
This document summarizes a presentation about running Kubernetes on AWS Elastic Kubernetes Service (EKS). It discusses: 1. The core components of Kubernetes and how they are implemented in EKS, with the control plane managed by AWS and worker nodes managed by the user. 2. How to set up an EKS cluster by provisioning the control plane with AWS services and launching worker nodes across Availability Zones using Auto Scaling Groups. 3. How user authentication uses IAM and authorization uses Kubernetes RBAC, and how they are integrated between AWS and Kubernetes.
![Meetup Zurich , Feb 2019
User authentication (IAM) / authorization (RBAC)
...exclusively managed by IAM
IAM policy attached to user/group
E.g. grant all EKS access
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"eks:*"
],
"Resource": "*"
}
]
}
IAM role, assumed by EKS, to access
other resources on your behalf
...exclusively managed by K8s RBAC, not
EKS specific
RBAC primitives:
Subjects Operations
Resources
Who can do what
where
!!
the user you used to create EKS cluster is
hardwired into system:masters group,
hence he can perform all activities
&
Authentication Authorization](https://image.slidesharecdn.com/cncf-meetupzurichfeb19-190225200316/85/Kubernetes-on-AWS-EKS-CNCF-Meetup-Zurich-Feb-2019-8-320.jpg)
Kubernetes on AWS => EKS || CNCF Meetup Zurich, Feb 2019
- 1. K8s on AWS: Holy Grail --- or --- Holy Moly ?
- 2. Meetup Zurich , Feb 2019 About Me ● BigData & Cloud Engineer/Consultant at ○ Hadoop, Kafka, AWS, Azure, Openshift ● OpenSource enthusiast ● Udemy instructor You can find me on ● LinkedIn: https://www.linkedin.com/in/gerdkoenig ● GitHub: https://github.com/gkoenig/ ● Twitter: https://twitter.com/gerd_koenig
- 3. Meetup Zurich , Feb 2019 Topics 1. K8s core components ...and how they are implemented in EKS 2. Setting up an EKS cluster 3. User authentication (IAM) / authorization (RBAC) 4. Network Overlay 5. Network policies by Calico 6. Persistent Volumes 7. Scaling worker nodes...incl Spot Instances
- 4. Meetup Zurich , Feb 2019 K8s core components Master Nodes (Control Plane) “Brain” of the cluster Worker Nodes (Data Plane) Container execution Communication API-server ⇔ kubelet
- 5. Meetup Zurich , Feb 2019 K8s core components….in EKS AZ B AZ CAZ A AWSMANAGED Master Node Master Node Etcd Etcd Etcd Master Node K8s worker nodes K8s worker nodes K8s worker nodes USER Shared responsibility: ● Control Plane/Masters => AWS ● Data Plane/Workers => User Control Plane ● HA setup, multi AZ ● Built upon EC2,ELB,ASG,NLB,VPC Source:https://www.udemy.com/amazon-eks-starter-kubernetes-on-aws/
- 6. Meetup Zurich , Feb 2019 Setting up EKS ● Provision Control Plane ○ via eksctl/CloudFormation/AWS console/aws-cli ○ Creates VPC/EC2/LB, assigns current IAM user the K8s master role ● Launch worker nodes ○ In ASG ○ across AZs ● Configure tools to talk to your cluster ○ kubectl, incl aws-iam-authenticator ● Deploy your K8s apps …… P A R T Y
- 7. Meetup Zurich , Feb 2019 Setting up EKS Lets get our hands dirty ….and quickly explore a running EKS
- 8. Meetup Zurich , Feb 2019 User authentication (IAM) / authorization (RBAC) ...exclusively managed by IAM IAM policy attached to user/group E.g. grant all EKS access { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "eks:*" ], "Resource": "*" } ] } IAM role, assumed by EKS, to access other resources on your behalf ...exclusively managed by K8s RBAC, not EKS specific RBAC primitives: Subjects Operations Resources Who can do what where !! the user you used to create EKS cluster is hardwired into system:masters group, hence he can perform all activities & Authentication Authorization
- 9. Meetup Zurich , Feb 2019 User authentication (IAM) / authorization (RBAC) ● Authentication is held by IAM ● Authorization is done by Kubernetes RBAC (native auth for K8s) ● This is done through a collaboration done between AWS and Heptio ● You can assign RBAC directly to IAM entities! ● By default, the role you assign to your K8s cluster has system:master permissions > kubectl IAM Kubernetes RBAC 2. Verify AWS Identity 3. Authorize AWS Identity With Kubernetes RBAC 1. Send AWS Identity 4. Allow / Deny
- 10. Meetup Zurich , Feb 2019 Network Overlay Source:https://itnext.io/kubernetes-is-hard-why-eks-makes-it-easier-for-network-and-security- architects-ea6d8b2ca965 VPC CNI plugin ● Pods running in same IP space as host (subnet of VPC), no additional vNIC ● ENIs will be attached to EC2 workers on demand (as no of pods grows) ● !! EC2 instance type limits max number of pods per host (=no of ENIs * max IPs per Interface) https://docs.aws.amazon.com/AWSEC2/latest/UserGui de/using-eni.html#AvailableIpPerENI ● IPs of ENIs are pre-warmed, ensure to have a big enough VPC
- 11. Meetup Zurich , Feb 2019 Networking policies via Calico Project Calico => OpenSource tool to limit network traffic flow based on specified policies Not just K8s, also DockerEE/OpenStack/bare-metal Calico on top of AWS-VPC-CNI to limit access in 2-Tier-app https://www.projectcalico.org Client Backend Frontend namespace Lets get our hands even dirtier
- 12. Meetup Zurich , Feb 2019 Persistent Volumes Deploying stateful apps => persistent volumes ● Block storage ● attached to one EC2 instance ● Can be re-attached, e.g. on EC2 failure ● use ReplicaSet over Deployment ● an EBS vol. lives just in one AZ When to use? Apps w/ local storage, not shared across app-instances ● Shared filesystem ● NFS ● Multi read/write ● Multi AZ availability When to use? Apps with storage shared across app-instances and AZs PV-1 (EBS) PV-3 (EBS) PV-2 (EBS) Node 1 Node 2 Pod 1 Pod 3 Pod 2 EFS Security Group Node 1 Node 2 Pod 1 Pod 3 Pod 2
- 13. Meetup Zurich , Feb 2019 Persistent Volumes Sample Scenario using EFS & EBS: Deploy Wordpress & MySQL MySQL pod ... Node 1 Node 2 Elastic filesystem (EFS) EFS provisioner storageclass pvc mysql Pvc wordpress EK S EFS /EB S WP pod WP pod WP pod EBS provisioner storageclass Elastic block store (EBS)
- 14. Meetup Zurich , Feb 2019 Scaling worker nodes...incl Spot Instances ● By default worker nodes are made of on-demand EC2 instances ● Spot Instances are ways cheaper and a good fit to scale your cluster and cover peaks, but: they can be terminated by AWS with short notice time ● Since Nov 2018: mixed types of EC2 instances in same ASG possible Spot + OnDemand && different sizing ● Node labelling required for intelligent scheduling to On-demand vs SpotInstance ● Additional _taints_ to further select where to run ● Quick Demo: Run sample deployment , which prefers pods on SpotInstances but does not require it spec: affinity: nodeAffinity: preferredDuringSchedulingIgnoredDuringExecution: - weight: 1 preference: matchExpressions: - key: lifecycle operator: In values: - Ec2Spot tolerations: - key: "spotInstance" operator: "Equal" value: "true" effect: "PreferNoSchedule"
- 15. Meetup Zurich , Feb 2019 <= ? => HA Control Plane managed by AWS Runs certified, upstream K8s So what ? May be overwhelming if new to AWS Instance type limits no of Pods (=>ENI)
- 16. Meetup Zurich , Feb 2019 Thank You !!!