EKS Workshop

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EKS - Kubernetes on AWS
Christoph Kassen
Solutions Architect - chrkas@amazon.de
Jonas Wagner
Solutions Architect

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What can you expect?

WHY DO WE LOVE CONTAINERS?
Packaging Distribution Immutable
infrastructure

Open source container
management platform
Helps you run
containers at scale
Gives you primitives
for building
modern applications
What is Kubernetes?

W h y d e v e l o p e r s l o v e K u b e r n e t e s
Vibrant and growing community
of users and contributors

Why developers love Kubernetes
A single extensible API
S C A L E P E R F O R M A N C E B R E A D T H

Cloud-native applications
M I C R O S E R V I C E
T O O L I N G
N A T I V E
A P P L I C A T I O N S

“Run Kubernetes for me.”
“Native AWS integrations”
“An open source Kubernetes
experience.”

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
ELASTIC CONTAINER SERVICE FOR KUBERNETES

Amazon Container Services

EKS is Kubernetes Certified

API
server
Cloud
controller
Controller
manager
Scheduler Add-onsKubeDNS
EKS control plane

Open Source Kubernetes Community
Kubernetes
https://github.com/kubernetes/kuber
netes
CNI plugin
https://github.com/aws/amazon-vpc-
cni-k8s
Heptio AWS Authenticator
https://github.com/heptio/authentic
ator
Virtual Kubelet
https://github.com/virtual-
kubelet/virtual-kubelet/
SIG AWS
https://github.com/kubernetes/com
munity/tree/master/sig-aws
Cloud Provider Working Group
https://github.com/kubernetes/com
munity/tree/master/wg-cloud-
provider
External-DNS
https://github.com/kubernetes-
incubator/external-dns
CoreOS ALB Ingress
https://github.com/coreos/alb-
ingress-controller
CODE
REVIEWS
FIXING
BUGS
IMPLEMENTING
NEW FEATURES

mycluster.eks.amazonaws.com
Availability
Zone 1
Availability
Zone 2
Availability
Zone 3
Kubectl
Workers

EC2 Worker
Nodes
EKS Control
Plane
Customer VPC EKS VPC
Network Load
Balancer
ENI
API Access
Kubectl
Exec/Logs
TLS
Static IPs
ENI Attachment
Autoscaling Group
EKS Architecture

Metrics
Nodes
Node exporter
Pod/Container
Kube-state-metrics
cAdvisor
Application
/metrics
JMX
Cluster-wide Aggregator
Prometheus, Heapster
Visualizer
Grafana, Kibana, Dashboard
Data Model
InfluxDB, Graphite
Alerting
AlertManager, Kapacitor

IAM authentication
with Kubernetes

~/.kube/config (with IAM)
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: REDACTED
server: https://1234.sk1.us-west-2.eks.amazonaws.com
name: eks
contexts:
- context:
cluster: eks
user: eks
name: eks
current-context: eks
kind: Config
users:
- name: eks
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
command: heptio-authenticator-aws
args:
- token
- -i
- eks
Config file is no longer
user-specific J

Access and Authentication
IAM ROLE
User X
IAM ROLE
Service Account Y
kubectl → K8s APIs → CRUD Operations on K8s
aws-cli → EKS Service APIs → CRUD Operations on Infra
K8s Master Nodes
K8s Master Nodes
K8s Master Nodes
API Server
Controller Mgr
kubelet
etcd
Cloud Controller Mgr.
Scheduler
Authentication
Webhook Tokens
Authorization
RBAC Mode
Admission Control
NamespaceLifecyle,LimitRanger
ServiceAccount,DefaultStorageClass,
ResourceQuota
AWS STS
client side
Heptio-aws-authenticator
server side

Worker provisioning
k u b e c t l
A W S A u t h
c o n f i g m a p & R B A C
W o r k e r s
R o l e
R o l e
config map

Native VPC networking
with CNI plugin
Pods have the same VPC
address inside the pod
as on the VPC
Simple, secure networking
Open source and
on Github
…{ }

Networking with CNI plugin
172.16.0.0/16
User X
Service
Account Y
Kubectl
K8s Node 2K8s Node 1
kubelet
kube-proxy
kubelet
kube-proxy
VPC Subnet per AZ
172.16.0.1/24
ENI ENIPrimary Private IP:
172.16.1.118
Secondary IPs:
172.16.1.147,
172.16.1.224….
Service: Front end
POD 2 POD 3
eth0
Service: Back end
POD 1 POD 4
eth0
ec2.associateaddress()
L3 RouteTable
veth0 Bveth0 A
eth0
172.16.1.147/32
eth0
172.16.1.224/32
CNI
K8s Master NodesK8s Master NodesK8s Master Nodes
API Server
Controller Manager
kubelet
etcd
Scheduler
kube-proxy

DNS, Services and ELB
172.16.0.0/16
User X
Service
Account Y
K8s Node 2K8s Node 1
kubelet
kube-proxy
kubelet
kube-proxy
VPC Subnet per AZ - 172.16.0.1/24
ENI ENI
Service: Front end
POD 2 POD 3
Service: Back end
POD 1 POD 4
CNI
K8s Master NodesK8s Master Nodes
K8s Master Nodes
API Server
Controller Manager
kubelet
etcd
Scheduler
kube-proxy
DNS
kubedns
dnsmasq
healthz
DNS Service – Static IP
POD 2 POD 2
kind: Service
type: LoadBalancer

Kubernetes Network
Policies enforce network
security rules
Calico is the leading
implementation of the
network policy API
Open source, active
development (>100
contributors)
Commercial support
available from Tigera

S T A G E
S E P A R A T I O N
“ T E N A N T ”
S E P A R A T I O N
F I N E - G R A I N E D
F I R E W A L L S
C O M P L I A N C E
Namespaces – without
network policy, they are
not network isolated
Reduce attack surface
within microservice-based
applications
Isolate dev, test, and prod E.g., PCI, HIPAA

Network Policy
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: web-allow-prod
spec:
podSelector:
matchLabels:
app: web
ingress:
- from:
- namespaceSelector:
matchLabels:
purpose: production
Select affected Pods
Define traffic that is
allowed

Lab 1

Hands-on! – Lab 1
AWS Workshop for Kubernetes
1. Create an EKS cluster
1. Follow the instructions at https://eksworkshop.com (beginner)
2. Use eksctl https://eksctl.io (advanced)
3. Follow the EKS docs
https://docs.aws.amazon.com/eks/latest/userguide/getting-
started.html (the “hard“ way)

Additional resources
https://github.com/ramitsurana/awesome-kubernetes
https://discuss.kubernetes.io/
TGIK Playlist:
https://www.youtube.com/playlist?list=PLvmPtYZtoXOEN
HJiAQc6HmV2jmuexKfrJ
https://aws.amazon.com/blogs/compute/tag/containers/

Kubernetes Concepts

Kubernetes concepts
port 8080 port 8080
ReplicaSet
#Pods—2
label selector: v1
ReplicaSet
#Pods—1
label selector: v2
Node
Docker
Pod
Containers

Kubernetes concepts
URI: /svc1/* URI: /svc2/*
DaemonSet
Daemon pod

Application Deployment

Pod definition example
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
selector:
matchLabels:
app: nginx
replicas: 2 # tells deployment to run 2 pods matching the template
template: # create pods using pod definition in this template
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:stable-alpine
ports:
- containerPort: 80

Deployments
❯ kubectl create -f nginx.yaml --record
❯ kubectl rollout history deployment/nginx
❯ kubectl expose deployment/nginx --port=80 --target-port=80 --
name=nginx --type=LoadBalancer
❯ kubectl describe svc nginx
❯ curl http://xyz.us-west-2.elb.amazonaws.com

Deployments - Helm
• Package management for k8s
• Chart - collection of files
• Description of k8s resources
• Flexible templating
• Tiller
• Installed on k8s cluster
• Client
• Cli tool to install/update Charts

Using helm
❯ helm search nginx
NAME CHART VERSION APP VERSION DESCRIPTION
stable/nginx-ingress 0.19.2 0.14.0 An nginx Ingress controller that uses ConfigMap...
stable/nginx-lego 0.3.1 Chart for nginx-ingress-controller and kube-lego
stable/gcloud-endpoints 0.1.0 Develop, deploy, protect and monitor your APIs ...
❯ helm install stable/nginx-ingress --name nginx-ingress --set
rbac.create=true
[displays README + information about deployment]
❯ helm list
NAME REVISION UPDATED STATUS CHART
NAMESPACE
Nginx-ingress 1 Mon May 21 18:30:17 2018 DEPLOYED nginx-ingress-0.19.2 default

Hosting Helm repositories
• Anywhere that serves HTTP can host a helm repo
• Host private Helm Repo with Chartmuseum
https://github.com/kubernetes-helm/chartmuseum
• There’s also a handy plugin for S3!
• This means IAM Role = auth for your repo J
• https://github.com/hypnoglow/helm-s3

Deploying Helm on EKS
Helm 2.9+ works with EKS out of the box
Helm needs RBAC permissions
❯ kubectl -n kube-system create serviceaccount tiller
❯ kubectl create clusterrolebinding tiller --clusterrole
cluster-admin --serviceaccount=kube-system:tiller
❯ helm init --service-account=tiller

Understanding deployments
# Columnar output
❯ kubectl get services # List all services in the namespace
❯ kubectl get pods --all-namespaces # List all pods in all namespaces
❯ kubectl get pods -o wide # List all pods in the namespace, with details
❯ kubectl get rc <rc-name> # Get a replication controller
# Verbose output
❯ kubectl describe nodes <node-name>
❯ kubectl describe pods <pod-name>
❯ kubectl describe pods/<pod-name> # Equivalent to previous
❯ kubectl describe pods <rc-name> # Lists pods created by<rc-name>using common prefix
# List Services Sorted by Name
❯ kubectl get services --sort-by=.metadata.name
# Get ExternalIPs of all nodes
❯ kubectl get nodes -o
jsonpath='{.items[*].status.addresses[?(@.type=="ExternalIP")].address}'

Interacting with Pods
❯ kubectl logs <pod-name> # dump pod logs (stdout)
❯ kubectl logs –f <pod-name> # stream pod logs (stdout)
❯ kubectl run –i --tty busybox—image=busybox -- sh # Run pod as interactive shell
❯ kubectl attach <podname> -i # Attach to Running Container
❯ kubectl port-forward <podname> <local>:<remote> # Forward port of Pod to localhost
❯ kubectl port-forward <servicename><port> # Forward port to service
❯ kubectl exec <pod-name> -- ls / # Run command in existing pod (1 container case)
❯ kubectl exec <pod-name> -c <container-name> -- ls /

Deployment Strategies

Rolling Update
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: my-app
labels:
app: my-app
spec:
replicas: 10
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1 # Numeric or percentage based value
maxUnavailable: 0
[...]

Blue / Green Deployment
kind: Deployment
metadata:
name: my-app-blue
labels:
app: my-app
spec:
replicas: 3
template:
metadata:
labels:
app: my-app
version: blue
[...]
Blue
kind: Deployment
metadata:
name: my-app-green
labels:
app: my-app
spec:
replicas: 3
template:
metadata:
labels:
app: my-app
version: green
[...]
Green

Blue / Green Deployment
Blue
kind: Service
metadata:
name: my-app
labels:
app: my-app
spec:
type: LoadBalancer
ports:
- name: http
port: 80
targetPort: http
selector:
app: my-app
version: blue
kind: Service
metadata:
name: my-app
labels:
app: my-app
spec:
type: LoadBalancer
ports:
- name: http
port: 80
targetPort: http
selector:
app: my-app
version: green
Green
kubectl patch service my-app -p '{"spec":{"selector":{"version":"green"}}}'

Canary Deployment
Production
kind: Deployment
metadata:
name: my-app-prod
labels:
app: my-app
spec:
replicas: 9
template:
metadata:
labels:
app: my-app
spec:
containers:
- name: my-app
image: images/container:v1
[...]
kind: Deployment
metadata:
name: my-app-canary
labels:
app: my-app
spec:
replicas: 1
template:
metadata:
labels:
app: my-app
spec:
containers:
- name: my-app
image: images/container:v2
[...]
More examples at https://container-solutions.com/kubernetes-deployment-strategies/
Canary

Load Balancing

Nginx PodsEC2 instances
kube-proxy
:32002
nginx-service
:32001
Internet
10001:8080
10002:8080
10003:8080
Request to NGINX Pod
{NLB}:443
NLB
NLB Forwards to the node
{node:32001}
Service Type – LoadBalancer (NLB)
k8s service ClusterIP
receives request
kube-proxy
load balances
to pods

Network Load Balancer
apiVersion: v1
kind: Service
metadata:
name: nginx
namespace: default
labels:
app: nginx
annotations:
service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
spec:
type: LoadBalancer
externalTrafficPolicy: Local
ports:
- name: http
port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
More options:
• Draining
• Logging
• SSL Certs
• Tagging
• Security groups
• Health checks
https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/providers/aws
/aws.go

Nginx Pods
EC2 instances
kube-proxy
:32001
nginx-service
:32003
Internet
Request to NGINX Pod
{ALB}:443
ALB
ALB Routes based on
the path.
/api
/home
10002:8080
Webapp Pods
10002:8080
Installation: https://github.com/pahud/eks-alb-ingress
Ingress Type – CoreOS ALB Ingress
kube-proxy
:32002
webapp-service
:32004
Load Balances
to pods
Proxies request
to the k8s service
ClusterIP

DNS

Automatic Route53 DNS creation for services
apiVersion: v1
kind: Service
metadata:
name: nginx
annotations:
# Uses https://github.com/kubernetes-incubator/external-dns
external-dns.alpha.kubernetes.io/hostname: nginx.highlyavailable.systems.
spec:
type: LoadBalancer
ports:
- port: 80
name: http
targetPort: 80
selector:
app: nginx

Automatic Route53 DNS creation for Ingress
kind: Ingress
metadata:
name: nginx
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: nginx.highlyavailable.systems
http:
paths:
- backend:
serviceName: nginx
servicePort: 80

Lab 2

Lab 2
1. https://github.com/aws-samples/aws-workshop-for-
kubernetes/tree/master/01-path-basics/103-kubernetes-
concepts
kubernetes/tree/master/03-path-application-
development/303-app-update
kubernetes/tree/master/04-path-security-and-
networking/405-ingress-controllers#alb-ingress-controller
development/307-app-management-with-helm

Scheduling

Controlling scheduling
Resource requirements
Resource filters

Limit resource usage
Container A Container B
limit
request
900m
600m
limit
request
800m
400m
⎲
⎳ Pod CPU and memory resources

Resource Quotas
apiVersion: v1
kind: Pod
metadata:
name: production
spec:
containers:
- name: nginx-pod
image: nginx
resources:
limits:
memory: "800Mi"
cpu: "800m" # 0.8 vCPU
requests:
memory: "600Mi"
cpu: "400m“ # 0.4 vCPU
Applied per Namespace
apiVersion: v1
kind: ResourceQuota
metadata:
name: production
spec:
hard:
requests.cpu: "1"
requests.memory: 1Gi
limits.cpu: "2"
limits.memory: 2Gi
ResourceQuota defined
both, so Pod must
define both
Pod Resource Request

Constraints
• Taints Node-level
• Tolerations Pod-level
Topology filters

Taints and Tolerations
# Taint node
$ kubectl taint nodes ip-10-0-32-12.us-west-2.compute.internal
skynet=false:NoSchedule
# Tolerations
kind: Pod
spec:
tolerations:
- key: skynet
operator: Equal
value: “false”
effect: NoSchedule
[...]
Match taint to
schedule onto
tainted node

Constraints
• Taints Node-level
• Tolerations Pod-level
Affinity/Anti-Affinity Topology filters

Affinity / Anti-Affinity
● Control scheduling onto nodes
○ Combine with Taints & Tolerations
● Distribute Pods across cluster
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: "beta.kubernetes.io/instance-type"
operator: In
values: [“r4.large",“r4.xlarge"]

App Auto-Scaling

App Auto-Scaling
# Install heapster
❯ kubectl create -f templates/heapster/heapster.yaml
❯ kubectl run webapp --image=trevorrobertsjr/webapp --requests=cpu=50m --
expose --port=8080
❯ kubectl autoscale deployment webapp --cpu-percent=5 --min=1 --max=10
❯ kubectl run -i --tty load-generator --image=busybox -- /bin/sh –c “while
true; do wget -q -O- http://webapp.default.svc.cluster.local:8080; done“
❯ kubectl get hpa –o wide
❯ kubectl delete hpa webapp
❯ kubectl delete deployment/load-generator deployment/webapp

Monitoring and Operations

Kubernetes Dashboard
• General purpose web-based UI for Kubernetes clusters
• Manage applications running in the cluster
• Manage the cluster

Kubernetes Dashboard

Prometheus, Node exporter, and Grafana
• Prometheus:
• Open-source systems monitoring and alerting
toolkit
• Collects metrics from monitored targets by
scraping metrics from HTTP endpoints
• Dynamically scrape new targets by adding
a ServiceMonitor
• Grafana:
• Open source, feature rich metrics dashboard
and graph editor
• Node exporter:

Prometheus, Node exporter, and Grafana

Prometheus installation
cd cluster-monitoring
kubectl apply -f templates/prometheus/prometheus-bundle.yaml
kubectl rollout status deployment/prometheus-operator -n monitoring
kubectl apply -f templates/prometheus/prometheus.yaml
kubectl get po -l prometheus=prometheus -n monitoring
kubectl port-forward $(kubectl get po -l prometheus=prometheus -n
monitoring -o jsonpath="{.items[0].metadata.name}") 9090 -n monitoring
open http://localhost:9090

Grafana installation
kubectl apply -f templates/prometheus/grafana-bundle.yaml
kubectl rollout status deployment/grafana -n monitoring
kubectl port-forward $(kubectl get pod -l app=grafana -o
jsonpath="{.items[0].metadata.name}" -n monitoring) 3000 -n monitoring
open http://localhost:3000/?orgId=1
kubectl delete -f templates/prometheus/prometheus-bundle.yaml

Lab 3

Hands-on! - Lab 3
kubernetes/tree/master/02-path-working-with-
clusters/205-cluster-autoscaling
kubernetes/tree/master/02-path-working-with-
clusters/201-cluster-monitoring

CI/CD pipelines

Jenkins – CI/CD with Kubernetes

AWS CodePipeline – CI/CD with Kubernetes

AWS CodePipeline – CI/CD with Kubernetes
• Continuous Deployment to Kubernetes using AWS CodePipeline,
AWS CodeCommit, AWS CodeBuild, Amazon ECR and AWS
Lambda
• CodeSuite - Continuous Deployment Reference Architecture for
Kubernetes

Deployment Tools
• Spinnaker
• Skaffold
• Jenkins X
• Argo
• ...
• kubectl
Overview: https://engineering.opsgenie.com/cloud-native-
continuous-integration-and-delivery-tools-for-kubernetes-
e6ea34d308c

Distributed Tracing

AWS X-Ray for Kubernetes

AWS X-Ray tracing

AWS X-Ray segment info

X-Ray for Kubernetes
AWS X-Ray X-Ray DaemonSet
X-Ray
trace k8s nodes running pods
Service
A
Service
B
Client
HTTP
Requests
AWS Console

Hands-on

Hands-on! - Lab 4
development/308-cicd-workflows/308-1-codesuite
development/305-app-tracing-with-jaeger-and-x-
ray/x-ray

Make sure to
stop&delete
resources!

Thank you!
Please leave 1-Minute feedback!

EKS Workshop

More Related Content

What's hot (20)

Similar to EKS Workshop (8)

More from AWS Germany (20)

Recently uploaded (20)

EKS Workshop