Kubernetes @ Nanit
0 likes276 views
This document discusses Kubernetes and its use at nanit for container orchestration. It provides an overview of Kubernetes features like service discovery, high availability, port management and compares them to alternative solutions like Amazon ECS. It also describes nanit's development, deployment and monitoring processes which involve developing locally, testing on Jenkins, deploying to Kubernetes clusters from a Makefile, and monitoring applications on Kibana and Grafana with alerts sent to Slack.
Kubernetes @ Nanit
- 1. Kubernetes @ nanit Chen Fisher, VP Engineering
- 8. Orchestration - Service Discovery - High Availability - Resource Management - Port Management API Clipper Focus Gateway Grant Mailman RTMP RabbitMQ S3 Stampa State Department Timeline Taper Algorithms
- 11. Service Discovery / ECS - Offers no service discovery - One can use external tools like Consul (https://aws.amazon.com/blogs/compute/service-discovery-via-consul-with-amazon-ecs/)
- 12. Service Discovery / Kubernetes - Has built-in service discovery (http://kubernetes.io/docs/user-guide/services/) - One can simply refer to a service by its name - Can group and isolate services by namespaces
- 13. High Availability / ECS - Uses ELB health checks - Limited to HTTP/TCP health checks - If not using ELB then only recovers on container exit - Must create ELB for every service to get health checks
- 14. High Availability / Kubernetes - Offers exec health check in addition to http - One can perform application level health check
- 15. Port Management / ECS - Maps an ELB to a service which exposes a container port on host - Cannot have more than one container with same exposed port on same host - Must manually manage container ports
- 16. - Assigns random port for every container - Maps ELB to the container’s random port Port Management / Kubernetes
- 17. Kubernetes - AWS - Kube 1.2.2 - Two clusters: Staging & Production - 20+ m4-large instances on production (pre-launch) - 100+ pods - RDS, DynamoDB outside of cluster
- 18. From Dev to Prod
- 19. Dev and test locally clojure, ruby, go, c/c++, elixir
- 20. Push to Github pull requests
- 21. Jenkins picks up and tests marks commit on Github
- 22. Manual merge to master
- 25. Deployment / directory structure ● nanit/ ○ api/ ■ app/ ■ Dockerfile ■ code... ■ kube/ ■ dep.yml ■ Makefile
- 26. apiVersion: extensions/v1beta1 kind: Deployment metadata: name: {{APP_NAME}} spec: replicas: {{REPLICAS}} template: metadata: labels: app: {{APP_NAME}} spec: volumes: - name: some-secret secret: secretName: some-secret containers: - name: {{APP_NAME}} image: {{IMAGE_NAME}} ... Deployment / kube yaml file
- 27. Deployment / Makefile APP_NAME=api IMAGE_TAG=$(shell git log -n 1 --pretty=format:%h app) IMAGE_NAME=nanit/$(APP_NAME):$(IMAGE_TAG) define generate-dep sed -e 's/{{IMAGE_NAME}}/$(IMAGE_NAME)/g' kube/dep.yml endef docker: sudo docker pull $(IMAGE_NAME) || (sudo docker build -t $(IMAGE_NAME) app && sudo docker push $(IMAGE_NAME)) deploy: docker $(call generate-dep) | kubectl apply -f - migrate: … test: ...
- 28. Deployment / config from Makefile: REPLICAS?=$(shell curl -s config/$(NANIT_ENV)/api/replicas) from kube yaml file: spec: replicas: {{REPLICAS}}
- 29. Post deployment (Monitoring & Alerts)
- 30. Post deployment (Monitoring & Alerting) - Currently using built in Kibana & Grafana - Alerting to Slack using proprietary service
- 31. Thank You!