A GitOps model for High Availability and Disaster Recovery on EKS
1 like710 views
The document presents a GitOps model for high availability and disaster recovery on EKS, emphasizing the simplification Kubernetes offers while integrating with cloud technologies. Paul Curtis, a principal solutions architect at Weaveworks, discusses the key principles of GitOps, which include making system states declarative and version-controlled, ensuring automated compliance and visibility. Upcoming webinars are scheduled to cover GitOps management for multi-cluster Kubernetes and governance for Kubernetes platforms.
![Paul--WeaveWorks:~ pcurtis$ eksctl create cluster
[ℹ] eksctl version 0.16.0
[ℹ] using region us-east-2
[ℹ] setting availability zones to [us-east-2b us-east-2c us-east-2a]
[ℹ] subnets for us-east-2b - public:192.168.0.0/19 private:192.168.96.0/19
[ℹ] subnets for us-east-2c - public:192.168.32.0/19 private:192.168.128.0/19
[ℹ] subnets for us-east-2a - public:192.168.64.0/19 private:192.168.160.0/19
[ℹ] nodegroup "ng-ae8a8623" will use "ami-020a35f771eac7c58"
[AmazonLinux2/1.14]
[ℹ] using Kubernetes version 1.14
[ℹ] creating EKS cluster "wonderful-rainbow-1586288088" in "us-east-2" region
with un-managed nodes
[✔] EKS cluster "wonderful-rainbow-1586288088" in "us-east-2" region is ready
( . . . . . . )](https://image.slidesharecdn.com/webinarhaanddroneks04082020-200408223033/85/A-GitOps-model-for-High-Availability-and-Disaster-Recovery-on-EKS-12-320.jpg)
![Paul--WeaveWorks:~ pcurtis$ EKSCTL_EXPERIMENTAL=true
eksctl enable repo
--git-url git@github.com:pfcurtis/my-eks-config
--git-email paulc@weave.works
--cluster wonderful-rainbow-1586288088
--region us-east-2
[ℹ] Generating manifests
[ℹ] Cloning git@github.com:pfcurtis/my-eks-config
Cloning into
'/var/folders/s0/2cthvv_n2vb9jzxtl_v535_m0000gn/T/eksctl-install-flux-clone-513640035'...
Warning: Permanently added the RSA host key for IP address '140.82.112.4' to the list of
known hosts.
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (3/3), done.
Already on 'master'
Your branch is up to date with 'origin/master'.
[ℹ] Writing Flux manifests
[ℹ] created "Namespace/flux"
[ℹ] Applying manifests
( . . . . . . )](https://image.slidesharecdn.com/webinarhaanddroneks04082020-200408223033/85/A-GitOps-model-for-High-Availability-and-Disaster-Recovery-on-EKS-15-320.jpg)
![[ℹ] Flux will only operate properly once it has write-access to
the Git repository
[ℹ] please configure git@github.com:pfcurtis/my-eks-config so
that the following Flux SSH public key has write access to it
ssh-rsa XXXXXXXXXXXXXX
( . . . . . . )](https://image.slidesharecdn.com/webinarhaanddroneks04082020-200408223033/85/A-GitOps-model-for-High-Availability-and-Disaster-Recovery-on-EKS-16-320.jpg)
![Usage: eksctl create cluster [flags]
General flags:
-n, --name string EKS cluster name
-r, --region string AWS region
--managed Create EKS-managed nodegroup
--fargate Create a Fargate profile
scheduling pods in the default and kube-system namespaces
onto Fargate](https://image.slidesharecdn.com/webinarhaanddroneks04082020-200408223033/85/A-GitOps-model-for-High-Availability-and-Disaster-Recovery-on-EKS-19-320.jpg)
A GitOps model for High Availability and Disaster Recovery on EKS
- 1. A GitOps model for High Availability and Disaster Recovery on EKS Paul Curtis, Principal Solutions Architect, Weaveworks
- 2. Webinar Platform - FAQs Using Zoom Questions? • You are in listen only mode • Q&A session will follow the presentation, please use the Q&A panel to submit questions • Hit escape to exit full screen Technical Issues - please visit Zoom Help https://support.zoom.us/hc/en-us/articles/206175806-Top-Questions
- 3. 3 Speaker Introduction Paul Curtis, Principal Solutions Architect, Weaveworks Paul is a Principal Solutions Architect at Weaveworks, where he provides pre- and post-sales technical expertise. Paul comes from the big data world and machine learning world, having spent seven years at MapR. Paul has served as Senior Operations Engineer for Unami, a startup founded to deliver on the promise of interactive TV for consumers, and was Systems Manager for Spiral Universe, a company providing school administration software as a service. He has also held senior support engineer positions at Sun Microsystems, as well as enterprise account technical management positions for both Netscape and FileNet. Earlier in his career, Paul worked in financial application development for Applix, IBM Service Bureau, and Ticketron. @pfcurtis_NY ✉ paulc@weave.works
- 4. Weaveworks ❤ GitOps Open Source Services Weave Kubernetes Platform Weaveworks enabling GitOps across the Kubernetes landscape • Key open source projects: flux, flagger, eksctl • Top 10 contributor to CNCF • GitOps thought leadership • Design, consulting and delivery of K8s • GitOps & Kubernetes training and quickstart • Helping teams optimise their platform • Manage 100’s of clusters with GitOps • Cloud and on-premise • Repeatability, flexibility and situational awareness 4
- 5. Building Out High Availability in a Data Center is Difficult and Costly
- 6. Building Out High Availability in a Data Center is Difficult and Costly Cloud Providers Simplify This, but Add Complexity in Other Areas
- 7. Kubernetes Makes the Application Layer Simple
- 8. Kubernetes Makes the Application Layer Simple EKS Makes Kubernetes Simpler
- 9. Kubernetes Makes the Application Layer Simple EKS Makes Kubernetes Simpler eksctl Makes EKS Simpler
- 10. 1 The entire system is described declaratively 2 The canonical desired system state is versioned (with Git) 3 Approved changes to the desired state are automatically applied to the system 4 Software agents ensure correctness and alert on divergence GitOps Principles 10
- 11. GitOps – An Operating Model for Cloud Native Deployment (clusters, apps) Monitoring Logging (Observability) Management (operations) Git Build GIT Test IDE Unifying Deployment, Monitoring and Management. Git as the single source of truth of a system’s desired state ALL intended operations are committed by pull request ALL diffs between intended and observed state with automatic convergence ALL changes are observable, verifiable, and auditable “Immutability Firewall” Kubernetes GitOps Continuous Integration 11
- 12. Paul--WeaveWorks:~ pcurtis$ eksctl create cluster [ℹ] eksctl version 0.16.0 [ℹ] using region us-east-2 [ℹ] setting availability zones to [us-east-2b us-east-2c us-east-2a] [ℹ] subnets for us-east-2b - public:192.168.0.0/19 private:192.168.96.0/19 [ℹ] subnets for us-east-2c - public:192.168.32.0/19 private:192.168.128.0/19 [ℹ] subnets for us-east-2a - public:192.168.64.0/19 private:192.168.160.0/19 [ℹ] nodegroup "ng-ae8a8623" will use "ami-020a35f771eac7c58" [AmazonLinux2/1.14] [ℹ] using Kubernetes version 1.14 [ℹ] creating EKS cluster "wonderful-rainbow-1586288088" in "us-east-2" region with un-managed nodes [✔] EKS cluster "wonderful-rainbow-1586288088" in "us-east-2" region is ready ( . . . . . . )
- 13. GitOps Makes Creating EKS Clusters Repeatable
- 14. GitOps Makes Creating EKS Clusters Repeatable eksctl Profiles Are the Key: Multiple Clusters Built from the Same Profile
- 15. Paul--WeaveWorks:~ pcurtis$ EKSCTL_EXPERIMENTAL=true eksctl enable repo --git-url git@github.com:pfcurtis/my-eks-config --git-email paulc@weave.works --cluster wonderful-rainbow-1586288088 --region us-east-2 [ℹ] Generating manifests [ℹ] Cloning git@github.com:pfcurtis/my-eks-config Cloning into '/var/folders/s0/2cthvv_n2vb9jzxtl_v535_m0000gn/T/eksctl-install-flux-clone-513640035'... Warning: Permanently added the RSA host key for IP address '140.82.112.4' to the list of known hosts. remote: Enumerating objects: 3, done. remote: Counting objects: 100% (3/3), done. remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0 Receiving objects: 100% (3/3), done. Already on 'master' Your branch is up to date with 'origin/master'. [ℹ] Writing Flux manifests [ℹ] created "Namespace/flux" [ℹ] Applying manifests ( . . . . . . )
- 16. [ℹ] Flux will only operate properly once it has write-access to the Git repository [ℹ] please configure git@github.com:pfcurtis/my-eks-config so that the following Flux SSH public key has write access to it ssh-rsa XXXXXXXXXXXXXX ( . . . . . . )
- 17. Paul--WeaveWorks:~ pcurtis$ kubectl get pods -A NAMESPACE NAME READY STATUS RESTARTS AGE flux flux-757ff78d8f-2gjhk 1/1 Running 0 34m flux flux-helm-operator-6878997bb5-fg9sr 1/1 Running 0 34m flux memcached-6bb74f94c4-9694m 1/1 Running 0 34m kube-system aws-node-bgsns 1/1 Running 0 39m kube-system aws-node-td442 1/1 Running 0 39m kube-system coredns-74dd858ddc-cjkkb 1/1 Running 0 45m kube-system coredns-74dd858ddc-lk2r7 1/1 Running 0 45m kube-system kube-proxy-b2qjw 1/1 Running 0 39m kube-system kube-proxy-k69bs 1/1 Running 0 39m
- 18. EKS Multiple Regions 18 git registry Flux Scope Flagger Workload Workload Workload ● Multi Region, Multi-AZ ● GitOps Managed Flux Scope Flagger Workload Workload Workload LB/Mesh
- 19. Usage: eksctl create cluster [flags] General flags: -n, --name string EKS cluster name -r, --region string AWS region --managed Create EKS-managed nodegroup --fargate Create a Fargate profile scheduling pods in the default and kube-system namespaces onto Fargate
- 20. Things to Consider: ● AWS Cloud Load Balancing or K8s Network/Service Mesh Load Balancing ● Persistent Storage: Stateful Data Across Multiple Clusters ● Authorization: IAM users, K8s Roles, and K8s RBAC (put these in the profile!)
- 21. Things to Consider: ● AWS Cloud Load Balancing or K8s Network/Service Mesh Load Balancing There are a number of ways to accomplish this today: Multi-Cluster Networking (Layer 2-ish): Weave Net, Amazon VPC-to-VPC Multi-Cluster Service Meshes: Istio, Cilium, Envoy (soon) Classic Load Balancing: Amazon Elastic Load Balancers + Route53 to Ingresses Choosing Will Depend on the Services Running: Do I need session awareness? Do I desire progressive delivery? Do I desire a single control plane?
- 22. Things to Consider: ● Persistent Storage: Stateful Data Across Multiple Clusters There are a number of ways to accomplish this today: Service Level Replication: Database replication, for example Shared Storage Replication: NetApp, MapR/HPE, Portworx Disaster Recovery only: Traditional file system block synchronization, “rsync”, Transport of file system snapshots, Old School backup/restore
- 23. ● Kubernetes application platform ● Management of cluster and applications ● Builds on GitOps and adds enterprise features ● Define clusters and components using a model based system ● Deploy new clusters using those definitions: multiple back-ends ● Alerting and operations built-in Weave Kubernetes Platform 23
- 24. Questions?
- 25. Upcoming webinars in April April 21: GitOps powered management for multi-cluster Kubernetes across any cloud We have partnered up with Solo.io to discuss common pitfalls and demonstrate how GitOps workflows along with Gloo and Service Mesh Hub can configure, operationalize and extend your application environment across clouds. April 22: A GitOps model for managing governance, risk and compliance for Kubernetes platforms Learn how GitOps workflows can provide a separation of concerns between development and deployment, automate transparency and auditability and reduce risk through rollback and logging.
- 26. Thank you! EKS Control: https://eksctl.io GitOps Enable EKS: https://eksctl.io/gitops-quickstart/setup-gitops/ Enterprise Ready GitOps with the Weave Kubernetes Platform: https://www.weave.works/blog/enterprise-ready-gitops-with-the-weave-ku bernetes-platform Paul Curtis @pfcurtis_NY ✉ paulc@weave.works