Journey from on prem to the cloud with kubernetes
- 1. April 20, 2021 Bob DeRosa Senior Devops Architect Journey From On Prem to the Cloud with Kubernetes
- 2. 1 © 2021 | Agenda • About Broadridge • The Project • Why we made the switch • Preparation • Process • What we should have done differently • Results • What’s Next • References • Questions
- 3. 2 © 2021 | About Broadridge • Global Fintech company with over $4.5 billion in revenues • Handle millions of trades a day involving trillions of dollars • Support communications that reach 75% of North American households • Manage shareholder voting in 120 countries. • Hosted nearly 2000 virtual shareholder meetings in 2020 • 10,000+ total employees, thousands of technical associates • www.broadridge.com Broadridge is the leading provider of investor communications
- 4. 3 © 2021 | The Project • Migrate our DevOps tools from on-prem into AWS • Large CloudBees CI (Jenkins) installation with tens of thousands of jobs that run regularly and thousands of deployments a day. • Monolithic design with around 10 masters and 1000 agents (VMs). • One team’s run-away job could affect other teams. • Adding resources took weeks. • A lot of time spent on maintenance and patching • Mostly VMWare some AWS some Solaris located all around the world
- 5. 4 © 2021 | Why We Made the Switch Why Kubernetes? • CloudBees CI optimized for K8s • Build agents work very well as pods. (Flexibility in scaling) • Ability to have more customized images for development teams. • Easier to patch and rollback. • Reduced infrastructure Why the cloud? • Corporate direction to move applications to the cloud • Better Scalability • Better Reliability • Infrastructure as a Service
- 6. 5 © 2021 | Preparation • If you don’t need to use Kubernetes, then don’t • Not the right environment for a lift and shift • Apply past lessons learned to the new design • Fast, cheap and good, pick two • Choose good! Take advantage of move to re-architect FAST CHEAP GOOD EXPENSIVE SLOW NOT POSSIBLE BAD
- 7. 6 © 2021 | Preparation Rockets are hard; so is Kubernetes! • Make sure you have the expertise to do this • Hire folks with K8s experience • Set expectations that this is going to take time • Build a Minimal Viable Product (MVP) • Start small, work with a few trusted customers to get it right and iterate
- 8. 7 © 2021 | Process Document, review, and test architecture • Do a quick POC to test assumptions • Use native services whenever possible • Plan for Disaster Recovery • Shared responsibility model (code and data are our responsibility) • Cloud is not magic • Ensure proper security, monitoring and backups • Aqua for security • Datadog for monitoring • Kasten K10 for backup Photo credit: unsplash
- 9. 8 © 2021 | Process • Storing code in Version Control • Part of CI/CD pipeline • Auditable • Repeatable • Push problems to the left Infrastructure/Configuration as Code The Noun Project icons
- 10. 9 © 2021 | Process • Expertise of vendors • Capture best practices as code • Terraform modules for EKS • Helm Charts for tools and ingress • Take advantage of 3rd party Helm Charts Don’t reinvent the wheel, just improve it Photo credit: langevo.blogspot.com/
- 11. 10 © 2021 | What We Should Have Done Differently Hire talent faster Less features in MVP Fail faster sooner
- 12. 11 © 2021 | Results •Reusable EKS Terraform Modules •Reusable Helm Charts •Wrap 3rd party Charts •Developed Automation to synchronize 3rd party Helm Charts •Leveraged this work to quickly deploy CloudBees CD
- 13. 12 © 2021 | What’s Next • Migrating internal customers to new platform • Scaling • 3rd party images • Automating testing • Other groups adopting code • People who gained skills with Infrastructure as code now helping on other projects. • Catalyst to rethink how our DevOps pipelines work. Photo credit: unsplash