Building Like Amazon
- 1. Building Like Amazon Leo Zhadanovsky Julie Lawson With: Moderated by: TO USE YOUR COMPUTER'S AUDIO: When the webinar begins, you will be connected to audio using your computer's microphone and speakers (VoIP). A headset is recommended. Webinar will begin: 11:00 am, PDT TO USE YOUR TELEPHONE: If you prefer to use your phone, you must select "Use Telephone" after joining the webinar and call in using the numbers below. United States: +1 (415) 655-0060 Access Code: 224-815-615 Audio PIN: Shown after joining the webinar --OR-- The Modern Tech Stack Webinar Series
- 2. AmazonWeb Services offers a broad set of global cloud-based products to equip technology leaders to build better and more powerful solutions. Partnering with CTO Universe,AWS is now offering an exclusive program to our members and audience. The program includes up to $100,000 USD of value AWS credits, free consulting sessions with AWS SolutionsArchitect to review your environment, strategies and optimize your cost, and other resources to help you get started on migrating to AWS. If you are interested in learning more, please reach out to aws-cto-program@amazon.com.
- 3. The Modern Tech Stack Webinar Series Click on the Questions panel to interact with the presenters https://www.ctouniverse.com/frs/14319860/make-your-good-devops-team-great-with-ci-cd
- 4. About Leo Zhadanovsky Leo Zhadanovsky, a Principal Solutions Architect at Amazon Web Services, spends his days (and occasional nights!) helping customers best leverage AWS services in order to help them build highly-available, scalable and elastic architectures to fulfill their business needs. As a speaker, Leo has delivered talks at conferences around the world, including Re:Invent, OmniTI Surge, and PuppetConf. Previously the Director of Systems Engineering at the Democratic National Committee, he’s also run the DNC's on-premise and cloud infrastructure, in use by the Obama campaign, state and local Democratic parties. This infrastructure was used to support hundreds of millions of dollars in online donations, and withstood significant amounts of election day traffic. In his free time, he enjoys travelling the world, going for a bike ride, and trying out new restaurants in whatever city he finds himself in. About Julie Lawson Julie Lawson majored in English at Boston University. She worked at the college radio station, WTBU, where she developed a passion for producing shows with good music and good stories. She started her career at a small publishing house in Los Angeles and went on to become Webinar Coordinator at Aggregage, where she produces webinars and facilitates BTS webinar functions. In her spare time, she enjoys going to the beach, camping and reading great books. The Modern Tech Stack Webinar Series
- 5. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Powerful ecosystem of tools CI/CDTools Monitoring &Tracing SDKs IDE IDE and DevOpsToolkits CLI and ScriptingTools Infrastructure as Code Web Apps MobileLanguages AWS CodeStar AWS CodePipeline AWS CodeDeploy AWS CodeCommit AWS CodeBuild AWS CloudFormation AWS Cloud Dev. Kit (CDK) AWS X-Ray Amazon CloudWatch AWS Cloud9 AWS Elastic Beanstalk Amazon Corretto AWS Amplify
- 6. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Internal and external customers across industries
- 7. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Customers tell us they need to: Build organizations, processes and systems that enable rapid innovation and delivery of high performing digital solutions for their customers. Gain ‘how-to’ knowledge, grounded in real-world practice.
- 8. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Architecture, software delivery, and operations By Amazon’s senior technical executives and engineers Real-world practices with detailed explanations Content available for free on the website The Amazon Builders’ Library How Amazon builds and operates software
- 9. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Architecture Software Delivery & Operations Curated articles to help you modernize and transform your business in the cloud Deep dive insights into practices that have worked well at Amazon and those that haven’t A Point of Reference: ‘First hand’ Lessons &Techniques
- 10. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. The Amazon Builders’ Library
- 11. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Lessons learned at Amazon for developing and delivering software Automate everything Decompose for agility (microservices, 2-pizza teams) Standardized tools Infrastructure as code Belts and suspenders (governance, templates)
- 12. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon’s Journey
- 13. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Our mission Our task was to improve: • Innovation • Speed • Agility • Safety What we did: • Decomposed for agility • Cultural and operational shift • Created tools for software delivery
- 14. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Just starting out This is how many web architectures started out, and it’s how Amazon started too… There any many bottlenecks, and scaling of the web server was an immediate factor Table 3Table 1Table 1 User Remote Facility Business Processing Web server Internet Table 2 Table 3 Database
- 15. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. User Remote Facility Business Processing Web Server Internet Table 1 Table 2 Table 3 Database Web DB Table 1 Web DB Table 3 Scaling v1 In 1998 the “Distributed Computing Manifesto” came out and we began breaking things down into separate components… This was a bit better, still not very scalable
- 16. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Going further Principles • Make units a small as possible (Primitives) • De-couple based on scaling factors, not functions • Each service operates independently “Communication is terrible!” —Jeff Bezos • APIs (contracts) between services
- 17. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Impact to our development
- 18. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Monolith development lifecycle
- 19. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Monolith development lifecycle
- 20. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Monolith development lifecycle MonitorBuild Test Release MonitorBuild Test Release MonitorBuild Test Release MonitorBuild Test Release MonitorBuild Test Release MonitorBuild Test Release This led to changes in organization
- 21. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Impact to our organization
- 22. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. “Two-pizza” teams • Own a service • Minimizes social constraints (Conway’s law) • Autonomy to make decisions Getting (re)organized
- 23. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Transformation timeline Monolithic application + teams Microservices + 2-pizza teams 2001 2002
- 24. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Teams Own Everything How can one team do all of this? • Planning • Security • Performance • Scalability • Deployment • Operation • Bugs • Documentation • Testing…
- 25. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Now we have…
- 26. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Today we have Modern Applications Modern Applications • Use independently scalable microservices (serverless, containers...) • Connect through APIs • Deliver updates continuously • Adapt quickly to change • Scale globally • Are fault tolerant • Carefully manage state and persistence • Have security built-in
- 27. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Measuring success
- 28. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. How we think about metrics Business metrics Operational metrics Input goals Enablement Growth Errors Features Principal reviews Usage Throttling Use cases Security training Feedback Failed deployments Performance Ops training Performance
- 29. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Enabling our teams through automation
- 30. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Software delivery at Amazon Dev/Test Code review Pre-production testing Production
- 31. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Software delivery at Amazon Dev/Test Code review Pre-production testing Production
- 32. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Dev/Test • Works on smallest thing possible • Only local code branches • Develops and tests locally • Explicitly defines dependencies (including those needed for testing) • Publishes changes for review Developer (or cloud) workstation AWS CodeCommit Developer AWS account Code review This part should be fast Gating Function
- 33. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Continuous Delivery at Amazon Dev/Test Code review Pre-production testing Production
- 34. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon Continuous Delivery: Deep Dive Source
- 35. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon Continuous Delivery: Deep Dive Source Build
- 36. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon Continuous Delivery: Deep Dive Source Build
- 37. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon Continuous Delivery: Deep Dive Source Build alpha Automated tests Pre-production
- 38. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon Continuous Delivery: Deep Dive Source Build alpha beta Automated tests Automated integration tests Automated load/perf tests Automated browser tests Pre-production
- 39. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon Continuous Delivery: Deep Dive Source Build alpha beta gamma Automated tests Automated integration tests Automated load/perf tests Automated integration tests Automated synthetic tests API smoke tests Automated browser tests Pre-production
- 40. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon Continuous Delivery: Deep Dive Source Build alpha beta gamma Automated tests Automated integration tests Automated load/perf tests Automated integration tests Automated synthetic tests API smoke tests Automated browser tests Pre-production Production
- 41. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon Continuous Delivery: Deep Dive Source Build alpha beta gamma Automated tests Automated integration tests Automated load/perf tests Automated integration tests Automated synthetic tests API smoke tests Automated browser tests Pre-production Production
- 42. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon Continuous Delivery: Deep Dive Source Build alpha beta gamma Automated tests Automated integration tests Automated load/perf tests Automated integration tests Automated synthetic tests API smoke tests Automated browser tests Pre-production Production
- 43. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon Continuous Delivery: Deep Dive Source Build alpha beta gamma Automated tests Automated integration tests Automated load/perf tests Automated integration tests Automated synthetic tests API smoke tests Automated browser tests Pre-production Production Synthetic Monitoring
- 44. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon Continuous Delivery: Deep Dive Source Build alpha beta gamma Automated tests Automated integration tests Automated load/perf tests Automated integration tests Automated synthetic tests API smoke tests Automated browser tests Pre-production Production Synthetic Monitoring Synthetic Monitoring
- 45. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon Continuous Delivery: Deep Dive Source Build alpha beta gamma Automated tests Automated integration tests Automated load/perf tests Automated integration tests Automated synthetic tests API smoke tests Automated browser tests Pre-production Production Synthetic Monitoring Synthetic Monitoring Synthetic Monitoring
- 46. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon Continuous Delivery: Deep Dive Source Build alpha beta gamma Automated tests Automated integration tests Automated load/perf tests Automated integration tests Automated synthetic tests API smoke tests Automated browser tests … Pre-production Production Synthetic Monitoring Synthetic Monitoring Synthetic Monitoring
- 47. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Amazon Continuous Delivery: Deep Dive Source Build alpha beta gamma Automated tests Automated integration tests Automated load/perf tests Automated integration tests Automated synthetic tests API smoke tests Automated browser tests App code Infrastructure as code Configuration OS patching … Pre-production Production Synthetic Monitoring Synthetic Monitoring Synthetic Monitoring
- 48. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Pipeline Blockers: time windows, pipeline policies, code coverage, code review, security scans, dependency updates, etc. Source Build alpha beta gamma Automated tests Automated integration tests Automated load/perf tests Automated integration tests Automated synthetic tests API smoke tests Automated browser tests App code Infrastructure as code Configuration OS patching … Pre-production Production Synthetic Monitoring Synthetic Monitoring Synthetic Monitoring
- 49. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Fully automated: this part should be CAREFUL Source Build alpha beta gamma Automated tests Automated integration tests Automated load/perf tests Automated integration tests Automated synthetic tests API smoke tests Automated browser tests App code Infrastructure as code Configuration OS patching … Pre-production Production Synthetic Monitoring Synthetic Monitoring Synthetic Monitoring
- 50. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. The Builders’ Library is a site composed of: • Engineering practices • Service Patterns • Team norms and practices • Helps to find Answers • Architectural Pillars • Design principles • Asking Questions Well-Architected is a tool composed of: Well-Architected +The Builders Library = Innovation Flywheel
- 51. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Just the beginning Along the way we have learned a lot about writing software That’s performant, safe, and scalable We have had to solve some really hard problems At massive scale We know our way is not the only way, and many of our solutions are not fancy But we know they work We are long obsessed with building things to help our customers Now we want to share the benefit of what we learned along the way
- 52. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark. Architecture, software delivery, and operations By Amazon’s senior technical executives and engineers Real-world practices with detailed explanations Content available for free on the website The Amazon Builders’ Library How Amazon builds and operates software https://aws.amazon.com/builders-library
- 53. The Modern Tech Stack Webinar Series Q&A Julie Lawson With: Moderated by: Principal Solutions Architect, AWS Linkedin page: /in/leozhadanovsky/ Twitter ID: @leozh Website: leozh.net Leo Zhadanovsky Webinar Coordinator, Aggregage Linkedin page: /in/Julie-Lawson/ Email: julie@aggregage.com https://www.ctouniverse.com/frs/14319860/make-your-good-devops-team-great-with-ci-cd
- 54. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.