Breaking the monolith (an example)
0 likes98 views
The document outlines a presentation by Massimo Re Ferrè, a developer advocate at Amazon Web Services, on modern software development practices that move away from monolithic applications to microservices using AWS technologies. It discusses the benefits of independent components, automation, and various compute options like EC2, Lambda, and container orchestration with ECS and EKS. The presentation also includes a case study on the refactoring and deployment of the Yelb application using different AWS services.
Breaking the monolith (an example)
- 1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. K Y I V 11.06.2019 Breaking the monolith Massimo Re Ferrè Developer Advocate Amazon Web Services
- 2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Who Am I ? Amazon Web Services (Developer Advocate) VMware (Solutions Architect and Technical Product Manager) IBM (Professional Services and Solutions Architect) Developer (many kilos ago) Email: mreferre@amazon.com Twitter: @mreferre Github: github.com/mreferre
- 3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. How software value gets created (in the old model) Monolithic application “Time to user”: months / years Very heavy manual integrations
- 4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. How software value gets created (in the new model) “Time to user”: hours / days Small independent components… End-to-end (hands off) automation …. with different release cycles
- 5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. At this point I have a couple of options
- 6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. At this point I have a couple of options
- 7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Demo?
- 8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. What are the compute options to run my code?
- 9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. What are the compute options to run my code? Instances Containers Lambda
- 10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS compute abstractions layers AWS Lambda Amazon EC2 Metal AWS Fargate Amazon EKS physicalseverVMcontainerfunction Levelofabstraction Provider space (managed by AWS) Consumer space (managed by the customer) Amazon ECS https://aws.amazon.com/blogs/architecture/compute-abstractions-on-aws-a-visual-story/
- 11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS compute abstractions layers AWS Lambda Amazon EC2 Metal AWS Fargate Amazon EKS physicalseverVMcontainerfunction Levelofabstraction Provider space (managed by AWS) Consumer space (managed by the customer) Amazon ECS https://aws.amazon.com/blogs/architecture/compute-abstractions-on-aws-a-visual-story/
- 12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. It all starts with an app https://github.com/mreferre/yelb
- 13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. It all starts with an app https://github.com/mreferre/yelb
- 14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. It all starts with an app https://github.com/mreferre/yelb
- 15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Yelb re-factoring and re-packaging
- 16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Watch out #1 (security posture)
- 17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Watch out #2 (stateful containers)
- 18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Watch out #3 (“diversity” for lack of a better term)
- 19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Yelb re-factoring and re-packaging
- 20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. EC2 option – in practice Amazon EC2 Amazon EC2 Amazon EC2 Amazon EC2 Ruby – App server Postgres Internet Client https://github.com/mreferre/yelb/blob/master/deployments/platformdeployment/AWS/EC2/yelb-cloudformation-EC2-distributed-deployment.json Raw deployment for demonstration purposes (not well architected)
- 21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Yelb EC2 setup is very sensitive to the Linux version Trying to use a different AMI may lead to issues with the setup scripts https://github.com/mreferre/yelb/blob/master/deployments/platformdeployment/AWS/EC2/yelb-cloudformation-EC2-distributed-deployment.json
- 22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Yelb-appserver setup Yelb-appserver Linux setup script How does appserver know where the data are? https://github.com/mreferre/yelb/blob/master/deployments/platformdeployment/AWS/EC2/yelb-cloudformation-EC2-distributed-deployment.json
- 23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Build during deployment (there is no build artifacts) web server root (the JS/HTML5 web site) proxy config build of the Angular app https://github.com/mreferre/yelb/blob/master/deployments/platformdeployment/Linux/yelb-ui.sh Yelb-ui Linux setup script
- 24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Yelb re-factoring and re-packaging
- 25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. We all know what Docker is don’t we? dockerfile docker build docker run Linux docker container Docker runtime Complete Decoupling container image
- 26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Aren’t these problems solved with VMs/instances? • Containers are lightweight (or at least they should be) • Containers are infrastructure agnostics • Containers have a dev friendly packaging mechanism (Dockerfile) • Containers are ideal for immutable deployments • Containers are easy to consume (docker run <image>) and share (FROM: <image>)
- 27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Yelb build of the Docker images (Yelb UI dockerfile): m https://github.com/mreferre/yelb/blob/master/yelb-ui/Dockerfile
- 28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Yelb Docker images
- 29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. EKS Vs ECS apiVersion: extensions/v1beta1 kind: Deployment metadata: name: yelb-ui spec: replicas: 1 template: metadata: labels: app: yelb-ui tier: frontend spec: containers: - name: yelb-ui image: mreferre/yelb-ui:0.3 ports: - containerPort: 80 TaskDefinitionYelbUi: Type: AWS::ECS::TaskDefinition Properties: Family: yelb-ui NetworkMode: awsvpc RequiresCompatibilities: - FARGATE ExecutionRoleArn: !Ref 'YelbECSTaskExecutionRole' Cpu: 256 Memory: 512 ContainerDefinitions: - Name: yelb-ui Essential: true Image: mreferre/yelb-ui:0.3 Environment: - Name: SEARCH_DOMAIN Value: !Ref 'Domain' PortMappings: - ContainerPort: 80 The same exact docker image, similar app packaging concepts, slightly different semantic
- 30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. EKS option – in practice Amazon EC2 Amazon EC2 Amazon EKS AWS Account Cust Account Auto Scaling Dedicated Control Plane Deployed with EKSCTL ( https://github.com/weaveworks/eksctl )
- 31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. EKS option – in practice Amazon EC2 Client https://github.com/mreferre/yelb/blob/master/deployments/platformdeployment/Kubernetes/yaml/cnawebapp-loadbalancer.yaml Amazon EC2 Amazon EKS ELB PodService PodService PodService PodService Yelb-ui Yelb-appserver Redis-server Yelb-db AWS Account Cust Account Auto Scaling Internal name resolution leverages standard out of the box K8s inner DNS resolution Dedicated Control Plane
- 32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. ECS option (ECS/EC2) – in practice Amazon EC2 Amazon EC2 AWS Account Cust Account Auto Scaling Multi-tenant Control Plane Amazon ECS
- 33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. ECS option (ECS/EC2) – in practice Client https://github.com/mreferre/yelb/blob/master/deployments/platformdeployment/AWS/ECS/yelb-cloudformation-ECS-deployment.yaml ELB TaskService TaskService TaskService TaskService Yelb-ui Yelb-appserver Redis-server Yelb-db Internal name resolution leverages ECS Service Discovery (aka AWS Cloud Map) Amazon EC2 Amazon EC2 AWS Account Cust Account Auto Scaling Multi-tenant Control Plane Amazon ECS
- 34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. ECS option (ECS/Fargate) – in practice Client https://github.com/mreferre/yelb/blob/master/deployments/platformdeployment/AWS/ECS/yelb-cloudformation-ECS-deployment.yaml ELB TaskService TaskService TaskService TaskService Yelb-ui Yelb-appserver Redis-server Yelb-db Internal name resolution leverages ECS Service Discovery (aka AWS Cloud Map) AWS Account Multi-tenant Control Plane AWS FargateAmazon ECS
- 35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. ECS option – in practice https://github.com/mreferre/yelb/blob/master/deployments/platformdeployment/AWS/ECS/yelb-cloudformation-ECS-deployment.yaml
- 36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Yelb re-factoring and re-packaging
- 37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Yelb setup on EC2 / Containers
- 38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Yelb setup on EC2 / Containers Amazon EC2 Pod/Task Amazon EC2 Pod/Task Amazon EC2 Pod/Task Amazon EC2 Pod/Task
- 39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Lambda/Serverless option – in practice Client restaurants Amazon DynamoDB Table Table cache S3 yelb-ui bucket Amazon API Gateway AWS Lambda Lambda #1 Lambda #2 Lambda #3 Lambda #4 Lambda #5 S3 Yelb-ui source Yelb-appserver source https://github.com/mreferre/yelb/tree/master/deployments/platformdeployment/AWS/Serverless
- 40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. How to refactor the UI for S3 hosting? Client S3 yelb-ui bucket Amazon API Gateway
- 41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. How to parametrize the yelb-appserver end-point?
- 42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. How to parametrize the yelb-appserver end-point?
- 43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. How to parametrize the yelb-appserver end-point? https://www.jvandemo.com/how-to-use-environment-variables-to-configure-your-angular-application-without-a-rebuild/
- 44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. How to parametrize the yelb-appserver end-point? https://www.jvandemo.com/how-to-use-environment-variables-to-configure-your-angular-application-without-a-rebuild/
- 45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. How to parametrize the yelb-appserver end-point? https://github.com/mreferre/yelb/blob/master/yelb-ui/clarity-seed-newfiles/src/env.js
- 46. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. The application monolith All APIs logic embedded in yelb-appserver.rb
- 47. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. The application monolith functions embedded in yelb-appserver.rb
- 48. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. The application monolith In other words my application tier was ONE FILE (yelb-appserver.rb)
- 49. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. The application monolith https://medium.com/datreeio/writing-portable-serverless-applications-252fd8623bce
- 50. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. The application monolith (now refactored) https://github.com/mreferre/yelb/tree/master/yelb-appserver Sinatra “choreography” (EC2/Docker) Lambda adapters True “business logic”
- 51. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. The application monolith (now refactored) https://github.com/mreferre/yelb/tree/master/yelb-appserver
- 52. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. The application monolith (now refactored) https://github.com/mreferre/yelb/tree/master/yelb-appserver
- 53. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. DB update function prior to the re-factor
- 54. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. DB update function after the re-factor
- 55. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. RDBMS to DDB may require more thinking (in reality) https://threadreaderapp.com/thread/1137002244157710336.html
- 56. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. The serverless setup https://github.com/mreferre/yelb/tree/master/deployments/platformdeployment/AWS/Serverless
- 57. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Conclusions and call to action(s)
- 58. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Useful links: • Demo App: https://github.com/mreferre/yelb • ECS workshop: https://ecsworkshop.com/ • EKS workshop: https://eksworkshop.com/ • Serverless Workshop: https://github.com/aws-samples/aws-serverless- workshops
- 59. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Who Am I ? Email: mreferre@amazon.com Twitter: @mreferre Github: github.com/mreferre