Serverless Architectures on AWS in practice - OSCON 2018
0 likes563 views
The document discusses serverless architectures on AWS, emphasizing the benefits of microservices, event-driven programming, and cost efficiency through Function as a Service (FaaS). It outlines various use cases, limitations, and best practices for implementing serverless solutions, including monitoring, logging, and avoiding common pitfalls. The author advocates for careful tool selection and understanding the implications of architectural choices in cloud environments.
Serverless Architectures on AWS in practice - OSCON 2018
- 1. Serverless Architectures on AWS in practice Manish Pandit 07/19/2018
- 6. If I had asked people what they wanted, they would have said: “faster horses” -Henry Ford
- 7. The Journey
- 8. About Me Manish Pandit Director of Platform Engineering @marqeta @lobster1234 lobster1234.github.io
- 12. Why? Scale technology to support the growing business Use public cloud capabilities so we can focus on product innovation
- 13. Break down the monolith
- 16. Microservices Independent development, deployment, and scaling of functionally isolated services.
- 17. Microservices “Loosely coupled, yet highly aligned”
- 18. Microservices You can have microservices without containers and serverless! Containers and Serverless are the most optimal way to package and deploy microservices.
- 20. Containers Distributed Applications/Services Economics of Scale - Optimal Resource Utilization Being lightweight, they’re fast to deploy and run
- 21. Serverless Function as a unit of deployment (FaaS) Pay for what you use, never for idle resources 100% abstraction from underlying resources
- 22. Serverless Components 1. Functions themselves 2. Events that trigger these functions, or handlers Event driven programming - Functions are event handlers You only pay when the handler runs
- 23. Lambda Amazon’s FaaS Supports Node, Python, Java 8, C#, Go Billed as GB-seconds
- 24. Lambda Pricing First 1M requests per month are free 400,000 GB-seconds per month are free START RequestId: 6bd9cf60-7e2b-11e8-9f4a-ebdea81e794d Version: $LATEST END RequestId: 6bd9cf60-7e2b-11e8-9f4a-ebdea81e794d REPORT RequestId: 6bd9cf60-7e2b-11e8-9f4a-ebdea81e794d Duration: 646.26 ms Billed Duration: 700 ms Memory Size: 128 MB Max Memory Used: 31 MB
- 25. Use Cases
- 27. Database Replication Jobs Get updates PutObject trigger apply
- 28. Async Events publish publish trigger trigger Service A Service B Service C ECS
- 29. File Processing Pipeline Split PGP & Transfer Payment Networks Banks trigger trigger N 1 Object + Metadata
- 30. Lambda limits Memory : 128-3008 MB with 64 MB increments Time : 300 seconds
- 31. Batch Jobs
- 32. AWS Batch : Managed Compute https://.. Submitter Job Job Job Batch Cloudwatch Event
- 33. Containers not Serverless Enough? = ECS minus the Cluster
- 34. Logging For Lambdas, Cloudwatch logs For ECS, filebeat agent on the host For Fargate, LogStashTcpSocketAppender
- 36. Testing Docker Compose Localstack Use environment variables for configuration (12-Factor App)
- 37. Gotchas.. - Lambda Cold start on both Python and Java - Due to VPC and ENI - Be aware of the retries - Async invocation gets two retries - Sync invocation gets no retries - Apply the Principle of Least Privilege to the Lambda and ECS Roles - S3 triggers - source and destination buckets
- 39. Tips - Declare connection pools outside of the handler’s scope - Write tests! - Test locally - the only error you should be expecting is IAM Permissions error. <role> is not authorized to perform: <operation> on resource: <resource> - Do not create Microliths - Set up billing alerts
- 40. Tips - Pick the best tool for the job - Do not pick the tool and try to fit the job into it - Beware of the shiny object syndrome - Take measured risks - Production is (mostly) unforgiving
- 42. “You’re free to choose, but you’re not free from the consequences of your choice” - Universal Paradox
- 43. Fin Manish Pandit Director of Platform Engineering @marqeta @lobster1234 lobster1234.github.io