Serverless Architectures and Continuous Delivery

Editor's Notes

• #2: Robin Weston. Work as a developer for ThoughtWorks, software consultancy Although I’m a dirty consultant. No ties to Amazon or any other technology. I’m not trying to sell you anything - Show of hands. Who has heard of or read about Serverless Architecture or Serverless before? Who has used them in anger?
• #3: - Try not to spend too much time as there’s lots on the internet. Want to focus on learnings - Had to cut a fair bit out of this talk to fit it into the time, so it’s going to be a joyous romp through the serverless world. Please come and talk to me afterwards if you want to chat about specific areas in more detail
• #4: Confusing name. We’ve somehow worked out how to execute code without using physical machines?! But it’s stuck. Marketing folks have got their sticky fingers on it. And you can probably get certifications in Serverless now too. Let’s look at a definition.
• #5: “Not use the usual server process”. Software doesn’t run on a server that you have access to. You don’t own the servers. They are abstracted away and managed for you. Can’t log into them. Many benefits to not managing servers. No servers randomly rebooting or going down. No snowflake servers. Not responsible for installing software on them. Even if using infra as code that’s extra code to maintain. “Third party services”. cloud accessible databases (like AWS hosted Elasticsearch cluster, Google Firebase), authentication services (Auth0, AWS Cognito). Outsourcing that capability Client-side logic, typically in ‘rich client’ applications like single page web apps, or mobile apps Functions as a Service - The term Serverless has also confusingly come to mean 2 things The definition above Functions as a service Mostly I’m going to talk about the second of these areas (FaaS) because it is the one that is newer, has significant differences to how we typically think about technical architecture, and has been driving a lot of the hype around Serverless.
• #6: Oversimplified diagram In top half, traditional server-based architecture. All our logic, authentication, and databases run on servers we are responsible for (in Cloud or on prem). Client calls the server In bottom diagram, serverless architecture, no server process. Logic split between client and multiple Functions as a Service. Authentication and database hosted and managed for us by 3rd parties. FaaS are the glue connecting services together
• #7: Function. A small block of code that does a specific job. Explicit interface. Input -> Output Stateless. Nothing is saved between separate runs of the function. Couldn’t write something to the disk and expect it to still be there Sandboxed. Similar approach to containers, except you don’t control the container Ephemeral Short lived Event-triggered Examples coming shortly Scalable by default Takes advantage of stateless nature to automatically run as many as are needed Fully managed by 3rd party Google cloud functions
• #8: Next evolution of cloud hosted software. To the left of green line, something you are responsible for managing. On the right hand side, delegated responsibility to cloud provider EC2 (deploy application to servers you manage), PaaS like Heroku or Azure websites (deploy application, servers abstracted away). Or containerization FaaS (deploy function, application and servers abstracted away). Evolution doesn’t necessarily mean better, just moving the slider further over to smaller deployable units and less control over underlying infrastructure
• #9: Don’t worry. Only 2 code samples. Again, loads on the internet for more reading AWS Lambda. Lambda function is an anonymous function not tied to an identifier Node Js example Simple function to add 2 numbers together Export single function - that’s your only interface. Function takes event, which is input. Event will differ depending on what triggered this function. Context is what you call when you’re done. Returns result to triggering system Go through code lines No surrounding boilerplate code. Pure business logic This is simple example, but you could have multiple files + include 3rd party package dependencies Underlying operating system is Amazon Linux Can also be written in Java, Python and now .NET. Similar interfaces. Or anything you can run from NodeJs on linux e.g. shell scripts, executables such as golang
• #10: Example event sources. Change in a Dynamo DB database table SNS notification e.g. from text message Change in an S3 bucket HTTP Request through API Gateway Schedule event (CRON job) Many more exist Lambda functions will scale to service the level of incoming events
• #11: Client in events industry (team in background). Build a standalone analytics and performance service with visible dashboard We built ran the system for 6 months, so can talk about the operational concerns too as well as the development side Deployments on average every 2-3 days
• #12: Scalable by default. Awesome! As many lambda functions as needed are created Main difference from PaaS. Like Heroku of Azure Web App. No slider Scaling is instant. No need for CPU or memory trigger, Auto Scaling groups + waiting for EC2 instances to spin up (or down). Or start new container instances With API gateway it’s very quick to get a scalable public API up and running HOWEVER, There are hard limits. Throttling. 100 concurrent executions. AWS says safety limit for costs. Manually apply for higher limit
• #13: Only pay for function throughput. Memory x runtime. Down to nearest 100ms Compare to normal application, you pay for uptime, even if it's not being used. For traditional resilient server based application on cloud you’d have 3 availability zones, > 1 instance per zone. 6 instances minimum AWS Lambda example costs. 30 million requests that use 128mb memory and run for 200ms. £10 (tenner) Free tier. 1 million free requests per month. Great for scheduled jobs, APIs with surging traffic profiles Not necessarily cheaper (although was for our client). Seen examples of well tuned applications that are cheaper Also, remember other AWS fees e.g. data transfers, use of other services e.g. S3
• #14: Local unit testing is great. Raw javascript or .NET. Simple entry and exit point We found that high level tests at the lambda entry point level were great. Generally no need to write lower level unit tests. If your tests get too complicated, then your lambda function is doing too much HTTP Functional tests much harder. Can’t run locally. Need to deploy then run in local developer only branch Solutions do exist (Claudia or AWS lambda docker, later on), but these aren’t replicating your production environments
• #15: No servers missing security patches. Automatically patched Smaller attack surface area (if you can’t log in, how can your attackers?) . No usernames or passwords, SSH keys etc. No users to escalate privilege Server lives for milliseconds so attack window very small No human sysadmins No faffing around with firewalls, ports etc File system is read only so many attack vectors unavailable Explain AWS Lambda IAM (Identity and Access Management) policy – only grant access things you need to from Lambda function. Most of security flaws around AWS Lambda are around misconfigured IAM rules with too much permission. Still have to manage some credentials e.g. AWS access keys for deployments. Store these securely.
• #16: How easy is it to see how your system is behaving, diagnose issues etc Out of the box observability is good, but you’ll need to customise and enhance
• #17: Logging Console.log or Console.WriteLine straight to Cloudwatch. Awesome. No physical log files. No need for logstash or other process running to move logs from A to B Not good enough though. Searching hard. No aggregation. No graphing of log data e.g. ERROR log events Elasticsearch + Kibana. ELK stack (or in this case EK) Aggregated logs (using another lambda + AWS Elasticsearch)
• #18: Monitoring. Out of the box! Describe graphs Easy to set up alerts based in these metrics. Perform action if metric hits threshold. Default graphs fine, but you’ll want more. In detailed graphs (e.g. specific HTTP status codes) or Business focussed metrics.
• #19: Deployments are easy, your build artefact is just upload a zip file of your code for JS. NuGet package for .NET Deployments are very quick (depending on size of deployment package), and are zero-downtime by default as running lambdas aren’t affected by those mid-deployment We found that batching all cloud services for a given domain service into single artefact and deploying that worked well. Artefact was a zip file containing API Gateway endpoints, lambda functions, alerts config. Kept them together in their own repo, built artefact once, and deployed same artefact through environments. Config per environment using JSON files or environment vars Amazon recently released SAM (Serverless Application Model). Define your application made up of API Gateway, AWS Lambda, Dynamo DB using JSON description format for easy creation and deployment AWS Lambda has function versioning and environments as first class concerns. Can specify lambda version when calling a function, allowing easy rollback to older versions. Also, can promote functions through environments with a single API call
• #20: Won’t go into too much detail. Provide some opinionated tooling around API Gateway and AWS Lambda. All command line driven. Amongst other things, provide help with: Deployment Testing (local and functional) Configuration Easy shims for other languages We actually rolled our own by calling AWS APIs directly as frameworks were immature and moving quickly. Wanted to reduce magic, keep things Good place to start for getting something up and running
• #22: Small deployable units. Forces you to think small from the outset. Simple deployment model. Whether you use one of Frameworks, AWS APIs, new SAM capability. Creating a deployment artefact and deploy through a pipeline is simple, quick, easy to automate No more “works on my machine”. Talked about not being able to test components locally as they only run in cloud. Can view this as a good thing, as it forces you to push your testing closer to production. Forces you to have conversations around monitoring, synthetic transactions etc. Also, continually exercising deployments. Enables focus on business value. Serverless architectures allow you to focus on core business value, and offload things which aren’t your core competency to Cloud Provider. You get a lot for free (like security, scalability etc), and even if it isn’t perfect, it gets you a lot closer than more traditional approaches. Our goal is not to continuously deliver code to production, but value. Ideally using as little code as possible Great, your company built a dynamically scalable and resilient log shipping and storage solution to service your applications, using containers and container orchestration, infrastructure as code and deployment pipelines. I just used a single Console.Log statement. Think how much it cost in money and time to build that system, and think of the ongoing maintenance cost. Think about how much competitive advantage my company gained by making use of our extra time and money
• #23: So you want to get started with a Serverless Architecture? What follows are a few pieces of advice from our team that we took away as learnings for future projects
• #24: Don’t go all in and replace your mission critical systems! Start by nibbling around the edges of your system
• #25: - Scheduled jobs e.g. archiving data Log streaming Webhook processing Chatbot Proof of Concepts - Run in production for a while, then evaluate
• #26: AWS Lambda imposes limits to guide you into the pit of success. AWS Lambda can provide you its benefits (e.g. scalable by default) if you play with its rules e.g. Functions should be quick and handle small amounts of data Example limits Timeouts. 5 minutes - Add queueing Data sizes. 6mb request and response size. 50mb deployment package Even thought it looks tempting to leap in and replace everything at once. Lower yourself in slowly so you don’t hit unexpected problems
• #27: Know your traffic profile. Serverless functions designed for small amounts of data, processed quickly. Don’t hit limits Don’t lift and shift your applications. You probably could take your existing application and stick it behind a single Lambda function, but you’ll pay a performance penalty in startup time every time that function is called. Decompose your systems into small functional units. Embrace serverless architectural concepts– this talk is not about that. AWS Lambda currently has low compliance regarding standards like PCI or ISO, as it is still shared infrastructure underneath. Might mean it’s not suitable for financial transactions etc, or you’d need to perform some data masking before processing it in Lambda Monitor and logging in from the start. Need to know how your system is performing and be able to diagnose problems easily Safely replace existing systems. Deploy in parallel and route traffic to both. Dark launch. Have high level functional tests
• #28: Lots of anger around Servlerless has come from NoOps. It scales and monitors itself. No need for your operations team NEXT SLIDE It isn’t NoOps. In fact, you’ll need operations skills and an operations mindset in your team more than ever Sure, you have no server maintenance + no auto-scaling configurations. But they’re just a small subset of modern day operations.
• #29: Probably preaching to the choir here, but try and make your team cross functional and has the skills required to operate your system. Need to set up and iterate on all the areas specified. Log aggregation Monitoring Alerts AWS infra as code (AWS APIs, Cloud Formation, Terraform) for services (including AWS Lambda itself). Plus all the above
• #30: Serverless architectures Tales from a world without servers: Enabling Continuous Delivery Severless starter tips One thing for sure. Serverless Architectures and FaaS are not going away. The cloud service offerings will only grow. As will the surrounding ecosystem. I now ask myself “is there a reason we CAN’T use a Serverless Architecture here?” So if you take advantage of these in a considered and well-executed manner, Serverless Architectures can be a game-changer.
• #31: Expand on definition - on Martin Fowler’s site by Badri Janakiraman. As a TW Employee I am contractually obliged to mentioned Martin Fowler at least once
• #32: In depth article by Mike Roberts - also on Martin Fowler’s site
• #33: - Simon Wardley – Why the fuss about serverless? I’ve tried to show both sides of the Serverless argument. Really opinionated piece that got a lot of attention Quotes “Serverless will fundamentally change how we build business around technology and how you code” “Containers — they are important but ultimately invisible subsystems and this is not where you should be focused.” “You thought devops was big but it’s chicken feed compared to this. This is where the action will be, it’ll be with you quicker than you realise and yes, you’ll have inertia. Now is not the time for building a DevOps team and heading towards IaaS, you’ve missed that boat. You should be catching this wave as fast as you can.” It’s basically the opinions I’m not quite brave enough to say yet
• #34: - Please come and find me afterwards if you want to chat some more Would love to hear your Serverless Architecture stories.