Cost Optimization - aws and cloud compute

Editor's Notes

• #2: Many customers are aware of how to be reactive when it comes to cutting down waste, however in order to sustain a cost optimized environment its important to consider proactive architectures This is where services such a service catalog come into play The goal of this presentation is to equip you with a basis of understanding around the implementation of service catalog Target audience – members of a CCOE
• #3: There are 3 different ways to purchase compute On-Demand: Pay-as-you-go, no commitments, best for fluctuating workloads Reserved Instance: Long term commitments that offer big savings over On-Demand prices. Best for always on workloads Savings Plan: Just like Reserved Instances, but monetary commitment based and compute can be used across Fargate and EC2 Spot Instances: Same as pay-as-you-go pricing as On-Demand, but at up to 90% off. EC2 can reclaim with a 2 minute warning. Best for stateless or fault tolerant workloads All four purchasing options use the same underlying EC2 instances and AWS infrastructure across 22 Regions [Poll] How many of you use Spot Instances?
• #5: SAY: Here are a few success stories where customers were able to optimize costs by using spot: Lyft – primarily uses Spot to reduce their computer costs. uses Spot to support their CI/CD pipeline and save up to 90% monthly Yelp – uses Spot for another key benefit – scale. With cheaper instances, they are able to throw more compute power at their testing requirements, reducing response time from two days to 30 minutes. Illumina takes the best of both worlds. All 3 of these customers are public references for us and their stories can be found on our website.
• #6: Explain How Spot placement score works References https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-placement-score.html
• #7: First let me highlight that more than 95% of the time a spot instance is terminated by the client or the job completes Meaning our client has requested a spot instance, we fulfilled it, it has completed its work and is no longer needed so the client terminates it Remember each spot pool has it’s own unique capacity and characteristics, but across our portfolio 95% of the time clients terminate the spot instance We also offer a tool called Spot Instance Advisor that provides a 30 day rolling average of interruption rates by instance per region which is helpful when configuring your requests. That said, Spot by nature is an interruptible service, so regardless of the frequency, you need to be able to handle interruptions if/when they occur. Logistically, the first thing that will happen is AWS will notify you we need to reclaim the instance This is done thru either a cloud watch alert or by pulling the metadata of the local instance When that warning is received, you’ll have 2 minutes to gracefully terminate the instance and migrate that work to elsewhere in your fleet We have different strategies that you can employ to handle interruptions and our team is here to help you make the right choices, but they essentially come down to 3 options” Option one is the most common and that is terminate the instance. In this case you want to check point and bring the instance up elsewhere in the fleet. WE can provide sample scripts to implement some LAMBDA script to accomplish this Another option is Stop/Start – this is often used for time-insensitive workloads and we can persist the EBS volume and then re-attach once the instance become available again. The third option is Hibernate. Again implemented much less often, but it is available. In this case we flush in-state memory to disk and then it functions much like opening and closing the lid of your laptop HIGHLIGHT: So depending on your workload and the requirements, we offer a variety of ways to elegantly manage an interruption should it occur. MENTION: It is not always possible for Amazon EC2 to send the rebalance recommendation signal before the two-minute Spot Instance interruption notice. Therefore, the rebalance recommendation signal can arrive along with the two-minute interruption notice. Rebalance recommendations are made available as a CloudWatch event and as an item in the instance metadata on the Spot Instance. Events are emitted on a best effort basis. MENTION: You should use at least 10 different instance types to minimize frequent rebalancing notifications. References https://aws.amazon.com/about-aws/whats-new/2020/11/introducing-ec2-instance-rebalance-recommendation-for-ec2-spot-instances/ https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/rebalance-recommendations.html
• #8: Being instance flexible is the most important element to a success ASG With instance weights you can specific the weight that and instance type would contribute to your application’s performance. For example, if you are flexible across different instance sizes within the C5 family, you can assign a weight of one to c5large, a weight of 2 to c5xlarge, or a weight of 4 to c5xxlarge, etc. For example, if your desired capacity is 100, it can either launch 100 c5larges or 50 c5xlarges or a combination of such Instance flexibility: (Time sensitive workloads) Mix instance types with similar capabilityes: num of vCPUs / Memory Time flexible: (time insensitive workloads) Workloads that require specific instance types, but can be flexible on completion times (e.g. batch Jobs with no SLA, ML training Jobs…) Region flexible: large size / very instance specific kind of workloads e.g. real time rendering with a specific g3 instance, can benefit of increased
• #10: 1/ As your cloud environments and usage scale, optimizing your costs and correctly scaling become essential. 2/ You can use Savings Plan with On-Demand and Spot Instances to mix our pricing models to best match your utilization and optimize your performance and costs. 3/ EC2 Fleet and Auto-Scaling allow you to scale across all these purchase models - Spot, On-Demand, RIs and Savings Plans.
• #11: SAY: Now that we’ve covered the key workloads where we see the widest adoption of Spot, let’s talk about how to launch Spot instances. Spot instances can be launched directly in the EC2 console under spot instances, but AWS provides tools called Spot Fleet and EC2 Fleet to help manage your spot workloads without additional management code. With Spot fleet you can automatically provision and scale instances across a variety of instance families to meet your needs. In addition, on-demand instances can be mixed in the instance types if you want a number of instances that are not running on spot and therefore will not have a probability of being interrupted. NEW FEATURE 15 Nov. 2022: Price capacity optimized https://aws.amazon.com/blogs/compute/introducing-price-capacity-optimized-allocation-strategy-for-ec2-spot-instances/
• #12: Substituted AWS Thinkbox Deadline with AWS Gamelift . Also mention: AWS Thinkbox Deadline References: https://docs.aws.amazon.com/gamelift/latest/developerguide/spot-tasks.html https://aws.amazon.com/about-aws/whats-new/2018/04/more-control-of-idle-spot-fleet-instances-with-thinkbox-deadline/
• #13: The most common use cases where we see successful Spot implementations are: Anything containerized – Big data frameworks like Apache Spark or Hadoop Batch processing Stateless web services Machine learning – Pytorch, Tensorflow or jobs that require heavy training Continuous Integration and Continuous Deployment (CI/CD) with Jenkins High performance computing (HPC) – genomics sequencing Anything fault-tolerant or stateless that can be instance flexible
• #14: SAY: Because many workloads running on containers already are stateless and fault-tolerant by design, they are great candidates for spot. ECS supports native integration with spot which can be used to spin up additional cluster nodes as needed and migrate running containers to a newly created ECS instance when an interruption notification is delivered Skyscanner is a one customer that was able to save 74% on their workloads by moving their ECS clusters to spot.
• #15: SAY: With discounts of up to 90% off, you can scale your compute environment by up to 10x while staying within budget. Integrate directly with EMR or Elastic MapReduce and can support any Hadoop or Spark clusters to process large amounts of data. Mobileeye was able to take advantage of this by parallelizing their tasks were able to do a job that took weeks in their datacenter, in hours running on spot at a much more cost efficient price.
• #16: SAY: CI/ CD is a growing use case with customers Spot provides inexpensive compute environments for developers to build, test, and deploy code changes There is also a Jenkins plug-in that spins up spot instances during automated build and testing Lyft is an interesting example as they were able to save 75% a month by just changing four lines of code that launched instances in their salt module.
• #17: SAY: Web Services is another common use case where we see a lot of usage of Spot instances as customers look for inexpensive solutions to maintain high performance during peak traffic periods. A very common architecture here is for customers to run a baseline of RIs for their typical traffic patterns and scale into Spot instances when traffic spikes. And as we’ll talk about shortly, EC2 Fleet and autoscaling groups can now support RIs, On-Demand and Spot in a single call or autoscaling group making this much easier to manage.
• #18: SAY: HPC or high performance computing workloads, like big data, can benefit from the scale that Spot instances can provide for workloads such as genomic sequencing, computational fluid dynamics and risk analytics. With Spot you can run multiple projects simultaneously and reduce the time to answers. A few of our key customers in this space include Illumina, who we discussed earlier, along with NASA, Autodesk and AdRoll. Integrated with AWS Batch, AWS CloudFormation and other AWS services
• #19: SAY: Spot is used by many leading technology companies in the world across a variety of industry verticals.
• #20: SAY: Spot is used by many leading technology companies in the world across a variety of industry verticals.