AWS DevDay Cologne - Resiliency and availability design patterns for the cloud
Download as PPTX, PDF0 likes70 views
The document outlines the importance of resiliency and availability in cloud software systems, highlighting key design patterns and best practices for building resilient architectures. It discusses the cost of downtime and techniques such as multi-AZ architecture, auto-scaling, and effective health checks to enhance system availability. Additionally, it emphasizes the use of AWS services like DynamoDB and Aurora for managing data resilience and performance at scale.
AWS DevDay Cologne - Resiliency and availability design patterns for the cloud
- 1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. V I E N N A 24.10.19 Resiliency and availability design patterns for the cloud Cobus Bernard Senior Technical Evangelist Amazon Web Services @cobusbernard cobusbernard cobusbernard B A R 3
- 2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Can you guess whatwillhappen?
- 3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Distributed Systems are hard
- 4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Failures areagivenand everythingwilleventuallyfail over time. Werner Vogels CTO – Amazon.com “ “
- 5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Resiliency:Abilityfor asystemtohandle and eventuallyrecover from unexpected conditions conditions
- 6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Partialfailure mode
- 7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Why do we build resilient software systems?
- 8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thecostof downtime Annual Fortune 1000 application downtime costs (IDC) $1.25 to $2.5B Average cost of a data breach (Ponemon Institute) $3.6M Cost/hr of a critical application failure (IDC) $500K to $1M Average cost/hr of downtime (Ponemon Institute) $474K Average cost per lost or stolen record (Ponemon Institute) $141
- 9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. How do we build resilient software systems?
- 10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. People Application Network & Data Infrastructure
- 11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Let’s talk aboutAvailability
- 12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Systemavailability Availability = Normal Operation Time Total Time MTBF** MTBF** + MTTR* = * Mean Time To Repair (MTTR) **Mean Time Between Failure (MTBF)
- 13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Reading homework
- 14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Availabilityinparallel A = 1 – (1 – Ax)2 Part X Part X
- 15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Availabilityinparallel Component Availability Downtime X 99% (2-nines) 3 days 15 hours Two X in parallel 99.99% (4-nines) 52 minutes Three X in parallel 99.9999% (6-nines) 31 seconds
- 16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Componentredundancyincreases availability significantly!
- 17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Global Infrastructure • 22 Regions with 69 Availability Zones • 3 Regions coming soon: Cape Town Jakarta and Milan • 100Gbps redundant network • 99.99% SLA
- 18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Fully-scaledAvailabilityZone
- 19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Highlyredundant regional network
- 20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Region and availabilityzones Region Availability zone a Availability zone b Availability zone c data center data center data center 1 or more data centers per AZ 2 or more AZs per region (new regions min 3) data center data center data center data center data center data center
- 21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. How about a global architecture?
- 22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Once upon a time … Origin
- 23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. And Now … Origin ~300ms
- 24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Origin Improve latency for end-users Origin
- 25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Improve availabilityand disasterrecovery Applications in US West Applications in US East Users from San Francisco Users from New York Service 1 Service 2 Service 3 Service 4 Service 1 Service 2 Service 3 Service 4
- 26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. So should we go for a global architecture?
- 27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Perfect your regional architecture first!
- 28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Let’s talk about Multi-AZ
- 29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Multi-AZ architecture Region Availability zone a Availability zone b Availability zone c Instances Instances Instances DB Instance DB instance standby Elastic Load Balancing (ELB)
- 30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Multi-AZ architecture Region Availability zone a Availability zone b Availability zone c Instances Instances Instances DB Instance DB instance standby Elastic Load Balancing (ELB)
- 31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Multi-AZ architecture Region Availability zone a Availability zone b Availability zone c Instances Instances Instances DB Instance DB instance standby Elastic Load Balancing (ELB)
- 32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Multi-AZ architecture Region Availability zone a Availability zone b Availability zone c Instances Instances Instances DB Instance DB instance new master Elastic Load Balancing (ELB)
- 33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Multi-AZ architecture • Enables fault-tolerant applications • AWS regional services designed to withstand AZ failures • Leveraged by AWS regional services such as Amazon S3, Amazon DynamoDB, Amazon Aurora, Amazon ELBs, etc. Region Availability zone a Availability zone b Availability zone c Instances Instances Instances DB Instance DB instance standby Elastic Load Balancing (ELB)
- 34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Let’s talk about auto scaling
- 35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Auto-Scaling FixedVariable
- 36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Availability zone 1 Auto Scaling group AWS Region Availability zone 2 Auto-scaling for self-healing Elastic Load Balancing (ELB) X
- 37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Let’s talk about the AWS responsibility models
- 38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWSoperationalresponsibilitymodels On-Premises Cloud Less More Compute Virtual Machine EC2 Elastic Beanstalk AWS LambdaFargate Databases MySQL MySQL on EC2 RDS MySQL RDS Aurora Aurora Serverless DynamoDB Storage Storage S3 Messaging ESBs Amazon MQ Kinesis SQS / SNS Analytics Hadoop Hadoop on EC2 EMR Elasticsearch Service Athena Firehose
- 39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Let’s talk about databases
- 40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Common resiliencyissueswithDatabases?? REPLICATION BACKUPSSCALING
- 41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
- 42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon DynamoDB: purpose-built for thecloud Fast and flexible key value database service for any scale Comprehensive security Encrypts all data by default and fully integrates with AWS Identity and Access Management for robust security Performance at scale Consistent, single-digit millisecond response times at any scale; build applications with virtually unlimited throughput Global database for global users and apps Build global applications with fast access to local data by easily replicating tables across multiple AWS Regions Serverless No hardware provisioning, software patching, or upgrades; scales up or down automatically; continuously backs up your data
- 43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR AZ 1 AZ 2 AZ 3 Network RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR Storage Node Leader PutItem
- 44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR AZ 1 AZ 2 AZ 3 RR RR RR RR RR RR RR RR RR RR RR RR RR RR RR Storage Node Leader GetItem Network
- 45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon.com, Nike, Netflix, Duolingo, Lyft, Airbnb, Samsung, Toyota, and Capital One depend on the scale and performance of DynamoDB to support their workloads. 10 trillion requests per day 20 million requests per second
- 46. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AmazonAurora: purpose-built for thecloud MySQL and PostgreSQL compatible relational database Availability and durability Fault-tolerant, self-healing storage; six copies of data across three AZs; continuous backup to S3 Fully managed Managed by RDS: no hardware provisioning, software patching, setup, configuration, or backups Highly secure Network isolation, encryption at rest/transit Performance and scalability 5x throughput of standard MySQL and 3x of standard PostgreSQL; scale-out up to 15 read replicas
- 47. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. BINLOG DATA DOUBLE-WRITELOG FRM FILES MYSQL WITH REPLICA EBS mirrorEBS mirror AZ 1 AZ 2 Amazon S3 EBS Amazon Elastic Block Store (EBS) Primary Instance Replica Instance 1 2 3 4 5 AZ 1 AZ 3 Primary Instance AZ 2 Replica Instance Replica Instance AMAZON AURORA TYPE OF WRITE AmazonAurora: distributed architecture
- 48. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Capital One, GE, Dow Jones, Netflix, Verizon, Expedia and Amazon depend on the scale and performance of Aurora to support their workloads. 5X the throughput of standard MySQL 15 low latency read replicas Failover in less than 30 seconds 3X the throughput of standard PostgreSQL
- 49. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Purpose-built databases Relational Key-value Document In-memory Graph Time-series Ledger DynamoDB NeptuneAmazon RDS Aurora CommercialCommunity Timestream QLDBElastiCacheDocumentDB
- 50. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Read-Writeseparation Master Read Replica Read Replica Read Replica Instance InstanceInstance
- 51. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. DatabaseFederation Users DB Products DB Master (Read) Replica Master (Read) Replica Instance InstanceInstance
- 52. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. DatabaseSharding User ShardID 002345 A 002346 B 002347 C 002348 B 002349 A CBA Master (Read) Replica Master (Read) Replica Master (Read) Replica Instance InstanceInstance
- 53. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Let’s talk about backups
- 54. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
- 55. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Backup service
- 56. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. PreventingAccidentalTableDeletion https://aws.amazon.com/blogs/database/preventing-accidental-table-deletion-in-dynamodb/
- 57. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. PreventingAccidentalTableDeletion (sql)
- 58. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Practice and testrecoveryfrom your backups!!
- 59. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Let’s talk about timeouts, backoff & retries!
- 60. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Users App DB Conn Pool INSERT INSERT INSERT INSERT What happens if the DB “slows down”? Timeout client side Timeout backend side ??
- 61. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. User 1 App DB Conn Pool INSERT Timeout client side = 10s Timeout backend side = default = Infinite Retry INSERT Retry INSERT ERROR: Failed to get connection from pool Retry
- 62. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. https://docs.microsoft.com/en-us/dotnet/api/system.net.httpwebrequest.timeout
- 63. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
- 64. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-configuration-properties.html
- 65. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. @timeout_decorator.timeout(5, timeout_exception=StopIteration) def timed_get(url): return requests.get(url) https://pypi.org/project/timeout-decorator/
- 66. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Set thetimeouts!
- 67. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. How elsecould wehaveprevented theerror? User 1 DB Conn Pool INSERT Retry INSERT Retry INSERT Retry ERROR: Failed to get connection from pool
- 68. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. User 1 DB Conn Pool INSERT Timeout client side = 10s Timeout backend side = 10s Wait 2s before Retry INSERT INSERT Wait 4s before Retry Wait 8s before Retry Wait 16s before Retry Backing off betweenretries Releasing connectionsBackoff
- 69. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. No jitter With jitter https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/ SimpleExponentialBackoffisnotenough:AddJitter
- 70. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Example: add jitter0-1000ms def get_item(self, url, n=1): MAX_TRIES = 12 try: res = requests.get(url) except: if n > MAX_TRIES: return None n += 1 time.sleep((2 ** n) + (random.randint(0, 1000) / 1000.0)) return self.get_item(url, n) else: return res
- 71. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Idempotent operation No additional effect if it is called more than once with the same input parameters.
- 72. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Let’s talk about health checking!
- 73. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Auto Scaling group Service A Availability zone 1 Auto Scaling group AWS Region Service A Availability zone 2 Service BService B database Email Probing for health Cluster
- 74. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Shallowhealthcheck Instance Cache node Email database Cluster Are you healthy? yes
- 75. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Shallowhealthcheck Instance Cache node Email database Cluster Are you healthy? yes
- 76. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Deep healthcheck Instance Cache node Email database Cluster Are you healthy? yes Are you healthy? yes yes yes yes
- 77. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Deep healthcheck Instance Cache node Email database Cluster Are you healthy? no Are you healthy? no yes yes yes
- 78. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Prioritize shallow health checks during hard times. Cache.
- 79. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Let’s talk about load shedding.
- 80. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
- 81. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
- 82. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Cheaply reject excess work
- 83. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
- 84. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Be careful when selecting the right metric
- 85. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Don’tbeoverlyoptimisticandtakeonmorethanyoucan. Findanoperationalmetrictorejectwhatyoucannottakein. Favorcachedandstaticcontent PrioritizeELBhealthcheck(shallow)pings Inanoverloadsituationyouhavepreciousresources,donotletany ofitgotowaste. Load Shedding
- 86. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Let’s talk aboutresiliency (chaos) engineering
- 87. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Fire Drills
- 88. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. GameDay atAmazon CreatingResiliencyThroughDestruction https://www.youtube.com/watch?v=zoz0ZjfrQ9s
- 89. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Chaosengineering https://github.com/Netflix/SimianArmy
- 90. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. “Chaos Engineeringis the discipline of experimentingon a distributedsystem in orderto buildconfidence in the system’s capabilitytowithstand turbulentconditionsin production.” http://principlesofchaos.org
- 91. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Failureinjection • Start small & build confidence • Application level • Host failure • Resource attacks (CPU, memory, …) • Network attacks (dependencies, latency, …) • Region attacks • “Paul” attack https://www.gremlin.comhttps://github.com/Netflix/SimianArmy https://chaostoolkit.org
- 92. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. STEADY STATE HYPOTHESIS RUN EXPERIMENT VERIFY FIX! PhasesofChaosEngineering
- 93. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. https://aws.amazon.com/wellarchitected
- 94. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thank you! © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. @cobusbernard cobusbernard cobusbernard