Scalable and Available, Patterns for Success
83 likes20,639 views
The document discusses patterns for building scalable and available systems. It covers key topics like: - Understanding performance vs scalability and the tradeoffs between them - Using patterns like event-driven architectures, load balancing, and parallel computing to improve scalability - Managing state through techniques like replication, sharding, caching and NoSQL databases - Ensuring high availability even during failures through failover and an eventual consistency model The document emphasizes measuring systems, understanding user needs, and mastering the tradeoffs between performance, scalability, availability and consistency.
Scalable and Available, Patterns for Success
- 1. Scalable & Available Patterns for Success
- 2. Derek Collison @derekcollison dcollison@vmware.com derek.collison@gmail.com
- 3. Special Thanks Jonas Bonér twitter: @jboner http://jonasboner.com/ http://www.slideshare.net/jboner/scalability- availability-stability-patterns
- 4. Background • Scalable Apps maintain performance under load • More requests, More users, More data • Available Apps maintain the experience during failures • Hardware failures, Network splits/partitioning • Simple Designs tend to scale better
- 9. Background Measure everything (can’t fix what you don’t know)
- 10. Background Don’t be a failure of your own success
- 11. Background Master the • Good Performance is good • Predictably Good Performance is king! Tradeoffs • Measure everything (can’t fix what you don’t know) • Understand app and your data!) (For your your data • Understand your user experience • Don’t be a failure of your own success
- 12. Master the Tradeoffs Performance vs Scalability
- 13. Master the Tradeoffs Latency vs Throughput
- 14. Master the Tradeoffs Availability vs Consistency
- 15. Lots of ways to skin a cat!
- 17. Performance vs Scalability
- 18. How do I know if I have a performance problem? If your system is slow for a single request/user
- 19. How do I know if I have a scalability problem? If your system is fast for a single request/user but slow for many users
- 20. Latency vs Throughput
- 21. You should strive for maximal throughput with acceptable latency
- 22. Performance vs Scalability Response Time Concurrent Requests
- 23. Know what to scale! • CPU or IO Bound? • Scale up or Scale out? • Waiting on IO? What? Disk/Net/Other System? • How many components are used per request? • Know who and what the slowest will be!
- 24. Scalability Patterns Behavior
- 25. Scalability Patterns: Behavior ✓Event-Driven Architectures ✓Load-Balancing ✓Parallel Computing
- 28. Messaging - Publish Subscribe 1:N Subscriber Publisher Subject Subscriber Subscriber
- 29. Messaging - Queuing 1:1 Subscriber Publisher Queue Subscriber Message #1 Subscriber
- 30. Messaging - Queuing 1:1 Subscriber Publisher Queue Subscriber Message #2 Subscriber
- 31. Messaging - Queuing 1:1 Subscriber Publisher Queue Subscriber Message #3 Subscriber
- 32. Messaging - Request Reply 1:1 Reply Subscriber Publisher Subject Subscriber Subscriber
- 33. Messaging Patterns ✓Addressing, discovery ✓Command and control ✓Load-balancing ✓N-way scalability
- 34. Messaging ✓Standards ✓ AMQP (wire) ✓ JMS (api) ✓Products ✓ RabbitMQ ✓ ZeroMQ ✓ ActiveMQ ✓ TIBCO ✓ MQSeries
- 35. Asynchronous and Non-Blocking ✓Don’t wait, go doing something else ✓Never block ✓All callbacks all the time can get messy! ✓Good language/framework support ✓functional closures ✓co-routines
- 36. Load Balancing ✓Multiple endpoints to perform work ✓Can be semantically aware ✓Chainable: DNS, hardware, software ✓Endpoints can be Hardware, VM, process, thread, co-routine, fiber, etc.
- 37. Load Balancing Selection ✓Random ✓Round Robin ✓Weighted ✓Dynamically “aware” ✓Least connections ✓Least loaded
- 38. Load Balancing Technologies ✓DNS Round Robin ✓Anycast ✓Reverse Proxies ✓Clustering ✓Hardware Load Balancers
- 39. Load Balancing Reverse Proxies ✓Nginx ✓HAProxy ✓Apache (mod_proxy) ✓Squid
- 40. Parallel Computing ✓Divide and Conquer ✓Worker queues ✓Map Reduce ✓UE = Unit of Execution ✓VM, process, thread, co-routine, fiber, callback
- 41. Parallel Computing Worker Queues ✓Good for offloading tasks ✓Need bounded time check in master ✓Async result processing ✓Fork/Join pattern
- 42. Parallel Computing MapReduce ✓Used internally at Google ✓Variation of Fork and Join ✓Distributed ✓Originally used for logs processing
- 43. Parallel Computing MapReduce ✓Google’s MapReduce ✓Hadoop ✓Amazon’s Elastic MapReduce ✓RIAK uses it internally for queries
- 44. Scalability Patterns State
- 45. Scalability Patterns: State Harder than scaling behavior
- 46. Scalability Patterns: State ✓Master Record ✓Replication ✓Sharding ✓Caching ✓NoSQL ✓Concurrency
- 47. Master Record ✓Normally Relational Databases (RDBMS) ✓NoSQL Databases emerging ✓Can’t lose this data ✓Scaling can be a challenge
- 48. Master Record: Scaling ✓Traditonally Scale Up ✓Technology will help here ✓SSD (50k-100k IOPs) ✓More memory/cores per box ✓Faster network connectivity ✓Clustering Appliances
- 49. Clustering Appliances 64 bit Infiniband SSD
- 50. Master Record: Scaling ✓Scaling Reads vs Writes? ✓Scaling Reads with Slaves ✓Synchronous (Speed of Light) ✓Asynchronous
- 51. Master Record: Scaling How do we scale OUT?
- 52. Master Record: Replication ✓Synchronous vs Asynchronous ✓Master / Slave Replication ✓Master / Master Replication ✓Tree Replication ✓Buddy Replication
- 53. Replication: Master / Slave
- 54. Replication: Master / Master
- 57. Sharding ✓Partitioning state ✓Requests need to know where to go ✓Distributed Hash ✓Load Balancer ✓Messaging
- 60. Sharding: Over-provision ✓Use N partitions ✓Use Y replicas ✓Use message based requests ✓First back wins ✓Therefore user wins (Google Search)
- 61. Master Record: RDBMS Do we really need an RDBMS?
- 62. Master Record: RDBMS Don’t underestimate RDBMS or the ability of a single machine
- 63. Master Record: RDBMS What about alternatives?
- 64. NoSQL ✓Key-Value ✓Column Databases ✓Document Databases ✓Graph Databases ✓Datastructure Databases
- 65. NoSQL ✓Key-Value: (Memcache, Redis, Riak) ✓Column Databases: (Cassandra, Vertica) ✓Document Databases: (MongoDB, CouchDB) ✓Graph Databases: (Neo4J, AllegroGraph) ✓Datastructure Databases: (Redis, Hazelcast)
- 66. NoSQL in the wild ✓Google: Bigtable, Colossus ✓Twitter: Redis ✓Amazon: Dynamo, SimpleDB ✓Yahoo: HBase (Hadoop) ✓Facebook: Cassandra, HBase
- 67. Caching ✓Cache early and often ✓Usually biggest bang for the buck ✓Referential Transparency ✓Polyglot APIs coming ✓NoSQL stores ✓Cache invalidation is still hard!
- 68. Caching ✓HTTP (HTML, JS, CSS, Images, Media) ✓Key/Value Data ✓Semantic Data structures
- 70. HTTP Caching CDN ✓Akamai ✓Limelight ✓Level3 ✓Digital Fountain (Qualcomm) ✓aiCache
- 71. HTTP Caching CDN
- 72. HTTP Caching ✓Lives in browsers, proxies, CDNs, apps ✓Hard to control, so do it right! ✓Master page controls other resources ✓master page not cached (at least too far) ✓read-only resources ✓change link in master page
- 75. Data Structure Caching ✓Standalone ✓Augment RDBMS ✓In Memory or on Disk
- 76. Data Structure Caching ✓Data Types ✓Strings, Hashes, Lists, Sets, Sorted Sets ✓Atomic Operations ✓Push, pop, ranges, set operations (intersect, union)
- 77. Caching Patterns ✓Write Through ✓Write Behind ✓Replicated ✓P2P
- 78. Cache Invalidation ✓TTL (Time to Live) ✓Bounded FIFO or LIFO ✓Explicit cache invalidation ✓Explicit non-use of read-only resource ✓Harder problem the more master items used
- 79. Scalability Key Points ✓The problem is not where you think ;) ✓Autoscaling is a myth ✓Can’t fix what you can’t measure ✓Scaling master record writes is hard ✓Scaling reads is more tractable ✓What is the opex cost of your choices?
- 81. What do you do when things go bad?
- 82. Availability Patterns Available vs Consistent
- 83. Availability Patterns Available
- 84. We have been here before, right?
- 85. Yes, we have been here before!?
- 86. Scalability Patterns Behavior
- 87. Scalability Patterns: Behavior ✓Event-Driven Architectures ✓Load-Balancing ✓Parallel Computing
- 88. Scalability Patterns State
- 89. Scalability Patterns: State ✓Master Record ✓Replication ✓Sharding ✓Caching ✓NoSQL ✓Concurrency
- 90. But let’s talk more about your data
- 91. Availability Patterns Available vs Consistent
- 94. You can only pick 2 Consistency Availability Partition Tolerance
- 95. Centralized Systems ✓If the system is centralized ✓no P (network partitions) ✓So you get both: ✓Availability ✓Consistency
- 96. Distributed Systems ✓If the system is distributed ✓you will have P! (network partitions) ✓So you get pick one: ✓Availability ✓Consistency
- 97. CAP in reality ✓There is only once choice to make: ✓When there is a network partition, which do you sacrifice? ✓Availability ✓Consistency
- 98. BASE
- 99. What is BASE?
- 101. Eventually Consistent ✓Great tradeoff for the right kind of data ✓Can’t be used everywhere ✓Works in more places than you think ✓Solved speed of light problem
- 102. Availability Patterns Failover
- 103. Availability Patterns: Failover ✓Failover is complex ✓Switch time is critical ✓Failback is equally as complex
- 104. Availability Patterns: Failover Copyright Michael Nygaard
- 105. Availability Patterns: Failback Copyright Michael Nygaard
- 106. Availability Patterns: Replication ✓Synchronous vs Asynchronous ✓Master / Slave Replication ✓Master / Master Replication
- 107. Availability Patterns: Redirection ✓DNS ✓Load Balancers ✓Secondary Sites
- 108. Availability Key Points ✓Always have a dial tone ✓Syntactically correct is good ✓Semantically correct is better ✓Be transparent
- 109. Background Beating the • Good Performance is good • Predictably Good Performance is king! dead horse • Measure everything (can’t fix what you don’t know) • Understand your data • Understand your user experience • Don’t be a failure of your own success
- 110. Background Understand • Good Performance is good • Predictably Good Performance is king! your data! • Measure everything (can’t fix what you don’t know) • Understand your data • Understand your user experience • Don’t be a failure of your own success
- 111. Background Understand • Good Performance is good • Predictably Good Performance is king! your user! • Measure everything (can’t fix what you don’t know) • Understand your data • Understand your user experience • Don’t be a failure of your own success
- 112. Background Understand • Good Performance is good the • Predictably Good Performance is king! • Measure everything (can’t fix what you don’t know) experience! • Understand your data • Understand your user experience • Don’t be a failure of your own success
- 113. Background Master the • Good Performance is good • Predictably Good Performance is king! Tradeoffs • Measure everything (can’t fix what you don’t know) • Understand app and your data!) (For your your data • Understand your user experience • Don’t be a failure of your own success
- 114. Thank You
- 115. Thank You
- 116. Questions?
- 117. Derek Collison @derekcollison dcollison@vmware.com derek.collison@gmail.com