Designing for the Cloud Tutorial - QCon SF 2009
Download as PPTX, PDF5 likes953 views
The document outlines an agenda for a tutorial on designing systems for the cloud, covering topics like cloud computing models, designing distributed services, data management approaches, and techniques for controlling infrastructure like containers and configuration management. It introduces concepts like MapReduce, NoSQL databases, and tradeoffs between consistency, availability and partitioning. The presenter's background and assumptions for the audience are also provided.
![Will the Real Slim Cloudy Please Stand Up?“I, for one, welcome our new outsourced overlords”Finer-grained outsourcingMetered resource usageAPIs & self-service UIs… but isn’t outsourcing often a shell game?See Distributed Computing Economics, Jim Gray (2003)“Scale without skill, availability without avarice”Insert constrained code [here]Magically scalable & availableGAE, Azure (some day)… but aren’t you locked in?](https://image.slidesharecdn.com/qconreco-091210121758-phpapp01/85/Designing-for-the-Cloud-Tutorial-QCon-SF-2009-12-320.jpg)
Designing for the Cloud Tutorial - QCon SF 2009
- 1. Designing for the Cloud:A TutorialStuart Charlton, CTO, Elastra
- 2. Tutorial ObjectivesWhat has cloud computing done to IT systems design & architecture?“The future is already here, it’s just not very evenly distributed” (Gibson)How should new systems be designed with the new constraints?Such as: parallelism, availability, on demand infraWhere can I find are practical frameworks, tools, and techniques, and what are the tradeoffs?Hadoop, Cassandra, Parallel DBs, Actors, Caches, Containers, and Configuration Management
- 3. About Your PresenterStuart CharltonCanadian, now in San FranciscoCTO, ElastraFocus on Customers, Products, Technology DirectionsIn prior lives... BEA Systems, Rogers Communications, Financial Services,global training & consultingRESTafarian and Data geekStu Says Stuffhttp://stucharlton.com/blog
- 4. Tutorial Agenda, in 4 WordsCloudsServiceDataControl4
- 5. Agenda – Part 1Clouds: Fear of a Fluffy PlanetWhat has changed, and what remains the same?Designing applications in this worldA Cloud Design Reference Architecture(aka. A cheat sheet to categorize thinking in the clouds)Service: Foundations for SystemsSolving Big Problems vs. Little ProblemsAmdahl’s Law & The Universal Scalability Law Actor-Based Concurrency: Dr. Strangelanguage, (or How I Learned to Stop Worrying and Love Erlang)
- 6. Agenda – Part 2Data: Management & AccessContrasting PhilosophiesPersistence vs. Management; Scale-Up vs. Scale-OutShared Disk vs. Shared NothingA survey of solutions (from clustered DBMS to K/V stores)Consistency, Availability, Partitioning (CAP) TradeoffsDeep dig into what these really implyControl: Containers, Configuration & ModelingThe Dev/Ops Tennis MatchThe Evolution of AutomationFrom Scripts to Runbooks to FSMs to HTNs
- 7. CaveatsAudience Assumption: IT Devs & ArchitectsSome exposure to cloud, but not necessarily advancedThe technology is a fast moving targetEspecially state of the specific tools & frameworksTheory vs. practiceI try to balance the two; both are essentialTime is limitedOnly scratching the surface of certain topicsMissing topics are usually full tutorials in their own rightMuch of the subject matter is up for debateAnd, this is a tutorial, not a workshop….
- 8. CloudsFear of a Fluffy Planet8
- 10. The Freedom!On Demand Infrastructure via API callsInside or outside my data centres (Private / Public Cloud)Pay-per-use pricing modelsGreat for temporary growth needsPlatform-as-a-ServiceScalability without Skill, Availability without AvariceLarge Scale, Always OnNew opportunities due to cheaper scale & availability
- 11. The Horror!Hype OverdriveCloud Running Shoes! Cloud Chewing Gum! GOOG!Werner Vogels Action Figures! (well, not quite yet)Standards SupportSo many to choose from!OCCI, vCloud + OVF, EC2, WBEM, WS-ManagementPlatform-as-a-ServiceWhat color would you like for your locked trunk’s interior?Crazy TalkNo SQL! Eventual Consistency! Infrastructure as Code!
- 12. Will the Real Slim Cloudy Please Stand Up?“I, for one, welcome our new outsourced overlords”Finer-grained outsourcingMetered resource usageAPIs & self-service UIs… but isn’t outsourcing often a shell game?See Distributed Computing Economics, Jim Gray (2003)“Scale without skill, availability without avarice”Insert constrained code [here]Magically scalable & availableGAE, Azure (some day)… but aren’t you locked in?
- 13. Will the Real Slim Cloudy Please Stand Up?“I like Big *aaS and I cannot lie”“My name is… what? Slim Cloudy!”Private, Public, or Community CloudsMultiple stack levels“Real” SOA, not just web services… haven’t I heard this before?Reduced lead times to changeAgile Operations / Lean ITRevolution in systems management… can we really change IT?
- 14. Designing Applications in this WorldDistributed & networked systems have triumphedThe fallacies must be taken seriously nowNetwork is unreliable, latency > 0, bandwidth is finite, topology might change, etc.Scale-out & fault tolerance: the new design centerVersus productive business logic, data management, etc.What’s old is newSome challengers to mainstream ideas are old ideas being reappliede.g. Erlang, Map/Reduce, distributed file systems, replication
- 15. Designing Applications in this WorldAutonomous services constitute most systemsFull-stack services, not just bits of codeDesign for constant operationsInterdependence + Distribution + Autonomy = PainFCAPS (Fault, Configuration, Accounting, Performance & Security Management) Security & PrivacyMulti-tenancy, data-in-transit vs. data-at-rest, etc.
- 16. Solving for one’s own problemsMainstream tools, platforms, and servers have not consistently caught upLOTS of software experimentation in:Web servers, containers, caches, databases, network configuration, systems managementThe danger is to view new solutions as the better way of doing things in generalIt’s possible; but stuff is changing quicklyNew territory always involves a level of reinventionThe tech world has not rebooted due to cloud computingBeware Fanbois/Fangrrls, Pundits & The Press
- 17. A Cloud Design Reference ArchitectureWeb – WebArch & RESTService, Data,& Control – this tutorialResource –virtualization,management &infrastructure cloudsWEBSERVICEDATACONTROLRESOURCE
- 18. ServiceOrganizing your computing domain forfaultscalemanagementWEBSERVICEDATACONTROLRESOURCE
- 19. DataStorage, retrieval,integrity, recovery givenDistributed systemsLarge scaleHigh availability(possible) Multi-tenancyWEBSERVICEDATACONTROLRESOURCE
- 20. ControlProvision, configuration, governance, and optimization of infrastructureResource brokeragePolicy constraintsDependency managementSoftware configurationAuthorization & AuditabilityWEBSERVICEDATACONTROLRESOURCE
- 22. Designing a Service, circa 1998-2008Multi-Tier Hybrid ArchitectureSome stateless, some stateful computingSession state is replicatedIndependent servers / applicationsLow-level redundancy (RAID, 2x NICs, etc.)“Put your eggs into a small number of baskets, and watch those baskets”General assumptionsFailure at the service layer shouldn’t lead to downtimeFailure at the data layer may be catastrophic
- 23. Designing a Service, circa 2008+Autonomous services Divide system into areas of functional responsibility (tiers irrelevant)Interdependent servers / applicationsSoftware-level redundancy andfault handling “Many, many servers breaking big problems down or distributinglots of little problems around”New realitiesPartial failure is a regular, normal occurrence; no excuse for downtime from any service
- 24. Breaking or bridging a problem across resourcesBig Problems (Parallel)Theory:Amdahl’s lawShared memory or disk vs. Shared nothingNew Practice:MapReduce (e.g. Hadoop), Spaces, Master/WorkerRetro: Linda, MPI, OpenMP, IPC or ThreadsLittle Problems (Concurrent)Theory: Actor-model & process calculiNew Practice: Lightweight Messaging, Spaces, Erlang & Scala ActorsRetro: IPC, Thread pools,Components (COM+/EJB),Big Messaging (MQ, TIB, JMS)
- 25. Case Study in “Big Problem” Solving:MapReduce & Apache HadoopInputRead your data from files as a K/V mapDistribute Mapping FunctionInput one (k,v) pairreturns new K/V listPartition & SortHandled by framework (eg. Hadoop)Provide a comparatorDistribute Reduce FunctionInput one (k, list of values) pairReturn a list of output valuesOutputSave the list as a file
- 26. ….But how fast can I get?Theory Interlude: Amdahl’s LawHow fast can I speed up a sequential process?Time = Serial part + Parallel part Thus, the speed up isWhere P is the % of the program that can be parallelN is the number of processorsWhat happens when P is 95%? -- Maximum of 20x How about 99.99%?
- 27. Gunther’s Universal Scalability LawIt gets worse…Most scale-outexperiencesretrogradebehavior at peak loadsCapacity(N) = N 1 + α (N − 1) + β N (N − 1) α is the contention β is the coherency delayhttp://www.perfdynamics.com/Manifesto/gcaprules.html
- 28. Case study in solving “little problems”Actors: The Basic IdeaProgrammable entities are concurrent, share nothing, communicate through messagesActors canSend messagesCreate other actorsSpecify how it responds to messagesVery lightweight (actors = objects)Usually no ordering guaranteesAt the language level
- 29. ErlangSupervisors: Assuming failure will occurFailures require cleanup & restartSupervisor relationships canensure the systemtolerates faultsHot-swap patchesFundamentally inthe language libraries
- 30. What kinds of failures? A Simplification.Exceptional ConditionsConditions that a programmer did not or should not handleTolerated through replication, fast failure, and/or restart(s)ExamplesHardware failures, network outages, “Heisenbugs”, rare software conditionsConditions that the programmer can handleHandled through cleanup or “catch” codeExamplesFile not found, type conversion, bad arithmetic (divide by zero),malformed inputError Conditions
- 32. Evolving the Database: Two PhilosophiesData Persistence Systemsand FrameworksDatabase Management Systems(DBMS)Goal: Store & retrieve data quickly, reliable, with minimal hassle to the programmerOften uses application tools & languages to manage & access dataFocused set of featuresGoal: Manage the access, integrity, security, and reliability of data, independently of applicationsHard separation of tools & languages (e.g. SQL, DBA tools)Broad set of features
- 33. Scaling the Database: Two PhilosophiesScale-UpScale-OutConcurrent processing & parallelism through hardwareSMP, NUMA, MPPRAID Arrays (SAN & NAS)Shared disk or memoryBenefit: It worked in the 90s.Drawback: Expensive, often bespoke, forklift upgradesConcurrent processing & parallelism through softwareCommodity hardwareSoftware provides the engineShared nothingBenefit: Linear scale, easy to standardize, easy to replicate / upgradeDrawback: Traditionally, the software sucked.33
- 34. … What happens when database clustering software stops sucking? (i.e. now)A flurry of programmer-oriented approachesPersistence engines rule the bleeding edge in 2009Key/Value Stores, JSON Document stores, etc.Declarative/Imperative impedance mismatch(the “Vietnam” of the software tools industry) gets conflated with distributed dataLots of practical confusionWhat are the tradeoffs with a widely scaled out database system?
- 35. Too many choices, with idiosyncratic design histories
- 37. When should I share components?Shared DiskShared NothingPartition compute across nodesStorage is shared through NAS or SANGood for:Mixed workloadSmall random access readsWorst case:Inter-node network chatter caps scalabilityDisk pings to propagate writes (e.g. Oracle pre-RAC)Partition data across nodesEach node owns its dataGood for:Read-mostlyParallel reads of huge data volumesConsistent writes go to one partitionWorst case:RepartitioningHotspot records don’t scaleWrites that span partitions
- 38. Modern Data Persistence Systems Object Persistence“Navigational databases in Java, Smalltalk, C++”GemStone, Versant, ObjectivityDistributed Key-Value Stores“Structured data with lesser need for complex queries”Consistent: BigTable, HBase, VoldemortEventually Consistent: Dynamo, CassandraDocument and/or Blob Stores“Indexed structured data + binaries/fulltext”CouchDB, BerkeleyDB, MongoDB
- 39. Clustered DBMS for TransactionsOracle Real Application Clusters (RAC)Shared disk, Replicated Memory (“Cache Fusion”)Limited by mesh interconnect to disk (partitioning possible)IBM DB2 Data Partitioning FeatureShared nothing database cluster, high number of nodesIBM DB2 pureScaleNew (Oct 2009) technology that ports IBM DB2 mainframe shared-disk clustering to the DB2 for open systemsMicrosoft SQL Server 2008“Federated” Shared Nothing Database a longtime feature
- 40. Clustered DBMS for Parallel QueriesTeradataThe old standard data warehouse, hardware + softwareNetezzaData warehousing appliance (hw + software)VerticaColumn-oriented, shared nothing clustered databaseMike Stonebraker’s new companyGreenplumColumn-oriented, shared nothing clustered databaseBased on PostgreSQL with MapReduce engine
- 41. Scaling to Internet-ScaleSingle Control DomainOne Database SiteConsistency is built-inScalable with tradeoffs among different workloadsScale to the limits of network bandwidth & manageabilityMain Example:Clustered DBMSMultiple Control DomainsMany Database SitesConsistency requires agreement protocolScalable only if consistency is relaxedNearly limitless (global) scaleMain Examples:DNS The Web39
- 42. How do I make consistency tradeoffs?Theory interlude: The CAP theoremConsistency (A+C in ACID)There’s a total orderingon all operations on the data;i.e. like a sequenceAvailabilityEvery request onnon-failed servers must havea responseTolerance to Network PartitionsAll messages might be lost between server nodesChoose at most two of these (as a spectrum).
- 43. CAP Tradeoffs: Consistency & AvailabilityThe common case.
- 44. Fault tolerance through replicas & fast fail + fast recoveryImplication:
- 45. network outage between servers might halt the system
- 46. generally requires a single domainof control
- 47. Examples that emphasize C+A:
- 48. Single-site cluster databases
- 49. Google BigTable
- 50. Hadoop’sHBase
- 51. Oracle RAC, IBM DB2 Parallel
- 52. Clustered file systems
- 53. Google File System & HDFS
- 54. Distributed Spaces & Caches
- 55. Coherence, Gigaspaces & TerracottaCAP Tradeoffs: Consistency & PartitionsCommon approach for traditional distributed systems
- 56. Implication:
- 57. multiple domains of control
- 58. clients can’t always read/write
- 59. failures degrade scale & performance due to negotiation
- 60. Examples that emphasize C+P:
- 61. Distributed shared nothing databases
- 62. Two-phase commit
- 63. Distributed locks & file systems
- 64. Chubby & Hadoop’sZooKeeper
- 65. Paxos & consensus protocols
- 66. Synchronous Log ShippingCAP Tradeoffs: Partitions & AvailabilityNew approach for Internet-scale systems
- 67. Implication:
- 68. multiple domains of control
- 69. reads & writes always succeed(eventually)
- 70. clients may read inconsistent (old or undone) data
- 71. Examples that emphasize A+P:
- 72. Internet DNS
- 73. Web Caching & Content Delivery Networks
- 74. Amazon Dynamo (and clones)
- 75. Cassandra (Facebook, Digg)
- 76. CouchDB (BBC)
- 77. Asynchronous log shippingSummary of the CAP TradeoffsMix & match the tradeoffs where appropriateGoogle’s search engine uses all three!The tradeoffs are a spectrum, and are not static choiceseg. there are adjustable levels of consistency to consider Strict, causal, snapshot / epoch, eventual, weak…The main tradeoff: writes to multiple sites / domains of control (with or without high availability)Single Domain (don’t tolerate network partitions), orAgreement Protocol (reduces availability), orRelaxed Consistency (stale/inaccurate data is possible)Weaker consistency is where the idea of a DBMS falters (it is contrary to its main purpose in life)
- 78. Please don’t throw out logical/relationaldata design! (unless you have to)“Future users of massive datasets should be protected from having to know how the data is organized in the computing cloud….…. Activities of users through web agents and most application programs should remain unaffected when the internal representation of data is changed and even when some aspects of the external representation are changed.”Paraphrasing Ed Codd – 39 years ago!
- 79. ControlContainers, Configuration, & Modeling
- 80. The Dev / Ops Game
- 81. Example:Why can’t these two servers communicate?Possible areas of problemsSecurityBad credentialsServer ConfigurationWrong IP or PortBad setup to listen or callNetwork ConfigurationWrong duplexBad DNS or DHCPFirewall ConfigurationPorts or protocols not open
- 82. Example:What do I need to do to make this change?Desired ChangeScale-out this clusterBut…Impacts on other systemsSecurity SystemsLoad BalancersMonitoringCMDB / Service DeskArchitecture issuesStateful or stateless nodesRepartitioning?Limits/constraints on scale out?49
- 83. Example:What is the authoritative reality?Desired StateConfiguration TemplateModelScriptWorkflowCMDBCodeCurrent StateOn the serverMight not be in a fileMight get changed at runtimeAnd when you do change…It may not actually changeIt might change to an undesirable settingIt might affect other settings that you didn’t think about50
- 84. Configuration Code, Files, and ModelsBottom UpScripts & RecipesHand-grown automationRunbooksWorkflow, policyFrameworksChefPuppet, CfengineBuild Dependency SystemsMavenTop DownModeled ViewpointsE.g. Microsoft Oslo, UML, Enterprise ArchitectureModular ContainersE.g. OSGi, Spring, Azure rolesConfiguration ModelsSML, CIMECML , EDML
- 85. An Evolution of AutomationScriptsFor automating common casesRun-Book AutomationScripts as visual workflowDeclarativeSeparate what you want from how you want it doneFinite State MachinesOrganize scripts into described states & transitionsHierarchical Task Networks (Planning)Assemble a plan by exploring hypothetical strategic paths
- 86. An Approach to Integrated Design and Ops53
- 87. Wrap-upCloudy, with a chance of …
- 88. Revisiting the Cloud Design Reference ArchitectureService – Big vs. Little ProblemsMapReduce & ActorsAmdahl’s LawData – persistence vs. mgtscale-up vs. scale-outCAP tradeoffsControl –containers, configuration, automationWEBSERVICEDATACONTROLRESOURCE
- 89. For More InformationHadoophttp://hadoop.apache.org/CAP Theorem Proof Paperhttp://people.csail.mit.edu/sethg/pubs/BrewersConjecture-SigAct.pdfGoogle’s papers on Distributed & Parallel Computinghttp://research.google.com/pubs/DistributedSystemsandParallelComputing.htmlNeil Gunther’s “Taking the Pith out of Performance” Bloghttp://perfdynamics.blogspot.com/A Comparison of Approaches to Large-Scale Data Analyticshttp://database.cs.brown.edu/sigmod09/benchmarks-sigmod09.pdfModel-Driven Operations for the Cloudhttp://www.stucharlton.com/stuff/oopsla09.pdf