Abstract image of a woman sitting on books and studying on a laptop

Availability, The Cloud and Everything (version 2, Surge2010)

L
logicalstack

The document discusses availability, distributed systems, and data-driven operations in cloud computing. It defines availability and reliability, noting that availability refers to accessibility while reliability refers to tolerating failures. It also discusses the costs and benefits of cloud computing, including issues like loss of control and single points of failure. Distributed systems concepts like redundancy, Erlang, and CAP theorem are covered. Automation, instrumentation, logging, and visualization are presented as important aspects of data-driven operations.

Technology
1 of 46
Downloaded 58 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
Availability, the Cloud and Everything Joe Williams Saturday, October 2, 2010
Me • Joe Williams • Infrastructure Engineer • Cloudant • @williamsjoe • joeandmotorboat.com Saturday, October 2, 2010
• Distributed database built on CouchDB • Real-time Search and Analytics • Sign Up! (Free to 256MB) • cloudant.com • http://github.com/cloudant/bigcouch Saturday, October 2, 2010
Bias • Distributed Databases (CouchDB) • Amazon EC2 • Chef • Erlang Saturday, October 2, 2010
Availability Saturday, October 2, 2010
Availability • What is Availability? Saturday, October 2, 2010
Availability Saturday, October 2, 2010
Availability “System availability refers to the accessibility of system services to users. A system is available if it is operational for an overwhelming fraction of the time. Unlike reliability, availability is instantaneous.” Saturday, October 2, 2010
Availability “System reliability refers to the property of tolerating constituent component failures, for the longest time. A system is perfectly reliable if it never fails.” Saturday, October 2, 2010
Availability • Reliability * Availability = Dependability Saturday, October 2, 2010
Availability • Availability & Reliability • Mean time to failures • Mean time to repair • Durability • Fault isolation • Fault tolerance Saturday, October 2, 2010
Availability • Uptime / Downtime • Perceived • Actual Saturday, October 2, 2010
Availability • Probabilistic Risk Assessment • Event Tree Analysis • Fault Tree Analysis Apthorpe (http://www.usenix.org/events/lisa01/tech/apthorpe/apthorpe.ps) Saturday, October 2, 2010
The Cloud Saturday, October 2, 2010
The Cloud “It never gets easier, you just go faster.” - Greg Lemond Saturday, October 2, 2010
The Cloud • Abstraction • Commoditization • Homogenous • Ephemeral Saturday, October 2, 2010
The Cloud • Costs • Loss of Control • Single Points of Failure • Network Partitions / Data Locality • Unreliable • Performance Saturday, October 2, 2010
The Cloud • Benefits • API to everything • Fast and Flexible Resource Mgmt • “Unlimited” Resources Saturday, October 2, 2010
The Cloud • Bootstrapping • Time and Effort Adam Jacob and Ezra Zygmuntowicz (http://blip.tv/file/2285124/) Saturday, October 2, 2010
The Cloud • Nodes are stateless and disposable. Saturday, October 2, 2010
The Cloud "Clouds are systems ... and with systems, you have to think hard and know how to deal with issues in that environment. The scale is so much bigger, and you don't have the physical control. But we think people should be optimistic about what we can do here. If we are clever about deploying cloud computing with a clear-eyed notion of what the risk models are, maybe we can actually save the economy through technology." - Security in the Ether By David Talbot - MIT Technology Review Jan/Feb 2010 Saturday, October 2, 2010
What’s Next • Distributed Systems • Automation • Data Driven Operations Saturday, October 2, 2010
Distributed Systems Baran (http://www.rand.org/pubs/research_memoranda/RM3420/) Saturday, October 2, 2010
Distributed Systems • RAID ain’t as redundant as it used to be. Leventhal (http://queue.acm.org/detail.cfm?id=1670144) Saturday, October 2, 2010
Distributed Systems • Redundancy • Duplication • Distribution Saturday, October 2, 2010
Distributed Systems • Alphabet Soup • ACID, CAP, BASE, 2PC, MVCC • Vector Clocks, Eventual Consistency • Dynamo, Paxos, Chandra, Byzantine Saturday, October 2, 2010
Distributed Systems • CAP == Availability Saturday, October 2, 2010
Distributed Systems • Erlang • Distributed • Concurrent • Fault Tolerant Saturday, October 2, 2010
Distributed Systems • Erlang • Supervision Trees Saturday, October 2, 2010
Distributed Systems • Erlang • Hot Code Upgrades • Distributed Upgrades are HARD Saturday, October 2, 2010
Distributed Systems • Future Work • Erlang Supervision Trees • PRA / FTA / ETA Apthorpe (http://www.usenix.org/events/lisa01/tech/apthorpe/apthorpe.ps) Saturday, October 2, 2010
Automation Saturday, October 2, 2010
Automation • Optimal use of the cloud. Saturday, October 2, 2010
Automation • Frequent deployment. Saturday, October 2, 2010
Automation • Tools • Chef • Puppet • Cfengine • Bcfg2 Saturday, October 2, 2010
Automation • Erlang + Chef (as of v0.8) • erl_call Provider Saturday, October 2, 2010
Data Driven Operations Saturday, October 2, 2010
Data Driven Operations “What gets measured, gets managed.” -Peter Drucker Saturday, October 2, 2010
Data Driven Operations • Instrumentation Saturday, October 2, 2010
Data Driven Operations • Logging Saturday, October 2, 2010
Data Driven Operations • Visualization Saturday, October 2, 2010
Data Driven Operations • Demo! Saturday, October 2, 2010
Data Driven Operations • Modeling • Analysis • Universal Law of Computational Scalability • Amdahl’s Law Saturday, October 2, 2010
Data Driven Operations • Modeling isn’t just for capacity planning. Montagne (http://queue.acm.org/detail.cfm?id=1862187) Saturday, October 2, 2010
The End Saturday, October 2, 2010
Questions? Joe Williams - @williamsjoe Saturday, October 2, 2010

More Related Content

PDF
Availability, the Cloud and Everything
logicalstack
 
PDF
Turning That UX Frown Upside Down
Steve Fisher
 
PPTX
Cloud Computing - Availability Issues and Controls
lylcheng88
 
PDF
Cloud Computing Security (Final Year Project) by Pavlos Stefanis
Pavlos Stefanis
 
PDF
Integrating Erlang with PHP
Alvaro Videla
 
PDF
A Security Analysis Framework Powered By An Expert System
Maher Gamal
 
PDF
Xtreme Deployment
Theo Schlossnagle
 
PDF
Enterprise Drupal
Chapter Three
 
Availability, the Cloud and Everything
logicalstack
 
Turning That UX Frown Upside Down
Steve Fisher
 
Cloud Computing - Availability Issues and Controls
lylcheng88
 
Cloud Computing Security (Final Year Project) by Pavlos Stefanis
Pavlos Stefanis
 
Integrating Erlang with PHP
Alvaro Videla
 
A Security Analysis Framework Powered By An Expert System
Maher Gamal
 
Xtreme Deployment
Theo Schlossnagle
 
Enterprise Drupal
Chapter Three
 

Similar to Availability, The Cloud and Everything (version 2, Surge2010) (14)

PDF
Calculating the ROI for XML and DITA topic-based authoring
Scriptorium Publishing
 
PDF
OpenStreetMap dongpo deng
AidIQ
 
PDF
M.Malone Simple Geo @ Social Developers Summit
Mediabistro
 
PDF
Batch Indexing & Near Real Time, keeping things fast
Marc Sturlese
 
PDF
Aegir one drupal to rule them all
Development Seed
 
PDF
Magic broker 2 #iot2010 presentation
Adrian Friday
 
PDF
iOS & Arduino
Nuno Morgadinho
 
PDF
Geoloqi: Non-visual augmented reality Open Source Bridge
Aaron Parecki
 
PDF
Geoloqi - Non-visual location based augmented reality with SMS and GPS - Ope...
Amber Case
 
PDF
Drupal security - Configuration and process
Gábor Hojtsy
 
PDF
DNSSEC Deployment at ROOT Zone
Mehmet Akcin
 
PDF
Gaelyk - SpringOne2GX - 2010 - Guillaume Laforge
Guillaume Laforge
 
PDF
Scaling webappswithrabbitmq
Alvaro Videla
 
PDF
Integrating php withrabbitmq_zendcon
Alvaro Videla
 
Calculating the ROI for XML and DITA topic-based authoring
Scriptorium Publishing
 
OpenStreetMap dongpo deng
AidIQ
 
M.Malone Simple Geo @ Social Developers Summit
Mediabistro
 
Batch Indexing & Near Real Time, keeping things fast
Marc Sturlese
 
Aegir one drupal to rule them all
Development Seed
 
Magic broker 2 #iot2010 presentation
Adrian Friday
 
iOS & Arduino
Nuno Morgadinho
 
Geoloqi: Non-visual augmented reality Open Source Bridge
Aaron Parecki
 
Geoloqi - Non-visual location based augmented reality with SMS and GPS - Ope...
Amber Case
 
Drupal security - Configuration and process
Gábor Hojtsy
 
DNSSEC Deployment at ROOT Zone
Mehmet Akcin
 
Gaelyk - SpringOne2GX - 2010 - Guillaume Laforge
Guillaume Laforge
 
Scaling webappswithrabbitmq
Alvaro Videla
 
Integrating php withrabbitmq_zendcon
Alvaro Videla
 
Ad

Recently uploaded (20)

PDF
August Patch Tuesday
Ivanti
 
PPTX
The various Industrial Revolutions .pptx
bandileshozi3
 
PDF
CloudStack 4.21: First Look Webinar slides
ShapeBlue
 
PDF
Hybrid model detection and classification of lung cancer
IAESIJAI
 
PDF
WOOl fibre morphology and structure.pdf for textiles
Rajendrakumar868651
 
PPTX
MicrosoftCybserSecurityReferenceArchitecture-April-2025.pptx
SilvioHayashi
 
PDF
Developing a website for English-speaking practice to English as a foreign la...
IAESIJAI
 
PDF
Video forgery: An extensive analysis of inter-and intra-frame manipulation al...
IAESIJAI
 
PDF
How ambidextrous entrepreneurial leaders react to the artificial intelligence...
IAESIJAI
 
PPTX
observCloud-Native Containerability and monitoring.pptx
anupsingh76937
 
PDF
Enhancing emotion recognition model for a student engagement use case through...
IAESIJAI
 
PDF
From MVP to Full-Scale Product A Startup’s Software Journey.pdf
KodekX
 
PPTX
Group 1 Presentation -Planning and Decision Making .pptx
MatthewLewis227954
 
PDF
Taming the Chaos: How to Turn Unstructured Data into Decisions
Safe Software
 
PPTX
Tartificialntelligence_presentation.pptx
ry7r52mzb4
 
PDF
1 - Historical Antecedents, Social Consideration.pdf
tuazon2030627
 
PDF
NewMind AI Weekly Chronicles – August ’25 Week III
NewMind AI
 
PDF
A comparative study of natural language inference in Swahili using monolingua...
IAESIJAI
 
PDF
sustainability-14-14877-v2.pddhzftheheeeee
VenkateshGovindaraja9
 
PPT
Geologic Time for studying geology for geologist
ssuser738f5a
 
August Patch Tuesday
Ivanti
 
The various Industrial Revolutions .pptx
bandileshozi3
 
CloudStack 4.21: First Look Webinar slides
ShapeBlue
 
Hybrid model detection and classification of lung cancer
IAESIJAI
 
WOOl fibre morphology and structure.pdf for textiles
Rajendrakumar868651
 
MicrosoftCybserSecurityReferenceArchitecture-April-2025.pptx
SilvioHayashi
 
Developing a website for English-speaking practice to English as a foreign la...
IAESIJAI
 
Video forgery: An extensive analysis of inter-and intra-frame manipulation al...
IAESIJAI
 
How ambidextrous entrepreneurial leaders react to the artificial intelligence...
IAESIJAI
 
observCloud-Native Containerability and monitoring.pptx
anupsingh76937
 
Enhancing emotion recognition model for a student engagement use case through...
IAESIJAI
 
From MVP to Full-Scale Product A Startup’s Software Journey.pdf
KodekX
 
Group 1 Presentation -Planning and Decision Making .pptx
MatthewLewis227954
 
Taming the Chaos: How to Turn Unstructured Data into Decisions
Safe Software
 
Tartificialntelligence_presentation.pptx
ry7r52mzb4
 
1 - Historical Antecedents, Social Consideration.pdf
tuazon2030627
 
NewMind AI Weekly Chronicles – August ’25 Week III
NewMind AI
 
A comparative study of natural language inference in Swahili using monolingua...
IAESIJAI
 
sustainability-14-14877-v2.pddhzftheheeeee
VenkateshGovindaraja9
 
Geologic Time for studying geology for geologist
ssuser738f5a
 
Ad

Availability, The Cloud and Everything (version 2, Surge2010)

  • 1. Availability, the Cloud and Everything Joe Williams Saturday, October 2, 2010
  • 2. Me • Joe Williams • Infrastructure Engineer • Cloudant • @williamsjoe • joeandmotorboat.com Saturday, October 2, 2010
  • 3. • Distributed database built on CouchDB • Real-time Search and Analytics • Sign Up! (Free to 256MB) • cloudant.com • http://github.com/cloudant/bigcouch Saturday, October 2, 2010
  • 4. Bias • Distributed Databases (CouchDB) • Amazon EC2 • Chef • Erlang Saturday, October 2, 2010
  • 6. Availability • What is Availability? Saturday, October 2, 2010
  • 8. Availability “System availability refers to the accessibility of system services to users. A system is available if it is operational for an overwhelming fraction of the time. Unlike reliability, availability is instantaneous.” Saturday, October 2, 2010
  • 9. Availability “System reliability refers to the property of tolerating constituent component failures, for the longest time. A system is perfectly reliable if it never fails.” Saturday, October 2, 2010
  • 10. Availability • Reliability * Availability = Dependability Saturday, October 2, 2010
  • 11. Availability • Availability & Reliability • Mean time to failures • Mean time to repair • Durability • Fault isolation • Fault tolerance Saturday, October 2, 2010
  • 12. Availability • Uptime / Downtime • Perceived • Actual Saturday, October 2, 2010
  • 13. Availability • Probabilistic Risk Assessment • Event Tree Analysis • Fault Tree Analysis Apthorpe (http://www.usenix.org/events/lisa01/tech/apthorpe/apthorpe.ps) Saturday, October 2, 2010
  • 15. The Cloud “It never gets easier, you just go faster.” - Greg Lemond Saturday, October 2, 2010
  • 16. The Cloud • Abstraction • Commoditization • Homogenous • Ephemeral Saturday, October 2, 2010
  • 17. The Cloud • Costs • Loss of Control • Single Points of Failure • Network Partitions / Data Locality • Unreliable • Performance Saturday, October 2, 2010
  • 18. The Cloud • Benefits • API to everything • Fast and Flexible Resource Mgmt • “Unlimited” Resources Saturday, October 2, 2010
  • 19. The Cloud • Bootstrapping • Time and Effort Adam Jacob and Ezra Zygmuntowicz (http://blip.tv/file/2285124/) Saturday, October 2, 2010
  • 20. The Cloud • Nodes are stateless and disposable. Saturday, October 2, 2010
  • 21. The Cloud "Clouds are systems ... and with systems, you have to think hard and know how to deal with issues in that environment. The scale is so much bigger, and you don't have the physical control. But we think people should be optimistic about what we can do here. If we are clever about deploying cloud computing with a clear-eyed notion of what the risk models are, maybe we can actually save the economy through technology." - Security in the Ether By David Talbot - MIT Technology Review Jan/Feb 2010 Saturday, October 2, 2010
  • 22. What’s Next • Distributed Systems • Automation • Data Driven Operations Saturday, October 2, 2010
  • 23. Distributed Systems Baran (http://www.rand.org/pubs/research_memoranda/RM3420/) Saturday, October 2, 2010
  • 24. Distributed Systems • RAID ain’t as redundant as it used to be. Leventhal (http://queue.acm.org/detail.cfm?id=1670144) Saturday, October 2, 2010
  • 25. Distributed Systems • Redundancy • Duplication • Distribution Saturday, October 2, 2010
  • 26. Distributed Systems • Alphabet Soup • ACID, CAP, BASE, 2PC, MVCC • Vector Clocks, Eventual Consistency • Dynamo, Paxos, Chandra, Byzantine Saturday, October 2, 2010
  • 27. Distributed Systems • CAP == Availability Saturday, October 2, 2010
  • 28. Distributed Systems • Erlang • Distributed • Concurrent • Fault Tolerant Saturday, October 2, 2010
  • 29. Distributed Systems • Erlang • Supervision Trees Saturday, October 2, 2010
  • 30. Distributed Systems • Erlang • Hot Code Upgrades • Distributed Upgrades are HARD Saturday, October 2, 2010
  • 31. Distributed Systems • Future Work • Erlang Supervision Trees • PRA / FTA / ETA Apthorpe (http://www.usenix.org/events/lisa01/tech/apthorpe/apthorpe.ps) Saturday, October 2, 2010
  • 33. Automation • Optimal use of the cloud. Saturday, October 2, 2010
  • 34. Automation • Frequent deployment. Saturday, October 2, 2010
  • 35. Automation • Tools • Chef • Puppet • Cfengine • Bcfg2 Saturday, October 2, 2010
  • 36. Automation • Erlang + Chef (as of v0.8) • erl_call Provider Saturday, October 2, 2010
  • 38. Data Driven Operations “What gets measured, gets managed.” -Peter Drucker Saturday, October 2, 2010
  • 39. Data Driven Operations • Instrumentation Saturday, October 2, 2010
  • 40. Data Driven Operations • Logging Saturday, October 2, 2010
  • 41. Data Driven Operations • Visualization Saturday, October 2, 2010
  • 42. Data Driven Operations • Demo! Saturday, October 2, 2010
  • 43. Data Driven Operations • Modeling • Analysis • Universal Law of Computational Scalability • Amdahl’s Law Saturday, October 2, 2010
  • 44. Data Driven Operations • Modeling isn’t just for capacity planning. Montagne (http://queue.acm.org/detail.cfm?id=1862187) Saturday, October 2, 2010
  • 46. Questions? Joe Williams - @williamsjoe Saturday, October 2, 2010