Dynamo: Not Just For Datastores
9 likes1,705 views
The document outlines the key features and concepts of the Dynamo distributed storage system, emphasizing its decentralized nature, fault tolerance, and models of consistency. It discusses core topics like CAP controls, consistent hashing, and the unique properties of Dynamo that support high availability and scalability for applications. Additionally, the document highlights the Riak Core framework and its abstractions, tools for building resilient distributed systems, and various optimizations and considerations for future developments.
![Dynamo: Consistent hashing
Figure: PrefsList for K is [C, D, E]](https://image.slidesharecdn.com/slides-110920110623-phpapp01/85/Dynamo-Not-Just-For-Datastores-28-320.jpg)
![Dynamo: Vector clocks
-type vclock() ::
[{actor(), counter()}]
A occurred-before B if counters in A are
less than or equal to those in B for each actor](https://image.slidesharecdn.com/slides-110920110623-phpapp01/85/Dynamo-Not-Just-For-Datastores-29-320.jpg)
Dynamo: Not Just For Datastores
- 1. Dynamo: not just datastores Susan Potter September 2011
- 2. Overview: In a nutshell Figure: "a highly available key-value storage system"
- 3. Overview: Not for all apps
- 4. Overview: Agenda Distribution & consistency Riak abstractions t - Fault tolerance Building an application N, R, W Considerations Dynamo techniques Oh, the possibilities!
- 5. Distributed models & consistency Are you on ACID? Are your apps ACID? Few apps require strong consistency Embrace BASE for high availability
- 6. Distributed models & consistency Are you on ACID? Are your apps ACID? Few apps require strong consistency Embrace BASE for high availability
- 7. Distributed models & consistency Are you on ACID? Are your apps ACID? Few apps require strong consistency Embrace BASE for high availability
- 8. Distributed models & consistency Are you on ACID? Are your apps ACID? Few apps require strong consistency Embrace BASE for high availability
- 9. Distributed models & consistency BASE Basically Available Soft-state Eventual consistency
- 10. Distributed models & consistency BASE Basically Available Soft-state Eventual consistency </RANT>
- 11. t-Fault tolerance: Two kinds of failure
- 12. t-Fault tolerance: More on failure Failstop fail in well understood / deterministic ways Byzantine fail in arbitrary non-deterministic ways
- 13. t-Fault tolerance: More on failure Failstop fail in well understood / deterministic ways t-Fault tolerance means t + 1 nodes Byzantine fail in arbitrary non-deterministic ways
- 14. t-Fault tolerance: More on failure Failstop fail in well understood / deterministic ways t-Fault tolerance means t + 1 nodes Byzantine fail in arbitrary non-deterministic ways
- 15. t-Fault tolerance: More on failure Failstop fail in well understood / deterministic ways t-Fault tolerance means t + 1 nodes Byzantine fail in arbitrary non-deterministic ways t-Fault tolerance means 2t + 1 nodes
- 16. CAP Controls: N, R, W N = number of replicas must be 1 ≤ N ≤ nodes R = number of responding read nodes can be set per request W = number of responding write nodes can be set per request Q = quorum "majority rules" set in stone: Q = N/2 + 1 Tunability when R = W = N =⇒ strong when R + W > N =⇒ quorum where W = 1, R = N or W = N, R = 1 or W = R = Q
- 17. CAP Controls: N, R, W N = number of replicas must be 1 ≤ N ≤ nodes R = number of responding read nodes can be set per request W = number of responding write nodes can be set per request Q = quorum "majority rules" set in stone: Q = N/2 + 1 Tunability when R = W = N =⇒ strong when R + W > N =⇒ quorum where W = 1, R = N or W = N, R = 1 or W = R = Q
- 18. CAP Controls: N, R, W N = number of replicas must be 1 ≤ N ≤ nodes R = number of responding read nodes can be set per request W = number of responding write nodes can be set per request Q = quorum "majority rules" set in stone: Q = N/2 + 1 Tunability when R = W = N =⇒ strong when R + W > N =⇒ quorum where W = 1, R = N or W = N, R = 1 or W = R = Q
- 19. CAP Controls: N, R, W N = number of replicas must be 1 ≤ N ≤ nodes R = number of responding read nodes can be set per request W = number of responding write nodes can be set per request Q = quorum "majority rules" set in stone: Q = N/2 + 1 Tunability when R = W = N =⇒ strong when R + W > N =⇒ quorum where W = 1, R = N or W = N, R = 1 or W = R = Q
- 20. CAP Controls: N, R, W N = number of replicas must be 1 ≤ N ≤ nodes R = number of responding read nodes can be set per request W = number of responding write nodes can be set per request Q = quorum "majority rules" set in stone: Q = N/2 + 1 Tunability when R = W = N =⇒ strong when R + W > N =⇒ quorum where W = 1, R = N or W = N, R = 1 or W = R = Q
- 21. Dynamo: Properties Decentralized no masters exist Homogeneous nodes have same capabilities No Global State no SPOFs for global state Deterministic Replica Placement each node can calculate where replicas should exist for key Logical Time No reliance on physical time
- 22. Dynamo: Properties Decentralized no masters exist Homogeneous nodes have same capabilities No Global State no SPOFs for global state Deterministic Replica Placement each node can calculate where replicas should exist for key Logical Time No reliance on physical time
- 23. Dynamo: Properties Decentralized no masters exist Homogeneous nodes have same capabilities No Global State no SPOFs for global state Deterministic Replica Placement each node can calculate where replicas should exist for key Logical Time No reliance on physical time
- 24. Dynamo: Properties Decentralized no masters exist Homogeneous nodes have same capabilities No Global State no SPOFs for global state Deterministic Replica Placement each node can calculate where replicas should exist for key Logical Time No reliance on physical time
- 25. Dynamo: Properties Decentralized no masters exist Homogeneous nodes have same capabilities No Global State no SPOFs for global state Deterministic Replica Placement each node can calculate where replicas should exist for key Logical Time No reliance on physical time
- 26. Dynamo: Techniques Consistent Hashing Vector Clocks Gossip Protocol Hinted Handoff
- 27. Dynamo: Techniques Consistent Hashing Vector Clocks Gossip Protocol Hinted Handoff
- 28. Dynamo: Consistent hashing Figure: PrefsList for K is [C, D, E]
- 29. Dynamo: Vector clocks -type vclock() :: [{actor(), counter()}] A occurred-before B if counters in A are less than or equal to those in B for each actor
- 30. Dynamo: Gossip protocol Figure: "You would never guess what I heard down the pub..."
- 31. Dynamo: Hinted handoff Figure: "Here, take the baton and take over from me. KTHXBAI"
- 32. riak_core: Abstractions Coordinator enforces consistency requirements, performing anti-entropy, gen_fsm, coordinates vnodes VNodes Erlang process, vnode to hashring partition, delegated work for its partition, unit of replication Watchers gen_event, listens for ring and service events, used to calculate fallback nodes Ring Manager stores local node copy of ring (and cluster?) data Ring Event Handlers notified about ring (and cluster?) changes and broadcasts plus metadata changes
- 33. riak_core: Abstractions Coordinator enforces consistency requirements, performing anti-entropy, gen_fsm, coordinates vnodes VNodes Erlang process, vnode to hashring partition, delegated work for its partition, unit of replication Watchers gen_event, listens for ring and service events, used to calculate fallback nodes Ring Manager stores local node copy of ring (and cluster?) data Ring Event Handlers notified about ring (and cluster?) changes and broadcasts plus metadata changes
- 34. riak_core: Abstractions Coordinator enforces consistency requirements, performing anti-entropy, gen_fsm, coordinates vnodes VNodes Erlang process, vnode to hashring partition, delegated work for its partition, unit of replication Watchers gen_event, listens for ring and service events, used to calculate fallback nodes Ring Manager stores local node copy of ring (and cluster?) data Ring Event Handlers notified about ring (and cluster?) changes and broadcasts plus metadata changes
- 35. riak_core: Abstractions Coordinator enforces consistency requirements, performing anti-entropy, gen_fsm, coordinates vnodes VNodes Erlang process, vnode to hashring partition, delegated work for its partition, unit of replication Watchers gen_event, listens for ring and service events, used to calculate fallback nodes Ring Manager stores local node copy of ring (and cluster?) data Ring Event Handlers notified about ring (and cluster?) changes and broadcasts plus metadata changes
- 36. riak_core: Abstractions Coordinator enforces consistency requirements, performing anti-entropy, gen_fsm, coordinates vnodes VNodes Erlang process, vnode to hashring partition, delegated work for its partition, unit of replication Watchers gen_event, listens for ring and service events, used to calculate fallback nodes Ring Manager stores local node copy of ring (and cluster?) data Ring Event Handlers notified about ring (and cluster?) changes and broadcasts plus metadata changes
- 37. riak_core: Coordinator Figure: Header of a coordinator
- 38. riak_core: Commands -type riak_cmd() :: {verb(), key(), payload()} Implement handle_command/3 clause for each command in your callback VNode module
- 40. riak_core: VNodes Figure: Sample handoff functions from RTS example app
- 41. riak_core: Project Structure rebar ;) also written by the Basho team, makes OTP building and deploying much less painful dependencies add you_app, riak_core, riak_kv, etc. as dependencies to shell project new in 1.0 stuff cluster vs ring membership, riak_pipe, etc.
- 42. riak_core: Project Structure rebar ;) also written by the Basho team, makes OTP building and deploying much less painful dependencies add you_app, riak_core, riak_kv, etc. as dependencies to shell project new in 1.0 stuff cluster vs ring membership, riak_pipe, etc.
- 43. riak_core: Project Structure rebar ;) also written by the Basho team, makes OTP building and deploying much less painful dependencies add you_app, riak_core, riak_kv, etc. as dependencies to shell project new in 1.0 stuff cluster vs ring membership, riak_pipe, etc.
- 44. Considerations Other:stuff() Interface layer riak_core apps need to implement their own interface layer, e.g. HTTP, XMPP, AMQP, MsgPack, ProtoBuffers Securing application riak_core gossip does not address identity/authZ/authN between nodes; relies on Erlang cookies Distribution models e.g. pipelined, laned vs tiered Query/execution models e.g. map-reduce (M/R), ASsociated SET (ASSET)
- 45. Considerations Other:stuff() Interface layer riak_core apps need to implement their own interface layer, e.g. HTTP, XMPP, AMQP, MsgPack, ProtoBuffers Securing application riak_core gossip does not address identity/authZ/authN between nodes; relies on Erlang cookies Distribution models e.g. pipelined, laned vs tiered Query/execution models e.g. map-reduce (M/R), ASsociated SET (ASSET)
- 46. Considerations Other:stuff() Interface layer riak_core apps need to implement their own interface layer, e.g. HTTP, XMPP, AMQP, MsgPack, ProtoBuffers Securing application riak_core gossip does not address identity/authZ/authN between nodes; relies on Erlang cookies Distribution models e.g. pipelined, laned vs tiered Query/execution models e.g. map-reduce (M/R), ASsociated SET (ASSET)
- 47. Considerations Other:stuff() Interface layer riak_core apps need to implement their own interface layer, e.g. HTTP, XMPP, AMQP, MsgPack, ProtoBuffers Securing application riak_core gossip does not address identity/authZ/authN between nodes; relies on Erlang cookies Distribution models e.g. pipelined, laned vs tiered Query/execution models e.g. map-reduce (M/R), ASsociated SET (ASSET)
- 48. Oh, the possibilities! What:next() Concurrency models e.g. actor, disruptor, evented/reactor, threading Consistency models e.g. vector-field, causal, FIFO Computation optimizations e.g. General Purpose GPU programming, native interfacing (NIFs, JInterface, Scalang?) Other optimizations e.g. Client discovery, Remote Direct Memory Access (RDMA)
- 49. Oh, the possibilities! What:next() Concurrency models e.g. actor, disruptor, evented/reactor, threading Consistency models e.g. vector-field, causal, FIFO Computation optimizations e.g. General Purpose GPU programming, native interfacing (NIFs, JInterface, Scalang?) Other optimizations e.g. Client discovery, Remote Direct Memory Access (RDMA)
- 50. Oh, the possibilities! What:next() Concurrency models e.g. actor, disruptor, evented/reactor, threading Consistency models e.g. vector-field, causal, FIFO Computation optimizations e.g. General Purpose GPU programming, native interfacing (NIFs, JInterface, Scalang?) Other optimizations e.g. Client discovery, Remote Direct Memory Access (RDMA)
- 51. Oh, the possibilities! What:next() Concurrency models e.g. actor, disruptor, evented/reactor, threading Consistency models e.g. vector-field, causal, FIFO Computation optimizations e.g. General Purpose GPU programming, native interfacing (NIFs, JInterface, Scalang?) Other optimizations e.g. Client discovery, Remote Direct Memory Access (RDMA)
- 52. # finger $(whoami) Login: susan Name: Susan Potter Directory: /home/susan Shell: /bin/zsh On since Mon 29 Sep 1997 21:18 (GMT) on tty1 from :0 Too much unread mail on me@susanpotter.net Now working at Assistly! Looking for smart developers!;) Plan: github: mbbx6spp twitter: @SusanPotter
- 53. Slides & Material https://github.com/mbbx6spp/riak_core-templates Figure: http://susanpotter.net/talks/strange-loop/2011/dynamo-not-just-for-datastores/
- 54. Examples Rebar Templates https://github.com/mbbx6spp/riak_core-templates Riak Pipes https://github.com/basho/riak_pipe Riak Zab https://github.com/jtuple/riak_zab
- 55. Questions? Figure: http://www.flickr.com/photos/42682395@N04/ @SusanPotter
- 56. Questions? Figure: http://www.flickr.com/photos/42682395@N04/ @SusanPotter