MongoDB London 2013: Basic Replication in MongoDB presented by Marc Schwering, 10gen
Download as PPTX, PDF0 likes841 views
The document discusses MongoDB replication and replica sets. It covers the lifecycle of replica sets including creation, initialization, failure, and recovery. It also discusses replica set roles and configuration options. Additionally, it addresses considerations for developing with replica sets like strong/delayed consistency, write concerns, tagging, and read preferences. Finally, it discusses operational considerations like maintenance/upgrades and different replica set deployment architectures.
![> conf = {
_id : "mySet",
members : [
{_id : 0, host : "A”, priority : 3},
{_id : 1, host : "B", priority : 2},
{_id : 2, host : "C”},
{_id : 3, host : "D", hidden : true},
{_id : 4, host : "E", hidden : true, slaveDelay : 3600}
]
}
> rs.initiate(conf)
Configuration Options](https://image.slidesharecdn.com/4-basicreplicationinmongodbmarcschwering-130517121152-phpapp02/85/MongoDB-London-2013-Basic-Replication-in-MongoDB-presented-by-Marc-Schwering-10gen-13-320.jpg)
![> conf = {
_id : "mySet”,
members : [
{_id : 0, host : "A”, priority : 3},
{_id : 1, host : "B", priority : 2},
{_id : 2, host : "C”},
{_id : 3, host : "D", hidden : true},
{_id : 4, host : "E", hidden : true, slaveDelay : 3600}
]
}
> rs.initiate(conf)
Configuration Options
Primary DC](https://image.slidesharecdn.com/4-basicreplicationinmongodbmarcschwering-130517121152-phpapp02/85/MongoDB-London-2013-Basic-Replication-in-MongoDB-presented-by-Marc-Schwering-10gen-14-320.jpg)
![> conf = {
_id : "mySet”,
members : [
{_id : 0, host : "A”, priority : 3},
{_id : 1, host : "B", priority : 2},
{_id : 2, host : "C”},
{_id : 3, host : "D", hidden : true},
{_id : 4, host : "E", hidden : true, slaveDelay : 3600}
]
}
> rs.initiate(conf)
Configuration Options
Secondary DC
Default Priority = 1](https://image.slidesharecdn.com/4-basicreplicationinmongodbmarcschwering-130517121152-phpapp02/85/MongoDB-London-2013-Basic-Replication-in-MongoDB-presented-by-Marc-Schwering-10gen-15-320.jpg)
![> conf = {
_id : "mySet”,
members : [
{_id : 0, host : "A”, priority : 3},
{_id : 1, host : "B", priority : 2},
{_id : 2, host : "C”},
{_id : 3, host : "D", hidden : true},
{_id : 4, host : "E", hidden : true, slaveDelay : 3600}
]
}
> rs.initiate(conf)
Configuration Options
Analytics
node](https://image.slidesharecdn.com/4-basicreplicationinmongodbmarcschwering-130517121152-phpapp02/85/MongoDB-London-2013-Basic-Replication-in-MongoDB-presented-by-Marc-Schwering-10gen-16-320.jpg)
![> conf = {
_id : "mySet”,
members : [
{_id : 0, host : "A”, priority : 3},
{_id : 1, host : "B", priority : 2},
{_id : 2, host : "C”},
{_id : 3, host : "D", hidden : true},
{_id : 4, host : "E", hidden : true, slaveDelay : 3600}
]
}
> rs.initiate(conf)
Configuration Options
Backup node](https://image.slidesharecdn.com/4-basicreplicationinmongodbmarcschwering-130517121152-phpapp02/85/MongoDB-London-2013-Basic-Replication-in-MongoDB-presented-by-Marc-Schwering-10gen-17-320.jpg)
![{
_id : "mySet",
members : [
{_id : 0, host : "A", tags : {"dc": "ny"}},
{_id : 1, host : "B", tags : {"dc": "ny"}},
{_id : 2, host : "C", tags : {"dc": "sf"}},
{_id : 3, host : "D", tags : {"dc": "sf"}},
{_id : 4, host : "E", tags : {"dc": "cloud"}}],
settings : {
getLastErrorModes : {
allDCs : {"dc" : 3},
someDCs : {"dc" : 2}} }
}
> db.blogs.insert({...})
> db.runCommand({getLastError : 1, w : "someDCs"})
Tagging Example](https://image.slidesharecdn.com/4-basicreplicationinmongodbmarcschwering-130517121152-phpapp02/85/MongoDB-London-2013-Basic-Replication-in-MongoDB-presented-by-Marc-Schwering-10gen-27-320.jpg)
MongoDB London 2013: Basic Replication in MongoDB presented by Marc Schwering, 10gen
- 1. Solutions Architect, 10gen Marc Schwering #MongoDBDays - @m4rcsch Replication and Replica Sets
- 2. Agenda • Replica Sets Lifecycle • Developing with Replica Sets • Operational Considerations
- 3. Why Replication? • How many have faced node failures? • How many have been woken up from sleep to do a fail-over(s)? • How many have experienced issues due to network latency? • Different uses for data – Normal processing – Simple analytics
- 5. Replica Set – Creation
- 6. Replica Set – Initialize
- 7. Replica Set – Failure
- 8. Replica Set – Failover
- 9. Replica Set – Recovery
- 10. Replica Set – Recovered
- 13. > conf = { _id : "mySet", members : [ {_id : 0, host : "A”, priority : 3}, {_id : 1, host : "B", priority : 2}, {_id : 2, host : "C”}, {_id : 3, host : "D", hidden : true}, {_id : 4, host : "E", hidden : true, slaveDelay : 3600} ] } > rs.initiate(conf) Configuration Options
- 14. > conf = { _id : "mySet”, members : [ {_id : 0, host : "A”, priority : 3}, {_id : 1, host : "B", priority : 2}, {_id : 2, host : "C”}, {_id : 3, host : "D", hidden : true}, {_id : 4, host : "E", hidden : true, slaveDelay : 3600} ] } > rs.initiate(conf) Configuration Options Primary DC
- 15. > conf = { _id : "mySet”, members : [ {_id : 0, host : "A”, priority : 3}, {_id : 1, host : "B", priority : 2}, {_id : 2, host : "C”}, {_id : 3, host : "D", hidden : true}, {_id : 4, host : "E", hidden : true, slaveDelay : 3600} ] } > rs.initiate(conf) Configuration Options Secondary DC Default Priority = 1
- 16. > conf = { _id : "mySet”, members : [ {_id : 0, host : "A”, priority : 3}, {_id : 1, host : "B", priority : 2}, {_id : 2, host : "C”}, {_id : 3, host : "D", hidden : true}, {_id : 4, host : "E", hidden : true, slaveDelay : 3600} ] } > rs.initiate(conf) Configuration Options Analytics node
- 17. > conf = { _id : "mySet”, members : [ {_id : 0, host : "A”, priority : 3}, {_id : 1, host : "B", priority : 2}, {_id : 2, host : "C”}, {_id : 3, host : "D", hidden : true}, {_id : 4, host : "E", hidden : true, slaveDelay : 3600} ] } > rs.initiate(conf) Configuration Options Backup node
- 21. Write Concern • Network acknowledgement • Wait for error • Wait for journal sync • Wait for replication
- 22. Unacknowledged
- 23. MongoDB Acknowledged (wait for error)
- 24. Wait for Journal Sync
- 26. Tagging • New in 2.0.0 • Control where data is written to, and read from • Each member can have one or more tags – tags: {dc: "ny"} – tags: {dc: "ny", subnet: "192.168", rack: "row3rk7"} • Replica set defines rules for write concerns • Rules can change without changing app code
- 27. { _id : "mySet", members : [ {_id : 0, host : "A", tags : {"dc": "ny"}}, {_id : 1, host : "B", tags : {"dc": "ny"}}, {_id : 2, host : "C", tags : {"dc": "sf"}}, {_id : 3, host : "D", tags : {"dc": "sf"}}, {_id : 4, host : "E", tags : {"dc": "cloud"}}], settings : { getLastErrorModes : { allDCs : {"dc" : 3}, someDCs : {"dc" : 2}} } } > db.blogs.insert({...}) > db.runCommand({getLastError : 1, w : "someDCs"}) Tagging Example
- 28. Wait for Replication (Tagging)
- 29. Read Preference Modes • 5 modes (new in 2.2) – primary (only) - Default – primaryPreferred – secondary – secondaryPreferred – Nearest When more than one node is possible, closest node is used for reads (all modes but primary)
- 30. Tagged Read Preference • Custom read preferences • Control where you read from by (node) tags – E.g. { "disk": "ssd", "use": "reporting" } • Use in conjunction with standard read preferences – Except primary
- 32. Maintenance and Upgrade • No downtime • Rolling upgrade/maintenance – Start with Secondary – Primary last
- 33. Replica Set – 1 Data Center • Single datacenter • Single switch & power • Points of failure: – Power – Network – Data center – Two node failure • Automatic recovery of single node crash
- 34. Replica Set – 2 Data Centers • Multi data center • DR node for safety • Can’t do multi data center durable write safely since only 1 node in distant DC
- 35. Replica Set – 3 Data Centers • Three data centers • Can survive full data center loss • Can do w= { dc : 2 } to guarantee write in 2 data centers (with tags)
- 36. Recent improvements • Read preference support with sharding – Drivers too • Improved replication over WAN/high-latency networks • rs.syncFrom command • buildIndexes setting • replIndexPrefetch setting
- 37. Just Use It • Use replica sets • Easy to setup – Try on a single machine • Check doc page for RS tutorials – http://docs.mongodb.org/manual/replication/#tutorials
- 38. Solutions Architect, 10gen Marc Schwering #MongoDBDays - @m4rcsch Thank You
Editor's Notes
- #7: Basic explanation2 or more nodes form the setQuorum
- #8: Initialize -> ElectionPrimary + data replication from primary to secondary
- #9: Primary down/network failureAutomatic election of new primary if majority exists
- #10: New primary electedReplication established from new primary
- #11: Down node comes upRejoins setsRecovery and then secondary
- #14: PrimaryData memberSecondaryHot standbyArbitersVoting member
- #15: PriorityFloating point number between 0..1000Highest member that is up to date wins Up to date == within 10 seconds of primaryIf a higher priority member catches up, it will force election and win Slave DelayLags behind master by configurable time delay Automatically hidden from clientsProtects against operator errorsFat fingeringApplication corrupts data
- #16: PriorityFloating point number between 0..1000Highest member that is up to date wins Up to date == within 10 seconds of primaryIf a higher priority member catches up, it will force election and win Slave DelayLags behind master by configurable time delay Automatically hidden from clientsProtects against operator errorsFat fingeringApplication corrupts data
- #17: PriorityFloating point number between 0..1000Highest member that is up to date wins Up to date == within 10 seconds of primaryIf a higher priority member catches up, it will force election and win Slave DelayLags behind master by configurable time delay Automatically hidden from clientsProtects against operator errorsFat fingeringApplication corrupts data
- #18: Analytics good for integrations with Hadoop, Storm, etc.
- #19: PriorityFloating point number between 0..1000Highest member that is up to date wins Up to date == within 10 seconds of primaryIf a higher priority member catches up, it will force election and win Slave DelayLags behind master by configurable time delay Automatically hidden from clientsProtects against operator errorsFat fingeringApplication corrupts data
- #20: ConsistencyWrite preferencesRead preferences
- #24: Not really fire and forget. This return arrow is to confirm that the network successfully transferred the packet(s) of data.This confirms that the TCP ACK response was received.
- #27: Presenter should mention:Default is w:1w:majority is what most people should use for durability. Majority is a special token here signifying more than half of the nodes in the set have acknowledged the write.
- #29: Using 'someDCs' so that in the event of an outage, at least a majority of the DCs would receive the change. This favors availability over durability.
- #30: Using 'allDCs' because we want to make certain all DCs have this piece of data. If any of the DCs are down, this would timeout. This favors durability over availability.
- #33: Upgrade/maintenanceCommon deployment scenarios
- #37: A good question to ask the audience : 'Why wouldn't you set w={dc:3}'… Why would you ever do that? What would be the complications?
- #38: Schemaoplog
- #42: rs.syncFromallows administrators to configure the member of a replica set that the current member will pull data from. Specify the name of the member you want to sync from in the form of [hostname]:[port].Replica Set Members will not Sync from Members Without Indexes Unless buildIndexes: falseTo prevent inconsistency between members of replica sets, if the member of a replica set has members[n].buildIndexes set to true, other members of the replica set will not sync from this member, unless they also have members[n].buildIndexes set to true. See SERVER-4160 for more information.New Option To Configure Index Pre-Fetching during ReplicationBy default, when replicating options, secondaries will pre-fetch Indexes associated with a query to improve replication throughput in most cases. The replIndexPrefetch setting and --replIndexPrefetch option allow administrators to disable this feature or allow the mongod to pre-fetch only the index on the _id field. See SERVER-6718 for more information.