Back tobasicswebinar part6-rev.
Download as PPTX, PDF5 likes1,398 views
This document provides an overview and agenda for a webinar on deploying MongoDB applications in production. It discusses replica sets, including their lifecycle and how they provide high availability and redundancy. It covers topics like write concern, tagging, and read preference modes when using replica sets. The document also discusses scaling MongoDB by sharding the database for large datasets that exceed a single server's resources. It provides examples of how data is partitioned and distributed across shards in a sharded cluster.
![{
_id : "mySet",
members : [
{_id : 0, host : "A", tags : {"dc": "ny"}},
{_id : 1, host : "B", tags : {"dc": "ny"}},
{_id : 2, host : "C", tags : {"dc": "sf"}},
{_id : 3, host : "D", tags : {"dc": "sf"}},
{_id : 4, host : "E", tags : {"dc": "cloud"}}],
settings : {
getLastErrorModes : {
allDCs : {"dc" : 3},
someDCs : {"dc" : 2}} }
}
> db.blogs.insert({...})
> db.runCommand({getLastError : 1, w : "someDCs"})
Tagging Example](https://image.slidesharecdn.com/backtobasicswebinar-part6-rev-140408100040-phpapp02/85/Back-tobasicswebinar-part6-rev-26-320.jpg)
Back tobasicswebinar part6-rev.
- 3. Solution Architect, MongoDB Sam Weaver #MongoDBBasics ‘Build an Application’Webinar Series Deploying your application in production
- 4. Agenda • Replica Sets Lifecycle • Developing with Replica Sets • Scaling your database
- 5. Q&A • Virtual Genius Bar – Use chat to post questions – EMEASolution Architecture / Support Team are on hand – Make use of them during the sessions!!!
- 6. Recap • Introduction to MongoDB • Schema design • Interacting with the database • Indexing • Analytics – Map Reduce – Aggregation Framework
- 8. Working Set Exceeds Physical Memory
- 9. Why Replication? • How many have faced node failures? • How many have been woken up from sleep to do a fail-over(s)? • How many have experienced issues due to network latency? • Different uses for data – Normal processing – Simple analytics
- 11. Replica Set – Creation
- 12. Replica Set – Initialize
- 13. Replica Set – Failure
- 14. Replica Set – Failover
- 15. Replica Set – Recovery
- 16. Replica Set – Recovered
- 20. Write Concern • Network acknowledgement • Wait for error • Wait for journal sync • Wait for replication
- 21. Unacknowledged
- 22. MongoDB Acknowledged (wait for error)
- 23. Wait for Journal Sync
- 25. Tagging • Control where data is written to, and read from • Each member can have one or more tags – tags: {dc: "ny"} – tags: {dc: "ny", subnet: "192.168", rack: "row3rk7"} • Replica set defines rules for write concerns • Rules can change without changing app code
- 26. { _id : "mySet", members : [ {_id : 0, host : "A", tags : {"dc": "ny"}}, {_id : 1, host : "B", tags : {"dc": "ny"}}, {_id : 2, host : "C", tags : {"dc": "sf"}}, {_id : 3, host : "D", tags : {"dc": "sf"}}, {_id : 4, host : "E", tags : {"dc": "cloud"}}], settings : { getLastErrorModes : { allDCs : {"dc" : 3}, someDCs : {"dc" : 2}} } } > db.blogs.insert({...}) > db.runCommand({getLastError : 1, w : "someDCs"}) Tagging Example
- 27. Wait for Replication (Tagging)
- 28. Read Preference Modes • 5 modes – primary (only) - Default – primaryPreferred – secondary – secondaryPreferred – Nearest When more than one node is possible, closest node is used for reads (all modes but primary)
- 29. Tagged Read Preference • Custom read preferences • Control where you read from by (node) tags – E.g. { "disk": "ssd", "use": "reporting" } • Use in conjunction with standard read preferences – Except primary
- 30. • SAFE writes acceptable for our use case • Potential to use secondary reads for comments, but probably not needed • Use tagged reads for analytics Our application
- 31. Scaling
- 32. Working Set Exceeds Physical Memory
- 33. • When a specific resource becomes a bottle neck on a machine or replica set • RAM • Disk IO • Storage • Concurrency When to consider Sharding?
- 34. Vertical Scalability (Scale Up)
- 35. Horizontal Scalability (Scale Out)
- 36. Partitioning • User defines shard key • Shard key defines range of data • Key space is like points on a line • Range is a segment of that line
- 37. Initially 1 chunk Default max chunk size: 64mb MongoDB automatically splits & migrates chunks when max reached Data Distribution
- 38. Architecture
- 39. What is a Shard? • Shard is a node of the cluster • Shard can be a single mongod or a replica set
- 40. Meta Data Storage • Config Server – Stores cluster chunk ranges and locations – Can have only 1 or 3 (production must have 3) – Not a replica set
- 41. Routing and Managing Data • Mongos – Acts as a router / balancer – No local data (persists to config database) – Can have 1 or many
- 43. Cluster Request Routing • Targeted Queries • Scatter Gather Queries • Scatter Gather Queries with Sort
- 44. Cluster Request Routing: Targeted Query
- 46. Request routed to appropriate shard
- 48. Mongos returns results to client
- 49. Cluster Request Routing: Non-Targeted Query
- 51. Request sent to all shards
- 52. Shards return results to mongos
- 53. Mongos returns results to client
- 54. Cluster Request Routing: Non-Targeted Query with Sort
- 55. Non-Targeted request with sort received
- 56. Request sent to all shards
- 57. Query and sort performed locally
- 58. Shards return results to mongos
- 59. Mongos merges sorted results
- 60. Mongos returns results to client
- 61. Shard Key
- 62. Shard Key • Shard key is immutable • Shard key values are immutable • Shard key must be indexed • Shard key limited to 512 bytes in size • Shard key used to route queries – Choose a field commonly used in queries • Only shard key can be unique across shards – `_id` field is only unique within individual shard
- 63. A suitable shard key for our app… • Occurs in most queries • Routes to each shard • Is granular enough to not exceed 64MB chunks • Any candidates? – Author? – Date? – _id? – Title? – Author & Date?
- 64. Summary
- 65. Things to remember • Size appropriately for your working set • Shard when you need to, not before • Pick a shard key wisely
- 66. Next Session – 17th April • Backup and Disaster Recovery • Backup and restore options
- 67. Thank you
Editor's Notes
- #9: Indexes should be contained in working set.
- #12: Basic explanation2 or more nodes form the setQuorum
- #13: Initialize -> ElectionPrimary + data replication from primary to secondary
- #14: Primary down/network failureAutomatic election of new primary if majority exists
- #15: New primary electedReplication established from new primary
- #16: Down node comes upRejoins setsRecovery and then secondary
- #18: ConsistencyWrite preferencesRead preferences
- #22: Not really fire and forget. This return arrow is to confirm that the network successfully transferred the packet(s) of data.This confirms that the TCP ACK response was received.
- #25: Presenter should mention:Default is w:1w:majority is what most people should use for durability. Majority is a special token here signifying more than half of the nodes in the set have acknowledged the write.
- #27: Using 'someDCs' so that in the event of an outage, at least a majority of the DCs would receive the change. This favors availability over durability.
- #28: Using 'allDCs' because we want to make certain all DCs have this piece of data. If any of the DCs are down, this would timeout. This favors durability over availability.
- #33: Indexes should be contained in working set.
- #35: From mainframes, to RAC Oracle servers... People solved problems by adding more resources to a single machine.
- #36: Large scale operation can be combined with high performance on commodity hardware through horizontal scalingBuild - Document oriented database maps perfectly to object oriented languagesScale - MongoDB presents clear path to scalability that isn't ops intensive - Provides same interface for sharded cluster as single instance
- #60: The mongos does not have to load the whole set into memory since each shard sorts locally. The mongos can just getMore from the shards as needed and incrementally return the results to the client.
- #63: _id could be unique across shards if used as shard key.we could only guarantee uniqueness of (any) attributes if the keys are used as shard keys with unique attribute equals true