Availability and scalability in mongo
Download as PPTX, PDF0 likes703 views
This document provides an overview of availability, scalability, and sharding in MongoDB. It discusses: 1) How replica sets provide availability and failover capabilities within a single data center. 2) The limitations of replica sets across multiple data centers in terms of write availability and consistency. 3) How sharding allows horizontal scaling across multiple servers and data centers to handle increased user and data growth beyond the capabilities of a single server. 4) The key components of a sharded cluster in MongoDB including config servers, mongos routers, shards, and how data is partitioned and distributed based on the shard key.
Availability and scalability in mongo
- 1. SoftwarePeople Md Khairul Anam Introduction to Availability & Scalability in MongoDB
- 2. Availability
- 3. Replica Set â Creation
- 4. Replica Set â Initialize
- 5. Replica Set â Failure
- 6. Replica Set â Failover
- 7. Replica Set â Recovery
- 8. Replica Set â Recovered
- 9. Replica Set Roles âą Heartbeats âą Priority Comparisons âą Optime âą Connections âą Networka Partitions Factors and Conditions that Affect Elections
- 12. Maintenance and Upgrade âą Rolling upgrade/maintenance â Start with Secondary â Primary last
- 13. Replica Set â 1 Data Center Single datacenter Single switch & power Points of failure: â Power â Network â Data center Automatic recovery of single node crash
- 14. Replica Set â 2 Data Centers Multi data center DR node for safety Canât do multi data center durable write safely since only 1 node in distant DC
- 15. Replica Set â 3 Data Centers Three data centers Can survive full data center loss Can do w= { dc : 2 } to guarantee write in 2 data centers
- 16. Questions?
- 17. Scalability
- 18. User Growth â 1995: 0.4% of the worldâs population â Today: 30% of the world is online (~2.2B) Data Set Growth â Facebookâs data set is around 100 petabytes â 4 billion photos taken in the last year (4x a decade ago Examining Growth
- 19. Read/Write Throughput Exceeds I/O Working Set In d e x e s D a t a Working Set Indexes Data Working Set Exceeds Physical Memory Vertical Scalability (Scale Up)
- 20. Horizontal Scalability (Scale Out)
- 21. Custom Hardware â Oracle Custom Software â Facebook + MySQL â Google MongoDB Auto-Sharding Adata store that is â Free â Publiclyavailable â Open Source(https://github.com/mongodb/mongo) â Horizontallyscalable â Applicationindependent Data Store Scalability Solutions
- 23. âą Shard is a node of the cluster âą Shard can be a single mongod or a replica set What is a Shard?
- 24. Config Server â Stores cluster chunk ranges and locations â Can have only 1 or 3 (production must have 3) â Not a replica set Meta Data Storage
- 25. Mongos â Acts as a router / balancer â No local data (persists to config database) â Can have 1 or many Routing and Managing Data
- 26. âą User defines shard key âą Shard key defines range of data âą Key space is like points on a line âą Range is a segment of that line Partitioning
- 27. âą Shard key is used to partition your collection âą Shard key must exist in every document âą Shard key must be indexed âą Shard key is used to route requests to shards What is a Shard Key
- 28. Shards and Shard Keys Shard Shard key range
- 29. âą Initially 1 chunk âą Default max chunk size: 64mb âą MongoDB automatically splits & migrates chunks when max reached Data Distribution
- 30. âą Targeted Queries âą Scatter Gather Queries âą Scatter Gather Queries with Sort Cluster Request Routing
- 31. Questions?
- 32. Thank You
Editor's Notes
- #4: Basic explanation 2 or more nodes form the set Quorum
- #5: Initialize -> Election Primary + data replication from primary to secondary
- #6: Primary down/network failure Automatic election of new primary if majority exists
- #7: New primary elected Replication established from new primary
- #8: Down node comes up Rejoins sets Recovery and then secondary
- #10: Primary Data member Secondary Hot standby Arbiters Voting member
- #16: A good question to ask the audience : 'Why wouldn't you set w={dc:3}'⊠Why would you ever do that? What would be the complications?
- #18: Consistency Write preferences Read preferences
- #20: A good question to ask the audience : 'Why wouldn't you set w={dc:3}'⊠Why would you ever do that? What would be the complications?