MongoDB.local Seattle 2019: Global Cluster Topologies in MongoDB Atlas
Download as PPTX, PDF0 likes260 views
The document discusses strategies for effectively utilizing MongoDB Atlas to manage global cluster topologies, emphasizing the importance of low latency and user engagement for over 3 billion new internet users annually. It outlines key factors such as global multi-region replication, read preferences, and the use of location attributes for data optimization. Additionally, it describes the path to scalability and global reach through incremental steps in deployment and application architecture.
![Data includes location attribute
{
_id: 123,
firstName: "Johannes",
lastName: "Doe",
activities: [
{
title: "Skiing",
attributes: ["downhill", "jumps"]
},
{
title: "Coding",
attributes: ["mongodb", "python"]
}
]
}
{
_id: 123,
location: "DE",
firstName: "Johannes",
lastName: "Doe",
activities: [
{
title: "Skiing",
attributes: ["downhill", "jumps"]
},
{
title: "Coding",
attributes: ["mongodb", "python"]
}
]
}](https://image.slidesharecdn.com/405pmglobalclustertopologiesinmongodbatlas-signarvaez-mongodb-190326143017/85/MongoDB-local-Seattle-2019-Global-Cluster-Topologies-in-MongoDB-Atlas-25-320.jpg)
![Data includes location attribute
{
_id: 123,
firstName: "Johannes",
lastName: "Doe",
activities: [
{
title: "Skiing",
attributes: ["downhill", "jumps"]
},
{
title: "Coding",
attributes: ["mongodb", "python"]
}
]
}
{
_id: 123,
location: "DE",
firstName: "Johannes",
lastName: "Doe",
activities: [
{
title: "Skiing",
attributes: ["downhill", "jumps"]
},
{
title: "Coding",
attributes: ["mongodb", "python"]
}
]
}](https://image.slidesharecdn.com/405pmglobalclustertopologiesinmongodbatlas-signarvaez-mongodb-190326143017/85/MongoDB-local-Seattle-2019-Global-Cluster-Topologies-in-MongoDB-Atlas-28-320.jpg)
![For example location and _id
{
_id: 123,
location: "CA",
firstName: "Jane",
lastName: "Doe",
activities: [
{
title: "Mountaineering",
attributes: ["glaciers"]
},
{
title: "Coding",
attributes: ["java", "mongodb"]
}
]
}](https://image.slidesharecdn.com/405pmglobalclustertopologiesinmongodbatlas-signarvaez-mongodb-190326143017/85/MongoDB-local-Seattle-2019-Global-Cluster-Topologies-in-MongoDB-Atlas-32-320.jpg)
![Query routing without
LATAM zone
{
location:
{$in: [ 'BR-PE', 'BR-PA']}
}](https://image.slidesharecdn.com/405pmglobalclustertopologiesinmongodbatlas-signarvaez-mongodb-190326143017/85/MongoDB-local-Seattle-2019-Global-Cluster-Topologies-in-MongoDB-Atlas-34-320.jpg)
![Get ISO Country code from HTTP Request
AWS – Lambda & CloudFront
• headers['cloudfront-viewer-country’]
• https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-examples.html#lambda-examples-redirecting-examples
Google Cloud –Functions
• country: req.headers["x-appengine-country"]
• region: req.headers["x-appengine-region"]
• https://cloud.google.com/appengine/docs/standard/go/reference/request-response-headers
• https://medium.com/mop-developers/free-ip-based-geolocation-with-google-cloud-functions-f92e20d47651
Azure – GeoLocation service (preview)
• IsoCode (country)
• https://docs.microsoft.com/en-us/rest/api/maps/geolocation/getiptolocationpreview
CloudFlare – Enable IP Geolocation
• HTTP_CF_IPCOUNTRY header
Akamai – EdgeScape (CDN)
• country_code and region_code headers](https://image.slidesharecdn.com/405pmglobalclustertopologiesinmongodbatlas-signarvaez-mongodb-190326143017/85/MongoDB-local-Seattle-2019-Global-Cluster-Topologies-in-MongoDB-Atlas-35-320.jpg)
![Query routing with
LATAM zone
{
location:
{$in: [ 'BR-PE', 'BR-PA']}
}](https://image.slidesharecdn.com/405pmglobalclustertopologiesinmongodbatlas-signarvaez-mongodb-190326143017/85/MongoDB-local-Seattle-2019-Global-Cluster-Topologies-in-MongoDB-Atlas-41-320.jpg)
MongoDB.local Seattle 2019: Global Cluster Topologies in MongoDB Atlas
- 2. Global Clusters Topologies Sig Narvaez Principal Solution Architect, MongoDB
- 3. 3.15 Billion new internet users since the first MongoDB code commit
- 4. Every year a US-sized population is coming online for the first time
- 5. You don’t have to be a household name to meaningfully reach them
- 6. Your users want… Interactive experiences Personalization Instant gratification Global Operational Data You want… Happy users To stay ahead of competition Global awareness
- 7. What does 100ms equate to?
- 8. What does 100ms equate to? 1% drop in sales Missed chance to engage a customer meaningfully
- 9. Global Latencies from AWS US West Americas US West – US East: 83ms
- 10. Global Latencies from AWS US West Americas US West – US East: 83ms EMEA US West – Frankfurt: 163ms LATAM US West – Sao Paulo: 190ms APAC US West – Singapore: 225ms
- 11. Options
- 14. Secondary Secondary Primary US West Frankfurt Singapore
- 15. Secondary Primary Frankfurt Frankfurt Application Driver US West Application Driver Singapore Application Driver EMEA Users Americas Users APAC Users Global Load Balancer Secondary US West Singapore
- 16. Global Multi-Region Replication Single connection string globally Cross-region high availability Advantages Disadvantages All writes go to preferred region ⅔ of users experience > 100ms Local reads require readPreference: nearest Nearest == primary consistent Nearest == secondary eventually consistent ok, or not, depending on the use case
- 18. Frankfurt Application Driver US West Application Driver Singapore Application Driver EMEA Users Americas Users APAC Users Global Load Balancer Secondary Secondary Primary Frankfurt SingaporeUS West Secondary Secondary Primary Secondary Secondary Primary
- 19. Cluster Per Region Easy today in MongoDB Atlas Independently scalable Advantages Disadvantages No global aggregation Application tier manages end user to connection string affinity
- 20. Can we get the best of both worlds?
- 21. Global Clusters with Global Writes
- 24. How Global Writes work
- 25. Data includes location attribute { _id: 123, firstName: "Johannes", lastName: "Doe", activities: [ { title: "Skiing", attributes: ["downhill", "jumps"] }, { title: "Coding", attributes: ["mongodb", "python"] } ] } { _id: 123, location: "DE", firstName: "Johannes", lastName: "Doe", activities: [ { title: "Skiing", attributes: ["downhill", "jumps"] }, { title: "Coding", attributes: ["mongodb", "python"] } ] }
- 26. Generally... an ISO 3166-1 country code e.g., US, DE, IN, or JP Or optionally and more specifically... an ISO-3166-2 subdivision code e.g., US-CA or US-NY Examples Description Potential Region Affinity US United States US East DE Germany Frankfurt IN India Mumbai JP Japan Tokyo US-CA California US West US-NY New York US East What is a location attribute?
- 27. US West Application Driver mongos mongos••• z Repl0 Repl1 Repl2 US West Zone Shard Frankfurt Application Driver Singapore Application Driver mongos Americas Users EMEA Users APAC Users Global Load Balancer MongoDB Atlas Global Cluster Repl0 Repl1 Repl2 Frankfurt Zone Shard Repl0 Repl1 Repl2 Singapore Zone Shard ••• z Repl0 Repl1 Repl2 Config Servers
- 28. Data includes location attribute { _id: 123, firstName: "Johannes", lastName: "Doe", activities: [ { title: "Skiing", attributes: ["downhill", "jumps"] }, { title: "Coding", attributes: ["mongodb", "python"] } ] } { _id: 123, location: "DE", firstName: "Johannes", lastName: "Doe", activities: [ { title: "Skiing", attributes: ["downhill", "jumps"] }, { title: "Coding", attributes: ["mongodb", "python"] } ] }
- 29. US West Application Driver mongos mongos••• z Repl0 Repl1 Repl2 US West Zone Shard Frankfurt Application Driver Singapore Application Driver mongos Americas Users EMEA Users APAC Users Global Load Balancer MongoDB Atlas Global Cluster Repl0 Repl1 Repl2 Frankfurt Zone Shard Repl0 Repl1 Repl2 Singapore Zone Shard ••• z Repl0 Repl1 Repl2 Config Servers Repl0 Repl1 Repl2 Frankfurt Zone Shard
- 31. MongoDB Atlas will be very opinionated... In the Data Explorer, you’ll be guided to set a shard key like {location : 1, <identifier> : 1} You’ll receive global-targeting query examples db.globalColl.findOne({ location : “US-WA”, <identifier> : 123, ... })
- 32. For example location and _id { _id: 123, location: "CA", firstName: "Jane", lastName: "Doe", activities: [ { title: "Mountaineering", attributes: ["glaciers"] }, { title: "Coding", attributes: ["java", "mongodb"] } ] }
- 33. How are locations mapped to zones?
- 34. Query routing without LATAM zone { location: {$in: [ 'BR-PE', 'BR-PA']} }
- 35. Get ISO Country code from HTTP Request AWS – Lambda & CloudFront • headers['cloudfront-viewer-country’] • https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-examples.html#lambda-examples-redirecting-examples Google Cloud –Functions • country: req.headers["x-appengine-country"] • region: req.headers["x-appengine-region"] • https://cloud.google.com/appengine/docs/standard/go/reference/request-response-headers • https://medium.com/mop-developers/free-ip-based-geolocation-with-google-cloud-functions-f92e20d47651 Azure – GeoLocation service (preview) • IsoCode (country) • https://docs.microsoft.com/en-us/rest/api/maps/geolocation/getiptolocationpreview CloudFlare – Enable IP Geolocation • HTTP_CF_IPCOUNTRY header Akamai – EdgeScape (CDN) • country_code and region_code headers
- 36. Provide a better experience to growing markets
- 38. When should I add a Zone? If latency of complete request exceeds threshold? Not really – There are many layers contributing to the latency outside of Atlas Metrics in Atlas? Maybe – performance metrics Maybe – mongos logs – cross-reference IP against IP-GeoLocation DB / Service It’s beyond Atlas - A combination of: • Data Set growing from certain countries & sub-divisions Use Aggregation Framework, BI Connector or Spark Connector for Analytics • CDN / API Management Metrics: AWS CloudFront / API Gateway, Akamai, etc. • Planned Expansion: New features targeting specific markets
- 39. When should I add a Zone? IP Origins? cross-reference client IP’s from mongos logs vs. IP-GeoLocation DB / Service It’s beyond Atlas - A combination of: • Data Set growing from certain countries & sub-divisions Use Aggregation Framework, BI Connector or Spark Connector for Analytics • CDN / API Management Metrics: AWS CloudFront / API Gateway, Akamai, etc. • Planned Expansion: New features targeting specific markets
- 41. Query routing with LATAM zone { location: {$in: [ 'BR-PE', 'BR-PA']} }
- 42. Chunk Migration – adding LATAM zone Chunk migration from Americas to LATAM
- 44. NA West Application Driver ••• z Repl0 Repl1 Repl2 NA West Zone Shard mongos NA East EMEA APAC North Global Load Balancer MongoDB Atlas Global Cluster NA West South America APAC South NA East Application Driver z Repl0 Repl1 Repl2 NA East Zone Shard mongos South America Application Driver z Repl0 Repl1 Repl2 South America Zone Shard mongos EMEA Application Driver z Repl0 Repl1 Repl2 EMEA Zone Shard mongos APAC North Application Driver z Repl0 Repl1 Repl2 APAC North Zone Shard mongos APAC South Application Driver z Repl0 Repl1 Repl2 APAC South Zone Shard mongos••• ••• ••• •••
- 46. Concise and global, it will look like Your driver will use this address to discover every mongos in the cluster globally Taking advantage of the MongoDB 3.6 driver support for SRV records … so you can deploy the same application code everywhere - symmetrically mongodb+srv://user:pass@globalCluster-rzeyq.mongodb.net
- 47. Resolving SRV record to mongos’s Blog Post: https://www.mongodb.com/blog/post/mongodb-3-6-here-to-SRV-you-with-easier-replica-set-connections $ python mongodb_srv_records.py globalcluster-twijg.mongodb.net globalcluster-shard-00-00-twijg.mongodb.net:27016 globalcluster-shard-00-01-twijg.mongodb.net:27016 globalcluster-shard-00-02-twijg.mongodb.net:27016 globalcluster-shard-01-00-twijg.mongodb.net:27016 globalcluster-shard-01-01-twijg.mongodb.net:27016 globalcluster-shard-01-02-twijg.mongodb.net:27016 globalcluster-shard-02-00-twijg.mongodb.net:27016 globalcluster-shard-02-01-twijg.mongodb.net:27016 globalcluster-shard-02-02-twijg.mongodb.net:27016 globalcluster-shard-03-00-twijg.mongodb.net:27016 globalcluster-shard-03-01-twijg.mongodb.net:27016 globalcluster-shard-03-02-twijg.mongodb.net:27016 globalcluster-twijg.mongodb.net: "authSource=admin"
- 48. An even better user experience?
- 49. What if I want low latency global reads?
- 50. What if I want low latency global reads?
- 51. What if I want low latency global reads?
- 52. Where this is going and limits
- 53. Massive reach MongoDB Atlas • in-country regions for 27% of the world population • with a much higher percent served by regions in nearby countries Available in 56 global cloud regions • AWS: 15 regions (38 availability zones) • Azure: 26 regions (26 availability zones w/ 60 Fault Domains) • GCP: 15 regions (38 Availability zones)
- 54. Africa • S. Africa announced by Azure; a long way from most of Africa’s population, however • Biggest economies in Africa: Nigeria, South Africa, Egypt; with Ethiopia now the fastest growing Middle East • We’ll see China • It’s complicated... There are gaps Russia • It’s also complicated… • Frankfurt remains best bet but also Finland on Google Cloud Special / Government / Compliance contexts
- 55. But MongoDB runs everywhere Over time the entire world will be well served by major cloud infrastructure You can always run your own MongoDB outside of where the big public cloud • MongoDB Ops Manager / MongoDB Cloud Manager • Live Migrate to MongoDB Atlas when ready So you can have a seamless development experience in any context
- 56. Imagine 2030
- 57. 2030 Projection GDP & relative position compared with today 1 China ↑ 17 Spain ↓ 2 United States ↓ 18 Canada ↓ 3 India ↑↑↑↑ 19 Egypt ↑↑↑ 4 Japan ↓ 20 Pakistan ↑↑↑ 5 Indonesia ↑↑↑ 21 Nigeria ↑↑ 6 Russia ↑ 22 Thailand ↑↑ 7 Germany ↓ 23 Australia ↓ 8 Brazil ↑ 24 Philippines ↑↑ 9 Mexico ↑ 25 Malaysia ↑↑ 10 United Kingdom ↓ 26 Poland ↓ 11 France ↓ 27 Argentina ↓ 12 Turkey ↑ 28 Bangladesh ↑↑↑ 13 Saudi Arabia ↑↑ 29 Vietnam ↑↑↑ 14 South Korea ↓ 30 South Africa ↑↑ 15 Italy ↓ 31 Colombia ↑↑ 16 Iran ↑↑ 32 Netherlands ↓
- 58. You can go global incrementally
- 59. Path to Global Over Time 1. Cluster – Single Region ← deploy today for free on Atlas ○ Focus on finding traction ○ Scale up when needed in a few minutes on Atlas
- 60. Path to Global Over Time 1. Cluster – Single Region ← deploy today for free on Atlas ○ Focus on finding traction ○ Scale up when needed in a few minutes on Atlas 2. Cluster – Multi Region HA ○ Get your application HA as well ○ Scale up when needed in a few minutes on Atlas
- 61. Path to Global Over Time 1. Cluster – Single Region ← deploy today for free on Atlas ○ Focus on finding traction ○ Scale up when needed in a few minutes on Atlas 2. Cluster – Multi Region HA ○ Get your application HA as well ○ Scale up when needed in a few minutes on Atlas 3. Cluster – add Global Read Regions ○ Route users to nearby application instance; use secondary reads ○ Scale up when needed in a few minutes on Atlas
- 62. Path to Global Over Time 1. Cluster – Single Region ← deploy today for free on Atlas ○ Focus on finding traction ○ Scale up when needed in a few minutes on Atlas 2. Cluster – Multi Region HA ○ Get your application HA as well ○ Scale up when needed in a few minutes on Atlas 3. Cluster – add Global Read Regions ○ Route users to nearby application instance; use secondary reads ○ Scale up when needed in a few minutes on Atlas 4. Enable Global Writes ○ Add a second write zone ○ Add location attribute to your data & shard your global collections in the Atlas UI
- 63. Path to Global Over Time 1. Cluster – Single Region ← deploy today for free on Atlas ○ Focus on finding traction ○ Scale up when needed in a few minutes on Atlas 2. Cluster – Multi Region HA ○ Get your application HA as well ○ Scale up when needed in a few minutes on Atlas 3. Cluster – add Global Read Regions ○ Route users to nearby application instance; use secondary reads ○ Scale up when needed in a few minutes on Atlas 4. Enable Global Writes ○ Add a second write zone ○ Add location attribute to your data & shard your global collections in the Atlas UI 5. Expand Reach of Global Cluster ○ Add more write zones ○ Or add shards to existing zones ○ Scale up any time
- 64. And you can follow your users...
- 65. Now go out and reach those 4+ billion internet users
- 66. Thanks
- 67. Sigfrido “Sig” Narvaez Principal Solution Architect, MongoDB sig@mongodb.com