MongoDB at MercadoLibre
4 likes1,499 views
This document discusses MercadoLibre's use of MongoDB. It provides an overview of MeLi's platform which receives 20 million requests per minute. It then discusses why MongoDB was chosen, highlighting its capabilities for horizontal scaling, high availability, performance and flexible data. Several use cases at MeLi are described such as storing social, orders, feedback and item data. Lessons learned are shared around storage, virtual machines, monitoring and planning actions. The document concludes that MeLi loves using MongoDB.
MongoDB at MercadoLibre
- 1. MongoDB At MercadoLibre Wh at h ave we le a rned
- 2. Overview About MeLi Why MongoDB? Use cases Lessons Learned Wrapping Up
- 4. MeLi - Facts! 20M requests / minute 50K requests / sec (peaks) 4 Gb bandwidth / sec
- 5. MeLi - Facts! 20M requests / minute 50K requests / sec (peaks) 4 Gb bandwidth / sec Our farm +1000 physical server +6000 virtual instances
- 6. MeLi - Inside Lot of fun here playing with new stuff :P
- 7. MeLi - Inside Lot of fun here playing with new stuff :P Our NoSQL stack Redis Memcached Hadoop
- 8. MeLi - Inside Lot of fun here playing with new stuff :P Our NoSQL stack Redis Memcached Hadoop .... and of course
- 9. Why MongoDB? Horizontal Scaling High Availability Performance Flexible Data!
- 10. Why MongoDB? Horizontal Scaling High Availability Performance Flexible Data!
- 11. Why MongoDB? (... and more) JavaScript(‘ing) Polyglot Open Source Awesome Community Support
- 12. Support :-)
- 13. Use Cases DB Documents Social +700M Orders +23M Feedback +3M Emails +61M Items +200M Logging +332M
- 14. Use Case: Social Store Meli’s users social data LOT of data (friends, likes, ...) > 500 Gb > 700 M documents 12 shards servers : 4 replica set Each replica set = 3 servers 80Gb per instance
- 15. Use Case: Social Sp ec @T ial ioB thk or st Store Meli’s users social data rac o ho LOT of data (friends, likes, ...) > 500 Gb > 700 M documents 12 shards servers : 4 replica set Each replica set = 3 servers 80Gb per instance
- 16. Use Case: Items 1 Item!
- 17. Use Case: Items 1 Item!
- 18. Use Case: Items 1 Item!
- 19. Use Case: Items 1 Item! N x N x N ... > 100k rows
- 20. Use Case: Items 1 Item!
- 21. Use Case: Logging Capped Collections! Info logs RabbitMQ messages Track certain use cases Errors logs Something bad happen... Recuperable errors with retry handler
- 22. Lessons Learned Storage File System NSF: Bad choice, Poor results Ext4: Far much better Mount Options Reducing Disk IO by mounting partitions with noatime and nodiratime Track disk performance iostat -xm 2 (%util column < 100%)
- 23. Lessons Learned VM’s Shard instances 4 cores >= 16 Gb ram Config server low cpu consumption 1Gb ram / 1Gb storage Mongos runs along with the webserver (client) 1 mongos x webserver
- 24. Lessons Learned Monitoring Disk usage (with zenoss) Ping health check curl --connect-timeout 3 --write-out %{http_code} --silent --output /dev/null http://localhost:28017/_status
- 25. Lessons Learned Monitoring MMS - It’s free!
- 26. Lessons Learned Plan your actions! Shard key Upgrades Cfg Servers + bad decisions = Lot of rework ...
- 27. Lessons Learned Plan your actions! Shard key Upgrades Cfg Servers + bad decisions = Lot of rework ... Study Think Plan Hands On!
- 28. Wrapping Up! Meli loves MongoDB!
- 29. The End Questions? { name : "Pablo Molnar", title : "Software Engineer", work : "MercadoLibre", location : "Buenos Aires", twitter : "@pablomolnar" }