Rapid Development with Schemaless Data Models
Download as PPTX, PDF0 likes1,076 views
The document discusses the MyEdu Profile Project's decision to use MongoDB over MySQL for its user profile data. It chose MongoDB to allow for rapid iteration and continuous change while minimizing downtime for new features. While a schema-less design is harder to maintain, the project employs patterns to control document structure. The profile has since undergone several iterations on MongoDB, demonstrating its ability to scale. Lessons include proper indexing and not thinking of data like MySQL.
Rapid Development with Schemaless Data Models
- 1. Rapid Development with Schema-less Data models The MyEdu Profile Project
- 2. Table of Contents • Technology Stack • What is the MyEdu Profile Project? • MongoDb vs MySQL decision • Version 1 using MongoDB • Mistakes we made along the way • What we have learned along the way • What we are doing next
- 4. The MyEdu Profile Project MyEdu is the leading academic and career network for college students and college recruiters. Millions of students have used MyEdu to graduate in less time, and reduce the cost of earning their degree, and get jobs and internships. Employers use MyEdu to build their brands with highly targeted groups of college students and hire the best talent for their companies.
- 5. We understood that change to the product was inevitable and continuous.
- 6. MongoDb vs MySQL decision • Technical goals for the user profile product – Move fast and iterate often. – Minimize the downtime required to launch new features – Make sure the new product would scale
- 7. MongoDb vs MySQL decision • Our decision to use MongoDB did not come without a lot of debate internally. In the end, the two main arguments were: – “We can do all that in MySQL.” – Schema-less design is going to be hard to maintain.
- 8. MongoDb vs MySQL decision • Argument 1: “We can do all this in MySQL.” – We could have built it in MySQL, but was going to get complicated fast. – Sure, it would start simple.
- 9. MongoDb vs MySQL decision • Well what if a user has more than one
- 10. MongoDb vs MySQL decision • Add more meta data about a user
- 11. MongoDb vs MySQL decision • All the sudden you have this schema and growing:
- 12. MongoDb vs MySQL decision • The result of the web service call to get the user profile would look something like:
- 13. MongoDb vs MySQL decision • The result of the web service call to get the user profile would look something like: Looks like a Mongo document to me
- 14. MongoDb vs MySQL decision • Argument 2: Schema-less design is going to be hard to maintain. • Lucky for us, the web services team had already been employing a Service / Mapper / Model pattern. • With this pattern we are able to control the structure of each mongo document with a defined data model.
- 15. Why we chose MongoDB? • It goes back to our product goals: – Move fast and iterate often. – Minimize the downtime required to launch new features – Make sure the new product would scale.
- 16. The MyEdu profile over the last 3 months • MyEdu Profile Project iteration 1
- 17. The MyEdu profile over the last 3 months • Iteration 2 – Adding more tiles
- 18. The MyEdu profile over the last 3 months • Iteration 3 – Tile Management w/ Duplicate Tile types
- 19. Why we chose MongoDB? • It goes back to our product goals: – Move fast and iterate often. – Minimize the downtime required to launch new features – Make sure the new product would scale.
- 20. A product that can Scale • With proper indexing alone we have over 600k mongo profile documents on just 3 member replica set. • We actually lowered our overall site speed time by 0.3 seconds and lowered our profile product speed 300% seconds when we introduced the MongoDB version.
- 21. Why we chose MongoDB? • It goes back to our product goals: – We want to move fast and iterate often. – We wanted to minimize the downtime required to launch new features – We wanted to make sure the new product would scale.
- 22. Mistakes we have made • Proper Indexing is super important. Bad Index • You have to stop thinking like MySQL.
- 23. What we have learned along the way • A Schema-less Data model has many benefits but the one we have found most useful so far is the ability to rapidly change our data to meet the needs of new product features and requirements. • The document object is a familiar structure for a web developer.
- 24. What we are doing next • Event tracking with MongoDB • Using Json Patch for doing updates