Building a Social Network with MongoDB
1 like3,785 views
This document discusses how Eventbrite built a social network using MongoDB to power social recommendations on their site. It describes how user, event, and order data is stored in MongoDB collections to enable querying neighbors and recommended events. Challenges around the dynamic and flexible nature of the social graph and performance needs are also outlined. The system provides recommendations by fetching a user's neighbors, their events, and scoring each possible event.
![User Data in MongoDB
13
{ "_id": 4558992,
"events" : {
"all_ids": [ 116706, 179487, 16389, 827496 ],
"curr_ids": [ 827496 ],
},
}
Past and current
attendance
Friday, December 3, 2010](https://image.slidesharecdn.com/eventbrite-mongosv-social-graph-120822011050-phpapp02-170405174034/85/Building-a-Social-Network-with-MongoDB-13-320.jpg)
![User Data in MongoDB
14
{ "_id": 4558992,
"events" : {
"all_ids": [ 116706, 179487, 16389, 827496 ],
"curr_ids": [ 827496 ],
},
"nns" : [
[ 2816442, 0.2 ],
[ 1615962, 0.047619047619047616 ],
],
} Nearest neighbors
(user_id, score)
Friday, December 3, 2010](https://image.slidesharecdn.com/eventbrite-mongosv-social-graph-120822011050-phpapp02-170405174034/85/Building-a-Social-Network-with-MongoDB-14-320.jpg)
![User Data in MongoDB
15
{ "_id": 4558992,
"events" : {
"all_ids": [ 116706, 179487, 16389, 827496 ],
"curr_ids": [ 827496 ],
},
"nns" : [
[ 2816442, 0.2 ],
[ 1615962, 0.047619047619047616 ],
],
"fb" : {
"_id" : 4808871,
"name" : "Brian Zambrano",
"location" : "San Francisco, California",
"friends" : [ 568876525, 569507467, 569559792 ],
},
}
Facebook data
Friday, December 3, 2010](https://image.slidesharecdn.com/eventbrite-mongosv-social-graph-120822011050-phpapp02-170405174034/85/Building-a-Social-Network-with-MongoDB-15-320.jpg)
![MongoDB Indexes
16
{ "_id": 4558992,
"events" : {
"all_ids": [ 116706, 179487, 16389, 827496 ],
"curr_ids": [ 827496 ],
},
"nns" : [
[ 2816442, 0.2 ],
[ 1615962, 0.047619047619047616 ],
],
"fb" : {
"_id" : 4808871,
"name" : "Brian Zambrano",
"location" : "San Francisco, California",
"friends" : [ 568876525, 569507467, 569559792],
},
}
Friday, December 3, 2010](https://image.slidesharecdn.com/eventbrite-mongosv-social-graph-120822011050-phpapp02-170405174034/85/Building-a-Social-Network-with-MongoDB-16-320.jpg)
![Events Collection
> db.events.findOne({_id: 799177})
{
"_id" : 799177,
"uid" : 2989008,
"title" : "MongoSV",
"venue" : {
"loc" : [
37.413042,
-122.071106
],
"state" : "CA",
"id" : 508093,
"city" : "Mountain View"
},
"logo" : "758915938.png",
"shortname" : "mongosv",
"start_date" : "Fri Dec 03 2010 01:00:00 GMT-0800 (PST)"
}
17
Friday, December 3, 2010](https://image.slidesharecdn.com/eventbrite-mongosv-social-graph-120822011050-phpapp02-170405174034/85/Building-a-Social-Network-with-MongoDB-17-320.jpg)
Building a Social Network with MongoDB
- 1. Building a Social Network with MongoDB Brian Zambrano MongoSV December 3, 2010 1 Friday, December 3, 2010
- 2. Eventbrite Brand Tenets 2 Friday, December 3, 2010
- 3. Eventbrite Brand Tenets 3 Friday, December 3, 2010
- 4. Social Recommendations 4 Friday, December 3, 2010
- 5. Eventbriteʼs Social Graph 5 Friday, December 3, 2010
- 6. Eventbriteʼs Social Graph 6 Friday, December 3, 2010
- 7. Neighbors 7 Friday, December 3, 2010
- 8. Challenges • Dynamic • Neighbors change often • Neighborsʼ events change often • Flexibility • Want to incorporate other social graphs • Product may evolve quickly • Performance • We need really fast reads • Frequent writes 8 Friday, December 3, 2010
- 9. Why MongoDB? • Performance • Flexible schema design • Easy to work with • We felt comfortable MongoDB would mature as our needs became more demanding 9 Friday, December 3, 2010
- 10. Providing Recommendations 1. User visits http://eventbrite.com/mytickets/ 2. Fetch neighbors 3. Fetch neighborsʼ events 4. Score each possible event 5. Return recommendations 10 Friday, December 3, 2010
- 11. MongoDB setup • One non-sharded replica set • Two DBs on Large EC2 instances • One arbiter • Three collections • Users • Events • Orders 11 Friday, December 3, 2010
- 12. User Data in MongoDB 12 { "_id": 4558992, } Unique User Id Friday, December 3, 2010
- 13. User Data in MongoDB 13 { "_id": 4558992, "events" : { "all_ids": [ 116706, 179487, 16389, 827496 ], "curr_ids": [ 827496 ], }, } Past and current attendance Friday, December 3, 2010
- 14. User Data in MongoDB 14 { "_id": 4558992, "events" : { "all_ids": [ 116706, 179487, 16389, 827496 ], "curr_ids": [ 827496 ], }, "nns" : [ [ 2816442, 0.2 ], [ 1615962, 0.047619047619047616 ], ], } Nearest neighbors (user_id, score) Friday, December 3, 2010
- 15. User Data in MongoDB 15 { "_id": 4558992, "events" : { "all_ids": [ 116706, 179487, 16389, 827496 ], "curr_ids": [ 827496 ], }, "nns" : [ [ 2816442, 0.2 ], [ 1615962, 0.047619047619047616 ], ], "fb" : { "_id" : 4808871, "name" : "Brian Zambrano", "location" : "San Francisco, California", "friends" : [ 568876525, 569507467, 569559792 ], }, } Facebook data Friday, December 3, 2010
- 16. MongoDB Indexes 16 { "_id": 4558992, "events" : { "all_ids": [ 116706, 179487, 16389, 827496 ], "curr_ids": [ 827496 ], }, "nns" : [ [ 2816442, 0.2 ], [ 1615962, 0.047619047619047616 ], ], "fb" : { "_id" : 4808871, "name" : "Brian Zambrano", "location" : "San Francisco, California", "friends" : [ 568876525, 569507467, 569559792], }, } Friday, December 3, 2010
- 17. Events Collection > db.events.findOne({_id: 799177}) { "_id" : 799177, "uid" : 2989008, "title" : "MongoSV", "venue" : { "loc" : [ 37.413042, -122.071106 ], "state" : "CA", "id" : 508093, "city" : "Mountain View" }, "logo" : "758915938.png", "shortname" : "mongosv", "start_date" : "Fri Dec 03 2010 01:00:00 GMT-0800 (PST)" } 17 Friday, December 3, 2010
- 18. Orders Collection > db.orders.find({_eid: 799177}) { "_id" : 17464215, "_uid" : 1111195, "_eid" : 799177 } { "_id" : 17575729, "_uid" : 6970539, "_eid" : 799177 } { "_id" : 17582343, "_uid" : 3092687, "_eid" : 799177 } { "_id" : 17588693, "_uid" : 2255017, "_eid" : 799177 } { "_id" : 17589589, "_uid" : 6976917, "_eid" : 799177 } { "_id" : 17601979, "_uid" : 885441, "_eid" : 799177 } { "_id" : 17603085, "_uid" : 2500199, "_eid" : 799177 } { "_id" : 17608289, "_uid" : 6984367, "_eid" : 799177 } { "_id" : 17681965, "_uid" : 628459, "_eid" : 799177 } { "_id" : 17684489, "_uid" : 7017999, "_eid" : 799177 } { "_id" : 17689673, "_uid" : 7020133, "_eid" : 799177 } { "_id" : 17728267, "_uid" : 7036607, "_eid" : 799177 } has more 18 Friday, December 3, 2010
- 19. Recommended Events Query Two + n queries 1. Get neighbors nns = db.users.find({_id : {$in : user.nn_ids}}) 2. Get possible event recommendations: db.events.find({_id : {$in : nns.events.all}}) n.For each event, get total attendee count db.orders.find({_eid : event_id}) 19 Friday, December 3, 2010
- 20. Recommended Events Query Two + n queries 1. Get neighbors nns = db.users.find({_id : {$in : user.nn_ids}}) 2. Get possible event recommendations: db.events.find({_id : {$in : nns.events.all}}) n.For each event, get total attendee count db.orders.find({_eid : event_id}) 20 Optimization opportunity: Embed orders in Event records Friday, December 3, 2010
- 21. Updating Neighbors Two queries, one update 1. Get all orders for a userʼs past events: uids = db.orders.find({_id : {$in : user.events.all}}) 2. Get all neighbors: nns = db.users.find({_id : {$in : uids}}) ➡Score neighbors 3. Update nn_ids db.users.update({_id : uid}, {$set : {nn_ids: nn}}) 21 Friday, December 3, 2010
- 22. Facebook Friendʼs Events Two queries 1. Get FB friends db.users.find({fb._id : {$in : fb.friends}}) 2. Get events FB friends are attending db.events.find({_id : {$in : fb_friends_events}}) 22 Friday, December 3, 2010
- 23. The Future • Incorporate other social networks • Iterate scoring algorithm • Count recommendation impressions 23 Friday, December 3, 2010
- 25. Thanks! Brian Zambrano <brianz@eventbrite.com> Eventbriteʼs new Facebook recommendations power social event discovery: http://bit.ly/gRVS7I Social Commerce: A First Look at the Numbers: http://bit.ly/gXeg9Q 25 Friday, December 3, 2010