Austin NoSQL 2011-07-06
- 1. How to choose the right NoSQL Technology Josh Butts - 7/6/2011
- 2. Agenda • Spoilers • Evaluation criteria • Winners & Losers • Gotchas • Questions & Discussion
- 3. Spoilers
- 4. Spoilers • We picked MongoDB and Redis
- 5. Spoilers • We picked MongoDB and Redis • We love Redis to death
- 6. Spoilers • We picked MongoDB and Redis • We love Redis to death • We’re in a platonic relationship with MongoDB
- 7. Spoilers • We picked MongoDB and Redis • We love Redis to death • We’re in a platonic relationship with MongoDB • Maybe one day it can be something more
- 9. Where to start? • First, why are you even considering this?
- 10. Where to start? • First, why are you even considering this? • Very, very clearly define the problem
- 11. Where to start? • First, why are you even considering this? • Very, very clearly define the problem • You already know SQL. Are you sure it isn’t the right choice?
- 13. Building Criteria • Make a list of real criteria
- 14. Building Criteria • Make a list of real criteria • These should be points which will help you differentiate
- 15. Building Criteria • Make a list of real criteria • These should be points which will help you differentiate • Hint: “Fast, flexible, good language support” are not useful
- 16. Building Criteria • Make a list of real criteria • These should be points which will help you differentiate • Hint: “Fast, flexible, good language support” are not useful • These are the common denominator
- 17. Use case #1 at Offers.com
- 18. Use case #1 at Offers.com • Log impressions of content on web pages
- 19. Use case #1 at Offers.com • Log impressions of content on web pages • Light caching
- 20. Use case #1 at Offers.com • Log impressions of content on web pages • Light caching • Data de-normalization
- 21. Our requirements
- 22. Our requirements • Reads & writes need to be blazing fast
- 23. Our requirements • Reads & writes need to be blazing fast • Data persistence is reasonably important
- 24. Our requirements • Reads & writes need to be blazing fast • Data persistence is reasonably important • Need to implement incrementally
- 25. The contenders
- 26. The contenders • In-memory key-value stores
- 27. The contenders • In-memory key-value stores • Memcached
- 28. The contenders • In-memory key-value stores • Memcached • Membase
- 29. The contenders • In-memory key-value stores • Memcached • Membase • Redis
- 30. The contenders • In-memory key-value stores • Memcached • Membase • Redis • Tokyo Tyrant
- 31. Why Redis
- 32. Why Redis • Redis is damn cool
- 33. Why Redis • Redis is damn cool • In memory + disk storage
- 34. Why Redis • Redis is damn cool • In memory + disk storage • As fast as Memcached
- 35. Why Redis • Redis is damn cool • In memory + disk storage • As fast as Memcached • “Remote data structures”
- 36. Why Redis • Redis is damn cool • In memory + disk storage • As fast as Memcached • “Remote data structures” • This is where the magic happens
- 38. Extras with Redis • Most libraries provide a session handler
- 39. Extras with Redis • Most libraries provide a session handler • Super-easy to install
- 40. Extras with Redis • Most libraries provide a session handler • Super-easy to install • Ability to treat it like memcached
- 41. Extras with Redis • Most libraries provide a session handler • Super-easy to install • Ability to treat it like memcached • memory model not quite the same
- 42. Extras with Redis • Most libraries provide a session handler • Super-easy to install • Ability to treat it like memcached • memory model not quite the same • In fact, not the same at all
- 43. Why not the others?
- 44. Why not the others? • Memcached has no disk persistance
- 45. Why not the others? • Memcached has no disk persistance • I can’t rely on the data (easily)
- 46. Why not the others? • Memcached has no disk persistance • I can’t rely on the data (easily) • Membase seems pointless-ish (compared to redis) for small-to- medium scale
- 47. Use Case #2 at Offers.com
- 48. Use Case #2 at Offers.com • Process (diff / update) 10 million SKU-level items from 100 partners
- 49. Use Case #2 at Offers.com • Process (diff / update) 10 million SKU-level items from 100 partners • Do this once per day
- 50. Use Case #2 at Offers.com • Process (diff / update) 10 million SKU-level items from 100 partners • Do this once per day • Deal with disparate data schemas
- 51. Use Case #2 at Offers.com • Process (diff / update) 10 million SKU-level items from 100 partners • Do this once per day • Deal with disparate data schemas • Deal with missing / null data nicely
- 52. The Contenders
- 53. The Contenders • Document-oriented Databases
- 54. The Contenders • Document-oriented Databases • MongoDB
- 55. The Contenders • Document-oriented Databases • MongoDB • CouchDB
- 56. The Contenders • Document-oriented Databases • MongoDB • CouchDB • Cassandra
- 57. Why Documents?
- 58. Why Documents? • Data arrives in various XML formats
- 59. Why Documents? • Data arrives in various XML formats • No one likes XML except Java devs
- 60. Why Documents? • Data arrives in various XML formats • No one likes XML except Java devs • Need flexible schema
- 61. Why Documents? • Data arrives in various XML formats • No one likes XML except Java devs • Need flexible schema • Need searching / querying by various fields
- 62. Why MongoDB
- 64. Why MongoDB • Fast • Outstanding connectivity
- 65. Why MongoDB • Fast • Outstanding connectivity • Schema free
- 66. Why MongoDB • Fast • Outstanding connectivity • Schema free • Good community support
- 67. Why MongoDB • Fast • Outstanding connectivity • Schema free • Good community support • Ad-hoc queries
- 68. Why MongoDB • Fast • Outstanding connectivity • Schema free • Good community support • Ad-hoc queries • Extremely low learning curve
- 69. Why MongoDB • Fast • Outstanding connectivity • Schema free • Good community support • Ad-hoc queries • Extremely low learning curve • They sent us stickers
- 70. Why not the others
- 71. Why not the others • We’re pretty sure Cassandra is just a Facebook troll project
- 72. Why not the others • We’re pretty sure Cassandra is just a Facebook troll project • CouchDB is brilliant but queries can be tedious without a plan
- 74. Downsides of Mongo • I still don’t trust that my data is 100% safe
- 75. Downsides of Mongo • I still don’t trust that my data is 100% safe • Aggregates and map/reduce are cumbersome compared to CouchDB
- 76. Downsides of Mongo • I still don’t trust that my data is 100% safe • Aggregates and map/reduce are cumbersome compared to CouchDB • Lack of good GUI tools for exploring data
- 77. Where we failed
- 78. Where we failed • We didn’t clearly articulate the problem
- 79. Where we failed • We didn’t clearly articulate the problem • We built a brilliant product catalog parser
- 80. Where we failed • We didn’t clearly articulate the problem • We built a brilliant product catalog parser • We forgot it had to get to MySQL eventually
- 81. Why is this harder than a traditional RDBMS?
- 82. Why is this harder than a traditional RDBMS? • Vendor lock in (model)
- 83. Why is this harder than a traditional RDBMS? • Vendor lock in (model) • Vendor lock in (feature set)
- 84. Why is this harder than a traditional RDBMS? • Vendor lock in (model) • Vendor lock in (feature set) • Vendor lock in (fanboyism)
- 85. How does SQL solve this?
- 86. How does SQL solve this? • ANSI SQL
- 87. How does SQL solve this? • ANSI SQL • Adapter patterns
- 88. How does SQL solve this? • ANSI SQL • Adapter patterns • Abstraction layers
- 89. How does SQL solve this? • ANSI SQL • Adapter patterns • Abstraction layers • ORMs
- 90. With NoSQL, you must choose!
- 91. With NoSQL, you must choose! • There’s no useful way to abstract NoSQL systems to the point of code portability
- 92. With NoSQL, you must choose! • There’s no useful way to abstract NoSQL systems to the point of code portability • ODMs might ease the pain
- 93. Questions?
Editor's Notes
- #2: \n
- #3: \n
- #4: \n
- #5: \n
- #6: \n
- #7: \n
- #8: \n
- #9: \n
- #10: \n
- #11: \n
- #12: \n
- #13: \n
- #14: \n
- #15: \n
- #16: \n
- #17: \n
- #18: \n
- #19: \n
- #20: \n
- #21: \n
- #22: \n
- #23: \n
- #24: \n
- #25: \n
- #26: \n
- #27: \n
- #28: \n
- #29: \n
- #30: \n
- #31: \n
- #32: \n
- #33: \n
- #34: \n
- #35: \n
- #36: \n
- #37: \n
- #38: \n
- #39: \n
- #40: \n
- #41: \n
- #42: \n
- #43: \n
- #44: \n
- #45: \n
- #46: \n
- #47: \n
- #48: \n
- #49: \n
- #50: \n
- #51: \n
- #52: \n
- #53: \n
- #54: \n
- #55: \n
- #56: \n
- #57: \n
- #58: \n
- #59: \n
- #60: \n
- #61: \n
- #62: \n
- #63: \n
- #64: \n
- #65: \n
- #66: \n
- #67: \n
- #68: \n
- #69: \n
- #70: \n
- #71: \n
- #72: \n
- #73: \n
- #74: \n
- #75: \n