Data Modeling with Cassandra Column Families
- 1. Needle Meet HaystackAdapting your data models for CassandraGary Dusbabek • Rackspace• ICOODB 2010
- 2. OutlineFirst Things FirstColumn FamiliesTrade OffsProcedures & Best PracticesInternals
- 3. It’s all about scalability
- 4. We can all be friends
- 14. 2.TradeOffs
- 15. No Transactions
- 16. No Adhoc Queries
- 17. No Joins
- 19. Don’tPanic!
- 21. 3. Procedures & Practices
- 22. Relational WayDefine entitiesNormalizeIdentify Many-to-manyQuery any way you want
- 24. Cassandra WayKnow your appQueries firstDenormalize
- 25. Know Your App
- 26. Queries First
- 27. Nobody is Normal
- 33. Does it feel strange?
- 34. 4. Internals
- 37. Partitioning
- 39. SummaryThe goal is to scale
- 40. ColumnFamilies != Relational tables
- 41. Trade-offs: you win some, you lose some
- 43. Queries first
- 45. Cassandra was built for thisLinkshttp://cassandra.apache.orghttp://wiki.apache.org/cassandrairc: #cassandra on freenodegdusbabek@gmail.com@gdusbabek
- 46. Image Creditshaystack http://www.flickr.com/photos/james_lumb/3921968993pyramids http://www.flickr.com/photos/gracewong/93631410scales http://www.flickr.com/photos/eflon/3465042138friends http://www.flickr.com/photos/ngmmemuda/4166182931television http://www.flickr.com/photos/angelrravelor/314306023columns http://www.flickr.com/photos/nostri-imago/3564300653devil http://www.flickr.com/photos/52890443@N02/4887855756angel http://www.flickr.com/photos/75001512@N00/4938623021transaction http://www.flickr.com/photos/neubie/2273635564queries http://www.flickr.com/photos/-bast-/349497988rings http://www.flickr.com/photos/baldur/4395738741indexes http://www.flickr.com/photos/waferboard/4137041591panic http://www.flickr.com/photos/pasukaru76/3998981988procedures "The Anatomy Lesson of Dr. NicolaesTuip" by Rembrandtrelational http://www.flickr.com/photos/35536700@N07/3292544674desert http://www.flickr.com/photos/waldenpond/4252575735jet http://www.flickr.com/photos/rmahle/709685queries http://www.flickr.com/photos/andreanna/2812118063blackboard http://www.flickr.com/photos/shonk/418180402normal http://www.flickr.com/photos/infrogmation/3180606117phonograph http://www.flickr.com/photos/shiyazuni/4770244591dodo http://www.flickr.com/photos/wheatfields/2071347416Internals http://www.flickr.com/photos/37hz/4057856826writing http://www.flickr.com/photos/stevendepolo/3877225152consistency http://www.flickr.com/photos/betsyweber/4962297050partitioning http://www.flickr.com/photos/featheredtar/3137028766slices http://www.flickr.com/photos/free-stock/4899674517summary http://www.flickr.com/photos/jkdsphotography/4061838798links http://www.flickr.com/photos/creative_stock/3397559016
Editor's Notes
- #4: It’s not about one data model vs. another.It’s not about one storage engine vs. another.Cassandra excels at replicating data and achieving high sustained write throughput.
- #5: The right tool for the right job
- #6: Shaped by distribution model
- #7: Shaped by distribution model
- #11: Shaped by distribution model
- #12: Sparse – do not have to exist in every row.
- #13: Flexible column namingYou define the sort orderNot required to have a specific column just because another row does
- #14: Look familiar?
- #15: Arise because of distribution model, not CF model.
- #16: * Atomic @ CF row. Not isolated.* Large trans apps push down to node (shared nothing)* Guaranteeing ACID constraints across nodes is a hard problem.
- #19: OTOH, you do get a lot of things:Data redundancyVery fast writes, fast reads
- #20: Relational>formally defined>correctQuery first>not formally defined>somehow incorrectYou get some things in exchange:ScalabilityAvailabilityReplication
- #21: Relational>formally defined>correctQuery first>not formally defined>somehow incorrectYou get some things in exchange:ScalabilityAvailabilityReplication
- #24: Focus on query & analysis.B+treesUpdate once*Cassandra typically becomes IO bound before becoming CPU bound.
- #25: Not set in stone.Your application may require a different approach.
- #26: Recognize non-starters: Is my dataset going to become Very Large? Will I need to sustain high write throughput?Also, what are the common operations? Optimize CFs for those operations.
- #27: *columns sorted. Choose keys and columns.you need to think about how you plan to slice your data.Related data is close to reduce io
- #28: DenormalizeUse the disk.Don’t be afraid to create another CF that duplicates some data.
- #30: Composite column namesPainful updates of denormalized partsFast reads & insertions
- #31: Key
- #32: Normal attributes
- #33: Composite column names.Pulling in relationshipsPainful updates. Denormalization is best when data doesn’t change.
- #36: Commit log – separate diskMemtableSstable