Modeling with Document Database: 5 Key Patterns
Download as PPTX, PDF11 likes2,498 views
This document discusses 5 key data modeling patterns for document databases: 1) One-to-many using embedded documents, 2) Many-to-many using references or embedded documents, 3) Trees using parent and child references, 4) Trees using materialized paths, and 5) Entity aggregation for polymorphic documents. It provides examples of each pattern and considerations for implementing them. The document also covers anti-patterns to avoid, such as large arrays and over-normalizing data.
![Overview
Patterns and Data Modeling
5 Key Patterns
Anti-Patterns to Avoid
Last suggestions
{
--------------
--------------
--------------
{ ------
------ }
}
[ ----------- ]
}
Document
{
--------------
--------------
--------------
{ ------
------ }
}
[ ----------- ]
}
Document
{
--------------
--------------
--------------
{ ------
------ }
}
[ ----------- ]
}
Document
{
--------------
--------------
--------------
{ ------
------ }
}
[ ----------- ]
}
Document](https://image.slidesharecdn.com/sullivan-documentdatamodelingpatterns-150403094656-conversion-gate01/85/Modeling-with-Document-Database-5-Key-Patterns-3-320.jpg)
![Pattern 2: Many-to-Many
Employees
({empID: 1783,
pname: “Michelle”,
lname:”Jones”
projects: [487,973, 287]}
{empID: 9872,
pname: “Bob”,
lname:”Williams”
projects: [487,973, 121]})
Projects
({projID:121,
projName:'NoSQL Pilot'',
team: [9872, 2431,
{projID:487,
projName:'Customer Churn
Analysis'',
team: [1873,9872]})
References
Minimizes redundancy
Preserves integrity
Reduces document growth
Requires multiple reads](https://image.slidesharecdn.com/sullivan-documentdatamodelingpatterns-150403094656-conversion-gate01/85/Modeling-with-Document-Database-5-Key-Patterns-11-320.jpg)
![Pattern 2: Many-to-Many
Employee
{empID: 1783,
pname: “Michelle”,
lname:”Jones”
projects: [
{projID:121,
projName:'NoSQL Pilot''},
{projID:487,
projName:'Customer Churn
Analysis''}
]}
Project
{projID:121,
projName:'NoSQL Pilot'',
team: [
{ empID: 1783,
fname: “Michelle”,
lname:”Jones”},
{ empID: 9872,
fname: “Bob”,
lname:”Williams”}
]}
Embedded Documents
Captures point in time data
One document read retrieves
data
Increases document growth](https://image.slidesharecdn.com/sullivan-documentdatamodelingpatterns-150403094656-conversion-gate01/85/Modeling-with-Document-Database-5-Key-Patterns-12-320.jpg)
![Pattern 3: Trees with References
Children Refs.
({orgUnitID:178,
orgUnitType: “Primary”,
orgUnitName:”P1”
children: [179,180]},
{orgUnitID:179,
orgUnitType: “Branch”,
orgUnitName:”B1”
children: [181,182]},
{orgUnitID:180,
orgUnitType: “Branch”,
orgUnitName:”B2”
children: [183,184]})
Parent Refs.
({orgUnitID:178,
orgUnitType: “Primary”,
orgUnitName:”P1”
parent: 177},
{orgUnitID:179,
orgUnitType: “Branch”,
orgUnitName:”B1”
parent: 178},
{orgUnitID:180,
orgUnitType: “Branch”,
orgUnitName:”B2”
parent: 178})](https://image.slidesharecdn.com/sullivan-documentdatamodelingpatterns-150403094656-conversion-gate01/85/Modeling-with-Document-Database-5-Key-Patterns-16-320.jpg)
![Materialized Paths
{orderItemId: 1873,
prodType: “Pens”
prodHierarchy:[ “Product_Categories”,
”Office Supplies”,
”Writing Instruments”,
”Pens”]
}](https://image.slidesharecdn.com/sullivan-documentdatamodelingpatterns-150403094656-conversion-gate01/85/Modeling-with-Document-Database-5-Key-Patterns-20-320.jpg)
Modeling with Document Database: 5 Key Patterns
- 1. Modeling with Document Databases: 5 Key Patterns Dan Sullivan, Principal DS Applied Technologies Enterprise Data World 2015 Washington, D.C. April 1, 2015
- 2. My Background Data Architect / Engineer NoSQL and relational data modeler Big data Analytics, machine learning and text mining Cloud computing Computational Biologist Author No SQL for Mere Mortals Contributor to TechTarget
- 3. Overview Patterns and Data Modeling 5 Key Patterns Anti-Patterns to Avoid Last suggestions { -------------- -------------- -------------- { ------ ------ } } [ ----------- ] } Document { -------------- -------------- -------------- { ------ ------ } } [ ----------- ] } Document { -------------- -------------- -------------- { ------ ------ } } [ ----------- ] } Document { -------------- -------------- -------------- { ------ ------ } } [ ----------- ] } Document
- 4. Patterns and Data Modeling
- 5. Schema-less <> Model-less Schema-less Document Databases No fixed schema Polymorphic documents ...however, not a Design Free for All Queries drives organization Performance Considerations Long-term Maintenance Middle Ground: Data Model Patterns Reusable methods for organizing data Model is implicit in document structures
- 6. Patterns Commonly used structure and organization Abstraction, not implementation specific Popularized by “Gang of Four” Applied to Relational Databases by David C. Hay Applies to NoSQL
- 7. Patterns Pattern 1: One-to-Many Pattern 2: Many-to-Many Pattern 3: Trees with References Pattern 4: Trees with Materialized Views Pattern 5: Entity Aggregation
- 8. Pattern 1: One-to-Many Embed Documents Multiple documents embedded “Many” attributes stored with “One” document Pros Single fetch returns primary and related data Might improve performance Simplifies application code Cons Increases document size Might degrade performance { OrderID: 1837373, customer : {Name: 'Jane Lox' Addr: '123 Main St' City: 'Boston' State: 'MA'}, orderItem:{ Sku: 38383838, Descr: 'Black chair'}, orderItem:{ Sku: 2872636, Descr: 'Glass desk'}, orderItem:{ Sku: 4747433, Descr: 'USB Drive 32GB''} }
- 9. One-to-Many Considerations Query attributes in embedded documents? Support for indexing embedded documents? Potential for arbitrary growth after record created? Need for atomic writes?
- 10. Patterns Pattern 1: One-to-Many Pattern 2: Many-to-Many Pattern 3: Trees with References Pattern 4: Trees with Materialized Views Pattern 5: Entity Aggregation
- 11. Pattern 2: Many-to-Many Employees ({empID: 1783, pname: “Michelle”, lname:”Jones” projects: [487,973, 287]} {empID: 9872, pname: “Bob”, lname:”Williams” projects: [487,973, 121]}) Projects ({projID:121, projName:'NoSQL Pilot'', team: [9872, 2431, {projID:487, projName:'Customer Churn Analysis'', team: [1873,9872]}) References Minimizes redundancy Preserves integrity Reduces document growth Requires multiple reads
- 12. Pattern 2: Many-to-Many Employee {empID: 1783, pname: “Michelle”, lname:”Jones” projects: [ {projID:121, projName:'NoSQL Pilot''}, {projID:487, projName:'Customer Churn Analysis''} ]} Project {projID:121, projName:'NoSQL Pilot'', team: [ { empID: 1783, fname: “Michelle”, lname:”Jones”}, { empID: 9872, fname: “Bob”, lname:”Williams”} ]} Embedded Documents Captures point in time data One document read retrieves data Increases document growth
- 13. Many-to-Many Considerations References Minimizes redundancy Preserves integrity Reduces document growth Requires multiple reads Embedded Documents Captures point in time data One document read retrieves data Increases document growth
- 14. Patterns Pattern 1: One-to-Many Pattern 2: Many-to-Many Pattern 3: Trees with References Pattern 4: Trees with Materialized Views Pattern 5: Entity Aggregation
- 15. Pattern 3: Trees with Parent & Child References Trees Single root document At most one parent No cycles Multiple Types Is-A Part-of
- 16. Pattern 3: Trees with References Children Refs. ({orgUnitID:178, orgUnitType: “Primary”, orgUnitName:”P1” children: [179,180]}, {orgUnitID:179, orgUnitType: “Branch”, orgUnitName:”B1” children: [181,182]}, {orgUnitID:180, orgUnitType: “Branch”, orgUnitName:”B2” children: [183,184]}) Parent Refs. ({orgUnitID:178, orgUnitType: “Primary”, orgUnitName:”P1” parent: 177}, {orgUnitID:179, orgUnitType: “Branch”, orgUnitName:”B1” parent: 178}, {orgUnitID:180, orgUnitType: “Branch”, orgUnitName:”B2” parent: 178})
- 17. Tree Considerations Children reference allow for top-down navigation Parent references allow for- bottom up navigation Combination allow for bottom-up and top-down navigation Avoid large arrays Consider need for point in time data
- 18. Patterns Pattern 1: One-to-Many Pattern 2: Many-to-Many Pattern 3: Trees with References Pattern 4: Trees with Materialized Views Pattern 5: Entity Aggregation
- 19. Pattern 4: Trees with Materialized Paths Full path from document to root is represented in document Implement with arrays or string Especially useful in hierarchical queries, i.e. a type and all its subtypes
- 20. Materialized Paths {orderItemId: 1873, prodType: “Pens” prodHierarchy:[ “Product_Categories”, ”Office Supplies”, ”Writing Instruments”, ”Pens”] }
- 21. Materialized Paths Considerations Support for multi-key indexing of arrays Use of regular expressions for pattern matching when string is used Ability to utilize indexes when string representation
- 22. Patterns Pattern 1: One-to-Many Pattern 2: Many-to-Many Pattern 3: Trees with References Pattern 4: Trees with Materialized Views Pattern 5: Entity Aggregation
- 23. Pattern 5: Entity Aggregation Entities with sub-types Relational models use multiple tables Document models use varying embedded documents Source of polymorphism
- 24. Entity Aggregation Polymorphic Documents {concertID: 132, locDescr:'Small Bar', price:”$30.00”, performerName:”Rolling Stones”} {concertID: 133, locDescr: “PDX Jazz Festival” price:”$75.00” festivalStart: 15-Feb festivalEnd: 25-Feb}
- 25. Entity Aggregation Considerations Aggregation vs Separate Collections High level branching Low level branching
- 26. Anti-Patterns
- 27. Anti-Patterns Large arrays Significant growth in document size Fetching more data than needed Fear of data duplication Thinking SQL, using NoSQL Normalizing without need
- 28. Closing Remarks Consider, is it worth it to loose: SQL Multi-statement transactions Triggers Let queries drive model Consider full life-cycle Exploit polymorphism
- 29. Questions?