Data Modeling Tricks for Neo4j
2 likes658 views
Max De Marzi is a Neo4j Field Engineer who provides tips and tricks for graph data modeling. The document discusses modeling concepts like: - Modeling with a property graph data model where nodes can have labels, properties, and relationships to other nodes. - How traditional relational databases struggle with relationships due to joins slowing performance as data grows, whereas graph databases can traverse relationships in real-time. - Examples of modeling different domains as graphs including acting, flights, twitter, forms, and supply chains. - Key advantages of graph databases over relational and other NoSQL databases for modeling flexible, high performance relationships between entities.
Data Modeling Tricks for Neo4j
- 1. Graph Data Modeling Tips and Tricks
- 2. github.com/maxdemarzi About 200 public repositories Max De Marzi Neo4j Field Engineer About Me ! 01 02 03 04 maxdemarzi.com @maxdemarzi About 160 blog posts
- 3. • Property Graph Data Model • The most important Slide about Neo4j you will ever see • Basics of Modeling • Modeling Acting • Modeling Flights • Modeling Twitter • Modeling Forms • Modeling Chains Agenda
- 5. Property Graph Model Components Nodes • Relate nodes by type and direction • Can have Properties • Can have Labels • Can have Properties name:”Dan” born: May 29, 1970 twitter:”@dan” name:”Ann” born: Dec 5, 1975 Since: Jan 10, 2011 brand: “Volvo” model: “V70” Car LOVES LIVES_WITH D RIVES Person O W NS Relationships Person
- 6. What you (probably) already know:
- 8. Joins are executed every time you query the relationship Executing a Join means to search for a key B-Tree Index: O(log(n)) Your data grows by 10x, your time goes up by one step on each Join More Data = More Searches Slower Performance The Problem 1 2 3 4
- 9. Relational Databases can’t handle Relationships Degraded Performance Speed plummets as data grows and as the number of joins grows Wrong Language SQL was built with Set Theory in mind, not Graph Theory Not Flexible New types of data and relationships require schema redesign Wrong Model They cannot model or store relationships without complexity1 2 3 4
- 10. NoSQL Databases can’t handle Relationships Degraded Performance Speed plummets as you try to join data together in the application Wrong Languages Lots of wacky “almost sql” languages terrible at “joins” Not ACID Eventually Consistent means Eventually Corrupt Wrong Model They cannot model or store relationships without complexity1 2 3 4
- 11. Same Data, Different Layout No more Tables, no more Foreign Keys, no more Joins
- 12. Double Linked List Relationship Layout
- 14. The Most Important Slide about Neo4j you will ever see
- 15. Fixed Sized Records “Joins” on Creation Spin Spin Spin through this data structure Pointers instead of Lookups 1 2 3 4
- 17. Swedes
- 18. Partitions Each Node’s relationships are partitioned by type and direction.
- 19. Real-Time Query Performance Relational and Other NoSQL Databases ResponseTime Connectedness and Size of Data Set 0 to 2 hops 0 to 3 degrees Few connections 5+ hops 3+ degrees Thousands of connections 1000x Advantage “Minutes to milliseconds” Neo4j
- 20. I don’t know the average height of all hollywood actors, but I do know the Six Degrees of Kevin Bacon But not for every query
- 21. Reimagine your Data as a Graph Better Performance Query relationships in real time Right Language Cypher was purpose built for Graphs Flexible and Consistent Evolve your schema seamlessly while keeping transactions Right Model Graphs simplify how you think 1 2 3 4 Agile, High Performance and Scalable without Sacrifice
- 23. Neo4j Property Graph The Whiteboard Model is the Physical Model A unified view for ultimate agility • Easily understood • Easily evolved • Easy collaboration between business and IT
- 24. http://www.apcjones.com/arrows/# It’s simple, it’s fast, it doesn’t run out of ink. Arrows
- 25. Nodes can have many relationships
- 26. Two Nodes can be related by more than one relationship type
- 27. Two Nodes can be related by the same relationship type more than once
- 28. Relationships can have properties
- 29. Don’t overcomplicate your model
- 30. Pick one or the other, not both
- 31. …but this is fine:
- 32. Modeling Acting
- 33. Just draw stuff and “walla” there is your data model Graphs are Whiteboard Friendly
- 34. Movie Property Graph Some Models are Easy
- 36. Should Roles be their own Node? Some Models are Easy but not for all Questions
- 37. Modeling Flights
- 38. Airports Nodes with Flying To Relationships How do you model Flight Data?
- 39. Maybe Flight should be its own Node? How do you model Flight Data?
- 40. Don’t we care about Flights only on particular Days? How do you model Flight Data?
- 41. What is this trick with the date in the relationship type? How do you model Flight Data?
- 42. We don’t need Airports if we model this way! How do you model Flight Data?
- 44. Group Destinations together! How do you model Flight Data?
- 45. OMG WAT! How do you model Flight Data?
- 46. Do not try and bend the data. That’s impossible.
- 47. Modeling Twitter
- 48. Cloning Twitter Building a News Feed 9:00 am @hipster This is what I had for breakfast! <Insert Image of squirrel food> 8:30 am @neo4j Automated tweet telling me about Graph Connect 2020 in NYC on Apr 20-22 8:12 am @ex-coworker Stuff I no longer care about. 8:03 am @someguy Inspirational Quote of the Day
- 49. How do others do it? Cloning Twitter
- 50. The Wrong Way Modeling a Twitter Feed
- 51. A Better Way Modeling a Twitter Feed
- 52. Fixed Sized Records “Joins” on Creation Spin Spin Spin through this data structure Pointers instead of Lookups 1 2 3 4 Neo4j Secret Sauce Yet Again
- 53. Make the Queries Scale …and the database scales with them. …and that’s why we don’t make any money.
- 54. SCALING OUT IS IN FASHION But when your model and your query match you don’t have to.
- 55. Modeling Forms
- 56. Fluid Schema How many arms does the T1000 have in this picture?
- 57. No more “Wall Sized” Schema Diagrams
- 58. For all forms, all fields, all values you’ll ever need. Any Form
- 59. Wait a minute • What if they fill out lots of forms? • What if they respond to the same form twice? • What if responses to forms are linked together? • Are we missing a concept?
- 60. Responses to Forms For all forms, all fields, all values you’ll ever need.
- 61. Modeling Chains
- 62. USE ISSUES Terminal ATM- skimming Data Breach Card Holder Card Issuer Fraudster USE $5MAKES $10 MAKES $2 MAKES MAKES $4000 AT Testing Merchants ATMAKES Tx
- 63. Tx $2000 TxTx Tx Tx TxTxTxTx Tx Tx Computer Store John
- 64. Tx $2000 TxTx Tx Tx TxTxTxTx Tx Tx Computer Store John
- 65. Tx $2000 TxTx Tx Tx TxTxTxTx Tx Tx Computer Store John
- 67. TxTx $2 TxTx Tx $2000 Tx Tx $25$10$4 TxTx Tx Tx TxTxTx Computer Store John Gas Station Sheila Robert $3 Karen TxTxTx Tx Tx TxTx $3000 Tx Jewelry StoreTx $3 TxTxTx Tx Tx TxTx TxTx TxTx TxTx Tx Tx TxTx $8 $12 Tx $1500 Furniture Store Tx Tx Tx
- 68. Hunger Games
- 69. 1. What is the Tool Max uses for Modeling? 2. Up to how many relationship types can I use? A. 8192 B. 32768 C. 65536 3. What is the secret to Neo4j? Questions
- 70. Thank you!