The Connected Data Imperative: Why Graphs? at Neo4j GraphDay New York City
4 likes713 views
The document outlines the agenda for a conference in New York City on April 18, 2017, focused on the transformative impact of graph databases, particularly Neo4j, across various industries. It discusses the evolution of data processing, highlighting the transition from traditional databases to graph databases, and presents key advantages of graph technology including real-time performance and enhanced agility for businesses. Additionally, it identifies five key financial graphs and their applications within organizations to improve analytics and decision-making.
![37
Example HR Query in SQL The Same Query using Cypher
MATCH (boss)-[:MANAGES*0..3]->(sub),
(sub)-[:MANAGES*1..3]->(report)
WHERE boss.name = “John Doe”
RETURN sub.name AS Subordinate,
count(report) AS Total
Project Impact
Less time writing queries
• More time understanding the answers
• Leaving time to ask the next question
Less time debugging queries:
• More time writing the next piece of code
• Improved quality of overall code base
Code that’s easier to read:
• Faster ramp-up for new project members
• Improved maintainability & troubleshooting
Key Ingredient #2 of 3:
A Productive and Powerful Graph Query Language](https://image.slidesharecdn.com/neo4jgraphdaynewyorkintrotalk-20170418ppr-final-170419170830/85/The-Connected-Data-Imperative-Why-Graphs-at-Neo4j-GraphDay-New-York-City-37-320.jpg)
The Connected Data Imperative: Why Graphs? at Neo4j GraphDay New York City
- 1. NEW YORK CITY April 18, 2017 09:00-09:30 09:30-10:15 10:15-11:00 11:00-11:30 11:30-12:30 12:30-13:30 13:30-17:00 Breakfast and Registration The Connected Data Imperative: Why Graphs Transform Your Data: A worked example Break Enterprise Ready: A Look at Neo4j in Production Lunch Training Session Agenda
- 2. Use of Graphs has created some of the most successful companies in the world C 34,3%B 38,4%A 3,3% D 3,8% 1,8% 1,8% 1,8% 1,8% 1,8% E 8,1% F 3,9%
- 4. The Connected Data Imperative: Neo4j in the Enterprise SOFTWARE FINANCIAL SERVICES RETAIL MEDIA & OTHER SOCIAL NETWORKS TELECOM HEALTHCARE
- 5. Takeaways for Today 1. Where graphs databases fit into your existing IT portfolio 2. What are others doing with graphs, particularly in Financial Services? 3. How can you use graphs to advance your own business
- 6. Latency & Freshness Function of your technology Batch- Precompute Real-Time Connectedness Function of your data & question Illustration by David Somerville based on the original by Hugh McLeod (@gapingvoid) Evolutions in Data Processing
- 7. Illustration by David Somerville based on the original by Hugh McLeod (@gapingvoid) Evolutions in Data Processing
- 8. Illustration by David Somerville based on the original by Hugh McLeod (@gapingvoid) Evolutions in Data Processing Phase I: “Data”
- 9. Data Management in 1979 Paper Forms Tiny RAM Spinning Platters (Low Capacity / Sequential IO)
- 10. The RDBMS Era (1979 - Present)
- 11. Key-Value, Column-Family, Document Database Aggregate-Oriented* NoSQL DBMSs The NoSQL Era (Circa 2009 - Present) Source: Martin Fowler, NoSQL Distilled https://martinfowler.com/books/nosql.html
- 12. Evolutions in Data Processing Phase II: “Information” Illustration by David Somerville based on the original by Hugh McLeod (@gapingvoid) RDBMS & Aggregate- Oriented NoSQL
- 13. Hordes of Data Hoardes of Data Data Management Circa 2005
- 14. Trending & Aggregation Finding Needles in Haystacks Data Management Circa 2005
- 15. Data Management Circa 2005 Commodity Server Farms Cheap & Abundant Storage
- 16. The Hadoop Era (2005 - Present)
- 17. Evolutions in Data Phase III: Data Relationships Illustration by David Somerville based on the original by Hugh McLeod (@gapingvoid) RDBMS & Aggregate- Oriented NoSQL Hadoop / MapReduce
- 18. The internet Data Management in 2017
- 19. Data Management in 2017
- 20. Data Management in 2017
- 21. Data Management in 2017 Dynamic Real-World Systems Abundant RAM SSD/Flash (High-Capacity Storage & Ultra-Fast Random I/O)
- 22. The Graph Era (Now & the Future)
- 23. The Graph Era (Now & the Future)
- 24. Illustration by David Somerville based on the original by Hugh McLeod (@gapingvoid) RDBMS & Aggregate- Oriented NoSQL Hadoop / MapReduce |<————————- Graph Database & ————————>| Graph Compute Engine Connected DataDiscrete Data A View of the Data Management Portfolio
- 25. Latency & Freshness (Function of your technology) Batch- Precompute Real-Time
- 26. Insight Action Data Professionals Direct Access to Data Customer + Employeers + Autonomous Devices Access via Applications “Data Warehousing/ Analytic/OLAP/Off-Line” “Real-Time / Transactional/ Operational/OLTP” Another View of the Data Management Portfolio Systems of Insight vs. Action
- 27. Data Professionals Direct Access to Data Customer + Employeers + Autonomous Devices Access via Applications “Data Warehousing/ Analytic/OLAP/Off-Line” “Real-Time / Transactional/ Operational/OLTP” Another View of the Data Management Portfolio Systems of Insight vs. Action
- 28. Real-Time Processing Recommendations based on activity from yesterday Batch Processing Overnight/Intermittent Loading and Calculations Results in lag between activity & knowledge response System-wide local pre-calculations are computationally inefficient Real-Time Writes & Writes Up-to-the-moment freshness “Just-in-time” processing most efficient for “local” queries Recommendations that reflects your latest activity Another View of the Data Management Portfolio Systems of Insight vs. Action
- 29. Latency & Freshness Batch- Precompute Real-Time Connectedness Illustration by David Somerville based on the original by Hugh McLeod (@gapingvoid) Neo4j Solves Connected, Real-Time Problems
- 31. UNIQUE ADVANTAGES OF A NATIVE GRAPH DATABASE
- 33. 33 A unified view for ultimate agility • Easily understood • Easily evolved • Easy collaboration between business and IT #1 Benefit: Project Agility The Whiteboard Model Is the Physical Model
- 34. Connectedness and Size of Data Set ResponseTime Relational and Other NoSQL Databases 0 to 2 hops 0 to 3 degrees Thousands of connections 1000x Advantage Tens to hundreds of hops Thousands of degrees Billions of connections Neo4j “Minutes to milliseconds” #2 Benefit: “Minutes to Milliseconds” Real-Time Query Performance
- 35. “We found Neo4j to be literally thousands of times faster than our prior MySQL solution, with queries that require 10-100 times less code. Today, Neo4j provides eBay with functionality that was previously impossible.” - Volker Pacher, Senior Developer “Minutes to milliseconds” performance Queries up to 1000x faster than RDBMS or other NoSQL #3 Benefit: “Minutes to Milliseconds” Real-Time Query Performance
- 36. At Write Time: data is connected as it is stored At Read Time: Lightning-fast retrieval of data and relationships via pointer chasing Index free adjacency Key Ingredient #1 of 3: Graph Optimized Memory & Storage
- 37. 37 Example HR Query in SQL The Same Query using Cypher MATCH (boss)-[:MANAGES*0..3]->(sub), (sub)-[:MANAGES*1..3]->(report) WHERE boss.name = “John Doe” RETURN sub.name AS Subordinate, count(report) AS Total Project Impact Less time writing queries • More time understanding the answers • Leaving time to ask the next question Less time debugging queries: • More time writing the next piece of code • Improved quality of overall code base Code that’s easier to read: • Faster ramp-up for new project members • Improved maintainability & troubleshooting Key Ingredient #2 of 3: A Productive and Powerful Graph Query Language
- 38. Graph Transactions Over ACID Consistency Graph Transactions Over Non-ACID DBMSs 38 Maintains Integrity Over Time Becomes Corrupt Over Time Key Ingredient #3 of 3: ACID Graph Writes
- 39. VALUE FROM GRAPHS IN FINANCIAL SERVICES
- 40. Asset Graph 1 Customer Graph2 Payment Graph3Master Data Graph 4 Entitlement Graph 5 THE FIVE GRAPHS OF FINANCE
- 41. #1 Asset Graph
- 43. #1 Asset Graph
- 45. #1 Asset Graph
- 46. Impact analysis Portfolio analytics Risk assessment Trading Dynamic Pricing Key Applications Asset Graph – Key Values Example Neo4j-customers
- 48. Manager Research VP Dallas Director United States Central Region CEO North America Strategy ANALYST Wholesale Banking Texas #2 Customer Graph
- 50. Upsell/Cross-Sell Customer Targeting Sales Operations Human Capital Management Key Applications Customer Graph – Key Values Example Neo4j-industries Manufacturing Health CareFinance Telecom Retail Human Resources
- 52. #3 Payment Graph SMB SMB SMB SMB SMB SMB SMB SMB Am ount: $18,000 Transactions: 10 Amount: $22,000 Transactions: 200 SOLD_TO SOLD_TO SOLD_TO Amount: $32,000Transactions: 170 Am ount: $22,000 Transactions: 200 SO LD_TO SMB SMB SMB Amount: $8,000 Transactions: 14 SOLD_TOAmount: $24,000Transactions: 11 SOLD_TO Amount: $17,000 Transactions: 300 SOLD_TO Am ount: $11,000 Transactions: 199 SOLD_TO Amount:$15,000 Transactions:10 SOLD_TOAmount:$15,000 Transactions:10 SOLD_TO
- 53. A Graph of Money Laundering #3 Payment Graph
- 54. INVESTIGATE Revolving Debt Number of Accounts INVESTIGATE Normal behavior Fraud Detection with Discrete Analysis Pros Stops rookies Cons False positives False negatives
- 55. Revolving Debt Number of Accounts Normal behavior Fraudulent pattern Fraud Detection with Connected Analysis Pros Detects fraud rings Reduces false negatives
- 56. Government Example Neo4j-customers Offers & Recommendations Network Effects Chargebacks Anti Fraud / Money Laundering Credit Risk Key Applications Payment Graph – Key Values
- 57. #4 Master Data Graph
- 58. Systems Planning, Impact Analysis, Data Governance, Micro-Services Enterprise Architecture | System of Systems #4 Master Data Graph
- 59. Extracts from “Graph databases for exploring metadata” by Jeremy Ponser #4 Master Data Graph Enterprise Metadata Graphs
- 60. #4 Master Data Graph VP Staff Staff StaffStaff DirectorStaffDirector Manager Manager Manager Manager Fiber Link Fiber Link Fiber Link Ocean Cable Switch Switch Router Router Service • Organizational Structures including sales territories, reporting structures, geography • Product Structures including product & feature hierarchies, time dimension • Network Inventories including configuration management, physical and logistics networks Enterprise Hierarchies
- 61. Example Neo4j-customers 360° View of the Customer Packaging & Product Bundling Recommendations Human Capital Management Key Applications Master Data Graph – Key Values
- 63. GraphConnect 2013: https://vimeo.com/76821847 Using Graph Databases in Real-Time to Solve Resource Authorization at Telenor
- 64. GraphConnect 2013: https://vimeo.com/76821847 Using Graph Databases in Real-Time to Solve Resource Authorization at Telenor
- 65. GraphConnect 2013: https://vimeo.com/76821847 Using Graph Databases in Real-Time to Solve Resource Authorization at Telenor
- 67. Compliance Faster onboarding Real-time provisioning Real-time deprovisioning The Value Entitlement Graph – Key Values
- 68. “Graph analysis is possibly the single most effective competitive differentiator for organizations pursuing data-driven operations and decisions after the design of data capture.” “By the end of 2018, 70% of leading organizations will have one or more pilot or proof-of-concept efforts underway utilizing graph databases.” Towards Graph Inevitability
- 69. #5 Entitlement Graph#4 Master Data Graph #3 Payment Graph#2 Customer Graph#1 Asset Graph The Five Graphs of Finance
- 70. NEW YORK CITY April 18, 2017 09:00-09:30 09:30-10:15 10:15-11:00 11:00-11:30 11:30-12:30 12:30-13:30 13:30-17:00 Breakfast and Registration The Connected Data Imperative: Why Graphs Transform Your Data: A worked example Break Enterprise Ready: A Look at Neo4j in Production Lunch Training Session Agenda
- 71. Key Analytics Patterns HDFS/MapReduce/Spark (Storage & Aggregation) Streaming (Filtering & Aggregation) Machine LearningGraph Computation