SlideShare a Scribd company logo

How Shutl Delivers Even Faster Using Neo4J

C4Media, profile picture
C4Media

The document covers the functionalities and enhancements of Shutl's e-commerce platform, particularly through the use of Neo4j graph databases. It details the improvements in quoting speed from the original version of the application and provides examples of querying data using the Cypher query language. The document emphasizes the importance of efficient querying and the role of graph databases in optimizing performance and user experience.

Technology
Related topics:
architecture-design • Graph Database Insights
1 of 168
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
How Shutl Delivers Even Faster Using Neo4J
Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /shutl-neo4j-graph-db InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month
Presented at QCon London www.qconlondon.com Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide
How Shutl Delivers Even Faster Using Neo4j Sam Phillips and Volker Pacher @samsworldofno @vpacher
Sam Phillips Volker Pacher
Graphs at Shutl
Graphs at Shutl • Graph databases are awesome
Graphs at Shutl • Graph databases are awesome • We’ve seen lots of the talks about modelling
Graphs at Shutl • Graph databases are awesome • We’ve seen lots of the talks about modelling • But querying is important too
Graphs at Shutl • Graph databases are awesome • We’ve seen lots of the talks about modelling • But querying is important too • So let’s talk about querying too!
Show of hands
Show of hands • Who has used graph databases before?
Show of hands • Who has used graph databases before? • Who has used Neo4j before?
Shutl
Shutl
ECOMMERCE IS QUICK & CONVENIENT
ECOMMERCE IS QUICK & CONVENIENT
PAYPAL FOR AWESOME DELIVERY
PAYPAL FOR AWESOME DELIVERY
PAYPAL FOR AWESOME DELIVERY Branded, super quick delivery that people trust, embedded in merchant websites
HUB & SPOKE A B
HUB & SPOKE A B
HUB & SPOKE A B Only cost effective means to deliver 10+ miles but slow and unpredictable
HUB & SPOKE Only cost effective means to deliver 10+ miles but slow and unpredictable A B
HUB & SPOKE Only cost effective means to deliver 10+ miles but slow and unpredictable A B POINT TO POINT A B
HUB & SPOKE Only cost effective means to deliver 10+ miles but slow and unpredictable A B POINT TO POINT A B Fast and predictable but cost prohibitive over longer distances
HUB & SPOKE 97% Courier, Express & Parcel Market
POINT TO POINT 3% Courier, Express & Parcel Market
POINT TO POINT +7,500 more! 3% Courier, Express & Parcel Market
POINT TO POINT
SHOP
Shutl generates a quote from each relevant carrier within platform SHOP $$ $$$ $ $$ $ $
Shutl generates a quote from each relevant carrier within platform Optimum picked based on price & quality rating SHOP $$ $$$ $ $$ $ $$
SHOP SHOP
On checkout, delivery sent via API into chosen carrier’s transportation system SHOP $$ SHOP
On checkout, delivery sent via API into chosen carrier’s transportation system Courier collects from nearest store and delivers to shopper SHOP $$
How Shutl Delivers Even Faster Using Neo4J
Delivery status updated in real-time, performance compared against SLA & carrier quality rating updated Better performing carriers get more deliveries & can demand higher prices
Delivery status updated in real-time, performance compared against SLA & carrier quality rating updated Better performing carriers get more deliveries & can demand higher prices
Delivery status updated in real-time, performance compared against SLA & carrier quality rating updated Better performing carriers get more deliveries & can demand higher prices
Track your order online…
FEEDBACK Quality paramount since we are motivated by LTV of shopper
FEEDBACK Quality paramount since we are motivated by LTV of shopper
FEEDBACK Shutl sends feedback email to consumer seconds after they have received delivery asking to rate qualitative aspects of experience
FEEDBACK Feedback streamed unedited to shutl.com/feedback & facebook
FEEDBACK
FEEDBACK
FEEDBACK
FEEDBACK
How Shutl Delivers Even Faster Using Neo4J
SHUTL IS NOW AN COMPANY
Version One Ruby 1.8, Rails 2.3 and MySQL
Version One Ruby 1.8, Rails 2.3 and MySQL
Version One Ruby 1.8, Rails 2.3 and MySQL • Well-known tale: built quickly, worked slowly, tough to maintain • Getting a quote for an hour time-slot took over 4 seconds
Here is the Shutl price calendar
Here is the Shutl price calendar To generate this in V1, the merchant site would have had to call Shutl to get available slots (2 seconds)
Here is the Shutl price calendar To generate this in V1, the merchant site would have had to call Shutl to get available slots (2 seconds) Then, they would have to call Shutl to generate a quote for each slot - for two days of store opening, that’s 20+ slots
Here is the Shutl price calendar To generate this in V1, the merchant site would have had to call Shutl to get available slots (2 seconds) Then, they would have to call Shutl to generate a quote for each slot - for two days of store opening, that’s 20+ slots So, that’s 2 + (20 x 4) seconds, 1:22 to generate the data for this calendar
Here is the Shutl price calendar To generate this in V1, the merchant site would have had to call Shutl to get available slots (2 seconds) Then, they would have to call Shutl to generate a quote for each slot - for two days of store opening, that’s 20+ slots So, that’s 2 + (20 x 4) seconds, 1:22 to generate the data for this calendar In V1, this UX could never have happened.
V2
• Broke app into services V2 • Services focused around functions like quoting, booking, and giving feedback • Key goal for the project was improving the speed of the quoting operation, which is where we used graph databases
V1 V2
V1 V2 • Quoting for 20 windows down from 82000 ms to 800 ms
V1 V2 • Quoting for 20 windows down from 82000 ms to 800 ms • Code complexity much reduced
V1 V2 • Quoting for 20 windows down from 82000 ms to 800 ms • Code complexity much reduced
A large part of the success of our rewrite was down to the graph database.
What is a graph anyway?
How Shutl Delivers Even Faster Using Neo4J
a simple graph a collection of vertices (nodes) connected by edges (relationships)
a short history the seven bridges of Königsberg (1735)! Leonard Euler
the seven bridges of Königsberg (1735)!
the seven bridges of Königsberg (1735)!
the seven bridges of Königsberg (1735)!
the seven bridges of Königsberg (1735)!
the seven bridges of Königsberg (1735)!
Euler walk each node has an even degree
Euler walk
Euler walk
Euler walk two nodes have an odd degree
Euler walk two nodes have an odd degree
Euler walk no two nodes have an odd degree
directed graph each relationship has a direction or one start node and one end node
property graph nodes contain properties (key, value) relationships have a type and are always directed relationships can contain properties too Person name: Sam Person name: Volker :friends :knows since: 2005 Person name: Megan :friends Company name: eBay :works_for :works_for
The Case for Graph Databases
relationships are explicit stored
additive domain modelling
whiteboard friendly
traversals of relationships are easy and very fast
DB performance remains relatively constant as queries are localised to its portion of the graph. O(1) for same query
a graph is its own index (constant query performance)
a graph is its own index (constant query performance)
a graph is its own index (constant query performance)
the case for Neo4j
standalone or embedded in jvm
ruby/jruby
ruby libraries - neo4j gem by Andreas Ronge (https://github.com/andreasronge/neo4j)
cypher
the neotech guys are awesome
Querying the graph: Cypher declarative query language specific to neo4j easy to learn and intuitive use specific patterns to query for (something that looks like ‘this’) inspired partly by SQL (WHERE and ORDER BY) and SPARQL (pattern matching) focuses on what to query for and not how to query for it switch from a mySQl world is made easier by the use of cypher instead of having to learn a traversal framework straight away
cypher clauses START: Starting points in the graph, obtained via index lookups or by element IDs. MATCH: The graph pattern to match, bound to the starting points in START. WHERE: Filtering criteria. RETURN: What to return. CREATE: Creates nodes and relationships. DELETE: Removes nodes, relationships and properties. SET: Set values to properties. FOREACH: Performs updating actions once per element in a list. WITH: Divides a query into multiple, distinct parts
cypher clauses START: Starting points in the graph, obtained via index lookups or by element IDs. MATCH: The graph pattern to match, bound to the starting points in START. WHERE: Filtering criteria. RETURN: What to return. CREATE: Creates nodes and relationships. DELETE: Removes nodes, relationships and properties. SET: Set values to properties. FOREACH: Performs updating actions once per element in a list. WITH: Divides a query into multiple, distinct parts
an example Person name: Sam Person name: Volker :friends :knows since: 2005 Person name: Megan :friends Company name: eBay :works_for :works_for Person name: Jim :friends :works_for Company name: neotech
find all the companies my friends work for MATCH (person{ name:’Volker’ }) -[:friends] - (person) - [:works_for]-> company RETURN company
find all the companies my friends work for MATCH (person{ name:’Volker’ }) -[:friends] - (person) - [:works_for]-> company RETURN company
find all the companies my friends work for MATCH (person{ name:’Volker’ }) -[:friends] - (person) - [:works_for]-> company RETURN company Person name: Sam Person name: Volker :friends :knows since: 2005 Person name: Megan :friends Company name: eBay :works_for :works_for Person name: Jim :friends :works_for Company name: neotech
find all the companies my friend’s friends work for MATCH (person{ name:’Volker’ }) - [:friends*2..2]-(person) - [:works_for] -> company RETURN company
find all the companies my friend’s friends work for MATCH (person{ name:’Volker’ }) - [:friends*2..2]-(person) - [:works_for] -> company RETURN company
find all the companies my friend’s friends work for MATCH (person{ name:’Volker’ }) - [:friends*2..2]-(person) - [:works_for] -> company RETURN company Person name: Sam Person name: Volker :friends :knows since: 2005 Person name: Megan :friends Company name: eBay :works_for :works_for Person name: Jim :friends :works_for Company name: neotech
find all my friends who work for neotech MATCH (person{ name:’Volker’ }) -[:friends] -(friends) - [:works_for]-> company WHERE company.name = ‘neotech’ RETURN friends
find all my friends who work for neotech MATCH (person{ name:’Volker’ }) -[:friends] -(friends) - [:works_for]-> company WHERE company.name = ‘neotech’ RETURN friends
find all my friends who work for neotech MATCH (person{ name:’Volker’ }) -[:friends] -(friends) - [:works_for]-> company WHERE company.name = ‘neotech’ RETURN friends Person name: Sam Person name: Volker :friends :knows since: 2005 Person name: Megan :friends Company name: eBay :works_for :works_for Person name: Jim :friends :works_for Company name: neotech
a good place to try it out: ! http://console.neo4j.org/ ! http://gist.neo4j.org/
coverage example Locality id = california Locality id = marin_county Locality id = 94901 :contains :contains Locality id = 94903 :contains :contains Locality id = 94902
coverage example Locality id = california Locality id = marin_county Locality id = 94901 :contains :contains Locality id = 94903 :contains :contains Locality id = 94902 :operates :operates Carrier id = carrier_1
coverage example Locality id = california Locality id = marin_county Locality id = 94901 :contains Store id = ebay_store :located :contains Locality id = 94903 :contains :contains Locality id = 94902 :operates :operates Carrier id = carrier_1
coverage example Locality id = california Locality id = marin_county Locality id = 94901 :contains Store id = ebay_store :located :contains Locality id = 94903 :contains :contains Locality id = 94902 :operates Carrier id = carrier_2 :operates :operates Carrier id = carrier_1
MATCH (store{ id:’ebay_store’ }) -[:located] -> (locality) <- [:operates]- carrier RETURN carrier the query
the query MATCH (store{ id:’ebay_store’ }) -[:located] -> (locality) <- [:operates]- carrier RETURN carrier
MATCH (store{ id:’ebay_store’ }) -[:located] -> (locality) <- [:operates]- carrier RETURN carrier the query Locality id = california Locality id = 94902 Locality id = marin_county Locality id = 94901 :contains Store id = ebay_store :located :contains Locality id = 94903 :contains :contains :operates :operates Carrier id = carrier_1
MATCH (store{ id:’ebay_store’ }) -[:located] -> () <- [:contains*0..2] - (locality) <- [:operates]- carrier RETURN carrier the query
MATCH (store{ id:’ebay_store’ }) -[:located] -> () <- [:contains*0..2] - (locality) <- [:operates]- carrier RETURN carrier the query
MATCH (store{ id:’ebay_store’ }) -[:located] -> () <- [:contains*0..2] - (locality) <- [:operates]- carrier RETURN carrier the query Locality id = california Locality id = marin_county Locality id = 94901 :contains Store id = ebay_store :located :contains Locality id = 94903 :contains :contains Locality id = 94902 :operates Carrier id = carrier_2 :operates :operates Carrier id = carrier_1
MATCH (store{ id:’ebay_store’ }) -[:located] -> () <- [:contains*0..2] - (locality) <- [:operates]- carrier RETURN carrier the query Locality id = california Locality id = marin_county Locality id = 94901 :contains Store id = ebay_store :located :contains Locality id = 94903 :contains :contains Locality id = 94902 :operates Carrier id = carrier_2 :operates :operates Carrier id = carrier_1
MATCH (store{ id:’ebay_store’ }) -[:located] -> () <- [:contains*0..2] - (locality) <- [:operates]- carrier RETURN carrier the query Locality id = california Locality id = marin_county Locality id = 94901 :contains Store id = ebay_store :located :contains Locality id = 94903 :contains :contains Locality id = 94902 :operates Carrier id = carrier_2 :operates :operates Carrier id = carrier_1
MATCH (store{ id:’ebay_store’ }) -[:located] -> () <- [:contains*0..2] - (locality) <- [:operates]- carrier RETURN carrier the query Locality id = california Locality id = marin_county Locality id = 94901 :contains Store id = ebay_store :located :contains Locality id = 94903 :contains :contains Locality id = 94902 :operates Carrier id = carrier_2 :operates :operates Carrier id = carrier_1
SELECT * FROM carriers LEFT JOIN locations ON carrier.location_id = location.id LEFT JOIN stores ON stores.location_id = carrier.location_id WHERE stores.name = ‘ebay_store’
SELECT * FROM carriers LEFT JOIN locations ON carrier.location_id = location.id OR carrier.location_id = location.parent_id LEFT JOIN stores ON stores.location_id = carrier.location_id WHERE stores.name = ‘ebay_store’
?
MATCH (store{ id:’ebay_store’ }) -[:located] -> () <- [:contains*0..2] - (locality) <- [:operates]- carrier RETURN carrier
representing dates/times root (0) :year_2014 Year: 2013 :year_2015 Year: 2015 :month_06 :month_05 :month_01 Month: 05 Month: 01 Month: 06 :day_24 :day_25 Day: 24 Day: 25 :day_26 Day: 26 :happens :happens :happens :happens Event 1 Event 2 Event 3
find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event
find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event
find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event root (0) :year_2014 Year: 2013 :year_2015 Year: 2015 :month_06 :month_05 :month_01 Month: 05 Month: 01 Month: 06 :day_24 :day_25 Day: 24 Day: 25 :day_26 Day: 26 :happens :happens :happens :happens Event 1 Event 2 Event 3
find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event root (0) :year_2014 Year: 2013 :year_2015 Year: 2015 :month_06 :month_05 :month_01 Month: 05 Month: 01 Month: 06 :day_24 :day_25 Day: 24 Day: 25 :day_26 Day: 26 :happens :happens :happens :happens Event 1 Event 2 Event 3
find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event root (0) :year_2014 Year: 2013 :year_2015 Year: 2015 :month_06 :month_05 :month_01 Month: 05 Month: 01 Month: 06 :day_24 :day_25 Day: 24 Day: 25 :day_26 Day: 26 :happens :happens :happens :happens Event 1 Event 2 Event 3
find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event root (0) :year_2014 Year: 2013 :year_2015 Year: 2015 :month_06 :month_05 :month_01 Month: 05 Month: 01 Month: 06 :day_24 :day_25 Day: 24 Day: 25 :day_26 Day: 26 :happens :happens :happens :happens Event 1 Event 2 Event 3
find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event root (0) :year_2014 Year: 2013 :year_2015 Year: 2015 :month_06 :month_05 :month_01 Month: 05 Month: 01 Month: 06 :day_24 :day_25 Day: 24 Day: 25 :day_26 Day: 26 :happens :happens :happens :happens Event 1 Event 2 Event 3
find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event root (0) :year_2014 Year: 2013 :year_2015 Year: 2015 :month_06 :month_05 :month_01 Month: 05 Month: 01 Month: 06 :day_24 :day_25 Day: 24 Day: 25 :day_26 Day: 26 :happens :happens :happens :happens Event 1 Event 2 Event 3
all together Locality id = california Locality id = marin_county :contains Locality id = 94901 :contains Store id = ebay_store :located :available {premium: 1.5} Carrier id = carrier_1 :operates root (0) :year_2014 Year: 2013 :month_05 Month: 05 :day_24 Day: 24 hour 09 :hour_09 hour 10 :hour_10 hour 11 :hour_11 :available {premium: 1}
all together MATCH (store{ id:’ebay_store’ }) -[:located] -> (locality) <- [:operates]- carrier - [available:available] -> () <- [:hour_10] - () <- [:day_24] - () [:month_05] - () [:year_2014] - () RETURN carrier, available.premium as premium
all together MATCH (store{ id:’ebay_store’ }) -[:located] -> (locality) <- [:operates]- carrier - [available:available] -> () <- [:hour_10] - () <- [:day_24] - () [:month_05] - () [:year_2014] - () RETURN carrier, available.premium as premium
all together MATCH (store{ id:’ebay_store’ }) -[:located] -> (locality) <- [:operates]- carrier - [available:available] -> () <- [:hour_10] - () <- [:day_24] - () [:month_05] - () [:year_2014] - () RETURN carrier, available.premium as premium Locality id = california Locality id = marin_county :contains Locality id = 94901 :contains Store id = ebay_store :located :available {premium: 1.5} Carrier id = carrier_1 :operates root (0) :year_2014 Year: 2013 :month_05 Month: 05 :day_24 Day: 24 hour 09 :hour_09 hour 10 :hour_10 hour 11 :hour_11 :available {premium: 1}
Other graph uses
Other graph uses • Recommendation engines
Other graph uses • Recommendation engines • Organisational analysis
Other graph uses • Recommendation engines • Organisational analysis • Graphing your infrastructure
Some gotchas
Some gotchas • There was a learning curve in switching from a relational mentality to a graph one
Some gotchas • There was a learning curve in switching from a relational mentality to a graph one • Tooling not as mature as in the relational world
Some gotchas • There was a learning curve in switching from a relational mentality to a graph one • Tooling not as mature as in the relational world • No out of the box solution for db migrations
Some gotchas • There was a learning curve in switching from a relational mentality to a graph one • Tooling not as mature as in the relational world • No out of the box solution for db migrations • Seeding an embedded database was unfamiliar
Testing was a challenge
Testing was a challenge • Setting up scenarios for tests was tedious
Testing was a challenge • Setting up scenarios for tests was tedious • Built our own tool based on the geoff syntax developed by Nigel Small
Testing was a challenge • Setting up scenarios for tests was tedious • Built our own tool based on the geoff syntax developed by Nigel Small • Geoff allows modelling of graphs in textual form and provides an interface to insert them into an existing graph
Testing was a challenge • Setting up scenarios for tests was tedious • Built our own tool based on the geoff syntax developed by Nigel Small • Geoff allows modelling of graphs in textual form and provides an interface to insert them into an existing graph (A) {“name”: “Alice”}
Testing was a challenge • Setting up scenarios for tests was tedious • Built our own tool based on the geoff syntax developed by Nigel Small • Geoff allows modelling of graphs in textual form and provides an interface to insert them into an existing graph (A) {“name”: “Alice”} (B) {“name”: “Bob”}
Testing was a challenge • Setting up scenarios for tests was tedious • Built our own tool based on the geoff syntax developed by Nigel Small • Geoff allows modelling of graphs in textual form and provides an interface to insert them into an existing graph (A) {“name”: “Alice”} (B) {“name”: “Bob”} (A) -[:KNOWS] -> (B)
Testing was a challenge • Setting up scenarios for tests was tedious • Built our own tool based on the geoff syntax developed by Nigel Small • Geoff allows modelling of graphs in textual form and provides an interface to insert them into an existing graph (A) {“name”: “Alice”} (B) {“name”: “Bob”} (A) -[:KNOWS] -> (B) • We created a Ruby dsl for modelling a graph and inserting it into the db that works with factory_girl
Testing was a challenge • Setting up scenarios for tests was tedious • Built our own tool based on the geoff syntax developed by Nigel Small • Geoff allows modelling of graphs in textual form and provides an interface to insert them into an existing graph (A) {“name”: “Alice”} (B) {“name”: “Bob”} (A) -[:KNOWS] -> (B) • We created a Ruby dsl for modelling a graph and inserting it into the db that works with factory_girl • Open source - https://github.com/shutl/geoff
Wrap Up
Wrap Up • Neo4j and graph theory enabled Shutl to achieve big performance increases in its most important operation - calculating delivery prices
Wrap Up • Neo4j and graph theory enabled Shutl to achieve big performance increases in its most important operation - calculating delivery prices • It’s a new tool based on tested theory, and cypher is the first language that allows you to query graphs in a declarative way (like SQL)
Wrap Up • Neo4j and graph theory enabled Shutl to achieve big performance increases in its most important operation - calculating delivery prices • It’s a new tool based on tested theory, and cypher is the first language that allows you to query graphs in a declarative way (like SQL) • Tooling and adoption is immature but getting better all the time
Thank you! ! Any questions? Sam Phillips Head of Engineering ! @samsworldofno http://samsworldofno.com sam@shutl.com Volker Pacher Senior Developer ! @vpacher https://github.com/vpacher volker@shutl.com
How Shutl Delivers Even Faster Using Neo4J
our
Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations/shutl-neo4j- graph-db

More Related Content

PDF
Building elements and experiences at Net-A-Porter
Robin Glen
 
PDF
Building for, perceiving and measuring performance for mobile web
Robin Glen
 
PDF
Dynamic Sound for Android
Tony Hillerson
 
PDF
Dynamic Sound for iOS Apps and Games
Tack Mobile
 
PDF
Scala in-practice-3-years by Patric Fornasier, Springr, presented at Pune Sca...
Thoughtworks
 
PDF
Beautiful APIs - SOSE2021 Keynote
Cesare Pautasso
 
PDF
Netapp Michael Galpin
rajivmordani
 
PDF
EmberConf 2016 – Idiomatic Ember (Speaker Notes)
Lauren Elizabeth Tan
 
Building elements and experiences at Net-A-Porter
Robin Glen
 
Building for, perceiving and measuring performance for mobile web
Robin Glen
 
Dynamic Sound for Android
Tony Hillerson
 
Dynamic Sound for iOS Apps and Games
Tack Mobile
 
Scala in-practice-3-years by Patric Fornasier, Springr, presented at Pune Sca...
Thoughtworks
 
Beautiful APIs - SOSE2021 Keynote
Cesare Pautasso
 
Netapp Michael Galpin
rajivmordani
 
EmberConf 2016 – Idiomatic Ember (Speaker Notes)
Lauren Elizabeth Tan
 

Viewers also liked (14)

PPTX
Laura Vision
Jotadani
 
PDF
Eventos sociales Determina el Tipo de Recepcion
erraticmasquera65
 
PPTX
Implantación ERP Hotel Baelo
Martín Serrano Rodríguez
 
PPTX
Mercadeo y publicidad informatica.
lufecago
 
PPTX
Empresas Organizadoras de Eventos
Caterine Ladino
 
PDF
Glovo - Propuesta de organización de evento
Anna Ortega Martínez
 
PPTX
Slide showoroca
Buzón E
 
PPTX
Decorfiestas Cartagena.
carcacol
 
PPT
Curso De Organizacion De Eventos Corporativos
CARLOS PEREZ
 
PPTX
Análisis de la organización comercial y direccion de ventas - jose puchades
Jose Puchades Vigar
 
PPT
Empresa de informatica
Ldecoree
 
PPT
EVENTOS SOCIALES
Stella Pereyra
 
PDF
Tipos De Eventos, Conceptos,Planificación
federicogalvis
 
PPTX
Presentacion Empresa Organizadora De Eventos
JOHANA COSTALES
 
Laura Vision
Jotadani
 
Eventos sociales Determina el Tipo de Recepcion
erraticmasquera65
 
Implantación ERP Hotel Baelo
Martín Serrano Rodríguez
 
Mercadeo y publicidad informatica.
lufecago
 
Empresas Organizadoras de Eventos
Caterine Ladino
 
Glovo - Propuesta de organización de evento
Anna Ortega Martínez
 
Slide showoroca
Buzón E
 
Decorfiestas Cartagena.
carcacol
 
Curso De Organizacion De Eventos Corporativos
CARLOS PEREZ
 
Análisis de la organización comercial y direccion de ventas - jose puchades
Jose Puchades Vigar
 
Empresa de informatica
Ldecoree
 
EVENTOS SOCIALES
Stella Pereyra
 
Tipos De Eventos, Conceptos,Planificación
federicogalvis
 
Presentacion Empresa Organizadora De Eventos
JOHANA COSTALES
 
Ad

Similar to How Shutl Delivers Even Faster Using Neo4J (20)

PDF
QCon 2014 - How Shutl delivers even faster with Neo4j
Volker Pacher
 
PDF
GraphConnect 2014 SF: How eBay and Shutl Deliver Even Faster Using Neo4j
Neo4j
 
PDF
Graphs in Action: In-depth look at Neo4j in Production
Neo4j
 
PDF
Cloud east shutl_talk
Volker Pacher
 
PPT
10. Graph Databases
Fabio Fumarola
 
PDF
managing big data
Suveeksha
 
PDF
Using graphs for recommendations
Rik Van Bruggen
 
PDF
Produktdatenmanagement mit Neo4j
Neo4j
 
PPTX
GraphTalk Frankfurt - Einführung in Graphdatenbanken
Neo4j
 
PDF
Graphs for Enterprise Architects
Neo4j
 
PPTX
Graphs in the Real World
Neo4j
 
PDF
how_graphs_eat_the_world
Ora Weinstein
 
PPTX
Introduction to Graph Databases
Max De Marzi
 
PPTX
Recommendations and Statistics with Graph Databases
Calin Constantinov
 
PDF
NOSQLEU - Graph Databases and Neo4j
Tobias Lindaaker
 
PDF
Optimizing Your Supply Chain with the Neo4j Graph
Neo4j
 
PPTX
Optimizing Your Supply Chain with Neo4j
Neo4j
 
PDF
Data Modeling with Neo4j
Neo4j
 
PDF
Optimizing the Supply Chain with Knowledge Graphs, IoT and Digital Twins_Moor...
Neo4j
 
PDF
Training Series - Intro to Neo4j
Neo4j
 
QCon 2014 - How Shutl delivers even faster with Neo4j
Volker Pacher
 
GraphConnect 2014 SF: How eBay and Shutl Deliver Even Faster Using Neo4j
Neo4j
 
Graphs in Action: In-depth look at Neo4j in Production
Neo4j
 
Cloud east shutl_talk
Volker Pacher
 
10. Graph Databases
Fabio Fumarola
 
managing big data
Suveeksha
 
Using graphs for recommendations
Rik Van Bruggen
 
Produktdatenmanagement mit Neo4j
Neo4j
 
GraphTalk Frankfurt - Einführung in Graphdatenbanken
Neo4j
 
Graphs for Enterprise Architects
Neo4j
 
Graphs in the Real World
Neo4j
 
how_graphs_eat_the_world
Ora Weinstein
 
Introduction to Graph Databases
Max De Marzi
 
Recommendations and Statistics with Graph Databases
Calin Constantinov
 
NOSQLEU - Graph Databases and Neo4j
Tobias Lindaaker
 
Optimizing Your Supply Chain with the Neo4j Graph
Neo4j
 
Optimizing Your Supply Chain with Neo4j
Neo4j
 
Data Modeling with Neo4j
Neo4j
 
Optimizing the Supply Chain with Knowledge Graphs, IoT and Digital Twins_Moor...
Neo4j
 
Training Series - Intro to Neo4j
Neo4j
 
Ad

More from C4Media (20)

PDF
Streaming a Million Likes/Second: Real-Time Interactions on Live Video
C4Media
 
PDF
Next Generation Client APIs in Envoy Mobile
C4Media
 
PDF
Software Teams and Teamwork Trends Report Q1 2020
C4Media
 
PDF
Understand the Trade-offs Using Compilers for Java Applications
C4Media
 
PDF
Kafka Needs No Keeper
C4Media
 
PDF
High Performing Teams Act Like Owners
C4Media
 
PDF
Does Java Need Inline Types? What Project Valhalla Can Bring to Java
C4Media
 
PDF
Service Meshes- The Ultimate Guide
C4Media
 
PDF
Shifting Left with Cloud Native CI/CD
C4Media
 
PDF
CI/CD for Machine Learning
C4Media
 
PDF
Fault Tolerance at Speed
C4Media
 
PDF
Architectures That Scale Deep - Regaining Control in Deep Systems
C4Media
 
PDF
ML in the Browser: Interactive Experiences with Tensorflow.js
C4Media
 
PDF
Build Your Own WebAssembly Compiler
C4Media
 
PDF
User & Device Identity for Microservices @ Netflix Scale
C4Media
 
PDF
Scaling Patterns for Netflix's Edge
C4Media
 
PDF
Make Your Electron App Feel at Home Everywhere
C4Media
 
PDF
The Talk You've Been Await-ing For
C4Media
 
PDF
Future of Data Engineering
C4Media
 
PDF
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
C4Media
 
Streaming a Million Likes/Second: Real-Time Interactions on Live Video
C4Media
 
Next Generation Client APIs in Envoy Mobile
C4Media
 
Software Teams and Teamwork Trends Report Q1 2020
C4Media
 
Understand the Trade-offs Using Compilers for Java Applications
C4Media
 
Kafka Needs No Keeper
C4Media
 
High Performing Teams Act Like Owners
C4Media
 
Does Java Need Inline Types? What Project Valhalla Can Bring to Java
C4Media
 
Service Meshes- The Ultimate Guide
C4Media
 
Shifting Left with Cloud Native CI/CD
C4Media
 
CI/CD for Machine Learning
C4Media
 
Fault Tolerance at Speed
C4Media
 
Architectures That Scale Deep - Regaining Control in Deep Systems
C4Media
 
ML in the Browser: Interactive Experiences with Tensorflow.js
C4Media
 
Build Your Own WebAssembly Compiler
C4Media
 
User & Device Identity for Microservices @ Netflix Scale
C4Media
 
Scaling Patterns for Netflix's Edge
C4Media
 
Make Your Electron App Feel at Home Everywhere
C4Media
 
The Talk You've Been Await-ing For
C4Media
 
Future of Data Engineering
C4Media
 
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
C4Media
 

Recently uploaded (20)

PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
PDF
Review of recent advances in non-invasive hemoglobin estimation
IAESIJAI
 
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
DatVoNgoc
 
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
SOFTTECHHUB
 
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
Mark Billinghurst
 
PDF
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
Libreria ERP
 
PDF
Per capita expenditure prediction using model stacking based on satellite ima...
IAESIJAI
 
PPTX
MYSQL Presentation for SQL database connectivity
Swati270511
 
PPTX
sap open course for s4hana steps from ECC to s4
sreeni2106invites
 
PPTX
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
23bcla24
 
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
PDF
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
PDF
MIND Revenue Release Quarter 2 2025 Press Release
MIND CTI
 
DOCX
The AUB Centre for AI in Media Proposal.docx
GAONE9
 
PDF
KodekX | Application Modernization Development
KodekX
 
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
PDF
Electronic commerce courselecture one. Pdf
OmerMohamed64
 
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
SynapseIndia
 
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
Review of recent advances in non-invasive hemoglobin estimation
IAESIJAI
 
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
20250228 LYD VKU AI Blended-Learning.pptx
DatVoNgoc
 
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
SOFTTECHHUB
 
Reach Out and Touch Someone: Haptics and Empathic Computing
Mark Billinghurst
 
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
Libreria ERP
 
Per capita expenditure prediction using model stacking based on satellite ima...
IAESIJAI
 
MYSQL Presentation for SQL database connectivity
Swati270511
 
sap open course for s4hana steps from ECC to s4
sreeni2106invites
 
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
23bcla24
 
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
MIND Revenue Release Quarter 2 2025 Press Release
MIND CTI
 
The AUB Centre for AI in Media Proposal.docx
GAONE9
 
KodekX | Application Modernization Development
KodekX
 
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
Electronic commerce courselecture one. Pdf
OmerMohamed64
 
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
SynapseIndia
 

How Shutl Delivers Even Faster Using Neo4J

  • 2. Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /shutl-neo4j-graph-db InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month
  • 3. Presented at QCon London www.qconlondon.com Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide
  • 4. How Shutl Delivers Even Faster Using Neo4j Sam Phillips and Volker Pacher @samsworldofno @vpacher
  • 7. Graphs at Shutl • Graph databases are awesome
  • 8. Graphs at Shutl • Graph databases are awesome • We’ve seen lots of the talks about modelling
  • 9. Graphs at Shutl • Graph databases are awesome • We’ve seen lots of the talks about modelling • But querying is important too
  • 10. Graphs at Shutl • Graph databases are awesome • We’ve seen lots of the talks about modelling • But querying is important too • So let’s talk about querying too!
  • 12. Show of hands • Who has used graph databases before?
  • 13. Show of hands • Who has used graph databases before? • Who has used Neo4j before?
  • 14. Shutl
  • 15. Shutl
  • 16. ECOMMERCE IS QUICK & CONVENIENT
  • 17. ECOMMERCE IS QUICK & CONVENIENT
  • 18. PAYPAL FOR AWESOME DELIVERY
  • 19. PAYPAL FOR AWESOME DELIVERY
  • 20. PAYPAL FOR AWESOME DELIVERY Branded, super quick delivery that people trust, embedded in merchant websites
  • 21. HUB & SPOKE A B
  • 22. HUB & SPOKE A B
  • 23. HUB & SPOKE A B Only cost effective means to deliver 10+ miles but slow and unpredictable
  • 24. HUB & SPOKE Only cost effective means to deliver 10+ miles but slow and unpredictable A B
  • 25. HUB & SPOKE Only cost effective means to deliver 10+ miles but slow and unpredictable A B POINT TO POINT A B
  • 26. HUB & SPOKE Only cost effective means to deliver 10+ miles but slow and unpredictable A B POINT TO POINT A B Fast and predictable but cost prohibitive over longer distances
  • 27. HUB & SPOKE 97% Courier, Express & Parcel Market
  • 28. POINT TO POINT 3% Courier, Express & Parcel Market
  • 29. POINT TO POINT +7,500 more! 3% Courier, Express & Parcel Market
  • 31. SHOP
  • 32. Shutl generates a quote from each relevant carrier within platform SHOP $$ $$$ $ $$ $ $
  • 33. Shutl generates a quote from each relevant carrier within platform Optimum picked based on price & quality rating SHOP $$ $$$ $ $$ $ $$
  • 35. On checkout, delivery sent via API into chosen carrier’s transportation system SHOP $$ SHOP
  • 36. On checkout, delivery sent via API into chosen carrier’s transportation system Courier collects from nearest store and delivers to shopper SHOP $$
  • 38. Delivery status updated in real-time, performance compared against SLA & carrier quality rating updated Better performing carriers get more deliveries & can demand higher prices
  • 39. Delivery status updated in real-time, performance compared against SLA & carrier quality rating updated Better performing carriers get more deliveries & can demand higher prices
  • 40. Delivery status updated in real-time, performance compared against SLA & carrier quality rating updated Better performing carriers get more deliveries & can demand higher prices
  • 41. Track your order online…
  • 42. FEEDBACK Quality paramount since we are motivated by LTV of shopper
  • 43. FEEDBACK Quality paramount since we are motivated by LTV of shopper
  • 44. FEEDBACK Shutl sends feedback email to consumer seconds after they have received delivery asking to rate qualitative aspects of experience
  • 45. FEEDBACK Feedback streamed unedited to shutl.com/feedback & facebook
  • 51. SHUTL IS NOW AN COMPANY
  • 52. Version One Ruby 1.8, Rails 2.3 and MySQL
  • 53. Version One Ruby 1.8, Rails 2.3 and MySQL
  • 54. Version One Ruby 1.8, Rails 2.3 and MySQL • Well-known tale: built quickly, worked slowly, tough to maintain • Getting a quote for an hour time-slot took over 4 seconds
  • 55. Here is the Shutl price calendar
  • 56. Here is the Shutl price calendar To generate this in V1, the merchant site would have had to call Shutl to get available slots (2 seconds)
  • 57. Here is the Shutl price calendar To generate this in V1, the merchant site would have had to call Shutl to get available slots (2 seconds) Then, they would have to call Shutl to generate a quote for each slot - for two days of store opening, that’s 20+ slots
  • 58. Here is the Shutl price calendar To generate this in V1, the merchant site would have had to call Shutl to get available slots (2 seconds) Then, they would have to call Shutl to generate a quote for each slot - for two days of store opening, that’s 20+ slots So, that’s 2 + (20 x 4) seconds, 1:22 to generate the data for this calendar
  • 59. Here is the Shutl price calendar To generate this in V1, the merchant site would have had to call Shutl to get available slots (2 seconds) Then, they would have to call Shutl to generate a quote for each slot - for two days of store opening, that’s 20+ slots So, that’s 2 + (20 x 4) seconds, 1:22 to generate the data for this calendar In V1, this UX could never have happened.
  • 60. V2
  • 61. • Broke app into services V2 • Services focused around functions like quoting, booking, and giving feedback • Key goal for the project was improving the speed of the quoting operation, which is where we used graph databases
  • 62. V1 V2
  • 63. V1 V2 • Quoting for 20 windows down from 82000 ms to 800 ms
  • 64. V1 V2 • Quoting for 20 windows down from 82000 ms to 800 ms • Code complexity much reduced
  • 65. V1 V2 • Quoting for 20 windows down from 82000 ms to 800 ms • Code complexity much reduced
  • 66. A large part of the success of our rewrite was down to the graph database.
  • 67. What is a graph anyway?
  • 69. a simple graph a collection of vertices (nodes) connected by edges (relationships)
  • 70. a short history the seven bridges of Königsberg (1735)! Leonard Euler
  • 71. the seven bridges of Königsberg (1735)!
  • 72. the seven bridges of Königsberg (1735)!
  • 73. the seven bridges of Königsberg (1735)!
  • 74. the seven bridges of Königsberg (1735)!
  • 75. the seven bridges of Königsberg (1735)!
  • 76. Euler walk each node has an even degree
  • 79. Euler walk two nodes have an odd degree
  • 80. Euler walk two nodes have an odd degree
  • 81. Euler walk no two nodes have an odd degree
  • 82. directed graph each relationship has a direction or one start node and one end node
  • 83. property graph nodes contain properties (key, value) relationships have a type and are always directed relationships can contain properties too Person name: Sam Person name: Volker :friends :knows since: 2005 Person name: Megan :friends Company name: eBay :works_for :works_for
  • 84. The Case for Graph Databases
  • 88. traversals of relationships are easy and very fast
  • 89. DB performance remains relatively constant as queries are localised to its portion of the graph. O(1) for same query
  • 90. a graph is its own index (constant query performance)
  • 91. a graph is its own index (constant query performance)
  • 92. a graph is its own index (constant query performance)
  • 93. the case for Neo4j
  • 96. ruby libraries - neo4j gem by Andreas Ronge (https://github.com/andreasronge/neo4j)
  • 98. the neotech guys are awesome
  • 99. Querying the graph: Cypher declarative query language specific to neo4j easy to learn and intuitive use specific patterns to query for (something that looks like ‘this’) inspired partly by SQL (WHERE and ORDER BY) and SPARQL (pattern matching) focuses on what to query for and not how to query for it switch from a mySQl world is made easier by the use of cypher instead of having to learn a traversal framework straight away
  • 100. cypher clauses START: Starting points in the graph, obtained via index lookups or by element IDs. MATCH: The graph pattern to match, bound to the starting points in START. WHERE: Filtering criteria. RETURN: What to return. CREATE: Creates nodes and relationships. DELETE: Removes nodes, relationships and properties. SET: Set values to properties. FOREACH: Performs updating actions once per element in a list. WITH: Divides a query into multiple, distinct parts
  • 101. cypher clauses START: Starting points in the graph, obtained via index lookups or by element IDs. MATCH: The graph pattern to match, bound to the starting points in START. WHERE: Filtering criteria. RETURN: What to return. CREATE: Creates nodes and relationships. DELETE: Removes nodes, relationships and properties. SET: Set values to properties. FOREACH: Performs updating actions once per element in a list. WITH: Divides a query into multiple, distinct parts
  • 102. an example Person name: Sam Person name: Volker :friends :knows since: 2005 Person name: Megan :friends Company name: eBay :works_for :works_for Person name: Jim :friends :works_for Company name: neotech
  • 103. find all the companies my friends work for MATCH (person{ name:’Volker’ }) -[:friends] - (person) - [:works_for]-> company RETURN company
  • 104. find all the companies my friends work for MATCH (person{ name:’Volker’ }) -[:friends] - (person) - [:works_for]-> company RETURN company
  • 105. find all the companies my friends work for MATCH (person{ name:’Volker’ }) -[:friends] - (person) - [:works_for]-> company RETURN company Person name: Sam Person name: Volker :friends :knows since: 2005 Person name: Megan :friends Company name: eBay :works_for :works_for Person name: Jim :friends :works_for Company name: neotech
  • 106. find all the companies my friend’s friends work for MATCH (person{ name:’Volker’ }) - [:friends*2..2]-(person) - [:works_for] -> company RETURN company
  • 107. find all the companies my friend’s friends work for MATCH (person{ name:’Volker’ }) - [:friends*2..2]-(person) - [:works_for] -> company RETURN company
  • 108. find all the companies my friend’s friends work for MATCH (person{ name:’Volker’ }) - [:friends*2..2]-(person) - [:works_for] -> company RETURN company Person name: Sam Person name: Volker :friends :knows since: 2005 Person name: Megan :friends Company name: eBay :works_for :works_for Person name: Jim :friends :works_for Company name: neotech
  • 109. find all my friends who work for neotech MATCH (person{ name:’Volker’ }) -[:friends] -(friends) - [:works_for]-> company WHERE company.name = ‘neotech’ RETURN friends
  • 110. find all my friends who work for neotech MATCH (person{ name:’Volker’ }) -[:friends] -(friends) - [:works_for]-> company WHERE company.name = ‘neotech’ RETURN friends
  • 111. find all my friends who work for neotech MATCH (person{ name:’Volker’ }) -[:friends] -(friends) - [:works_for]-> company WHERE company.name = ‘neotech’ RETURN friends Person name: Sam Person name: Volker :friends :knows since: 2005 Person name: Megan :friends Company name: eBay :works_for :works_for Person name: Jim :friends :works_for Company name: neotech
  • 112. a good place to try it out: ! http://console.neo4j.org/ ! http://gist.neo4j.org/
  • 113. coverage example Locality id = california Locality id = marin_county Locality id = 94901 :contains :contains Locality id = 94903 :contains :contains Locality id = 94902
  • 114. coverage example Locality id = california Locality id = marin_county Locality id = 94901 :contains :contains Locality id = 94903 :contains :contains Locality id = 94902 :operates :operates Carrier id = carrier_1
  • 115. coverage example Locality id = california Locality id = marin_county Locality id = 94901 :contains Store id = ebay_store :located :contains Locality id = 94903 :contains :contains Locality id = 94902 :operates :operates Carrier id = carrier_1
  • 116. coverage example Locality id = california Locality id = marin_county Locality id = 94901 :contains Store id = ebay_store :located :contains Locality id = 94903 :contains :contains Locality id = 94902 :operates Carrier id = carrier_2 :operates :operates Carrier id = carrier_1
  • 117. MATCH (store{ id:’ebay_store’ }) -[:located] -> (locality) <- [:operates]- carrier RETURN carrier the query
  • 118. the query MATCH (store{ id:’ebay_store’ }) -[:located] -> (locality) <- [:operates]- carrier RETURN carrier
  • 119. MATCH (store{ id:’ebay_store’ }) -[:located] -> (locality) <- [:operates]- carrier RETURN carrier the query Locality id = california Locality id = 94902 Locality id = marin_county Locality id = 94901 :contains Store id = ebay_store :located :contains Locality id = 94903 :contains :contains :operates :operates Carrier id = carrier_1
  • 120. MATCH (store{ id:’ebay_store’ }) -[:located] -> () <- [:contains*0..2] - (locality) <- [:operates]- carrier RETURN carrier the query
  • 121. MATCH (store{ id:’ebay_store’ }) -[:located] -> () <- [:contains*0..2] - (locality) <- [:operates]- carrier RETURN carrier the query
  • 122. MATCH (store{ id:’ebay_store’ }) -[:located] -> () <- [:contains*0..2] - (locality) <- [:operates]- carrier RETURN carrier the query Locality id = california Locality id = marin_county Locality id = 94901 :contains Store id = ebay_store :located :contains Locality id = 94903 :contains :contains Locality id = 94902 :operates Carrier id = carrier_2 :operates :operates Carrier id = carrier_1
  • 123. MATCH (store{ id:’ebay_store’ }) -[:located] -> () <- [:contains*0..2] - (locality) <- [:operates]- carrier RETURN carrier the query Locality id = california Locality id = marin_county Locality id = 94901 :contains Store id = ebay_store :located :contains Locality id = 94903 :contains :contains Locality id = 94902 :operates Carrier id = carrier_2 :operates :operates Carrier id = carrier_1
  • 124. MATCH (store{ id:’ebay_store’ }) -[:located] -> () <- [:contains*0..2] - (locality) <- [:operates]- carrier RETURN carrier the query Locality id = california Locality id = marin_county Locality id = 94901 :contains Store id = ebay_store :located :contains Locality id = 94903 :contains :contains Locality id = 94902 :operates Carrier id = carrier_2 :operates :operates Carrier id = carrier_1
  • 125. MATCH (store{ id:’ebay_store’ }) -[:located] -> () <- [:contains*0..2] - (locality) <- [:operates]- carrier RETURN carrier the query Locality id = california Locality id = marin_county Locality id = 94901 :contains Store id = ebay_store :located :contains Locality id = 94903 :contains :contains Locality id = 94902 :operates Carrier id = carrier_2 :operates :operates Carrier id = carrier_1
  • 126. SELECT * FROM carriers LEFT JOIN locations ON carrier.location_id = location.id LEFT JOIN stores ON stores.location_id = carrier.location_id WHERE stores.name = ‘ebay_store’
  • 127. SELECT * FROM carriers LEFT JOIN locations ON carrier.location_id = location.id OR carrier.location_id = location.parent_id LEFT JOIN stores ON stores.location_id = carrier.location_id WHERE stores.name = ‘ebay_store’
  • 128. ?
  • 129. MATCH (store{ id:’ebay_store’ }) -[:located] -> () <- [:contains*0..2] - (locality) <- [:operates]- carrier RETURN carrier
  • 130. representing dates/times root (0) :year_2014 Year: 2013 :year_2015 Year: 2015 :month_06 :month_05 :month_01 Month: 05 Month: 01 Month: 06 :day_24 :day_25 Day: 24 Day: 25 :day_26 Day: 26 :happens :happens :happens :happens Event 1 Event 2 Event 3
  • 131. find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event
  • 132. find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event
  • 133. find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event root (0) :year_2014 Year: 2013 :year_2015 Year: 2015 :month_06 :month_05 :month_01 Month: 05 Month: 01 Month: 06 :day_24 :day_25 Day: 24 Day: 25 :day_26 Day: 26 :happens :happens :happens :happens Event 1 Event 2 Event 3
  • 134. find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event root (0) :year_2014 Year: 2013 :year_2015 Year: 2015 :month_06 :month_05 :month_01 Month: 05 Month: 01 Month: 06 :day_24 :day_25 Day: 24 Day: 25 :day_26 Day: 26 :happens :happens :happens :happens Event 1 Event 2 Event 3
  • 135. find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event root (0) :year_2014 Year: 2013 :year_2015 Year: 2015 :month_06 :month_05 :month_01 Month: 05 Month: 01 Month: 06 :day_24 :day_25 Day: 24 Day: 25 :day_26 Day: 26 :happens :happens :happens :happens Event 1 Event 2 Event 3
  • 136. find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event root (0) :year_2014 Year: 2013 :year_2015 Year: 2015 :month_06 :month_05 :month_01 Month: 05 Month: 01 Month: 06 :day_24 :day_25 Day: 24 Day: 25 :day_26 Day: 26 :happens :happens :happens :happens Event 1 Event 2 Event 3
  • 137. find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event root (0) :year_2014 Year: 2013 :year_2015 Year: 2015 :month_06 :month_05 :month_01 Month: 05 Month: 01 Month: 06 :day_24 :day_25 Day: 24 Day: 25 :day_26 Day: 26 :happens :happens :happens :happens Event 1 Event 2 Event 3
  • 138. find all events on a specific day START root=node(0) MATCH root - [:year_2014] -> () -[:month_05] -> ()- [:day_24] -> () - [:happens] -> event RETURN event root (0) :year_2014 Year: 2013 :year_2015 Year: 2015 :month_06 :month_05 :month_01 Month: 05 Month: 01 Month: 06 :day_24 :day_25 Day: 24 Day: 25 :day_26 Day: 26 :happens :happens :happens :happens Event 1 Event 2 Event 3
  • 139. all together Locality id = california Locality id = marin_county :contains Locality id = 94901 :contains Store id = ebay_store :located :available {premium: 1.5} Carrier id = carrier_1 :operates root (0) :year_2014 Year: 2013 :month_05 Month: 05 :day_24 Day: 24 hour 09 :hour_09 hour 10 :hour_10 hour 11 :hour_11 :available {premium: 1}
  • 140. all together MATCH (store{ id:’ebay_store’ }) -[:located] -> (locality) <- [:operates]- carrier - [available:available] -> () <- [:hour_10] - () <- [:day_24] - () [:month_05] - () [:year_2014] - () RETURN carrier, available.premium as premium
  • 141. all together MATCH (store{ id:’ebay_store’ }) -[:located] -> (locality) <- [:operates]- carrier - [available:available] -> () <- [:hour_10] - () <- [:day_24] - () [:month_05] - () [:year_2014] - () RETURN carrier, available.premium as premium
  • 142. all together MATCH (store{ id:’ebay_store’ }) -[:located] -> (locality) <- [:operates]- carrier - [available:available] -> () <- [:hour_10] - () <- [:day_24] - () [:month_05] - () [:year_2014] - () RETURN carrier, available.premium as premium Locality id = california Locality id = marin_county :contains Locality id = 94901 :contains Store id = ebay_store :located :available {premium: 1.5} Carrier id = carrier_1 :operates root (0) :year_2014 Year: 2013 :month_05 Month: 05 :day_24 Day: 24 hour 09 :hour_09 hour 10 :hour_10 hour 11 :hour_11 :available {premium: 1}
  • 144. Other graph uses • Recommendation engines
  • 145. Other graph uses • Recommendation engines • Organisational analysis
  • 146. Other graph uses • Recommendation engines • Organisational analysis • Graphing your infrastructure
  • 148. Some gotchas • There was a learning curve in switching from a relational mentality to a graph one
  • 149. Some gotchas • There was a learning curve in switching from a relational mentality to a graph one • Tooling not as mature as in the relational world
  • 150. Some gotchas • There was a learning curve in switching from a relational mentality to a graph one • Tooling not as mature as in the relational world • No out of the box solution for db migrations
  • 151. Some gotchas • There was a learning curve in switching from a relational mentality to a graph one • Tooling not as mature as in the relational world • No out of the box solution for db migrations • Seeding an embedded database was unfamiliar
  • 152. Testing was a challenge
  • 153. Testing was a challenge • Setting up scenarios for tests was tedious
  • 154. Testing was a challenge • Setting up scenarios for tests was tedious • Built our own tool based on the geoff syntax developed by Nigel Small
  • 155. Testing was a challenge • Setting up scenarios for tests was tedious • Built our own tool based on the geoff syntax developed by Nigel Small • Geoff allows modelling of graphs in textual form and provides an interface to insert them into an existing graph
  • 156. Testing was a challenge • Setting up scenarios for tests was tedious • Built our own tool based on the geoff syntax developed by Nigel Small • Geoff allows modelling of graphs in textual form and provides an interface to insert them into an existing graph (A) {“name”: “Alice”}
  • 157. Testing was a challenge • Setting up scenarios for tests was tedious • Built our own tool based on the geoff syntax developed by Nigel Small • Geoff allows modelling of graphs in textual form and provides an interface to insert them into an existing graph (A) {“name”: “Alice”} (B) {“name”: “Bob”}
  • 158. Testing was a challenge • Setting up scenarios for tests was tedious • Built our own tool based on the geoff syntax developed by Nigel Small • Geoff allows modelling of graphs in textual form and provides an interface to insert them into an existing graph (A) {“name”: “Alice”} (B) {“name”: “Bob”} (A) -[:KNOWS] -> (B)
  • 159. Testing was a challenge • Setting up scenarios for tests was tedious • Built our own tool based on the geoff syntax developed by Nigel Small • Geoff allows modelling of graphs in textual form and provides an interface to insert them into an existing graph (A) {“name”: “Alice”} (B) {“name”: “Bob”} (A) -[:KNOWS] -> (B) • We created a Ruby dsl for modelling a graph and inserting it into the db that works with factory_girl
  • 160. Testing was a challenge • Setting up scenarios for tests was tedious • Built our own tool based on the geoff syntax developed by Nigel Small • Geoff allows modelling of graphs in textual form and provides an interface to insert them into an existing graph (A) {“name”: “Alice”} (B) {“name”: “Bob”} (A) -[:KNOWS] -> (B) • We created a Ruby dsl for modelling a graph and inserting it into the db that works with factory_girl • Open source - https://github.com/shutl/geoff
  • 162. Wrap Up • Neo4j and graph theory enabled Shutl to achieve big performance increases in its most important operation - calculating delivery prices
  • 163. Wrap Up • Neo4j and graph theory enabled Shutl to achieve big performance increases in its most important operation - calculating delivery prices • It’s a new tool based on tested theory, and cypher is the first language that allows you to query graphs in a declarative way (like SQL)
  • 164. Wrap Up • Neo4j and graph theory enabled Shutl to achieve big performance increases in its most important operation - calculating delivery prices • It’s a new tool based on tested theory, and cypher is the first language that allows you to query graphs in a declarative way (like SQL) • Tooling and adoption is immature but getting better all the time
  • 165. Thank you! ! Any questions? Sam Phillips Head of Engineering ! @samsworldofno http://samsworldofno.com sam@shutl.com Volker Pacher Senior Developer ! @vpacher https://github.com/vpacher volker@shutl.com
  • 167. our
  • 168. Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations/shutl-neo4j- graph-db