A Gremlin ate my graph
2 likes879 views
The document provides an in-depth exploration of Gremlin and its application in analyzing graph data, specifically focusing on a dataset derived from WordPress function calls. It discusses the structure of graphs, including vertices and edges, and demonstrates various Gremlin queries for filtering, transforming, and counting nodes. Additionally, it highlights the state of Gremlin as well as related tools and resources for implementation.
![g.v(1).id => 1
g.v(1).name => apply_filter
g.v(1).version => true
g.v(1).compat => [4.0, 4.1, 4.2, 4.3]
// non-existing properties
g.v(1).isFedAfterMidnight => null
Properties
Graph is schemaless
apply_filter](https://image.slidesharecdn.com/gremlin2015-151031093346-lva1-app6891/85/A-Gremlin-ate-my-graph-7-320.jpg)
![g.v(30).out(‘CALLS’)
.retain([g.v(30)])
.name => get_category_parents
Is it Recursive
get_permalink get_category_
link
get_category_
parents
id : 30
CALLS
CALLS
CALLS](https://image.slidesharecdn.com/gremlin2015-151031093346-lva1-app6891/85/A-Gremlin-ate-my-graph-18-320.jpg)
![Is it Recursive
g.v(30).in(‘CALLS’)
.retain([g.v(30)])
.name => get_category_parents
get_permalink get_category_
link
get_category_
parents
id : 30
CALLS
CALLS
CALLS](https://image.slidesharecdn.com/gremlin2015-151031093346-lva1-app6891/85/A-Gremlin-ate-my-graph-19-320.jpg)
![g.v(47).out(‘CALLS’).except([g.v(47)])
.out('CALLS').retain([g.v(47)])
.name
=> wp_trash_comment
Ping-Pong Function
CALLS
wp_trash
_comment
id: 47
wp_delete
_comment
id : 148
CALLS
CALLS
CALLS
CALLS
CALLS
CALLS
CALLS
CALLS](https://image.slidesharecdn.com/gremlin2015-151031093346-lva1-app6891/85/A-Gremlin-ate-my-graph-20-320.jpg)
![Sampling
g.V[0..3].name => do_activate_header
do_action
wpmu_activate_stylesheet
comment_footer_die
g.V[0..3].id => 1
3
5
4](https://image.slidesharecdn.com/gremlin2015-151031093346-lva1-app6891/85/A-Gremlin-ate-my-graph-24-320.jpg)
![Get Linking Index
g.V.transform{['name':it.name,
'links':it.in('CALLS').count()]}
=> ...
{'name':wpmu_signup_stylesheet, 'links':0}
{'name':show_blog_form, 'links':7}
{'name':validate_blog_form, 'links':3}
{'name':show_user_form, 'links':4}](https://image.slidesharecdn.com/gremlin2015-151031093346-lva1-app6891/85/A-Gremlin-ate-my-graph-34-320.jpg)
![Get Called Index
g.V.transform{['name':it.name,
'links':it.in('CALLS').count()]
}
.order{ it.a.links <=> it.b.links}
=> {'name':get_post, 'links':191}
{'name':get_option, 'links':218}
{'name':_deprecated_function, 'links':296}
{'name':__, 'links':442}
{'name':apply_filters, 'links':598}](https://image.slidesharecdn.com/gremlin2015-151031093346-lva1-app6891/85/A-Gremlin-ate-my-graph-35-320.jpg)
![Most linked Function
groupCount(m)
m = [:];
g.V.groupCount(m);
m;
=> {
v[1] = 1,
v[2] = 1,
...
v[47] = 1,
}](https://image.slidesharecdn.com/gremlin2015-151031093346-lva1-app6891/85/A-Gremlin-ate-my-graph-36-320.jpg)
![Most linked Function
groupCount(m){key}{value}
m = [:];
g.V.groupCount(m){it.name}
{it.in('CALLS').count()};
m;
=> {
wp_restore_image = 1,
press_this_media_buttons = 0,
WP_Filesystem = 3,
...
}](https://image.slidesharecdn.com/gremlin2015-151031093346-lva1-app6891/85/A-Gremlin-ate-my-graph-37-320.jpg)
![Most linked Function
groupCount(m){key}{value}
m = [:];
g.V.groupCount(m){it.name}
{it.in('CALLS').count()};
m.sort{ -it.value }[0..2];
=> {
apply_filters = 598,
__ = 442,
_deprecated_function = 296
}](https://image.slidesharecdn.com/gremlin2015-151031093346-lva1-app6891/85/A-Gremlin-ate-my-graph-38-320.jpg)
![Most linked Function
m = [:];n = [:];
g.V.groupCount(m){it.name}
{it.in('CALLS').count()}
.groupCount(n){it.name}
{it.out('CALLS').count()};
n.sort{ -it.value}[0..2];
=> {
redirect_canonical = 60,
export_wp = 47,
edit_post = 36](https://image.slidesharecdn.com/gremlin2015-151031093346-lva1-app6891/85/A-Gremlin-ate-my-graph-39-320.jpg)
![SideEffect Steps
sideEffect : emit incoming but allows for side
computation
m = [:];n = [:];
g.V.groupCount(m){it.name}
{it.in('CALLS').count()}
.groupCount(n){it.name}
{it.out('CALLS').count()}
.count();
=> 2692](https://image.slidesharecdn.com/gremlin2015-151031093346-lva1-app6891/85/A-Gremlin-ate-my-graph-40-320.jpg)
A Gremlin ate my graph
- 1. A gremlin ate my graph Barcelona, Spain, October, 30th 2015
- 2. Agenda Discover the Graph Steps with a gremlin The state of the Gremlin
- 3. Damien Seguy CTO at Exakat PHP code as Dataset Speaker
- 5. What is gremlin? V :Vertices, or nodes or objects E : Edges, or links or relations G : graph, or the dataset
- 6. The Vertices g.vrepresents all the vertices g.v(1) is one of the nodes Vertices always have an id g.v(1) => v(1) 1
- 7. g.v(1).id => 1 g.v(1).name => apply_filter g.v(1).version => true g.v(1).compat => [4.0, 4.1, 4.2, 4.3] // non-existing properties g.v(1).isFedAfterMidnight => null Properties Graph is schemaless apply_filter
- 8. Vertice discovery Use map to discover the graph g.v(2).map => {name=wp_die, leaf=true} wp_die
- 9. The Edges Edges have id, properties also : start, end and label g.E represents all the edges g.e(1) is the edge with Id 1
- 10. g.e(1) => e(1) g.e(1).map => { } g.e(1).id => 1 g.e(1).label => CALLS Edge discovery wp_diewp_ajax_fetch _list CALLS Edges link the vertices
- 11. g.e(5).outV => v(2) g.e(6).outV => v(3) g.e(1).inV => v(4) Running on the Edges 2 3 4 1 5 6 7 Directed graph
- 12. g.v(1).out => v(2) v(3) g.v(1).in => v(4) g.v(1).both => v(2) v(3) v(4) Following Edges 2 3 4 1 5 6 7 g.v(1).inE => e(7)
- 13. g.v(1).out.id => 2 3 g.v(2).in.in.id => 4 g.v(1).both.id => 2 3 4 Chaining 2 3 4 1 5 6 7
- 14. Wordpress Calls Graph The graph of all Wordpress internal function calls function :name function :name CALLS
- 17. g.v(19).out(‘CALLS’).name => wp_hash_password wp_cache_delete g.v(19).in(‘CALLS’).name => reset_password wp_check_password Calling home wp_set_ password reset_ password wp_hash_ password wp_check_ password wp_cache_ delete CALLS CALLS CALLS CALLS
- 18. g.v(30).out(‘CALLS’) .retain([g.v(30)]) .name => get_category_parents Is it Recursive get_permalink get_category_ link get_category_ parents id : 30 CALLS CALLS CALLS
- 19. Is it Recursive g.v(30).in(‘CALLS’) .retain([g.v(30)]) .name => get_category_parents get_permalink get_category_ link get_category_ parents id : 30 CALLS CALLS CALLS
- 20. g.v(47).out(‘CALLS’).except([g.v(47)]) .out('CALLS').retain([g.v(47)]) .name => wp_trash_comment Ping-Pong Function CALLS wp_trash _comment id: 47 wp_delete _comment id : 148 CALLS CALLS CALLS CALLS CALLS CALLS CALLS CALLS
- 21. Up to now nodes and vertices : basic blocs in and out (and both) : navigation except(), retain(), in(‘label’) : filtering Starting at the vertices
- 22. Traversing the graph Finding nodes in the graph that satisfy criteria Traversing involves listing nodes, following links, applying filters, processing data until all conditions are met Starting point : g.V and g.E
- 23. Counting g.V.count() => 2691 g.E.count() => 9013 function functionCALLS
- 24. Sampling g.V[0..3].name => do_activate_header do_action wpmu_activate_stylesheet comment_footer_die g.V[0..3].id => 1 3 5 4
- 26. Dying Functions g.V.out('CALLS') .has('name','wp_die') .count() => 84 ???? wp_dieCALLS
- 27. g.V.out('CALLS') .has('name','wp_die') .name => Dying Functions ???? wp_dieCALLS PROCESSING wp_die wp_die wp_die wp_die wp_die wp_die wp_die wp_die wp_die
- 28. g.V.has('name','wp_die') .in('CALLS') .name => wp_ajax_trash_post wp_ajax_delete_post wp_ajax_delete_meta wp_ajax_delete_link wp_ajax_delete_tag wp_ajax_delete_comment wp_ajax_oembed_cache wp_ajax_imgedit_preview we_ajax_fetch_list Dying Functions ???? wp_dieCALLS PROCESSING
- 29. g.V.as('start') .out('CALLS') .has('name','wp_die') .back('start') .name => wp_ajax_trash_post wp_ajax_delete_post wp_ajax_delete_meta wp_ajax_delete_link wp_ajax_delete_tag wp_ajax_delete_comment wp_ajax_oembed_cache wp_ajax_imgedit_preview Dying Functions ???? wp_dieCALLS PROCESSING
- 30. Relay Functions g.V.filter{ it.out('CALLS').count() == 1} .count() => 650 CALLS CALLS CALLS
- 31. Closures Steps often offer possibility for closure Closure is between {} , uses ‘it’ as current node, is written in Groovy (or else) Closure often have a standard default behavior, so they are sometimes stealth
- 32. Applied to properties Non standard functions g.V.filter{ it.name != it.name.toLowerCase()} .count() => 73
- 33. Leaf and Roots LEAF ROOT g.V.filter{ it.out('CALLS').any() == false} .count() => 407 g.V.filter{ it.in('CALLS').any() == false} .count() => 1304
- 34. Get Linking Index g.V.transform{['name':it.name, 'links':it.in('CALLS').count()]} => ... {'name':wpmu_signup_stylesheet, 'links':0} {'name':show_blog_form, 'links':7} {'name':validate_blog_form, 'links':3} {'name':show_user_form, 'links':4}
- 35. Get Called Index g.V.transform{['name':it.name, 'links':it.in('CALLS').count()] } .order{ it.a.links <=> it.b.links} => {'name':get_post, 'links':191} {'name':get_option, 'links':218} {'name':_deprecated_function, 'links':296} {'name':__, 'links':442} {'name':apply_filters, 'links':598}
- 36. Most linked Function groupCount(m) m = [:]; g.V.groupCount(m); m; => { v[1] = 1, v[2] = 1, ... v[47] = 1, }
- 37. Most linked Function groupCount(m){key}{value} m = [:]; g.V.groupCount(m){it.name} {it.in('CALLS').count()}; m; => { wp_restore_image = 1, press_this_media_buttons = 0, WP_Filesystem = 3, ... }
- 38. Most linked Function groupCount(m){key}{value} m = [:]; g.V.groupCount(m){it.name} {it.in('CALLS').count()}; m.sort{ -it.value }[0..2]; => { apply_filters = 598, __ = 442, _deprecated_function = 296 }
- 39. Most linked Function m = [:];n = [:]; g.V.groupCount(m){it.name} {it.in('CALLS').count()} .groupCount(n){it.name} {it.out('CALLS').count()}; n.sort{ -it.value}[0..2]; => { redirect_canonical = 60, export_wp = 47, edit_post = 36
- 40. SideEffect Steps sideEffect : emit incoming but allows for side computation m = [:];n = [:]; g.V.groupCount(m){it.name} {it.in('CALLS').count()} .groupCount(n){it.name} {it.out('CALLS').count()} .count(); => 2692
- 41. Nature of Steps filter step : emit input if condition is satisfied : has(), filter(), retain(), except() map step : transform input into another object : in(), out(), BOTH() sideEffect step : emit input but allows for side computation : transform, groupCount, each, sideEffect Branch and flatMap
- 42. Lonely Functions g.V.filter{ it.in('CALLS').any() == false} .filter{ it.out('CALLS').any() == false} .sideEffect{ it.lonely = true; } .count() => 184 SELECT, UPDATE AND COUNT
- 43. Updating a node g.V.sideEffect{ incoming = it.in('CALLS').count(); } .each{ it.setProperty('incoming', incoming); it.setProperty('outgoing', it.out('CALLS').count()); }
- 44. Updating The Graph // removing deprecated functions g.V.filter{ it.out('CALLS') .has('name', '_deprecated_function' .any() } .each{ it.bothE.each{ g.removeEdge(it); } g.removeVertex(it); }
- 45. State of Gremlin
- 49. Gremlin Variants
- 50. Gremlin For PHP https://github.com/PommeVerte/gremlin-php Get up and running with Tinkerpop 3 and PHP : https://dylanmillikin.wordpress.com/2015/07/20/ get-up-and-running-with-tinkerpop-3-and-php/ Using with Neo4j : REST API Older API : neo4jPHP, rexpro-php
- 51. Thanks dseguy@exakat.io @exakat http://www.slideshare.net/dseguy/ on the http://2015.phpconference.es//