Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono

A Common Graph
Database Access Layer
for .NET and Mono

Achim Friedland <achim@graph-database.org>, Aperis GmbH FOSDEM 2011 - Mono devroom

Graphs

Any structure having vertices and edges...
2

...but different representations possible.

Adjacency matrix vs. Incidence matrix vs. Adjacency list vs.
Edge list vs. Classes, Index-based vs. Index-free Adjacency,
Dense vs. Sparse graphs, On-disc vs. In-memory graphs...

3

The problem...

• Different graph representations...
• have different levels of expressivity
• can be very application speciﬁc
• hard to optimize a single one for every
use-case
• Multiple APIs from different vendors

4

So, where to ﬁnd a common API?

5

Graph Commons

• Edges are ﬁrst-class citizens
• Information gathering by traversing
Indices are less important than in relational DBs

• Index-free adjacency, so the cost of
traversing an edge is in-depended from the
size of the graph
( ...at least when you ignore the GC ;)

6

The Property-Graph Model
The most common graph model within
the NoSQL GraphDB space

edge label
Id: 1 Id: 2
Friends
name: Alice name: Bob
since: 2009/09/21
age: 21 age: 23
edge
vertex
properties
properties

• directed: Each edge has a source and destination vertex
• attributed: Vertices and edges carry key/value pairs
• edge-labeled: The label denotes the type of relationship
• multi-graph: Multiple edges between any two vertices allowed
7

A Property Graph Model Interface for .NET and Mono

// Use a class-based in-memory graph
var graph = new InMemoryGraph();

var v1 = graph.AddVertex(new VertexId(1));
var v2 = graph.AddVertex(new VertexId(2));
v1.SetProperty("name", "Alice");
v1.SetProperty("age" , 21);
v2.SetProperty("name", "Bob");
v2.SetProperty("age" , 23);

var e1 = graph.AddEdge(v1, v2, new EdgeId(1), "Friends");
e1.SetProperty(“since”, ”2009/09/21”);

8

Or, if you prefer the Dynamic Language Runtime...

// Use a class-based in-memory graph
var graph = new InMemoryGraph();

var v1 = graph.AddVertex().AsDynamic();
var v2 = graph.AddVertex().AsDynamic();
v1.name = "Alice";
v1.age = 21;
v2.name = "Bob";
v2.age = 23;

var e1 = graph.AddEdge(v1, v2, "Friends").AsDynamic();
e1.since = ”2009/09/21”;

9

Current status...

• About 70% of the current JAVA original
• e.g. Transactions, Auto-Indexing still missing
• Expect ﬁrst release within the next 4-6 weeks

10

The future...

• Different implementations for different
graph representations
• Implementations for remote graphs
(Rexster and Neo4J REST in separate projects)

• Extensions, e.g. for semantic graphs

11

Ok, but how to query a property-
graph?

12

A data ﬂow framework for property graph models

: IEnumerator<E>, IEnumerable<E>

S AbstractPipe<S, E> E

Source Elements Emitted Elements

13

A simple pipe yielding all objects without modiﬁcation

public class IdentityPipe<S> : AbstractPipe<S, S> {
public override Boolean MoveNext() {
if (_InternalEnumerator == null)
return false;
if (_InternalEnumerator.MoveNext()) {
_CurrentElement = _InternalEnumerator.Current;
return true;
}
return false;
}
}

14

A simple pipe yielding all strings turned to uppercase

public class ToUpperPipe<String> : AbstractPipe<String, String> {
return false;
_CurrentElement = _InternalEnumerator.Current.ToUpper();
return true;
}
return false;
}
}

15

A simple pipe yielding the vertex property “name”

public class GetNameProperty : AbstractPipe<IVertex, String> {
return false;
_CurrentElement = _IVertex.GetProperty(“name”);
return true;
}
return false;
}
}

16

Allow the creation of state or side effects within a pipe

S ISideEffectPipe<in S, out E, out T> E
Source Emitted
Elements T Elements

Side Effect

17

Count the total number of elements passed through the pipe

public class CountPipe<S> : AbstractPipe<S,S>, ISideEffectPipe<S,S,Int64 {
[...]
return false;
_CurrentElement = _InternalEnumerator.Current;
Interlocked.Increment(ref _Counter);
return true;
}
return false;
}
public Int64 SideEffect { get { return _Counter; }}
}

18

Create complex pipes by combining pipes to pipelines

S Pipeline<S, E> E
pipe1<S,A> pipe2<B,C> pipe3<C,E>
Source Emitted
Elements Elements

19


// Friends-of-a-friend
var pipe1 = new VertexEdgePipe(VertexEdgePipe.Step.OUT_EDGES);
var pipe2 = new LabelFilterPipe("Friends", ComparisonFilter.EQUALS);
var pipe3 = new EdgeVertexPipe(EdgeVertexPipe.Step.IN_VERTEX);
var pipe7 = new PropertyPipe("name");

var pipeline = new Pipeline(pipe1,pipe2,pipe3,pipe4,pipe5,pipe6,pipe7);
pipeline.SetSource(new SingleEnumerator(
graph.GetVertex(new VertexId(1))));

foreach (var _foaf in pipeline.Take(5)) {
Console.WriteLine(_foaf);
}

20


• Think of “LINQ for graphs”
• ...but without the syntactic sugar.
• Very Powerful, but also very “noisy” :(

21

Current status & the future

• Expect ﬁrst release with the next weeks

22

And what about ad hoc querying for
exploring a property-graph?

23

Ad hoc Query Language for graphs

• Ad Hoc querying a property graph
• User-friendly wrapper around pipes
• “perl for graphs” ;)

24

Ad hoc Query Language for graphs

// Friends-of-a-friend
var pipe7 = new PropertyPipe("name");

var pipeline = new Pipeline(pipe1,pipe2,pipe3,pipe4,pipe5,pipe6,pipe7);
pipeline.SetSource(new SingleEnumerator(
graph.GetVertex(new VertexId(1))));

g:id-v(1)/outE[@label='Friends']/inV/outE
[@label='Friends']/inV/@name
25

Current status & the future

• Reimplementation not yet started :(
• Will probably be based on the DLR
• Expect ﬁrst release with the next 2-3 months

26

But...

• Do not expect Gremlin as the one-and-only
query language for each use-case!
• More application speciﬁc Query Languages
could be implemented using pipes
e.g. sones GQL, OrientDB SQL

27

How to access a remote graph?
How to expose a graph via the network?

28

A HTTP/REST interface for
Blueprints Property Graph Models

• Exposes your application-embedded Blueprints
graph via HTTP/REST
• Vertices and edges are REST resources
• The rexster client library allows you to access most
JAVA-based GraphDBs
Neo4J, OrientDB available; DEX, InﬁniteGraph coming soon...

29

Common CRUD Operations...

30


31


32

Default resource representation: JSON

curl -H Accept:application/json http://localhost:8182/graph1/vertices/1
{
"version" : "0.1",
"results" : {
"_type" : "vertex",
"_id" : "1",
"name" : "Alice",
"age" : 21
},
"query_time" : 0.014235
}

33

Current status

• Expect ﬁrst release with the next 3-4 weeks
• Rexster Shell might take some more time...

34

Future Resource Representations

• Link-aware, self-describing hypermedia (see Neo4J)
• e.g. ATOM, XML + XLINK, RDFa
• Application speciﬁc protocols
• GEXF for exporting a graph to GEPHI
• GraphML/GDF/... graph formats
• MediaRSS your photo graph traversal via Cooliris

35

The GraphDB Graph...

OrientDB for Documents ThinkerGraph &
Gremlin for Ad Hoc

InﬁniteGraph for Neo4J for GIS
Clustering

InfoGrid for WebApps In-Memory for
Caching

OrientDB for Ad Hoc
Neo4J for HA

36

Still hungry for more?

FOSDEM 2011 GraphDB dinner!
Meet us in front of Le Roy d’Espagne
Grand Place 1, Brussels
20:00 PM

37

Questions?

http://www.graph-database.org
http://www.twitter.com/graphdbs

38

Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono

More Related Content

What's hot (18)

Viewers also liked (20)

Similar to Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono (20)

More from Achim Friedland (17)

Recently uploaded (20)

Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono