Meetup: An Introduction to InfiniteGraph, and Connecting the Dots in Big Data.

Graph Database Overview and Feature Update Darren Wood Chief Architect, InfiniteGraph

History Objectivity – Massively scalable, distributed object oriented database Used in Government (DoD, Intelligence) Machine generated data such as sensor, acoustic… OEM Markets Either complex data models, or high ingest or both Significant technical advantage in highly connected (many-to-many) data models Copyright © InfiniteGraph

Graph Databases Optimized around data relationships Relationships as first class citizens Super fast traversal between entities Rich/flexible annotation of connections Small focused API (typically not SQL) Natively work with concepts of Vertex/Edge SQL has no concept of “navigation” Most attempts based in SQL are convoluted Copyright © InfiniteGraph

Distributed Graph Must Haves High performance distributed persistence Ability to deal with remote data reads (fast) Intelligent local cache of subgraphs Distributed navigation processing Distributed, multi-source concurrent ingest Write modes supporting both strict and eventual consistency Copyright © InfiniteGraph

Some Code Copyright © InfiniteGraph Vertex alice = myGraph.addVertex(new Person(“Alice”)); Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); bob.addEdge(new Call(timestamp), charlie); Alice Carlos Charlie Bob Meets Calls Pays Calls

Physical Storage Comparison Copyright © InfiniteGraph Meetings P1 Place Time P2 Alice Denver 5-27-10 Bob Calls From Time Duration To Bob 13:20 25 Carlos Bob 17:10 15 Charlie Payments From Date Amount To Carlos 5-12-10 100000 Charlie Met 5-27-10 Alice Called 13:20 Bob Payed 100000 Carlos Charlie Called 17:10 Rows/Columns/Tables Relationship/Graph Optimized

Query and Navigation Queries – but not as you know them More like a rules based search and discovery Asynchronous Results Copyright © InfiniteGraph Alice Carlos Charlie Bob Meets Calls Pays Calls “ Find all paths between Alice and Charlie” “ Find all paths between Alice and Charlie – within 2 degrees” “ Find all paths between Alice and Charlie – events in May 2010”

Navigation Example Copyright © InfiniteGraph // Create a qualifier that describes the target vertex Qualifier findCharliePredicate = new VertexPredicate(personType, "name == ’Charlie'" ); // Construct a navigator which starts with Alice and uses a result qualifier // to find all paths in the graph to Charlie Navigator charlieFinder = alice.navigate( Guide.SIMPLE_BREADTH_FIRST, // default guide Qualifier.ANY, // no path constraints findCharliePredicate , // find paths ending with Charlie myResultHandler); // fire results to supplied handler // Start the navigator charlieFinder.start();

Management of Large Data Graphs Graphs grow quickly Billions of phone calls / day in US Emails, social media events, IP Traffic Financial transactions Some analytics require navigation of large sections of the graph Each step (often) depends on the last Must distribute data and go parallel Copyright © InfiniteGraph

Basic Architecture Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Extensions Blueprints User Apps Objectivity/DB Distributed Database Session / TX Management Placement

Accelerated Ingest Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Extensions Session / TX Management Placement Standard Blocking Ingest/Placement (MDP Plugin) Objectivity/DB App-1 (Ingest V 1 ) App-2 (Ingest V 2 ) App-3 (Ingest V 3 ) V 1 V 2 V 3 App-1 (E 1 2 { V 1 V 2 }) App-2 (E 23 { V 2 V 3 }) App-3 E 12 E 23

Accelerated Ingest Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Extensions Session / TX Management Placement (Standard) Placement (Accelerated) V 1 V 2 V 3 E 12 E 23 Distributed Pipelines Staging Containers Pipeline Containers E(1->2) E(3->1) E(2->3) E(2->1) E(2->3) E(3->1) E(1->2) E(3->2) E(1->2) E(2->3) E(3->1) E(2->1) E(2->3) E(3->1) E(3->2) E(1->2)

InfiniteGraph Visualizer Really nice flexible graph viewer Browser style navigation and history Full index support – search your data Display connections around a selected point Fully customize display to your data model Full data view via selection Copyright © InfiniteGraph

Indexing Framework Focused on providing choice ! Manual Indexes for grouping data Automatic Indexes for cross population Query interface with qualification language Pluggable query operators External index support (Lucene) Copyright © InfiniteGraph

Meetup: An Introduction to InfiniteGraph, and Connecting the Dots in Big Data.

More Related Content

Viewers also liked (19)

Similar to Meetup: An Introduction to InfiniteGraph, and Connecting the Dots in Big Data. (20)

More from InfiniteGraph (20)

Recently uploaded (20)

Meetup: An Introduction to InfiniteGraph, and Connecting the Dots in Big Data.