Smart Data Webinar: Emerging Data Management Options

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Emerging Data Management Options
Adrian Bowles, PhD

Founder, STORM Insights, Inc.

info@storminsights.com

Basic Life Advice
“When the map and the terrain disagree, believe the terrain.”
Gause and Weinberg (Exploring Requirements)
It is the pervading law of all things organic, and inorganic, of all things physical
and metaphysical, of all things human and all things superhuman, of all true
manifestations of the head, of the heart, of the soul, that the life is recognizable in
its expression, that form ever follows function. That is the law.
Louis Sullivan: The Tall Ofﬁce Building Artistically Considered, 1896

How You Think About a Domain…
…inﬂuences your choice of maps and models…

rules and representations…and required operations.

” To solve really hard problems, we'll have to use several different representations.
This is because each particular kind of data structure has its own virtues and
deficiencies, and none by itself would seem adequate for all the different functions
involved with what we call common sense.”
Marvin Minsky

What Do You Want/Need to Store?

How much? How complex? How fast?
What Do You Want/Need to DO With What You Store?
Do you need a graph database?
Options Include…
Files, tables, trees, queues, stacks, lists…

Hierarchical

RDBMS

Object DBMS

NoSQL

Graph

Perception/
NLP
Problem Solving
& Learning
Simple:
deterministic,
retrieve/calculate
Complex:
probabalistic
hypothesize, test,
rank, select
Creative:
discover, generate
ORGANIZED
Memory*
Input Class/Type
Visual
Text
Image
Aural
Speech
Music
Cues
Noise
Informative
Touch
Temperature
Tactile
Texture
Taste
Smell
Response Types
Visible (to the environment)
Verbal/NL Text
Behavioral (system changes)
Haptics/Touch/Proprioception
Invisible
Memory updates
*Corpus including data in taxonomies, ontologies, trees…

Graphs 101
A graph is a structure with vertices and edges.

Graphs 101
a
e
dc
b
Old Post Road
Cross Highway
Main Street
Shinbone Alley
Elk Road

Graphs 101
a
e
dc
b
Old Post Road
Cross Highway
Main Street
Shinbone Alley
Elk Road
Old Post Road Paved
Old Post Road 11 miles
Elk Road Dirt
Elk Road 2 miles
Cross Highway toll road
Cross Highway 250 miles
Main Street 1 mile
Shinbone Alley .5 miles
a bus stop
b gas station
b Shell
c Elementary school
d House
e Ofﬁce building
May be labeled, edges may be directed, all may
be stored/processed by properties
represented as key/value pairs.

Copyright (c) 2014-2016 by STORM Insights Inc. All Rights reserved.
Obvious structure is easy to process…
but most of the interesting stuff isn’t obvious to a computer.
Vertices,
edges,
properties
should
represent data
with higher-
level structure.

You Probably Already Think In Graphs if…
You watch detective shows
You know trivia about movies
You remember relationships between people
You took a biology class

You took a biology class or played 20 questions (“animal, mineral or vegetable?”)
Wikipedia contributors. "Taxonomy (biology)." Wikipedia,
The Free Encyclopedia. Wikipedia, The Free Encyclopedia,
11 May. 2016. Web. 12 May. 2016.

You watch detective shows
Typical crazy wall whiteboard - from Fargo.
A screen from IBM I2 Coplink

You know trivia about movies
IMDB

You remember relationships between people
Family Tree
LinkedIn Tree

Anonymized look at my
desk/wall on a typical day.

Processes Can Be Represented As Graphs

A taxonomy represents the formal structure of classes or types of objects within a domain. Taxonomies are generally hierarchical and provide names
for each class in the domain. They may also capture the membership properties of each object in relation to the other objects. The rules of a specific
taxonomy are used to classify or categorize any object in the domain, so they must be complete, consistent, and unambiguous. This rigor in
specification should ensure that any newly discovered object must fit into one, and only one, category or object class.

1952 DSM I
1968 DSM II
Pervasive Developmental Disorder (PDD)
Childhood onset PDD Infantile Autism Atypical Autism
1980 DSM III
Taxonomies Evolve
The History of Autism in the Diagnostic & Statistical Manual of the American Psychiatric Association
PDD-NOS Autistic Disorder
(Not Otherwise Speciﬁed)
1987 DSM III-R
PDD-NOS Autistic Disorder Asperger Disorder Childhood Disintegrative Disorder Rett Syndrome
1994 DSM IV
2000 DSM IV-TR
Autism Spectrum Disorder (ASD)
2013 DSMV

1952 DSM I
1968 DSM II
Childhood onset PDD Infantile Autism Atypical Autism
1980 DSM III
PDD-NOS Autistic Disorder
(Not Otherwise Speciﬁed)
1987 DSM III-R
PDD-NOS Autistic Disorder Asperger Disorder Childhood Disintegrative Disorder Rett Syndrome
1994 DSM IV
2000 DSM IV-TR
Autism Spectrum Disorder (ASD)
2013 DSMV
An ontology provides more detail than a taxonomy, although the boundary between them in practice is somewhat fuzzy. An ontology should
comprehensively capture the common understanding – vocabulary, definitions, rules - of a community as it applies to a specific domain.

Key Concept… Graphs have well known mathematical properties:
e.g. If you represent a graph as a matrix M, then values in Mn
represent the number of paths of length n in the original graph.
a
e
dc
b
a b c d e
a 1
b 1
c 1
d 1
e 1
M =

a
e
dc
b
a b c d e
a 1
b 1
c 1
d 1
e 1
M2 =

a
e
dc
b
a b c d e
a 1
b 1
c 1
d 1
e 1
M3 =

The Market is Ready for You Now With Options
Commercial
Open Source
As a Service

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.Wikipedia contributors. "Graph database." Wikipedia, The Free Encyclopedia. Wikipedia, The Free Encyclopedia, 11 May. 2016. Web. 12 May. 2016.

Property graph
RDF
RDF - Resource Description Framework, W3C specs for
metadata modeling, now used in knowledge management

No SQL? (as opposed to NoSQL) No problem
Gremlin - Open source (Apache2 license)
- a graph traversal language, supported by Titan, Neo4j,
HadoopGiraph, Hadoop Spark, IBM…
Cypher - Neo4j, Objectivity…
Emerging graph query/traversal languages
SPARQL - Open source (SPARQL Protocol and RDF Query Language)

This chart is representative of the
market, but incomplete.

Apache TinkerPop, TinkerPop, Apache, Apache feather logo, and Apache TinkerPop project logo are
either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries.
Apache TinkerPop™ is a graph computing framework for both
graph databases (OLTP) and graph analytic systems (OLAP).
“A graph is a structure composed of vertices and edges. Both vertices and edges
can have an arbitrary number of key/value-pairs called properties. Vertices denote
discrete objects such as a person, a place, or an event. Edges denote relationships
between vertices. For instance, a person may know another person, have been
involved in an event, and/or was recently at a particular place. Properties express
non-relational information about the vertices and edges. Example properties include
a vertex having a name, an age and an edge having a timestamp and/or a weight.
Together, the aforementioned graph is known as a property graph and it is the
foundational data structure of Apache TinkerPop.”
Apache TinkerPop™ is an open source, vendor-agnostic, graph computing
framework distributed under the commercial friendly Apache2 license. When a data
system is TinkerPop-enabled, its users are able to model their domain as a graph
and analyze that graph using the Gremlin graph traversal language.

Getting Started…
Why choose a graph database?
Speed to delivery when the data is naturally modeled as a graph

Simpliﬁes multi-hop queries

Visualization? Baked-in
Ask Yourself
Do you need an on-premise solution, or to manage your own database?

Lots of options, Neo4J is the market leader

Do you want graphs as a service?

IBM oﬀering graph as a service through BlueMix (in Beta now)

Upcoming Webinar Dates & Topics
June 9 Advances in Natural Language Processing (NLP)  
July 13 Modern AI and The Future of Work (With Steve Ardire)  
adrian@storminsights.com Twitter @ajbowles Skype ajbowles
A hat-tip to Kamille Nixon…

Upcoming Webinar Dates & Topics
June 9 Advances in Natural Language Processing (NLP)  
July 13 Modern AI and The Future of Work (With Steve Ardire)  
adrian@storminsights.com Twitter @ajbowles Skype ajbowles

Smart Data Webinar: Emerging Data Management Options

More Related Content

Viewers also liked (19)

Similar to Smart Data Webinar: Emerging Data Management Options (20)

More from DATAVERSITY (20)

Recently uploaded (20)

Smart Data Webinar: Emerging Data Management Options