Graph Gurus Episode 25: Unleash the Business Value of Your Data Lake with Graph Analytics

© 20190 TigerGraph. All Rights Reserved
Graph Gurus 25
Unleash the Business Value of Your Data Lake
with Graph Analytics
1

Today’s Presenter
2
David Schexnayder
Senior Sales Engineer
● 20+ years in tech industry
● 7+ years with Hadoop / Big Data
● Bachelor of Science from North Carolina State
University

Some Housekeeping Items
● Although your phone is muted we do want to answer your questions -
submit your questions at any time using the Q&A tab in the menu
● The webinar is being recorded and will uploaded to our website shortly
(https://www.tigergraph.com/webinars-and-events/) and the URL will be
emailed you
● If you have issues with Zoom please contact the panelists via chat
3

Our Agenda
4
Introduction1
4
3
2 Data Lakes - Characteristics and Challenges
Why Graph DB and Analytics
Why TigerGraph
5 How TigerGraph Fits
6 Resources

Relational Database Key-Value Database Graph Database
Customer
XXXXXX
Product
XXXXXXXX
Supplier
XXXXXXXX
Location
XXXXXXXX
Order
XXXXXXXXX
Product
Customer
Supplier
Location
KEY VALUE
XXXXX
Order
Customer
Produc
t
Supplier
• Rigid schema
• High performance for transactions
• Poor performance for deep analytics
• Highly fluid schema/no schema
• High performance for simple transactions
• Poor performance deep analytics
• Flexible schema
• High performance for complex transactions
• High performance for deep analytics
Location 1 = Delivery Location
Location 2 = Warehouse
Location 2
Product
Payment
PURCHASED
RESIDES
SHIPSTO
PURCHASED
SHIPS FROM
A
C
C
EPTED
MAKES
The Evolution of Databases
XXXXX
XXXXX
XXXXX Location 1
N
O
TIFIES
5

Characteristics of Data Lakes
Up to Petabytes of data from diverse sources
• Non-relational and relational data
• Web logs
• Social media feeds
• Collected data
Multiple engines to process and analyze data
• Spark
• Hive
• Impala
Data stored in a variety of formats
• NoSQL Databases (HBase, Cassandra)
• Columnar Storage (Parquet, ORC)
• Flat files, sequence files
6

Common Components of Data Lakes
7

Challenges of Data Lakes
● Data sprawl and duplication
● Gaining real-time insights very difficult
● Security and data governance
● Table joins are still expensive!
8

Graph to the Rescue
TigerGraph provides the ability for businesses to put the data
in your lake to work:
• Faster insights into your connected data
• Hybrid Transactional and Analytical Processing (HTAP)
• Connect multiple data sources expressed in graph
• Distributed efficient storage and parallel processing
9

Graph Can Accelerate Your Use Cases
TigerGraph can run optimized use cases to put the data in
your lake to work:
• Fraud and Anti Money Laundering
• Next-generation recommendation engines
• Customer journey / Customer 360
• Master Data Management
• Entity Resolution
• IT Optimization
10

Blocking Fraud Before Payment Is Made
● 30% increase in fraud detection
● Integrated 20+ data sources,
handling 2 billion real-time
events per day, 1 trillion edges
● Distributed across 40 standard
servers (20 with 2x replication)
● Reduced working cycle from
one hour to minutes
● Business logic implemented in
10k of C++ lines
11

Increase Revenues with Recommendations
Person A Person C
Shops at Store Y
Bought item X online
from Store Y
Knows/likes/follows B B commented (to C)
Person B
C (also) shops at store Y
Is located at Z
Item X is stored at
Warehouse W
Item X has
tag/feature F
Customer
Visit Click Path
Would you like…?
B likes hobby H
12

Building a Customer 360° Data Hub
Business Challenge
• Combine all available data for customer with
transactions to improve business outcomes
Solution
• Build on top of current investments in master data
management, data warehouse/Hadoop data lake
and NoSQL repositories
• Find new relationships among data to drive better
fraud and money laundering detection, credit risk
scoring and monitoring, product & service
marketing, cross-sell and up-sell recommendation
for higher revenue & profits
• Analyze temporal (Time Series) and spatial data to
find new patterns and insights
• Expand schema (attributes/fields, relationships) to
accommodate new data sources & use cases
13

Entity Resolution for MDM with TigerGraph
14

Metadata Management with Graph
● Metadata is “Data about other Data”
● Types of Metadata
○ Structure - Organizational units, operations, IT infrastructure, stages/tates, data
collections, database schema
○ Processes - business, operational, software, data processing
○ Process flow = directional (“Directed”) graph
● Uses
○ Establish and manage policies and processes
○ Manage data lineage, integration and sharing
○ Ensure compliance (financial, GDPR, HIPAA, etc.)
○ Security, access control
○ Identity and eliminate redundancies and inefficiencies
15

Improved Energy Management System
• Business Challenge: Monitor complex energy
infrastructure to detect and manage power
overloads and outages
• Solution:
• Model power system using real-time
operational Graph to accelerate state
estimation & power flow calculation
(no data preparation needed)
• Leverage massively parallel computing
in Graph for bus ordering &
admittance graph forming to balance
load spikes
• Visualize energy computation results in
Graph for contingency analysis and
action plan
• Business Value: Deliver faster than real-time
EMS (ie, EMS capable of completing
execution within a SCADA sample cycle of 5
seconds)
16

Delivering Better Outcomes
Customers
Supplier
Employee
Device
IoT Signals
Orders
Payments
Shipments
Invoices
Visits
Downloads
Master Data
Operational Data
Data
Warehouse
Data Mart Data Lake NoSQL
Historical
Data
Queries / Lookups,
Comprehensive Graph Patterns
and Algorithms
Graph-Computed
Features
Batch and Streaming
17

Hadoop Interoperability
Several ingestion tools to integrate Your data lake:
● Spark via JDBC
● HDFS Connector
● S3 Connector
● JDBC from RDBMS
18

Testing Setup and Configuration
• Cloudera Quickstart VM 5.13.0 and example data
• TigerGraph Developer Edition 2.5.0
19

Hadoop Workflow
• Use sqoop to pull transactional data from RDBMS
• Build Hive tables and store on HDFS as parquet
• Query data with Impala, Hive, Spark
20

Spark and TigerGraph Data Pipeline
Static
Data
Sources
TigerGraph
JDBC
Driver
Streaming
Data
Sources

JDBC Driver (1.2)
• Type 4 driver
• Support Reads and Writes bi-directional data flow to
TigerGraph
• Read: Converts ResultSet to DataFrame
• Write: Load DataFrame and files to vertex/edge in
TigerGraph
• Supports REST endpoints of built-in, compiled and
interpreted GSQL queries
• Open Source:
https://github.com/tigergraph/ecosys/tree/master/etl/tg-jdbc-driver
22

JDBC Driver Installation on Cloudera
• Upgrade Java to 1.8.0 openJDK
• Convert to parcels
• Install Spark 2.4.0 parcel (optional)
• Build the TigerGraph JDBC driver (from github)
• Install the JDBC driver and add it to ‘classpath.txt’
23

Relational Data Schema
24

Graph Data Schema
25

Hadoop —> TigerGraph Workflow
• Use Apache Spark to read the data on HDFS in parquet
format
• Use the TigerGraph Spark JDBC connector to load the data
into the graph
• Explore the graph
26

Demo!
27

What Makes TigerGraph Fast and Scalable?
Designed from ground up for computational parallelism
• Native graph storage
• Parallel graph computation engine
• Deep link analytics in real time
• 10X+ compression
• Distributed graph for scaleout using multiple machines
28

Q&A
Please submit your questions via the Q&A tab in Zoom

Additional Learning Resources
Start Free at TigerGraph Cloud Today!
https://www.tigergraph.com/cloud/
Test Drive Online Demo
https://www.tigergraph.com/demo
Download the Developer Edition
https://www.tigergraph.com/download/
Guru Scripts
https://github.com/tigergraph/ecosys/tree/master/guru_scripts
Join our Developer Forum
https://groups.google.com/a/opengsql.org/forum/#!forum/gsql-users
30

Coming to a City Near You
Let us know if you would like to help organize a Graph Gurus
Comes To You workshop in your city
https://info.tigergraph.com/graph-gurus-request
31

Thank You

Graph Gurus Episode 25: Unleash the Business Value of Your Data Lake with Graph Analytics

More Related Content

What's hot (20)

Similar to Graph Gurus Episode 25: Unleash the Business Value of Your Data Lake with Graph Analytics (20)

More from TigerGraph (20)

Recently uploaded (20)

Graph Gurus Episode 25: Unleash the Business Value of Your Data Lake with Graph Analytics