Graph Gurus Episode 28: In-Database Machine Learning Solution for Real-Time Recommendations

Graph Gurus Episode 28
In-Database Machine Learning Solution
for Real-Time Recommendations

© 2020 TigerGraph. All Rights Reserved
Today’s Host
David Ronald
Director of Product Marketing
● 18+ years in tech industry
● Prior work in artificial intelligence, natural linguistic
programming and telecommunications technology
● BSc in Applied Physics from Strathclyde University, MSc
in Optoelectronic & Laser Devices from St Andrews

Today’s Presenters
Changran Liu
Solution Architect
● BS in Mechanical Engineering, Tsinghua University
● MS & PhD in Mechanical Engineering, Stanford
University
● PhD minor in Philosophy focused on applications
of mathematical logic in artificial intelligence
Mingxi Wu
VP of Engineering
● 19+ years in data management industry &
research
● BS in Computer Science from Fudan University
● MS & Ph.D in Computer Science from
University of Florida

Some Housekeeping Items
● Although your phone is muted we do want to answer your questions -
submit your questions at any time using the Q&A tab in the menu
● The webinar is being recorded and will uploaded to our website shortly
(https://www.tigergraph.com/webinars/) and the URL will be emailed
you
● If you have issues with Zoom please contact the panelists via chat
4

Outline
● Why Do ML in Graph Database
● Recommendation Systems
● Demo
● Latent factor model (model based)
○ Intuition
○ Implementation

Current Situation
training data
model
request
results
Database:
● data storage
● data update
● preprocess data
Machine learning platform
● model training
● model validation
Applications:
● recommendation
● fraud detection
● ...

Current Situation
training data
model
request
results
Database:
● data storage
● data update
● preprocess data
● model training
Applications:
● recommendation
● fraud detection
● ...
The whole training set needs
to be transferred

Current Situation
training data
model
request
results
Database:
● data storage
● data update
● preprocess data
● model training
Applications:
● recommendation
● fraud detection
● ...
Data is stale when it’s used
for training

Current Situation
training data
model
request
results
Database:
● data storage
● data update
● preprocess data
● model training
Applications:
● recommendation
● fraud detection
● ...
Learning platform is not
scaled-out

The Challenge For In-database ML
training data
model
request
results
Database:
● data storage
● data update
● preprocess data
● model training
Applications:
● recommendation
● fraud detection
● ...
● SQL is declarative, not good for iterative algorithms
● Relational model prevents users get some useful features that spanning multiple hops.
● Many databases are not real-time mutable, so data is stale.

In-situ ML in TigerGraph Database:
● Native graph storage and PG model
● Coded once, auto scale-out & scale-up
● Support real-time update
● GSQL Turing-complete language
○preprocess data
○model training: flow-control, accumulator, pattern match
○model validation
Solution: In Graph Database ML with GSQL
request
results
Applications:
● recommendation
● fraud detection
● ...

Recommendation Systems

Movie Recommendation
movie features
users ratings
Goals:
● Predict users' ratings for movies they haven't
seen, based on previous ratings
● Recommend movies to users based on rating
prediction

User-Rate-Movie Graph
● Content based method
Toy story
● Disney
● ...
Iron man
● Marvel
● Action
● ...
Alice
● Disney fan
● Marvel fan
● ...
Bob
● Marvel fan
● ...
rating: 5
rating: 5
rating:4.5
rating:?

Toy story
● Disney
● ...
Iron man
● Marvel
● Action
● ...
Alice
● Disney fan
● Marvel fan
● ...
Bob
● Marvel fan
● ...
rating: 5
rating: 5
rating:4.5
rating:?
● K-nearest neighbors

Toy story
● Disney
● ...
Iron man
● Marvel
● Action
● ...
Alice
● Disney fan
● Marvel fan
● ...
Bob
● Marvel fan
● ...
rating: 5
rating: 5
rating:?
● Latent factor (model-based)

● Latent factor (model-based)
● Hybrid method
● ...
Toy story
● Disney
● ...
Iron man
● Marvel
● Action
● ...
Alice
● Disney fan
● Marvel fan
● ...
Bob
● Marvel fan
● ...
rating: 5
rating: 5
rating:?

Demo

MovieLens Data
● Dataset of 100k ratings and 40k tags that 1k users gave to 17k movies
● Each rating is a quadruplet of the form <user, movie, rating, date>
● Each movie is tagged with multiple different terms
● The user and movie fields are integer IDs, while grades are from 0 to 5
stars
● https://grouplens.org/datasets/movielens/

Root Mean Square Error (RMSE)

Results
TF-IDF method
(content based)
RMSE: 0.91239
Latent factor model
(model based)
RMSE: 0.96869
hybrid model
RMSE: 0.90368
Root Mean Square Error (RMSE) =

Movie Rating Prediction (Latent factors model)
Movie Alice Bob Carol Dave
Love at last 5 5 0 0
Romance forever 5 ? ? 0
Cute puppies of love ? 4 0 ?
Toy story ? ? ? 5
Sword vs. karate 0 0 5 ?
Nonstop car chases 0 0 5 4
● Each movie has a latent
factor vector: θ(j)
● Each user has a latent
factor vector: x(i)
● Predict the user j’s rating
to movie i by: (θ(j)
)T
x(i)
θ(1)
= [5, 0] θ(2)
= [5, 0] θ(3)
= [0, 5] θ(4)
= [0, 5]
x(1)
= [0.9, 0]
x(2)
= [1, 0.1]
x(3)
= [0.9, 0]
x(4)
= [0.1, 1]
x(5)
= [0.1, 1]
x(6)
= [0, 0.9]
4.5
5
4.5
0.5
0.5
0

Movie Rating Prediction (Latent factors model)
Movie Alice Bob Carol Dave
Love at last 5 5 0 0
Romance forever 5 ? ? 0
Cute puppies of love ? 4 0 ?
Toy story ? ? ? 5
Sword vs. karate 0 0 5 ?
Nonstop car chases 0 0 5 4
θ(1)
= [5, 0]
● Each movie has a latent
factor vector: θ(j)
● Each user has a latent
factor vector: x(i)
● Predict the user j’s rating
to movie i by: (θ(j)
)T
x(i)
θ(2)
= [5, 0] θ(3)
= [0, 5] θ(4)
= [0, 5]
x(1)
= [0.9, 0]
x(2)
= [1, 0.1]
x(3)
= [0.9, 0]
x(4)
= [0.1, 1]
x(5)
= [0.1, 1]
x(6)
= [0, 0.9]
action
romance
4.5
5
4.5
0.5
0.5
0

Schema and Graph
User Movie
LIST<FLOAT>: theta LIST<FLOAT>: x
FLOAT: rating
rate
User 2
Movie 1
Movie 2
Movie 3
User 1
rating: 4
rating: 5
rating: 5
rating: 3

Training
Split data
Initialize latent factor
vectors
diff. between prediction and
label
converged?
no
finish
yes
update latent vectors
using gradient descent
(splitData.gsql)
(initialization.gsql)
(training_validation.gsql)

GSQL Training Block
USERs = SELECT s FROM USERs:s -(rate:e)-> MOVIE:t
ACCUM
DOUBLE prediction = dotProduct(s.@theta,t.@x),
DOUBLE delta = prediction-e.rating,
s.@Gradient += product(t.@x,delta),
t.@Gradient += product(s.@theta,delta)
POST-ACCUM
s.@theta += product(s.@Gradient,-alpha),
t.@x += product(t.@Gradient,-alpha);
Dave
Romance
forever
Love at
last
Nonstop
car chases
Alice
rating: 5
rating: 5
rating: 0
rating: 4
θ = [1.5, 1.7]
θ = [1.0, 1.5]
x = [2.0, 2.3]
x = [2.0, 1.3]
x = [1.0, 1.3]

GSQL Training Block
ACCUM
POST-ACCUM
Dave
Romance
forever
Love at
last
Nonstop
car chases
Alice
rating: 5
rating: 5
rating: 0
rating: 4
θ = [1.5, 1.7]
θ = [1.0, 1.5]
x = [2.0, 2.3]
x = [2.0, 1.3]
x = [1.0, 1.3]
prediction: 6.9
prediction: 5.2
prediction: 4.0
prediction: 3.0

GSQL Training Block
ACCUM
POST-ACCUM
Dave
Romance
forever
Love at
last
Nonstop
car chases
Alice
ẟ: 1.9
ẟ: 0.2
ẟ: 4.0
ẟ: -1.1
θ = [1.5, 1.7]
θ = [1.0, 1.5]
x = [2.0, 2.3]
x = [2.0, 1.3]
x = [1.0, 1.3]

GSQL Training Block
ACCUM
POST-ACCUM
Dave
Romance
forever
Love at
last
Nonstop
car chases
Alice
ẟ: 1.9
ẟ: 0.2
ẟ: 4.0
ẟ: -1.1
θ = [1.5, 1.7]
grad(θ) = [4.2, 4.7]
θ = [1.0, 1.5]
grad(θ) = [6.9, 3.8]
x = [2.0, 2.3]
x = [2.0, 1.3]
x = [1.0, 1.3]

GSQL Training Block
ACCUM
POST-ACCUM
Dave
Romance
forever
Love at
last
Nonstop
car chases
Alice
ẟ: 1.9
ẟ: 0.2
ẟ: 4.0
ẟ: -1.1
θ = [1.5, 1.7]
grad(θ) = [4.2, 4.7]
θ = [1.0, 1.5]
grad(θ) = [6.9, 3.8]
x = [2.0, 2.3]
grad(x) = [2.9, 3.2]
x = [2.0, 1.3]
grad(x) = [4.3, 6.3]
x = [1.0, 1.3]
grad(x) = [-1.1, -1.6]

GSQL Training Block
ACCUM
POST-ACCUM
Dave
Romance
forever
Love at
last
Nonstop
car chases
Alice
ẟ: 1.9
ẟ: 0.2
ẟ: 4.0
ẟ: -1.1
θ = [1.5, 1.7]
θ’ = [1.46, 1.65]
θ = [1.0, 1.5]
θ’ = [0.93, 1.46]
x = [2.0, 2.3]
x’ = [1.97, 2.27]
x = [2.0, 1.3]
x’ = [1.96, 1.24]
x = [1.0, 1.3]
x’ = [1.01, 1.32]
* alpha = 0.01

GSQL Training Block
ACCUM
POST-ACCUM
Dave
Romance
forever
Love at
last
Nonstop
car chases
Alice
ẟ: 1.9
ẟ’: 1.6
ẟ: 0.2
ẟ’: -0.1
ẟ: 4.0
ẟ’: 3.6
ẟ: -1.1
ẟ’: -1.1
θ = [1.5, 1.7]
θ’ = [1.46, 1.65]
θ = [1.0, 1.5]
θ’ = [0.93, 1.46]
x = [2.0, 2.3]
x’ = [1.97, 2.27]
x = [2.0, 1.3]
x’ = [1.96, 1.24]
x = [1.0, 1.3]
x’ = [1.01, 1.32]

Summary
● User-rate-item relation can be
represented as a graph
● The Latent factor model can be
trained in TigerGraph database
● The hybrid recommendation model
can be conveniently implemented
using TigerGraph
● The solution for recommendation
system can easily be adapted for
link prediction or entity resolution
problems.

Q&A
Please submit your questions via the Q&A tab in Zoom
38

More Questions?
Join our Developer Forum
https://groups.google.com/a/opengsql.org/forum/#!forum/gsql-users
Sign up for our Developer Ofﬁce Hours (every Thursday at 11 AM PST)
https://info.tigergraph.com/officehours
39

Additional Resources
Start Free at TigerGraph Cloud Today
https://www.tigergraph.com/cloud/
Test Drive Online Demo
https://www.tigergraph.com/demo
Download the Developer Edition
https://www.tigergraph.com/download/
Guru Scripts
https://github.com/tigergraph/ecosys/tree/master/guru_scripts
40

Upcoming Graph Guru Events
41
Coming to Seattle, San Francisco, Atlanta and more.
View the full list of events, or request your own, here:
https://www.tigergraph.com/graphguruscomestoyou/
Virtual Healthcare Roundtable: Transforming
Healthcare with Graph Database and Analytics
https://info.tigergraph.com/healthcare-roundtable

Graph Gurus Episode 28: In-Database Machine Learning Solution for Real-Time Recommendations

More Related Content

What's hot (19)

Similar to Graph Gurus Episode 28: In-Database Machine Learning Solution for Real-Time Recommendations (20)

More from TigerGraph (20)

Recently uploaded (20)

Graph Gurus Episode 28: In-Database Machine Learning Solution for Real-Time Recommendations