Training Week: Create a Knowledge Graph: A Simple ML Approach

Neo4j, Inc. All rights reserved 2021
1
Creating a Knowledge Graph with
Neo4j:
A Simple Machine Learning Approach
Clair J. Sullivan, PhD
Data Science Advocate
https://medium.com/@cj2001
@CJLovesData1

All materials for this demonstration are available on the
workshop GitHub repo:
https://github.com/cj2001/nodes2021_kg_workshop
3
(I will put this link up several times!)

To run today’s code:
1. Jupyter or Google Colab
◦ We will have some dependencies to manage in either
◦ If you are bringing your own Jupyter, you probably want to create a virtual
environment for this workshop
2. Neo4j Sandbox
◦ https://dev.neo4j.com/sandbox
We can either populate the database manually, or I will show how to download
a pre-populated one...

5

6
By the end of this workshop you will be able to...
Apply data science
and machine
learning to
knowledge graph
Vectorize
knowledge graph
(create graph
embeddings)
Create a knowledge
graph
Get some text and
extract relevant
information

7
By the end of this workshop you will be able to...
Apply data science
and machine
learning to
knowledge graph
Vectorize
knowledge graph
(create graph
embeddings)
Create a knowledge
graph
Get some text and
extract relevant
information
Natural Language Processing (NLP) Graph Data Science Library + Basic ML

8
Two Key
Concepts 1. There is no proverbial “silver bullet” with
Natural Language Processing (NLP)
2. The quality of what you get out of a
knowledge graph depends on the quality of
what you put into it

9
https://yashuseth.blog/2019/10/08/introduction-question-answering-knowledge-graphs-kgqa/

• “Things not strings”
• What knowledge graphs are useful for
◦ Search
◦ Question answering
◦ Recommendation engine
• Can be generated a lot of different ways
◦ Co-occurrence
◦ Resource Description Framework (RDF)
◦ Subject-Verb-Object (SVO)
10
Introduction to knowledge graphs

11
Word co-occurrence
https://covid19biblio.com/2020/04/28/keyword-co-occurrence-network-graph-for-the-overall-research-ﬁeld-on-covid-19-up-to-april-27th-2020/

12
RDF triples
https://en.wikipedia.org/wiki/Resource_Description_Framework#Examples

13
SVO triples

• Named Entity Recognition (NER)
• SVO / SPO triples
◦ ...but verbs can be difficult to reliably detect via NLP!
• Very language dependent
• Very topic-area dependent
14
NLP considerations for knowledge graph creation

15

16
Barack Hussein Obama II is an American politician and
attorney who served as the 44th president of the United
States from 2009 to 2017.

17
Barack Hussein
Obama II is an
American politician
and attorney who
served as the 44th
president of the
United States from
2009 to 2017.
Text Lemma Tag POS DEP is_stop
Barack Barack NNP PROPN compound FALSE
Hussein Hussein NNP PROPN compound FALSE
Obama Obama NNP PROPN compound FALSE
II II NNP PROPN nsubj FALSE
is be VBZ AUX ROOT TRUE
an an DT DET det TRUE
American american JJ ADJ amod FALSE
politician politician NN NOUN attr FALSE
and and CC CCONJ cc TRUE
attorney attorney NN NOUN conj FALSE
who who WP PRON nsubj TRUE
served serve VBD VERB relcl FALSE
as as IN ADP prep TRUE
the the DT DET det TRUE
44th 44th JJ ADJ amod FALSE
president president NN NOUN pobj FALSE
of of IN ADP prep TRUE
United United NNP PROPN compound FALSE
States States NNP PROPN pobj FALSE
from from IN ADP prep TRUE
2009 2009 CD NUM pobj FALSE
to to IN ADP prep TRUE
. . . PUNCT punct FALSE
https://github.com/explosion/spaCy/blob/master/spacy/glossary.py

18
Barack Hussein
Obama II is an
American politician
and attorney who
served as the 44th
president of the
United States from
2009 to 2017.

19
Barack Hussein
Obama II is an
American politician
and attorney who
served as the 44th
president of the
United States from
2009 to 2017.

• spacy
• Wikipedia Python package
• Google Knowledge Graph
• Pywikibot
• Neo4j
◦ Awesome Procedures on Cypher (APOC)
◦ Graph Data Science (GDS) Library
◦ Cypher
20
An introduction to the tools we will use today

Clone the GitHub repository at (OPTIONAL)
https://github.com/cj2001/nodes2021_kg_workshop

Method 1: The NLP Only
Approach
22

23
Some ways we could get this done: NLP only approach
Advantage: limitless verbs
Drawback: entity disambiguation

word2vec
https://www.kdnuggets.com/2019/01/
burkov-self-supervised-learning-word-
embeddings.html
https://medium.com/swlh/word2vec-in-practice-for-
natural-language-processing-a179b3286a21

25
Overview of workﬂow

26
NLP workﬂow

27
Create a Google Knowledge API key
https://developers.google.com/knowledge-graph/how-tos/authorizing

28
{...
"@type": "ItemList",
"itemListElement": [
{
"@type": "EntitySearchResult",
"result": {
"@id": "kg:/m/0dl567",
"name": "Taylor Swift",
"@type": [
"Thing",
"Person"
],
...
"detailedDescription": {
"articleBody": "Taylor Alison Swift is an American singer-songwriter and
actress. Raised in Wyomissing, Pennsylvania, she moved to Nashville, Tennessee, at the
age of 14 to pursue a career in country music. ",
"url": "http://en.wikipedia.org/wiki/Taylor_Swift",
...
}
Enhance the existing data with Google Knowledge Graph

29
Detailed knowledge graph data model

Method 2: The NLP Lite
Approach
31

Some ways we could get this done: NLP “lite”
Advantage: entity disambiguation
Drawback: must specify which
verbs you are interested in

Create a PyWikiBot token
https://heardlibrary.github.io/digital-scholarship/host/wikidata/bot/

34

Machine Learning on Graphs
36

37
Why machine learning with (knowledge) graphs?
• Traditional ML uses a relational database-type model
◦ All data points are are independent of each other
• Example: churn prediction based on user behavior
• Graphs (and graph databases) treat relationships as a “first class citizen”
◦ Models can include homophily
• Example: churn prediction includes the churn of neighbors within the graph
◦ Models can also include the same data as the traditional, independent data
point models
Example: making a better
recommendation engine

word2vec
38
https://www.geeksforgeeks.org/python-word-embedding-using-word2vec/
Note: word2vec typically
creates one vector per word.
The spacy implementation of
vectorization takes a
document (sentence) and
averages the word vectors
across the sentence.

node2vec
https://snap.stanford.edu/node2vec/

node2vec
40
Embedding dimension: 10

Node similarity via embeddings
Embedding dimension: 300

Visualizing embeddings with t-SNE

Where to go from here?
44

45
Two Key
Concepts 1. There is no proverbial “silver bullet” with
Natural Language Processing (NLP)
2. The quality of what you get out of a
knowledge graph depends on the quality of
what you put into it

What could we do from here?
• Add nodes to the graph!
• Various embedding optimization techniques
• Add data for creating embeddings
◦ Ex: Word vectors from text descriptions
• Different embedding/modeling techniques
◦ GraphSAGE
◦ GNN

Problems we could solve
• Community/cluster detection
• Node classification, link prediction
• Graph-to-graph classification
• Unstructured text, NLP
• Question answering systems

48
Thank you!
https://medium.com/@cj2001
@CJLovesData1

Training Week: Create a Knowledge Graph: A Simple ML Approach

More Related Content

What's hot (20)

Similar to Training Week: Create a Knowledge Graph: A Simple ML Approach (20)

More from Neo4j (20)

Recently uploaded (20)

Training Week: Create a Knowledge Graph: A Simple ML Approach