![REFERENCES
39
[1] M. Nickel, K. Murphy, V. Tresp, and E. Gabrilovich, “A review of relational machine learning for knowledge
graphs,” Proceedings of the IEEE, vol. 104, pp. 11-33, 2016.
[2] N. Lao, T. Mitchell, and W. W. Cohen, “Random walk inference and learning in a large scale knowledge base,”
in Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2011, pp. 529-539.
[3] “An Introduction to Statistical Relational Learning” at: www.cs.kuleuven.be/~lucdr/salvador.pdf.
[4] “sampling procedure” at: https://www.slideshare.net/anasomoray/chapter-4-sampling-procedure
[5] “Non-Hierarchical Clustering” at: https://courses.cs.washington.edu/courses/cse527/01au/oct25/oct25.html
[6] https://en.wikipedia.org/
[7] N. Feldman, Learning and overgeneralization patterns in a connectionist model of the German plural: Cite seer,
2005.
[8] “Introduction to: Open World Assumption vs Closed World Assumption”, at:
http://www.dataversity.net/introduction-to-open-world-assumption-vs-closed-world-assumption/
[9] “Introducing the Knowledge Graph ”, at: https://www.youtube.com/watch?v=mmQl6VGvX-c
[10] “What is Linked Data?”, at: https://www.w3.org/standards/semanticweb/data
[11] “Resource Description Framework (RDF)”, at: https://www.w3.org/TR/rdf-concepts/](https://image.slidesharecdn.com/knowledgeautosaved-180516042745/85/A-Review-of-Relational-Machine-Learning-SRL-for-Knowledge-Graphs-39-320.jpg)
A Review of Relational Machine Learning(SRL) for Knowledge Graphs
- 1. A Review of Relational Machine Learning for Knowledge Graphs Present By: Yalda Akbarzadeh May 2018
- 2. A Review of Relational Machine Learning for Knowledge Graphs 1. INTRODUCTION 2. KNOWLEDGE GRAPHS A. Knowledge representation B. Open vs. closed world assumption C. Knowledge base construction 3. STATISTICAL RELATIONAL LEARNING FOR KNOWLEDGE GRAPHS D. Uses of knowledge graphs E. Main tasks in knowledge graph construction and curation A. Probabilistic knowledge graphs B. Statistical properties of knowledge graphs C. Types of SRL models 4. LATENT FEATURE MODELS D. Multi-layer perceptrons B. Other tensor factorization models C. Matrix factorization methods A. RESCAL: A bilinear model E. Neural tensor networks F. Latent distance models G. Comparison of models 8. MARKOV RANDOM FIELDS C. Path Ranking Algorithm B. Rule Mining and Inductive Logic Programming A. Similarity measures for uni- relational data 6. COMBINING LATENT AND GRAPH FEATURE MODELS 7. TRAINING SRL MODELS ON KNOWLEDGE GRAPHS 5. GRAPH FEATURE MODELS A. Additive relational effects model B. Other combined models A. Penalized maximum likelihood training B. Where do the negative examples come from? C. Pairwise loss training D. Model selection B. Inference A. Representation C. Learning D. Discussion 9. KNOWLEDGE VAULT 10. EXTENSIONS AND FUTURE WORK D. Discussion B. Hard constraints A. Non-binary relations C. Generalizing to new entities and relations D. Querying probabilistic knowledge graphs 12. APPENDIX 11. CONCLUDING REMARKS D. Querying probabilistic knowledge graphs
- 3. A Review of Relational Machine Learning for Knowledge Graphs 1. INTRODUCTION 2. STATISTICAL RELATIONAL LEARNING FOR KNOWLEDGE GRAPHS 3. LATENT FEATURE MODELS 4. GRAPH FEATURE MODELS 5. COMBINING LATENT AND GRAPH FEATURE MODELS 6. TRAINING SRL MODELS ON KNOWLEDGE GRAPHS 7. MARKOV RANDOM FIELDS EXTENSIONS AND FUTURE WORK, COCLUSION REFERENCES
- 4. Traditional machine Learning Algorithms feature vector, numeric or categorical attribute Input Mapping Output Observation tower Class labels, Regression score, Unsupervised cluster id Latent vector (embedding) 1. INTRODUCTION A. Relational Machine Learning 4
- 5. feature vector, numeric or categorical attribute Representation of an Object Traditional machine Learning Algorithms Graph, Node and Labeled edges Berlin Munich Düsseldorf 478 Km. 486 Km. 505Km. 1. INTRODUCTION A. Relational Machine Learning 5
- 6. Some Canonical Tasks >> Collective classification >> Linkprediction >> Link-based clustering prediction of missing edge clustering nodes based on their connectivity pattern prediction of properties of nodes 1. INTRODUCTION A. Relational Machine Learning 6
- 7. >> PREDICT(new facts (edges) |existing fact) = LEARN SRL Methods(KGs ) Two main SRL Models 1. INTRODUCTION A. Relational Machine Learning 7
- 8. 1. INTRODUCTION B. KNOWLEDGE GRAPHS 8
- 9. >> Knowledge graphs model information in the form of entities and relationships between them. Linked Data (subject, predicate, object) :: (SPO) (Leonard Nimoy, played, Spock) Leonard Nimoy was an actor who played the character Spock in the science-fiction movie Star Trek Semantic Web Linked Data Resource Description Framework subject object predicate 1. INTRODUCTION B. KNOWLEDGE GRAPHS i. Knowledge representation 9
- 10. >> closed world assumption (CWA), non-existing triples indicate false relationships. Linked Data>> KGs are known to be very incomplete >> RDF and the Semantic Web make the open-world assumption. >> open world assumption (OWA), a non-existing triple is interpreted as unknown. 1. INTRODUCTION B. KNOWLEDGE GRAPHS ii. Open vs. closed world assumption 10 Existing triples are facts What about non-existing triples?
- 11. Linked Data 1. INTRODUCTION B. KNOWLEDGE GRAPHS iii. Knowledge base construction 11
- 12. Linked Data 1. INTRODUCTION B. KNOWLEDGE GRAPHS iv. Uses of knowledge graphs 12
- 13. Linked Data 1. INTRODUCTION B. KNOWLEDGE GRAPHS v. Main tasks in KGs construction and curation Link prediction Entity resolution Link-based clustering Kate Ted Jack is sister of isfatherof isfatherof Jack Bonn uni. Worksin uni. Of Bonn Ted Studiesin TjLu Rj Ed Lj Rt Kj Jd Pj 13
- 14. 2. STATISTICAL RELATIONAL LEARNING FOR KNOWLEDGE GRAPHS 14
- 15. 2. STATISTICAL RELATIONAL LEARNING FOR KNOWLEDGE GRAPHS A. Probabilistic KGs 15
- 16. 2. STATISTICAL RELATIONAL LEARNING FOR KNOWLEDGE GRAPHS B. Statistical properties of KGs 16 • Denotes that query answer should follow a type. • Tendency of entities to be related to other entities with similar characteristic. • All the same blocks members have the same relation with other block.
- 17. 2. STATISTICAL RELATIONAL LEARNING FOR KNOWLEDGE GRAPHS C. Types of SRL models 17 M1) latent features and additional parameters M2) observed graph features and additional parameters M3) existence of local interactions (Markov Random Fields)
- 18. 2. STATISTICAL RELATIONAL LEARNING FOR KNOWLEDGE GRAPHS C. Types of SRL models 18 of M1 and M2:
- 19. 3. LATENT FEATURE MODELS 19 prestigious awardlatent features --> Ex) Guinness received the Academy Award good actor <- good actor -> <- prestigious award -> Relationships between entities can be derived from interactions of their latent features.
- 20. 3. LATENT FEATURE MODELS A. RESCAL 20 >> RESCAL is a bilinear relational latent feature model which explains triples via pairwise interactions of latent features. This importance is used here: Relationships between entities can be derived from interactions of their latent features.
- 21. 3. LATENT FEATURE MODELS B. Matrix factorization methods 21 >> Approach for learning from knowledge graphs is based on matrix factorization memory complexity Lose information Reshape
- 22. 3. LATENT FEATURE MODELS C. Multi-layer-perceptrons 22 L:
- 23. 3. LATENT FEATURE MODELS C. Multi-layer-perceptrons 23 >> E-MLP model : 02 03 04 Σ
- 24. 3. LATENT FEATURE MODELS C. Multi-layer-perceptrons 24 It can reduce the number of required parametersIt can reduce the number of required parameters Advantage of E-MLP Disadvantage of E-MLP 02 03 04 Σ
- 25. 3. LATENT FEATURE MODELS D. Neural tensor networks 25 02 03 04 >> Neural Tensor Network (NTN): Combines traditional MLPs with bilinear models
- 26. 3. LATENT FEATURE MODELS E. Latent distance models 26 02 03 04 >> Latent distance models or latent space models in social network analysis: Derive the probability of relationships from the distance between latent representations of entities #--------------------------------------------------------------------------------------------------------------------------------------
- 27. 4. GRAPH FEATURE MODELS A. Similarity measures for uni-relational data 27 >> Existence of an edge can be predicted by extracting features from the observed edges in the graph. >> GFMs Widely used for link prediction in uni-relational graphs. >> intuition: similar entities are likely to be related (homophily) the similarity of entities can be derived from: • neighborhood of nodes • existence of paths between nodes >> Approaches to measure the similarity of entities: John Mary Parent of Anne Parent of (John, marriedTo, Mary) Predict Local similarity Global similarity Quasi-local similarity
- 28. 4. GRAPH FEATURE MODELS B. Path Ranking Algorithm 28 >> Using random walks of bounded lengths for predicting links in multi-relational knowledge graphs.
- 29. 4. GRAPH FEATURE MODELS B. Path Ranking Algorithm 29 EXAMPLES OF PATHS LEARNED BY PRA ON FREEBASE TO PREDICT WHICH COLLEGE A PERSON ATTENDED Relation Path Weight (draftedBy, school) 2.62 (sibling(s), sibling, education, institution) 1.88 (parents, education, institution) 1.37 (children, education, institution) 1.85 >> key idea in PRA: Use path probabilities as features for predicting the probability of missing edges.
- 30. 5. COMBINING LATENT AND GRAPH FEATURE MODEL 30 >>Latent feature models: well-suited for modeling global relational patterns via newly introduced latent variables. They are computationally efficient if triples can be explained with a small number of latent variables. >>Graph feature models: are well-suited for modeling local and quasi-local graphs patterns. They are computationally efficient if triples can be explained from the neighborhood of entities or from short paths in the graph
- 31. 5. COMBINING LATENT AND GRAPH FEATURE MODEL Additive Relational Effects models 31
- 32. 6. TRAINING SRL MODELS ON KNOWLEDGE GRAPHS A. Penalized maximum likelihood training 32 02 03 04 posterior priorlikelihood
- 33. 6. TRAINING SRL MODELS ON KNOWLEDGE GRAPHS B. Where do the negative examples come from? 33 >> Most knowledge graphs only contain positive training examples.
- 34. 6. TRAINING SRL MODELS ON KNOWLEDGE GRAPHS 34 02 03 04
- 35. 7. MARKOV RANDOM FIELDS 35 3-vertex cliques
- 36. 7. MARKOV RANDOM FIELDS 36 02 03 04 >> Form of Markov Random Fields (MRFs):
- 37. EXTENSIONS AND FUTURE WORK 37 A. Non-binary relations B. Hard constraints: types, functional constraints, and others C. Generalizing to new entities and relations D. Querying probabilistic knowledge graphs E. Trustworthiness of knowledge graphs
- 38. Conclusion 38 review of state-of-the-art statistical relational learning (SRL) methods applied to very large knowledge graphs KGs have found important applications in question answering, structured search, exploratory search showed how to create a truly massive, machine-interpretable “semantic memory” of facts
- 39. REFERENCES 39 [1] M. Nickel, K. Murphy, V. Tresp, and E. Gabrilovich, “A review of relational machine learning for knowledge graphs,” Proceedings of the IEEE, vol. 104, pp. 11-33, 2016. [2] N. Lao, T. Mitchell, and W. W. Cohen, “Random walk inference and learning in a large scale knowledge base,” in Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2011, pp. 529-539. [3] “An Introduction to Statistical Relational Learning” at: www.cs.kuleuven.be/~lucdr/salvador.pdf. [4] “sampling procedure” at: https://www.slideshare.net/anasomoray/chapter-4-sampling-procedure [5] “Non-Hierarchical Clustering” at: https://courses.cs.washington.edu/courses/cse527/01au/oct25/oct25.html [6] https://en.wikipedia.org/ [7] N. Feldman, Learning and overgeneralization patterns in a connectionist model of the German plural: Cite seer, 2005. [8] “Introduction to: Open World Assumption vs Closed World Assumption”, at: http://www.dataversity.net/introduction-to-open-world-assumption-vs-closed-world-assumption/ [9] “Introducing the Knowledge Graph ”, at: https://www.youtube.com/watch?v=mmQl6VGvX-c [10] “What is Linked Data?”, at: https://www.w3.org/standards/semanticweb/data [11] “Resource Description Framework (RDF)”, at: https://www.w3.org/TR/rdf-concepts/