Odsc 2019 entity_reputation_knowledge_graph
- 1. ENTITY CO-OCCURENCE and ENTITY REPUTATION FROM UNSTRUCTURED DATA USING KNOWLEDGE GRAPH By Venkatraman.J Senior data software engineer, Metapack, London
- 2. AGENDA • Motivation • Knowledge graph basic introduction • Problem statement • Data flow architecture • Conclusion
- 3. MOTIVATION • Most of the Data is very unstructured in nature. • Not every problem needs ML/Deep learning models. • Large amount of datasets are connected in this world. • Connected or Linked data provides valuable insights quickly.
- 4. KNOWLEDGE GRAPH • Knowledge graphs encode structured information of entities and their rich relations. • Captures relationship between individual items providing a model and access patterns that can be processed automatically by machines. • Entities are represented using nodes and relationships as edges between entities. • Data representation is also named as triples – (Subject, Predicate, Object) • Very much compared to Ontology where Ontology captures relationships between concepts, data and entities within a particular domain. Eg. Dbpedia, Yago, WordNet. • Use cases powered by knowledge graph – Improving search relevance, Question answering applications, Recommendation engines.
- 6. PROBLEM STATEMENT • Individuals review products on social media. • Actionable insight to find out how the product is doing in market.
- 7. KNOWLEDGE GRAPH CONSTRUCTION • NLP techniques to do Information extraction techniques to extract entities and relationship across entities • Entities are represented using nodes and relationships among entities as edges • Entities in twitter feeds are persons, products and location. • Relationships are likes and dislikes of individual person about a product, relation to other users.
- 8. GRAPH INFERENCE • Centrality algorithms – Degree, Pagerank, Closeness. • Degree centrality – Used for determining popular nodes in the graph. • Degree centrality measures the number of incoming and outgoing relations from a node. Entities that have the highest degree centrality score are considered very popular.
- 10. LEARNING • Tweets are easy to get but quality of data is very poor and too noisy. • Graph querying is not same as SQL. Querying works as pattern matching. Understand the internals of query language is needed to write efficient queries and debug problems. • Understand the data model represented in graph. • Start with small graph and iterate on before building a bigger one. • Neo4j is ACID compliant like RDBMS, watch out for multiple writers writing to Database. • Deploy containerized applications and orchestrate using Docker swarm or Kubernetes to scale up.
- 11. CONCLUSION • Identify the problems that can be solved using graph theory and connected data. • Scoring via Graph model can augment or support the results received from ML/DL models. • Papers related to knowledge graph: http://ceur-ws.org/Vol-2306/paper9.pdf https://aclweb.org/anthology/D18-2024 Questions?