Learning Semantic Relationships between Entities in Twitter
Download as PPTX, PDF2 likes1,109 views
The document discusses the discovery of semantic relationships between entities on Twitter to enhance personalized web experiences. It outlines a framework that uses both tweet-based and news-based strategies for detecting relationships, emphasizing the importance of temporal constraints and the type of entities involved. The findings suggest that combining different strategies increases accuracy and highlights areas for future work, including adaptive faceted search on Twitter.
Learning Semantic Relationships between Entities in Twitter
- 1. Learning Semantic Relationships between Entities in TwitterICWE, Cyprus, June 22, 2011IlknurCelik, Fabian Abel, Geert-Jan HoubenWeb Information Systems, TU Delft
- 2. What we do: Science and Engineering for the Personal Webdomains: news social mediacultural heritage public datae-learningPersonalizedRecommendationsPersonalized SearchAdaptive Systems Analysis and User ModelingSemantic Enrichment, Linkage and Alignmentuser/usage dataSocial Web
- 3. 60,000,000number of tweets published per day
- 4. 1number of tweets per day that are interesting for me
- 6. Issues with Multiple Keywords Search
- 7. Let’s try to search with One Keyword
- 8. Page 1
- 9. Page 2
- 10. Page 3
- 11. Page 60!!Music ArtistNext Saturday @thatsimpsonguyaka Guilty Simpson will be performing atArea51 in my hometwonEindhoven. #realliveshit #iwillspinrecordsabout 9 hours ago via Blackberrytweet I was looking forLocations
- 12. Is there an easier way?Faceted Search can helpCurrent Query:Expand Query:Results:Yskiddd: Next saturday@thatsimpsonguy aka Guilty Simpson will be performing at Area51 in my homeytown Eindhoven. #realliveshit#iwillspinrecords2Usee123: Cool #EV3door7980 !!! http://bit.ly/igyyRhL sanmiquelmusic: This Saturday I'm joining @KrusadersMusic to IntentsEindhovenMusicLocations more...Events more...Music Artists:+ Guilty Simpson+ Bryan Adams+ Elton John+ Golden Earring+ Rihanna+ The eagles+ 3 Doors Downmore...
- 13. Location:Eindhoven Music Artist:Guilty SimpsonLocation:Area51Semantic relationships between entities are essential to realize such applications.
- 14. Relation Discovery FrameworkRelation Discovery Frameworktemporal constraintsrelation typetyped relationsmicroblogpostsEntity extraction &semantic enrichmentLocation APerson ALocation APerson AisLocatedInRelation discoveryLocation BGroup APerson AGroup AinvolvedInnews articlesEvent Aweighting schemesourceselectionApplications Browsing support
- 16. Schema enrichmentEntity Extraction and Semantic Enrichmentpowered byJulian Assange@bob: JulianAssange got arrested http://bit.ly/5d4r2tJulian AssangeTweet-basedenrichmentJulian AssangeJulian Assange JulianAssangearrestedJulianAssange, the founder ofWikiLeaks, is under arrest inLondon…News-basedenrichmentJulian AssangeLondonWikiLeaksLondonWikiLeaks
- 17. Relation Learning Strategiesentitiestime periodRelation: relation(e1, e2, type, tstart, tend, weight)RelationLearningstrategy: Input:entity e1 and e2, time period (tstart, tend)Challenge:inferweightand type of the relationfor the givenWeightingaccording to co-occurrence frequency:Tweet-based: count co-occurrence in tweetsNews-based: count co-occurrence in newsTweet-News-based: count co-occurrence in both tweets and newstype/label of relationrelatedness
- 18. Research QuestionsWhich strategy performs best in detecting relationships between entities?Does the accuracy depend on the type of entities which are involved in a relation?How do the strategies perform for discovering relationships which have temporal constraints (trending relationships)?
- 19. Datasetmore than: 20,000Twitter users2months10,000,000WikiLeaks founder, Julian Assange, under arrest in Londontweets75,000newstimeDec 15Jan 15Nov 15
- 21. Tweets and news articles per day50,000-400,000tweets per day100-1000 news articlesper day
- 22. Entities referenced per day10,000-100,000entity ref. in tweetsper day5,000-20,000 entity ref.in newsper day~40% tweets do not mention any (recognizable) entity72.6% of the top 1000 mentioned entities in Twitter are also mentioned in the mainstream news media99.3% of the news articles mention at least one (recognizable) entity
- 23. Number of Distinct Entities per Entity Types39 types of entities
- 24. Performance of Relation Learning Strategies
- 25. Our Ground Truth of true relationsBased on DBpedia:We mapped entities to their corresponding DBpedia resourcesNo appropriate DBpedia URIs for more than 35% of the entitiesWe analyzed whether there is a direct relation between two entitiesBased on user study:Participants judged whether two entities are really:related (62.6% were rated as related)related in the given time period (57.3% were rated as related)Overall: 2588 judgmentsThank you!
- 26. 1. Which strategy performs best in detecting relationships between entities?
- 27. Accuracy of relation discovery Combining both tweet-based and news-based strategies allows for highest accuracyBased on user studyBased on DBpedia
- 28. F-Measure@kCombined strategy (and news-based) increase in performance.Tweet-based strategy saturates quickly
- 29. 2. Does the accuracy depend on the type of entities which are involved in a relation?
- 30. Does the accuracy depend on the type of entities?87% precision92% Relationships which involve events can be discovered with high precision26% precision23%
- 31. Does the accuracy depend on the type of entities? (cont.)Relationships between events can be detected with highest precision.Relationships between persons/groups are difficult to detect.
- 32. 3. How do the strategies perform for discovering relationships which have temporal constraints?
- 33. Relationships with temporal constraintsTweet-based strategy performs better in discovering relationships that are valid only for a specific period in time
- 34. Where do relationships emerge faster?Speed of strategies is domain-dependenttime difference (in days) of first occurrence of relationshipNews is fasterTwitter is faster
- 35. Conclusions and Future WorkWhat we did: relation discovery framework based on TwitterFindings:Strategy that considers both tweets and (linked) news articles allows for highest accuracyPerformance varies for different domains (e.g. event-relationships can be detected with highest precision)Tweet-based strategy allows for detecting relationships, which have a restricted temporal validity, with high precision (and fast)Ongoing work: Adaptive Faceted Search on Twitterhttp://wis.ewi.tudelft.nl/tweetum/
- 36. Relation Discovery for Adaptive Faceted SearchCurrent Query:2. Analyze (temporal)relationships of entities that appear in the user profile to adapt facet ranking.Expand Query:Results:Yskiddd: Next saturday@thatsimpsonguy aka Guilty Simpson will be performing at Area51 in my homeytown Eindhoven. #realliveshit#iwillspinrecords2Usee123: Cool #EV3door7980 !!! http://bit.ly/igyyRhL sanmiquelmusic: This Saturday I'm joining @KrusadersMusic to IntentsEindhovenMusicLocations more...Events more...Music Artists:+ Guilty Simpson+ Bryan Adams+ Elton John+ Golden Earring+ Rihanna+ The eagles+ 3 Doors Downmore...1. Analyze (temporal)relationships of entities of the “current query” to adapt facet ranking.user
- 37. Thank you!IlknurCelik, Fabian Abel, Geert-Jan HoubenTwitter: @perswebhttp://wis.ewi.tudelft.nl/tweetum/
- 38. The Social WebHelp me to tackle the information overload! Who is this? What are his personal demands? How can we make him happy?Recommend me news articles that now interest me!Help me to find interesting (social) media!Give me personalized support when I do my online training!Personalize my Web experience!Do not bother me with advertisements that are not interesting for me!
Editor's Notes
- #4: Motivation:Information overloadPersonalised “better” search
- #5: Why do people search on Twitter rather than Google?Real time info & opinion about almost anything
- #6: Example: HT’11 @Eindhoven, looking for some entertainment events...http://search.twitter.com/http://search.twitter.com/advanced
- #7: Space limitation + selecting keywords (abbreviations –shorthand notations + colloquial expressions)
- #12: Highlight 60
- #13: Very time consuming and overwhelming indeed!
- #15: entity extraction and semantic enrichment and relation discovery.
- #19: large dataset of more than 10 million tweets and 70,000 news articles
- #21: 100-1000 news articles per day50,000 and 400,000 tweets per dayTwo of the minima were caused by temporary unavailability of the Twitter monitoring service.
- #22: approximately 10,000-100,000 entity references per day for tweetsapproximately 5,000-20,000 entity references per day for News~40% of the tweets had no entities99.3% of the news articles had at least one entityoverlap of entities: 72.6% of the top 1000 mentioned entities in Twitter are also mentioned in the news media.
- #23: 39 different entity typesPersons, locations and organizations were mentioned most often, followed by movies, music albums, sport events and political eventswe analyzed specific types of relations such as relationships between persons and locations or organizations and events in detail
- #30: Person/Group-Event relationships cover relations between persons and political events, persons and sport events, organizations and sport events, etcinteresting to see that the Tweet+News-based strategy discovers relationships, between persons/groups and events with higher precision 0.92 and 0.87 regarding P@10 and P@20 than people's relations to products (0.23 and 0.26) or locations (0.73 and 0.6).
- #31: relations between entities that are of the same typerelationships between two events can also be discovered with high precision, followed by relations between locations...
- #33: Twitter is more appropriate for inferring relationships, which have temporal constraints, than the news media.Tweet-based strategy improves precision (P@5) by 22.7%
- #34: relationships between persons and movies or music albums emerge much faster (14.7 and 5.1 days respectively) in Twitter than in the traditional news media.
- #35: Our framework extracts typed entities from enriched tweets/news and provides strategies for detecting semantic (trending) relationships between entities. We:investigated the precision and recall of the relation detection strategies,analyzed how the strategies perform for each type of relationships andWhich strategy performs best in detecting relationships between entities?Does the accuracy depend on the type of entities which are involved in a relation?How do the strategies perform for discovering relationships which have temporal constraints, and how fast can the strategies detect (trending) relationships?evaluated the quality and speed for discovering trending relationships that possibly have a limited temporal validity.
- #36: Very time consuming and overwhelming indeed!