How useful are semantic links for the detection of implicit references in cscl chats
Download as PPT, PDF1 like732 views
The document discusses the usefulness of semantic links for detecting implicit references in chat conversations from computer-supported collaborative learning activities. It finds that on a macro level, the distribution of explicit links between utterances is similar to the distribution of semantic similarity scores, showing semantic similarity can be useful. However, at a micro level analyzing individual links, only around 40% of explicit links could be explained by semantic similarity alone, suggesting semantic similarity misses important links and additional techniques are needed.
How useful are semantic links for the detection of implicit references in cscl chats
- 1. Authors University Politehnica of Bucharest How Useful are Semantic Links for the Detection of Implicit References in CSCL Chats? Traian Rebedea traian.rebedea@cs.pub.ro Costin-Gabriel Chiru Gabriel-Marius Guțu
- 2. Overview • Introduction • Background • Research Question • Semantic Similarity • Macro-level: Are Semantic Links Useful? • Micro-level: Are Semantic Links Useful? • Conclusions 19.08.15 RoEduNet Conference 2014 – Chișinău, R. Moldova 2
- 3. Introduction • Chat conversations are used in a large range of Computer-Supported Collaborative Learning (CSCL) activities – Debating and solving difficult problems in small groups – Multi-party conversations – Several discussion threads running in parallel • Discourse analysis does not provide a theory suitable for processing multi-party chat conversations 19.08.15 RoEduNet Conference 2014 – Chi inău, R. Moldovaș 3
- 4. Background • New theories propose the use of conversation (or coherence) graphs for chat analysis • There is a existence of a multitude of links – explicit or implicit – between utterances that might explain the evolution of discussion threads • Each utterance has an inner (or specific) voice of the participant which uttered it, but also complex echoes from previous voices • Determining and analyzing this linkage between voices would provide a powerful method for analyzing learning and knowledge building – Individual level – Group level 19.08.15 RoEduNet Conference 2014 – Chișinău, R. Moldova 4
- 5. Background • The quality of individual thinking can be improved by improving the quality of dialogue – “individual thinking skills originate in conversations, where we learn to reason, to evaluate, to join in creative play and to provide relevant information” (Wegerif, 2006) • Difficult to determine the quality of a conversation, especially in online multi-party discussions – Proposed that the degree of inter-animation in a conversation can be used to assess its quality – Inter-animation assumes that meaning arises not from a single utterance, but rather from the interactions between utterances 19.08.15 RoEduNet Conference 2014 – Chișinău, R. Moldova 5
- 6. Background • Inter-animation of voices in a conversation may be represented through the links between the utterances – Explicit links – Many links remain implicit • The discovery of these implicit links is a difficult task, mainly because of the multitude distinct types of links that may arise – Lexical links – Continuation of utterances (which may be considered broken utterances or syntactic links) – Semantic links – Pragmatic and conversation specific references 19.08.15 RoEduNet Conference 2014 – Chișinău, R. Moldova 6
- 7. Research Question • How useful is semantic similarity alone for detecting links arising in CSCL chat conversations? • Many CSCL applications that are processing either conversations or written texts are mainly using (lexical, semantic) similarity relatedness methods for assessing the cohesion of the analyzed texts • They may miss important links that are not detectable using semantic similarity alone • Cohesion vs. Coherence19.08.15 RoEduNet Conference 2014 – Chișinău, R. Moldova 7
- 8. Semantic Similarity • Semantic similarity (or relatedness) between proximal words is one of the characteristics of the context in most coherent discourses • Knowledge-based methods (“strong” semantics) – Rely on general, domain and discourse ontologies • Corpus-based methods (“weak” semantics) – Do not define any underlying semantic model between the words or concepts, but rather exploit the probability distribution and the statistics of two words co-appearing together in a given discourse unit (e.g. paragraph) 19.08.15 RoEduNet Conference 2014 – Chișinău, R. Moldova 8
- 9. Knowledge-based Methods • Primarily use lexical resources built by linguists, for example dictionaries, thesauri and lexical ontologies • The most popular methods for computing the semantic similarity or relatedness between two words or concepts are defined for ontologies => semantic distances • Most of them have been especially constructed for the linguistic ontology (or lexical database) WordNet – synonyms, antonyms, related words, hypernyms, hyponyms, meronyms and holonyms 19.08.15 RoEduNet Conference 2014 – Chișinău, R. Moldova 9
- 10. Corpus-Based Methods • Instead of using human-assembled linguistic knowledge, these techniques process large amounts of text corpora and then use the statistics of words co- appearance • Latent Semantic Analysis (LSA) – Perform SVD to reduce the dimensionality of the term- document matrix – Keep only the largest singular values (usually k = 100..300) – This reduced dimensionality space is also called the latent semantic space and may be used to compute the similarity between words, word sets and texts by using cosine similarity for the document vectors in this reduced space • Other techniques have been proposed: Latent Dirichlet Allocation (LDA), Explicit Semantic Analysis (ESA), etc. 19.08.15 RoEduNet Conference 2014 – Chișinău, R. Moldova 10
- 11. Distribution of Explicit Links in Chats • Corpus of 8 chat conversations analyzed to compute statistics with regard to the usage of explicit links – Multi-party discussions in groups of 4-5 students – Using Concert Chat software (allows explicit referencing) 19.08.15 RoEduNet Conference 2014 – Chișinău, R. Moldova 11
- 12. Distribution of Explicit Links in Chats • Complete distribution of explicit links given the distance between the two utterances 19.08.15 RoEduNet Conference 2014 – Chișinău, R. Moldova 12
- 13. Are Semantic Links Useful? • Macro-level analysis: Compare the distribution of the explicit links to the semantic similarity between the current utterance and utterances that are at a certain distance • Computing the semantic similarity between two utterances uses the similarities between all the pairs of words in each utterance 19.08.15 RoEduNet Conference 2014 – Chișinău, R. Moldova 13 ) )( ))(*)1,((max )( ))(*)2,((max (* 2 1 )2,1( 2 2 1 1 ∑ ∑ ∑ ∑ ∈ ∈ ∈ ∈ += Tw Tw Tw Tw widf widfTwSim widf widfTwSim TTsim
- 14. Are Semantic Links Useful? LSA WordNet – Resnik distance 19.08.15 RoEduNet Conference 2014 – Chișinău, R. Moldova 14
- 15. Are Semantic Links Useful? • Micro-Level Analysis • How effective is LSA-based semantic similarity for detecting links between utterances? • Simple greedy method for choosing the link of the current utterance: – Compute the semantic similarity between the utterance and the previous 20 utterances – Choose as a link the pair which has the best (largest) similarity score 19.08.15 RoEduNet Conference 2014 – Chișinău, R. Moldova 15
- 16. Are Semantic Links Useful? ID Utterance User Link Sematic score 148 ok.... so what we were talking about before the connection issue? Mona 149 as long as it depends on an internet connection... Mona LINK TO 147 150 meeting board Cristi LINK TO 148 score(150, 148) = 0.18 = max 151 about different stages of a project... a client must know about them..so wiki is a good solution Corina LINK TO 148 152 meeting? Corina LINK TO 150 score(152, 150) = 1.00 = max 153 ok Stefan LINK TO 151 154 so we agree that wiki is a good solution when we want to present a product/the evolution of a project to a client ? Corina score(154, 151) = 0.5 = max 155 what about selling our products..what technologies we should use for this? Diana … 166 to sell our products blog is the best solution Corina LINK TO 155 score(166, 155) = 0.38 = max … 169 A blog would be a good way to advertise our products Mona LINK TO 166 score(169, 166) = 0.48 = max … 180 A forum would be useful for offering solutions to some problems that our customers have Mona 181 I agree...also other people can offer solutions, not only us Corina LINK TO 180 score(181, 180) = 0.27 = max 19.08.15 RoEduNet Conference 2014 – Chișinău, R. Moldova 16
- 17. Are Semantic Links Useful? • Can explicit links be identified using only the LSA semantic similarity between utterances? 19.08.15 RoEduNet Conference 2014 – Chișinău, R. Moldova 17
- 18. Conclusions • Two important conclusions arise from this study • The distribution of explicit links is very similar to that of the semantic similarity scores, thus proving that, on a macro-level, both links and semantic similarity measures have a similar behavior • This is not always true when analyzing local elements: an individual link between two utterances is not always explained by semantic similarity • Only around 40% of the explicit links in our conversations could be explained based mainly on semantic similarity measures 19.08.15 RoEduNet Conference 2014 – Chi inău, R. Moldovaș 18
- 19. Thank you! Questions? Discussion 19.08.15 RoEduNet Conference 2014 – Chi inău, R. Moldovaș 19 This work has been funded by the Sectorial Operational Programme Human Resources Development 2007-2013 of the Romanian Ministry of European Funds through the Financial Agreement POSDRU/159/1.5/S/132397 and by the FP7 LTfLL project.