![Data Preprocessing
The goal was to deduce whether the text is subjective or not.
Example: knowing whether the writer uses few or many punctuation
marks would definitely denote some sort of subjectivity.
Discretization was done using clustering. The k-means algorithm was
performed on each attribute with k=3 or k=4.
The values in each attribute were normalized using Min-Max normalization
which mapped them to the range [0;1].](https://image.slidesharecdn.com/aframeworkforemotionminingfromtextinonlinesocialnetworksfinal-121019005518-phpapp01/85/A-framework-for-emotion-mining-from-text-in-online-social-networks-final-16-320.jpg)
A framework for emotion mining from text in online social networks(final)
- 1. A Framework for Emotion Mining from Text in Online Social Networks Published in: Data Mining Workshops (ICDMW), 2010 IEEE International Conference Mohamed Yassine Hazem Hajj Department of Electrical and Computer Engineering American University of Beirut Beirut, Lebanon E-mail: {mmy18, hh63}@aub.edu.lb Advisors: Yin-Fu Huang Student: Chen-Ting Huang
- 2. Abstract & Introduction A new framework is proposed for characterizing emotional interactions in social networks, and then using these characteristics to distinguish friends from acquaintances. The framework includes a model for data collection, database schemas, data processing and data mining steps. The paper presents a new perspective for studying friendship relations and emotions’ expression in online social networks
- 3. Abstract & Introduction Text mining techniques are performed on comments retrieved from a social network. The purpose is not to identify specific emotions but rather to tell if the text contains emotions or not. In other words, if the text is subjective reflecting the writer’s affect and emotional state or if it is factual and objective.
- 4. Related Work SentiWordNet: a publicly available lexical resource for opinion mining NTUSD Face Book
- 5. Social Network – text example Post Comment
- 6. Row Data Collection Row data collection
- 7. Row Data Collection Posts The texts to be mined for emotions Comments User Information Friends
- 9. Lexicons Development This step deals with the informal language of social network a. Social acronyms a. Emotions
- 10. Lexicons Development This step deals with the informal language of social network c. Foreign Languages Lexicons: It is also necessary to take into consideration the use of foreign languages.
- 11. Feature generation Feature generation
- 12. Feature generation Computes new features from available raw data collected in step 1 to assess subjectivity of text. It uses word-matches to existing affective lexicons (by SetiWordNet) and employs new lexicons developed in step 2 Comments collected from step 1 along with the features computed in this step will be stored in the Sentiment Mining Database
- 13. Feature generation In order to assess the subjectivity of the text, several features need to be computed. a. Affective lexicon based on SentiWordNet b. Based on intentional misspelling errors and grammatical markers (such as punctuation and capitalized letters) c. Based on social acronyms, interjections and emoticons.
- 14. Data Preprocessing Data preprocessing
- 15. Data Preprocessing Feature selection was performed based on a correlation measure to remove redundant attributes.
- 16. Data Preprocessing The goal was to deduce whether the text is subjective or not. Example: knowing whether the writer uses few or many punctuation marks would definitely denote some sort of subjectivity. Discretization was done using clustering. The k-means algorithm was performed on each attribute with k=3 or k=4. The values in each attribute were normalized using Min-Max normalization which mapped them to the range [0;1].
- 17. Training Model for Text Subjectivity Creating a Training model for text subjectivity
- 18. Training Model for Text Subjectivity The training was done in step 5 using the k-means clustering algorithm with k =3. a. Objective or factual texts, b. Moderately subjective texts c. Subjective texts The output of the model was the centroids of the three clusters and is used in step 6 for classifying the comments with respect to subjectivity.
- 19. Text subjectivity classification Text subjectivity classification
- 20. Text subjectivity classification This step uses the centroids generated in the previous steps. Employs the k-nearest neighbor algorithm with k=1 to classify all comments into one of three subjectivity levels. The manual annotation was then compared to the predicted result by the model.
- 21. Model Evaluation eg: Comment 1 • Repeated letters: ‘o’ ‘i’ ’l’ ’u’ ’.’ ’o’ ’h’ • Emotion: ‘ :P ’ ‘ :) ’ • Affective words: • Acronyms: mwahhh
- 22. Model Evaluation k-means algorithm & Min-Max normalization
- 23. Model Evaluation
- 24. Model Evaluation Evaluating the model was done by applying the same above procedure on the sample of 850 comments. 84% 81%
- 25. Friendship Classification Friendship Classification
- 26. Friendship Classification We collected the comments shared for 1213 pairs of users. We used the output of step 6 which is text subjectivity classification in order to get the subjectivity measure of all comments shared between the pair. SVM algorithm was used to predict whether the pair consists of close friends or just acquaintances. Every user reported a list of his/her close friends. Using 10-fold cross- validation, the SVM model reported an accuracy of 87%.
- 27. Conclusion It presents a new perspective for studying friendship relations and emotions’ expression in online social networks We developed new lexicons that cover common expressions used by online users. The model predicted the right class with 88% accuracy. When the resulting model was used to predict relationship strength between two users, the prediction reported an accuracy of 87%.
Editor's Notes
- #3: 藉由
- #4: 在這篇 paper 中主要的目標便是從 social network 的文字中提取 emotional content
- #5: SentiWordNet 為 wordnet 的同義詞集提供 3 中層次的情緒分級 : positivity, negativity, objectivity.
- #7: Word match
- #11: 一些在 social network 常見的本土外語表達方式 , 在論文中會把這部分納入考量目的是在於嘗試處理外語的內容
- #22: 提出 3 個分別代表「 subjective 」「 moderately subjective 」 「 objective 」的 comment 作例子
- #27: 使用 SVM 來根據每一對底下的 subjectivity measure 實作 friendship 預測
- #28: 這篇 paper 藉由情緒探看台提供 1. 包含 emotion, acronyms