#lak2013, Leuven, DC slides, #learninganalytics
0 likes499 views
1) The document summarizes a proposed social semantic recommender system for learning that would provide recommendations to users in an online learning environment based on their social connections and activities. 2) An empirical study evaluated different recommender algorithms on several datasets to determine the most accurate algorithm, and found the extended T-index algorithm performed best. 3) Further proposed research includes a user evaluation study and pilot study to test the system with users and continue improving the recommender system and evolving the social network.
#lak2013, Leuven, DC slides, #learninganalytics
- 1. A Social Semantic Recommender for Learning Soude Fazeli, PhD candidate Dr. Hendrik Drachsler Prof. Dr. Peter Sloep page 1
- 2. The doctoral study is funded by • NELLL (Netherlands Laboratory for Lifelong Learning at the OUNL) • Open Discovery Space (ODS) A socially-powered, multilingual open learning infrastructure to boost the adaptation of eLearning Resources in Europe Run-time: 2011-2015 page 2
- 3. A social space for learning page 3
- 4. page 4
- 6. Link to Learning Analytics (LA) • Duval (2011) introduced recommenders as a solution • To deal with the “paradox of choice” • To turn the abundance from a problem into an asset for learning • Several domains try to find patterns in a large amount of data • Educational data mining, Big Data, and Web analytics • Recommender systems and personalization as an important part of LA research, Greller and Drachsler (2012) page 6
- 7. A proposed recommender system for learning Based on the framework proposed by Manouselis & Costopoulou (2007) For more details, please refer to Fazeli, S., Drachsler, H., Brouns, F. and Sloep, P. (2012) page 7 !
- 9. State-of-the-art educational recommenders • Manouselis et al. (2010) • Testing multi-attribute recommenders within Learning Resource Exchange (http://lreforschools.eun.org) • Cechinel et al. (2012) • Several memory-based collaborative filtering algorithms on the MERLOT repository (http://www.merlot.org) • Koukourikos et al. (2012) • Using sentiment analysis techniques to enhance collaborative filtering algorithms within MERLOT dataset • Sparsity! • Verbert et al. (2011) • Different algorithms on several datasets: MACE, Travel well, MovieLens • Manouselis et al. (2012) • Organic.Edunet (http://portal.organic-edunet.eu/) and a synthetic dataset page 9 including the real data plus some simulated data
- 10. (Golbeck, 2009; Kamvar et al., 2003; Ziegler & Golbeck, 2007; Massa & Avesani, 2004; Lathia et al., 2008; Fazeli et al., 2010) page 10
- 11. A social recommender system: T-index approach (Fazeli et al., 2010) • Creates trust relationships between users • Based on the ratings information • Proposes T-index concept • To measure trustworthiness of users • To improve the process of finding the nearest neighbours • Inspired by H-index • Used to evaluate the publications of an author • Based on results, T-index improves • Prediction accuracy of generated recommendations • Structure of trust networks of users page 11
- 12. Trust in recommender systems • Trustworthy users == like-minded users • A new trust relationship between two thus far unconnected users is inferred if and only if: • Condition 1: • page 12 • mutual trust value between intermediate users is higher than a certain threshold Condition 2: • The number of intermediate users is lower than an upper bound; in this study the upper bound is 2
- 13. Social trust in recommender systems rated rated Bob Alice Carol page 13 rated rated if A trusts B and B trusts C, then A trusts C if and only if condition 1 is met and condition 2 is met
- 15. • RQ1: How to generate more accurate and thus, more relevant recommendations by using the social data originating from social activities of users within an online environment? • RQ2: Can the use of the inter-user trust relationships that originally come from the social activities of users within an online environment, help user networks evolve? page 15
- 16. Proposed research 1. Requirement analysis • Literature review • Interview study 2. Data-driven study 3. User evaluation study 4. Pilot study page 16
- 17. 1. Requirement analysis • Goal • Investigating main needs and requirements of users in an online social environment • Method • • A summer school for European teachers in Greece, July 2012 Asking the participants to fill in a questionnaire regarding • The importance or usefulness of the activities within an online social environment • The use of recommender systems. • Description • • 33 teachers participated from 14 countries (Portugal, Germany, France, Finland, Greece, Austria, Poland, Lithuania, Spain, Hungary, Romania, Cyprus, Ireland, Serbia and the US) “sharing content on Facebook, Twitter, etc. or by email” important, useful or not • Expected outcomes • A list of the most important needs and requirements of teachers within an online social environment like the ODS portal page 17
- 18. 1. Requirement analysis 1.1. Use case diagram page 18 !
- 19. 1. Requirement analysis 1.2. Results How much the teachers find the online social activities important/useful How much teachers find the detailed requirements important/ page 19 useful !
- 20. Proposed research 1. Requirement analysis • Literature review • Interview study 2. Data-driven study 3. User evaluation study 4. Pilot study page 20
- 21. 2. Data-driven study • Goal • To find out the most suitable recommender system algorithm for a social online platform like ODS platform • Method • • An offline empirical study of candidate algorithms including the extended Tindex algorithm Datasets: • TravelWell, Mace, OpenScout, MovieLens (as a standard dataset for comparison) • Mendeley, MERLOT • Variables to be measured • Performance: Precision accuracy, recall, F-measure (F1) • Network analysis: degree centrality • Expected outcomes • Which of the recommender algorithms best performs and thus, is suitable for social online platforms like ODS platform page 21
- 22. 2. Data-driven study 2.1. F1 result 0.1" 0.09" 0.08" 0.07" 0.06" 0.05" 0.04" 0.03" 0.02" 0.01" 0" OpenScout% 0.14" 0.12" 0.1" 0.08" Tanimoto3Jaccard"(CF1)" 0.06" Loglikelihood"(CF2)" Euclidean"(CF3)" 0.04" Euclidean"(CF3)" Graph4based"(CF4)" 0.02" Graph3based"(CF4)" Tanimoto4Jaccard"(CF1)" Loglikelihood"(CF2)" F1@10% F1@10% MACE% 0" 3" 5" 7" 3" 10" 5" Travel%well% 10" MovieLens% 0.25" 0.08" 0.2" 0.06" Tanimoto3Jaccard"(CF1)" 0.04" Loglikelihood"(CF2)" Euclidean"(CF3)" 0.02" Graph3based"(CF4)" 0" 3" 5" 7" size%of%neighborhood%(n)% page 22 10" F1@10% 0.1" F1@10% 7" size%of%neighborhood%(n)% size%of%neighborhood%(n)% Tanimoto0Jaccard"(CF1)" 0.15" Loglikelihood"(CF2)" 0.1" Euclidean"(CF3)" 0.05" Graph0based"(CF4)" 0" 3" 5" 7" 10" size%of%neighborhood%(n)% F1 of the extended T-index and Tanimoto algorithms for different datasets, based on the size of neighborhood
- 23. 2. Data-driven study 2.2. user network page 23
- 24. 2. Data-driven study 2.3. Degree centrality 250" 200" 150" MovieLens" degree% OpenScout" 100" MACE" Travel"well" 50" 0" u1" u2" u3" u4" u5" u6" u7" u8" u9" u10" Top)10%central%users% Degree distribution of top-10 central users for different datasets page 24
- 25. Proposed research 1. Requirement analysis • Literature review • Interview study 2. Data-driven study 3. User evaluation study 4. Pilot study page 25
- 26. 3. User evaluation study • Goal • To study usability of developed prototype by evaluating users’ satisfaction • Method • • Questionnaire Adapting the user-centric evaluation proposed by Pu et al. (2011) in the context of recommender systems • Variables to be measured • Quality of recommendations based on accuracy, novelty, and usefulness • Expected outcomes • page 26 Initial feedback by end-users on users’ satisfaction as an input for pilot study
- 27. Proposed research 1. Requirement analysis • Literature review • Interview study 2. Data-driven study 3. User evaluation study 4. Pilot study page 27
- 28. 4. Pilot study • Goal • To deploy the final release • To test it under realistic operational conditions with the end-users • Method • Evaluating performance of the designed recommender system algorithm • Study the structure of the built users network • Variables to be measured • Prediction precision and recall, and F-measure (F1) • Effectiveness in terms of total number of visited, bookmarked, or rated • learning objects for two groups of users (pre and post study) Degree centrality distribution to study how the structure of users network changes • Expected outcomes • • page 28 Empirical data on performance of the used recommender algorithm The visualization of teachers’ networks
- 29. Conclusion • The aim is to support user in social platforms to find the most suitable content or people • Recommender systems as a solution • How to deal with the sparsity problem by use of social data of users page 29
- 30. Ongoing and Further work • Data set study (May 2013) • • Testing more datasets (Mendeley, MERLOT) Testing other recommender algorithms (loglikelihood for implicit indicators, Pearson, Euclidian for explicit indicators) • Go online with the ODS platform (June 2013) • User evaluation study (September 2013) page 30
- 31. Soude Fazeli PhD candidate Open University of the Netherlands Centre for Learning Sciences and Technologies (CELSTEC) PO-‐Box 2960 6401 DL Heerlen, The Netherlands email: soude.fazeli@ou.nl page 31