Using the Social Web to Supplement Classical Learning
1 like721 views
The document discusses supplementing traditional classroom learning with social web technologies like blogs and chat. Students were assigned team projects using these tools. Tutors then evaluated student work using tools to analyze chat conversations and blog posts. The tools helped reduce grading time. Surveys found students responded positively to this approach, which engaged them in collaborative learning communities. Over 700 students participated in total across multiple years.
Using the Social Web to Supplement Classical Learning
- 1. Using the Social Web to Supplement Classical Learning Stefan Trausan-Matu, Vlad Posea, Traian Rebedea & Costin Chiru “ Politehnica” University of Bucharest, Department of Computer Science and Engineering
- 2. Overview Context Learning Scenario Monitoring & Evaluation Tools Chat Evaluation Blog Analysis Results Conclusions
- 3. Context Web 2.0 (Social Web) People participate in communities: create and share content, discuss, rank, … Classical teaching and learning paradigm Attendance, interest, moving to involving the students more and more in discussion E-learning LMS, LCMS, Authoring Tools, Learning Reporting, Assessment
- 4. Social Web & Learning ?
- 5. Solution Supplement classical learning by using the Social Web Classroom-based learning + Web 2.0 assignments E-learning + Web 2.0 => E-learning 2.0 Questions that arise: What technologies to use ? How to support the interaction of the students ? Assessment ?
- 6. Learning Scenario Human-Computer Interaction course Theory of interface design and evaluation, usability, user’s psychology Laboratory and assignments: focus on Web standards and technologies Experiments performed from 2005 to 2009 Over 150 (senior year) students each year
- 7. Old Learning Scenario Standard scenario (prior to 2005) Students were learning the theoretical aspects in the course and solving 4-5 assignments using Web technologies for the laboratory Problems: Great number of technologies and standards Specifications and APIs changing very fast Some of them were still under development (beta) The students worked mostly individually
- 8. Old Learning Scenario Problems: Difficult to provide support for each technology and each programming language Time consuming to keep the documentation up to date Sparse documentation for beta APIs Students were not sufficiently involved / individuality
- 9. New Learning Scenario Assignments using Web 2.0 technologies: Modified Virtual Company learning scenario: website + blog – teams of 4 students Twitter, FB accounts, Youtube, Slideshare Chat conferences debating a given topic – 1-3 assignments for small groups of 3-8 students Discussion forums – S2S, S2T Informal learning in a formal context Competition & Collaboration
- 10. Monitoring & Evaluation Tools Large volume of data produced by the students Social data (including conversations) is difficult to assess (time consuming) Several objectives: To help tutors assess the work of the students To discover the useful artifacts produced by the students Tools that are used by tutors
- 11. Chat Evaluation Tools Assess both collaboration and quality/content Two distinct tools: Polyphony Analysis & ASAP NLP + WordNet + SNA techniques Improve the quality of the evaluation process and reduce the amount of time needed by the tutors Tested for the first time in 2008 Polyphony (Bakhtin): implicit references between utterances, inter-animation, discussion threads
- 12. Chat Evaluation Tools For supporting the tutors: Extraction of the most important topics in the chat using a grouping algorithm based on synsets Visualization of the conversation Evaluation module that assigns a value to each utterance in the chat and based on these values it provides a score for each participant Interactive component that permits inspecting the utterances and references in the conversation.
- 15. Blog Analysis Posts are collected using ATOM In 2009 – more than 2000 posts Clustering of the posts according to their subject Looking for concepts related to the course and lab (using Freebase and DBPedia, not WN) Classification of the posts might be possible, but no training data was available
- 16. Results - blogs Important result: the collaborative knowledge constructed by the groups of students Analyzed the created content in order to establish their actual value for the community Categories of posts: personal experiences, studied technologies, new technologies and miscellaneous
- 17. Results – blogs Misc: 48% - 2008, 28% - 2009 The most useful: 1. personal experiences; 2. new technologies Example: community found out how to tackle a bug that existed into one software tool used for the assignments (CTTE) % Personal Experiences New Technologies Studied Technologies 2008 23 43 34 2009 18 21 61
- 18. Results - chat Tutors used the statistics provided by the tools in order to have a better understanding of a chat conference in less time. NLP -> content Threads + SNA -> collaboration Preliminary test: 4 chats * 4 students Grading error: 10.1% Polyphony, 20% ASAP Correlation: .85 Polyphony for 14 students Time reduced by more than 30%
- 19. Conclusions Complex learning experiment using Web 2.0 technologies and collaboration in a formal context Learning community – artifacts Evaluation: 78% of the students considered this approach to be good or very good Great involvement (128 blog posts, hours of chat per session) Over 700 students
- 20. THANK YOU! Questions & Feedback
Editor's Notes
- #2: Hello! (My name is Traian Rebedea and) I am a Phd student and a teaching assistant at the PUB – Department of Computer Science, working in the CSCL team (under the supervision of prof TM). I shall present you the approach and investigations undertaken by my colleagues and I of using the social web to supplement classical learning / teaching for a course at our department. Before I continue, I must say that this is an interesting subject for me as a researcher and as a (“future”) teacher as well and I have tried to present our work from both perspectives. Let’s start…
- #3: … with the overview. The first part of my presentation introduces the context of our experiments, with the associated problems and the proposed solution (that might not be perfect, but we believe that it is for sure an improvement). Next I shall present the new learning scenario that was developed by adopting some social web technologies into the course, in parallel with the old one. But, this new scenario raised several problems that we have tried to tackle by developing tools for assisting the tutors in their new tasks: they have developed for chat conversations, on one hand, and for blog analysis, on the other hand. The final part of my talk offers several results that have been achieved both by using the new learning scenario and the developed tools. Several conclusions end my presentations.
- #4: The Social Web (or Web 2.0) provides (complex) web applications that allow people to create and share content, to discuss, to rank and group themselves into communities (and more…). Basically, it is used to support and enhance the interaction between people over the Internet. Everybody has heard about it, knows, uses (at least some applications) and seems to be happy with it. The latter sentence seems to be true for the classical teaching and learning paradigm, especially in formal F2F higher education (at least in Romania). But there are more and more problems that arise by using this paradigm: as the students are used to being more social (raised partly by the Web 2.0 environment), attendance starts to be an issue, the same is true for stimulating the interest and more. In order to solve them, the teachers are moving towards involving the students more and more in the discussion, to stimulate interaction… the social part of learning. A similar perspective exists in e-learning, where there has been much focus on how to construct Learning Management System, LCMS, …., but there has been little support until recently to integrate Web 2.0 technologies and applications (especially existing ones).
- #5: On one side, we have a lot of implemented Web 2.0 technologies. One the other, we have the usual teaching / learning paradigm, either classical or e-learning. Should we combine them? How to combine them?
- #6: A lot of other questions are of interest.
- #7: During this course the students were taught the theory of interface design and evaluation, usability, user’s psychology and they had to apply their theoretical knowledge in practice developing individually some Web 2.0 applications for the laboratory.
- #8: Because the students were in their senior year, they were allowed to work with their favorite programming languages and technologies. Due to this characteristic, the number of technologies (APIs, tools, libraries) with which we actually worked during the lab was very large and the documentation was often sparse. The technologies were changing very fast (for example the APIs for interacting with social web sites like delicious.com) and they were not the same from one year to another. Moreover some of them were in a beta phase and had different types of bugs. There were also a rather large number of types of applications that the students were able to choose from, in order to have a higher degree of personalization for the assignments. We felt that the students could benefit more if they worked in teams if possible in ways that are close to a company environment, as they were in the last year of studies.
- #10: - They were asked to construct a web site for their team using the technologies and the theoretical principles learnt in the course. Also they were asked to use a blog to share their personal experiences with the technology and to discuss about the new and interesting stuff they discovered during this course. An interesting starting point for this approach was the Virtual Company learning scenario - the students work in project teams and develop competences by working collaboratively on real tasks from real customers. The Virtual Company scenario is very good for our students but we needed to adapt it because we were not able to use real customers and tasks for such a large number of students. Therefore we asked the students to consider their teams as their own virtual companies and their assignments as their company’s projects. Topic: comparison between different social web technologies (wikis, blogs, discussion forums and chat). After studying individually this topic, the students formed teams to discuss the subject using chat: in the first part of the conversation, each student had to support one of these technologies by presenting its features and advantages and criticize the others by invoking their flaws. In the final part of the chat, they also had to discuss how they could integrate all these technologies in a single online collaboration platform. This way, in a single conversation the students first engaged into a debate those results are then used for building collaboratively a solution to a given problem. - The result of competition is evolution (genetics, economics, but also social)
- #12: The evaluation should consider two distinct components: assessment of the degree of collaboration and involvement in the chat and determining the quality of the utterances issued by a participant in the conversational context. Both aspects are complex to measure without computer assistance, especially the latter. Both employ natural language processing and social network analysis techniques, but they use different approaches. Polyphony Analysis uses Bakhtin’s ideas of voices and polyphony in order to discover implicit references between the utterances of the chat, thus constructing discussion threads. Discourse analysis techniques including cue phrases and speech acts detection were used in order to achieve this task. Moreover, temporal constraints and semantic similarities based on Wordnet were also utilized.
- #16: As most of these terms don’t appear in a regular linguistic ontology such as Wordnet we have used a public knowledge base called Freebase for identifying computer science-related terms. The terms were searched using the Freebase web services and Freebase’s own query language MQL. The query tried to identify if the term had one of the following standard Freebase domains/types: {computer, software, internet}. The script had a success rate of just over 80%, managing to identify most of the common used abbreviations in the blog posts such as the popular web standards and technologies like “RDF”, ”RSS”, ”XML”, etc.
- #17: We had 45 blogging teams in 2007-2008 and 96 in 2008-2009. These teams produced over 700 posts in the first year and 2200 in the second one. This content wasn’t always of top quality and it wasn’t always useful so we needed to define some criteria for interesting and useful posts and blogs. - The first thing we did was differentiate the blogs on the quantity of content. The number of posts, for a 3 months time span evolved from 4 posts to 130 with an average of 16.5 in the first year and of 23.6 in 2009.
- #19: Preliminary tests were conducted on a group of 4 chat sessions involving teams of 4 members each that were analyzed separately by four tutors – two of them using the tools and the other two without any software assistance. The grading error for Polyphony is 10.1%, twice better than ASAP, and quite close to the error rates of the tutors. Nevertheless, the correlation between the average tutor grade and the grades provided by the two systems are significantly poorer compared to those of the tutors. Still, the correlation obtained by Polyphony Analysis is encouraging for a subset of three chats out of the four - .85, only slightly worse than the average tutor correlation. The improvement in time needed for the evaluation of a chat session is also hopeful as the time required for analysis was reduced by more than 30% for the tutors employing the analysis tools. More information: http://cscl2009.blogspot.com/2009/04/computer-assisted-evaluation-of-cscl.html
- #20: The results show that this way of learning was appreciated by the students who worked hard and produced an important quantity of valuable content.