SCRIBL Data- Scalable, Real-Time, Individual Behavior and Learning Data
Download as PPTX, PDF2 likes5,286 views
The document discusses a conference on cyberlearning tools for STEM education. It provides details on an online discussion forum for conference sessions and shares data on individual behavior and learning from wikis. Research questions are posed about making wikis effective learning environments, whether certain student populations benefit more, and how to design a study to examine digital divides in tool usage.
![Dimensions of the Digital Divide
1st Digital Divide: Access 2nd Digital Divide: Usage
• Schools • Attewell (2003): ―[There exists a]
– 3.8 Students/Computer in real possibility that computing for
schools with <35% already-disadvantaged children
students eligible for FRPL may be dominated by games at
home and unsupervised drill-
– 4.0 students/computer in
and-practice or games at
schools with >75% student
school, while affluent children
eligible for FRPL
enjoy educationally richer fare
• Anywhere with more adult involvement‖
– 86 % of students living in • Jenkins (2007) Participation Gap
households making <$30K
Attewell, P. (2003). Beyond the digital divide. In P. Attewell, & N.
use the internet Seel (Eds.), Disadvantaged teens and computer
technologies (pp. 15-34). Munster, Germany: Waxmann.
– 97% of students living in Jenkins, H.; Clinton K., Purushotma R., Robison A. and Weigel
M.(2007), Confronting the challenges of participatory culture:
households making >$70K Media education for the 21st century. Chicago, Il.:
MacArthur Foundation.
use the internet](https://image.slidesharecdn.com/cytse2011-111118125628-phpapp01/85/SCRIBL-Data-Scalable-Real-Time-Individual-Behavior-and-Learning-Data-29-320.jpg)
![Fear, anxiety and worry about student
exposure to the Internet
Barbara: There was - the superintendent said, "Do want
you want but I never want us on the frontpage with some
bad headline" [chuckle]. I thought those are kind of
interesting. That's why he told our technology committee.
Our main goal is to not make the frontpage with the bad
headline attached.
Jim: And as far as I know but I don't want to be the person
to be in the newspaper for... Look at what this kid was
doing on the internet in a school. I don't want that](https://image.slidesharecdn.com/cytse2011-111118125628-phpapp01/85/SCRIBL-Data-Scalable-Real-Time-Individual-Behavior-and-Learning-Data-52-320.jpg)
SCRIBL Data- Scalable, Real-Time, Individual Behavior and Learning Data
- 1. Cyberlearning Tools for STEM Education Conference 2011 Join the online discussion backchannel for this session at: www.cyberlearningSTEM.org/sessions
- 2. SCable Real-time SCRIBL Data Justin Reich Individual Harvard Graduate School of Education Behavior and Cyberlearning Tools Learning for STEM conference 3/8/11 Data bjr795@mail.harvard.edu
- 6. MrBoyersClass.Pbworks.com Page Saves by Day 100 90 80 70 60 50 Page Saves 40 30 20 10 0 0 50 100 150 200 250 300 Days
- 7. Scatter Plot of Wiki Page Saves by Day (n=1,799)
- 8. Moving average of wiki development measured in page saves ALLPS 30 20 10 0 0 100 200 300 400 500 600 700 800 900 day
- 10. SCable Real-time Individual Behavior and SCRIBL Data Learning Data
- 11. # of Time/Scale Web 2.0 Research Cases State Space Modeling 1,000K Usage Simulations Statistics 100K Policy Semantic Makers Analysis 10K Surveys 1,000 School Content Interviews Leaders Analysis 100 Discursive 10 Analysis Teachers Biometric Design Observational 1 Analysis Research Research Seconds Days Weeks Months Years 11 Duration of data collection and capture
- 12. Research with SCRIBL Data • We can study SCRIBL data with microscopes and telescopes – Plan for interdisciplinarity – New methods are needed • Operationalize time • Don’t invent new platforms, meter widely adopted platforms
- 14. Learner Analytic Approaches Netflix vs.Pandora
- 15. PBworks wiki lifetimes in seconds/days 1 0.9 (n=179,851) Estimated Survival Probability 0.8 Estimated Seconds Days 0.7 Lifetimes 25% 250 <1 0.6 50% 123,613 1.4 0.5 75% 5,282,874 61.1 0.4 0.3 0.2 All PBWorks Wikis 0.1 0 0 20000000 40000000 60000000 80000000 10000000 (231) (463) (694) (926) (1157) Time in seconds (days)
- 16. Learner Analytic Approaches Netflix vs.Pandora
- 17. Does wiki persistence differ in Title I and non-Title I schools? (n=259) Non-Title I Eligible Title I Eligible
- 18. Wiki Opportunities for Students to Develop 21st Century Skills • Expert thinking: – Do students use academic content knowledge in wiki activities? – Do students reflect on the process/product? • Collaboration: – Do students concatenate text on pages? – Do they substantively edit each others work and co-create pages? • New Media Literacy: Wiki Quality Instrument – Do students use formatting? 25 Questions – Do they hyperlink? – Do they embed multimedia? Scale of 1-25
- 19. Do wikis provide opportunities for students to develop 21st century skills? 25 Are great wikis born or made? 20 Wiki Quality Score 15 10 5 0 0 50 100 150 200 250 300 350 400 450 Days
- 20. Do wikis created in high SES schools provide more opportunities for 21st century skill development? 25 20 15 10 5 High SES Low SES 0 0 50 100 150 200 Days 250 300 350 400 450
- 21. Do wikis created in different subject areas provide different levels of opportunity to develop 21st C skills 25 20 Wiki Quality Score Social Studies English 15 Science Computer Science 10 Math 5 0 0 50 100 150 200 250 300 350 400 450 Days
- 22. SCable Real-time Individual Behavior and SCRIBL Data Learning Data
- 23. Educators routinely try to gather information about their students’ learning on the basis of what students do in class. But for any question posed in the classroom, only a few students respond. Educators’ insight into what the remaining students do and do not understand is informed only by selected students’ facial expressions of interest, boredom, or puzzlement. To solve this problem, a number of groups are exploring the use of various technologies to ―instrument‖ the classroom in an attempt to find out what students are thinking.
- 24. Back Deck
- 25. Classroom Wiki Research Questions • How do we make them good? – What best practices, attitudes and resources produce wiki learning environments that promote and nurture 21st century skills? • Do only certain kids get the good ones? – What is the distribution of high quality wikis across schools serving different student populations? 25
- 26. Open Education Resources • Twin Hopes for OER – (Excellence) Teachers will use free, online tools and resources to create student- centered learning environments that prepare students for 21st century life – (Equity) Since these materials are free, poor students will disproportionately benefit. Brown, J. S., & Adler, R. P. (2008). Minds on fire: Open education, the long tail, and learning 2.0. Educause Review, 43(1), 16-32. Bonk, C. J. (2009). The world is open : How web technology is revolutionizing education (1st ed.). San Francisco, Calif.: Jossey-Bass.
- 27. Narrowing gaps or a rising tide? Non-poor students Non-poor students 21st Century Skills Poor students Poor students
- 28. Remapping the Digital Divide • How can we design a study to examine whether or not poor students disproportionately benefit from the availability of free online tools? – Theoretical Framework – Operationalizing the theoretical framework – Research Design
- 29. Dimensions of the Digital Divide 1st Digital Divide: Access 2nd Digital Divide: Usage • Schools • Attewell (2003): ―[There exists a] – 3.8 Students/Computer in real possibility that computing for schools with <35% already-disadvantaged children students eligible for FRPL may be dominated by games at home and unsupervised drill- – 4.0 students/computer in and-practice or games at schools with >75% student school, while affluent children eligible for FRPL enjoy educationally richer fare • Anywhere with more adult involvement‖ – 86 % of students living in • Jenkins (2007) Participation Gap households making <$30K Attewell, P. (2003). Beyond the digital divide. In P. Attewell, & N. use the internet Seel (Eds.), Disadvantaged teens and computer technologies (pp. 15-34). Munster, Germany: Waxmann. – 97% of students living in Jenkins, H.; Clinton K., Purushotma R., Robison A. and Weigel M.(2007), Confronting the challenges of participatory culture: households making >$70K Media education for the 21st century. Chicago, Il.: MacArthur Foundation. use the internet
- 30. How can we operationalize usage? Usage as persistence • Wiki lifetime: number of days of activity of a wiki community – Birth: Creation of wiki subdomain (e.g. ReichWorldHistory.pbworks.com) – Death: Final wiki edit • After a 90 day observational period 30
- 31. How can we operationalize usage? Usage as opportunities to develop 21st century skills • Participation • Expert Thinking • Complex Communication • New Media Literacy 31
- 32. Research Design
- 33. What are our research questions? RQ #1) Persistence: Are wikis created in schools serving affluent students used for greater lengths of time than wikis created in poor schools? RQ#2) Participation: Do wikis created in schools serving affluent students provide more opportunities for students to develop 21st century skills 33
- 34. Which wikis are in my sample? • Dataset – All179,853 publicly-viewable education-related wikis started on the PBworks platform between June 2005 and August of 2008. – Does not include ―private‖ wikis (~70,000) • Sample – Randomly sampled 1,799 wikis (1%) – Coded to identify 259 U.S. based, K-12 wikis from specific public schools • Detailed usage statistics provided by PBworks.com • Demographic school level data from the Common Core of Data (National Center for Education Statistics, 2007-2008) 34
- 35. What are our data analytic strategies?
- 36. RQ#1) Are wikis created in non-poor schools used for greater lengths of time than wikis created in poor schools? • Estimate survival functions of wiki groups by Title I status using Kaplan-Meier estimation; use Wilcoxon’s test to test for differences where S(ti) is the estimated survival probability in any of t time periods, which are delineated by instances of wiki death ni is the number of wikis still active at the beginning of time period ti; di is the number of wikis that become inert during time period ti.
- 37. RQ#2) Do wikis created in non-poor schools exhibit more evidence of collaboration and student involvement than wikis created in poor schools? Estimate wiki quality trajectories using the multilevel model for change.
- 38. Findings
- 39. How long do K-12 wikis persist? (n=411) Estimated Seconds Days 1 Lifetimes All PBworks 0.9 25% 250 <1 0.8 50% 123,613 1.4 Survival Probability 0.7 75% 5,282,874 61.1 0.6 K-12 Wikis 25% 2,721 <1 0.5 50% 763,195 8.8 0.4 75% 12,590,074 145.7 0.3 All PBWorks Wikis 0.2 K-12 Wikis 0.1 0 0 20000000 40000000 60000000 80000000 10000000 (231) (463) (694) (926) (1157) Time in seconds (days)
- 40. What subjects are wikis used for? (n=411) English / Language Arts 120 Social Studies 70 Science 61 Computer Science/ Technology 60 Math 45 Library 26 Art 22 Contained Elementary 20 Modern FL 10 Health/PE 8 Business 6 ESL 5 Classics 4 Education 2 0 20 40 60 80 100 120 140
- 41. Does subject area predict persistence? (n=411) 50% 75% Lifetime Lifetime Subject Computer Science 28 198 English/Language Arts 27 198 Science 18 152 Social Studies 6 56 Math 4 33 No Subject 1 8
- 42. What Grade Levels are K-12 wikis used in? (n=411) K-5 109 6--8 118 9--12 180 Higher Ed 8 Unknown 83 0 50 100 150 200
- 43. Does grade level predict persistence? (n=411) • No
- 44. Do wikis from non-poor schools persist longer than wikis from poor schools? (n=259) Kaplan Meier Survival Estimates of Wikis from Poor and Non-Poor Schools Non-poor schools (n=146) Poor schools (n=110) Days 44
- 45. Summary statistics of wiki lifetimes in poor and non-poor schools (n=259) • Day 1 Mortality: – Wikis from poor schools: 40% – Wikis from non-poor schools: ~20% • Median Lifetime: – Wikis from poor schools: 7 days – Wikis from non-poor schools: 58 days • 25% Lifetime: – Wikis from poor schools: 73 days – Wikis from non-poor schools: 259 days 45
- 46. RQ#2) Do wikis created in non-poor schools exhibit more evidence of collaboration and student involvement than wikis created in poor schools?
- 47. Title I eligible Non-Title I eligible (n=110) (n=146) Concatenation 12 24 Copyediting 10 15 Co-construction 7 7 Commenting 16 22 Collaborative Sum= 0 82 (75%) 110 (75%) Collaborative Sum= 1 17 (15%) 13 (9%) Collaborative Sum= 2 7 (6%) 17 (12%) Collaborative Sum= 3 2 (2%) 3 (2%) Collaborative Sum= 4 2 (2%) 3 (2%) 2 Goodness of fit test ( 2=4.2, df=4, p=.38) Student Involvement 28 (25%) 50 (34%) 2 Goodness of fit test ( 2=2.28, df=1, p=.13) Student Involvement and at least 1 16 (15%) 20 (14%) Collaborative Behavior 2 Goodness of fit test ( 2=0.04, df=1, p=.85)
- 48. Discussion • Participatory behavior is rare across all wikis, but both student involvement and collaboration can be found in wikis from both poor and non-poor schools • Wikis from non-poor schools persist longer than wikis from poor schools. • The Open Education Resources strategy of promoting free online tools and resources may, counter-intuitively, expand the second digital divide—in the absence of targeted interventions. 48
- 49. How can we begin to explain these patterns? • What might explain our empirical findings? – What obstacles do poor schools have in using wikis? • Qualitative Research – Interviews with 50+ wiki-using teachers, many drawn at random from our PBworks samples. – 35+ focus groups with students – Classroom observations in 12 schools in MA, CT, ME, NH, CA, GA, VA
- 50. What obstacles exist for wiki use in poor schools? • Differences in resources? • Differences in school culture?
- 51. Differences in networked technology Low Poverty Medium Poverty High Poverty resources among public school districts Districts Districts Districts (>20%) (<10%) (11-20%) Provide teachers with their own server space for posting their own Web pages or class materials (Elementary 90% 81% 74% Secondary) 92% 84% 74% Provide students with electronic storage space 76% 60% 50% on a server 92% 85% 72% Provide students with online access to the 82% 69% 66% library catalogue 92% 82% 72% Provide students online access to databases (for 71% 58% 53% library resources) 79% 67% 57% Employ an individual responsible for education technology leadership (Full Time 60% 48% 47% Part Time 26% 35% 33% None) 13% 17% 20% SOURCE: U.S. Department of Education, National Center for Education Statistics, Fast Response Survey System (FRSS), “Educational Technology in Public School Districts, Fall 2008,” FRSS 93, 2008.
- 52. Fear, anxiety and worry about student exposure to the Internet Barbara: There was - the superintendent said, "Do want you want but I never want us on the frontpage with some bad headline" [chuckle]. I thought those are kind of interesting. That's why he told our technology committee. Our main goal is to not make the frontpage with the bad headline attached. Jim: And as far as I know but I don't want to be the person to be in the newspaper for... Look at what this kid was doing on the internet in a school. I don't want that
- 53. Framework for Intervention: Cycle of Experiment and Experience Experiment Fear - Growth+ Review Plan (Experience) Institutional Capacity+
- 54. The atmosphere at our school in general is to cautiously open up to the possibilities. So, we’re not being pushed to use technology. If we find our way to it, and if we find obstacles that are there and we need things to change to get access to certain things, the administration generally will make that happen. But this, I think in terms of the environment, security is more important than openness in general and sometimes that leads us into a few obstacles…. We haven't gotten to the point where we're making a lot of the stuff public yet. Is that possible down the road? I think with a little bit of experience, with a little bit of, we improve the comfort level then, we can start to say, “Let's make a blog that invites other people throughout the world. Whoever wants to come visit and check it out and be part of the conversation; let's figure how to do that.” I don't think I should... I'm not really in that place that right now. I'm not sure what the response would be if I did. I think we all sort of need to build that comfort level piece first. But I also see that it could be pretty awesome to move in that direction.
- 55. Dear Justin, I am in the library right now and sat down to do some work on the Wikispace to get it ready for next year. However, upon sitting down, I discovered that over the summer this website has been blocked by the City. I spoke to the librarian about appealing blocked content, but he says that they are not unblocking any sites at this time. Consequently, it does not look like we are going to be doing this project this year. I will be trying to put together something else for this unit, but at this point, I don't know what we will be doing and doubt it will be appropriate for your project because it will not be on wikispaces.com. I am very sorry; it was quite a surprise to me.
- 56. Framework for Inequities in wiki usage Wikis created in non- Intervention: suggest that Web 2.0 poor schools persist Promoting a Cycle of tools may than and longer exacerbate Experiment wikis nd digital schools created in poor divide the 2Experience Experiment Fear - Growth+ Non-poor students 21st Century Skills Review Poor students Plan (Experience) Institutional Capacity+
- 57. What Next? 1. Develop an instrument to measure Wiki Quality 2. Correlate wiki quality profiles with teacher attitudes, practices, and resources 3. Develop computational tools to automate those analyses at scale 4. ? 5. Profit
- 58. Acknowledgements • Hewlett Foundation Open Education Resources for grant support • PBworks for data support • Hunter Gehlbach, Stone Wiske, Laura Schifter, Anna Savaadra, and other readers of this paper. • Benjamin Mako Hill for coming up the river to offer his thoughts and critique! 58