SlideShare a Scribd company logo
Patterns and Outcomes
ofYouth Engagement in
Collaborative Information
Seeking withVaried
Resources During Guided
Discovery-Based
Learning.
UT-Austin Guest Lecture
Rebecca Reynolds, Assistant Professor
School of Communication & Information
Library and Information Science
Rutgers University
• Theoretical debates in the learning and information
sciences around learning, structure and agency
• Current landscape of educational technology at this
present historical moment (map technological terrain;
proliferation into K-12, without research evidence).
• Research findings from Globaloria, a game design learning
innovation that includes pilot locations in East Austin, TX
• Intersections of learning sciences, information science
theory in light of findings, and questions invited.
Agenda for Today’s Talk
Socio-technical systems / social informatics perspectives and
technology determinism (understanding reciprocal interrelationships
between humans and machines in both design and application; shaping
both the technical and the social conditions of work, such that efficiency
and humanity do not contradict each other)
Learning sciences, social constructivism, Constructionism,
design-based research (LS developed as a reaction against the
random control trial as the gold standard research methodology in
evaluation and ed policymaking, and the standardized testing paradigm
as the deterministic driver for curriculum development instead of vice
versa)
Both share roots in social-constructivist theory; both foster
more human / community-centered design of innovation
Parallels Between Socio-technical Systems Perspectives and
the Goals of the “Learning Sciences”
To the extent that “constructivist” teaching pedagogies
facilitate exploration and “discovery-based” learning…
Along come Kirschner, Sweller & Clark (2006):
Why Minimal Guidance During Instruction Does Not Work: An Analysis of
the Failure of Constructivist, Discovery, Problem-Based, Experiential, and
Inquiry-Based Teaching (Cited by 2759)
• Staunch objection to constructivist learning on basis of
cognitive load research
• If educators / designers have a learning objective in a
knowledge domain they are trying to meet, asking learners
to search for their own resources and make sense of the
information they find is de-motivating and frustrating;
detracts from learning, rather than contributing.
Setting Up a Straw Man Argument Against Constructivism
in Education . . . . The Issue of Cognitive Load
• Objection criticized for over-simplifying “constructivism”
and minimizing role of intervention design, scaffolding and
guidance present in such interventions (e.g., Hmelo-Silver,
Duncan, Chinn, 2007, cited by 808)
• Rich research evidence for effectiveness of PBL, IPjBL &
IBL…(e.g., scholars such as Hmelo-Silver, Martin, Kapur & Kinzer,
Blumenfeld, Eccles, Kuhlthau, Eisenberg, Chu)
• We also have a history of positive results in Constructionist
contexts regarding student computational thinking,
engagement, affect, meta-cognition outcomes (e.g., Harel &
Papert, 1991; Harel, 1991; Kafai, 1995; Bruckman & Resnick, 1995; Kafai &
Resnick, 1996; Urrea, 2001, 2002; Cavallo, 2004; Kafai & Ching, 2004; Kafai,
2006; Peppler, Kafai & Chiu, 2007; Klopfer, 2008; Reynolds, 2008).
“Constructivist” learning interventions come in many shapes
and sizes. . .
Counter-arguments to Kirschner, Sweller, Clark
What About Learning in the Context of
Information Seeking?
Inquiry-based learning “in the wild” tends to be less-structured, occurring more so
in informal, learner-driven information search contexts without a human guide
(unless one is lucky enough to have a personal librarian!).
While learning during information seeking is often the underlying goal of search
(explicit or implicit), if we are to believe Kirschner, Sweller & Clark,
• Open ended, unstructured exploration and resource use without an expert
guide is not optimal way to learn things
• If someone really wants to learn material, better suited to do so in guided,
mentored, scaffolded, problem-based scenarios designed by experts. . .
(schools/teachers)
• Informal/Formal . . . Extent of structure are factors
Any information scientist will advocate there is an appropriate role for
independent research in the knowledge creation process
 Key Factor in Success: Learner Expertise (Novices vs. Experts)
EducationalTechnologies as Information
Systems. . . Mapping theTerrain
• While information search systems tend to be more generic, archival, multi-use, targeted
towards experts….
• Educational technologies as information environments tend to be more specific to
formal learning settings involving lessons and discrete learning objectives, and users
with lesser extents of expertise/domain knowledge (e.g., youth).
What comes to mind?
 Hardware (Computers/laptops/tablets/mobiles; Smartboards; robotics; other devices)
 Software and web services:
 MS Office and related multimedia productivity tools
 Learning management systems
 Games
 Simulations
 Mobile apps
 Game design platforms (Unity; Flash; HTML5, etc.)
EducationalTechnologies as Information
Systems. . .
 Some software and web services (LMS’s, games, sims) have been designed specifically to
teach a particular knowledge domain. . . The learning objective drives the design; design
is emergent from the learning objective.
 Others are more generic . . . Multi-purpose . . . Educator populates the content [e.g.,
course shells]
 Still others provide “canned” curriculum . . . Publishing company products distributing
existing print material e.g., text book content, in electronic format (not much of a
departure in pedagogy per se; transformation of print to digital content, with multiple
choice tests added on to the end).
Contexts of E-Learning Implementation: Varying… informal (e.g., after-school) vs. formal
(during school time, for credit and a grade); includes line distance education, blended e-
learning, fusion environments in which instructor leads in FTF classes, leveraging online
environments/experiences/content. . . Role of educator varies across.
A few quick examples…
Games designed around key learning
objectives (e.g., GLS games)
• .
Simulations in the school subject
domains. . .
• .
Learning Management System
Course Shells for K-12
Generic Course Management
Generic Course Resource Sharing
Pre-Packaged, Standards-Driven Curriculum Content from Existing Publishers, Delivered
in IncreasinglyComplex Modalities and Online Distribution Channel Models for K-12
Research Paradigms, Challenges
IES: Effects
data on research
effectiveness
WhatWorks:
“Blended learning”
12 results
Clearinghouses: So
useful in theory,
so challenging to
SUSTAIN
Research Paradigms, Challenges
Learning sciences scholarship: Smaller-N studies
• While yielding many necessary innovations, SO FAR most of
these smaller N programs featuring “ideal” features don’t scale.
More generically designed LMS Solutions:
• Scaling, but relatively little research evidence base for
effectiveness
• Schools are diving in, making do, based on sales claims.
The diffusion of K–12 blended learning is far outpacing the
availability of useful research
So MuchVariation. . .
How do we know what is effective?
• Considering Kirschner et al’s critiques  Design innovations for
specific knowledge domains that are refined closely to meet
explicit learning objectives would be higher quality.
• And yet. . . LMS shell systems proliferate, and perhaps… a lowest
common denominator of innovation.
• To what extent do less-guided systems support learning?
• Role of research . . . .
My Research Focus:
Issues of structure and agency
in the context of educational technology design
and learner engagement.
Globaloria has served as a rich and complex test
case in which to investigate these questions. . .
Where does Globaloria fit in the
theoretical / design features landscape?
 Design-based research-driven learning innovation developed by PhD
researchers with many iterations of R & D. “Guided Discovery” . . .
 Learning objectives: Introductory CS principles, digital literacy, 6-CLAs, through
game design for MS and HS students [not in US core curriculum]
 Pragmatic, semi-structured: Many teachers are not yet domain experts in
CS/digital literacy, but the need for teaching children in this NOW is clear. .
.[industry job market; live/livelihood skills Scholarship on CS Education,
computational thinking]
 Globaloria LMS has pre-populated game design tutorials, innovations in
content delivery, scope & sequencing (rather than mere digitization of existing
print textbook materials). Theoretically driven by Constructionism.
 Guide AND autonomy-supportive: Electives; teachers w varying levels of
expertise; does require self-driven learning and discovery but within a closed
system that offers many rich resources in an organized sequence
INTERVENTION: Guided discovery-based game design program
and curriculum offered by the WorldWide Workshop. MS, HS
teachers and students gain experience and expertise in a range of
agentive digital practices.
Domains of Learning and Expertise
• Constructionist digital literacy (skills needed in knowledge economy =>
6-CLAs)
• Computational thinking through game design in Flash and programming in
Actionscript
• Core curricular subject matter:
o When game subjects are linked to core curriculum and students deepen
knowledge about topic through online research and design
• STEM career interests: Technology & Engineering; Computer Science
• Motivation, Affect, Attitudes, Life Choices, New Possibilities and Horizons
E-Learning Environment as Information System
Inquiry, Collaboration, Creation during a
Game Design Course
My research investigates MS and HS student engagement in
collaborative information-seeking behavior, within a pilot game
design learning program involving a wiki-based LMS as a “coordinating
representation” and productive social media platform.
 Design affordances / constraints of the environment
 Student inquiry and collaborative processes
 Learning outcomes: successes, struggles / challenges
• This testbed environment and program is richly and deeply
integrated into the schools in which it is being piloted (students
engage daily, for credit and a grade, for a full year +)
• Findings may be generalizable to wider phenomena of shell LMS
proliferation.... Which are much *less* structured, technologically.
Broad,Top-Level Research Question
Is Globaloria the best way of teaching these domains of
knowledge to middle schoolers and high schoolers?
 Not many existing programs yet. . . Against what can we compare?
 Flexibility/Adaptability of curriculum = Some non-uniformity
 Learn what we can about this implementation, share, compare when
possible
Existing Effects findings
 Globaloria participation increases science, social studies, and reading
WESTEST standardized test scores, compared with matched case non-
participants in West Virginia
 Quasi-experimental research with match-case controls [Chadwick & Gore (2010)
, Chadwick & Gore (2011) , Ho, Gore & Chadwick (2012) , Ho, Gore & Chadwick
(2013)
 Globaloria participation increases student engagement and self-efficacy in
the “6 Contemporary Learning Abilities” (a framework of 6 dimensions of
digital expertise specified in Reynolds & Harel Caperton (2009), Harel
Caperton (2010), and tested in non-experimental pre/post design by
Reynolds (2011, 2013).
 Globaloria attenuates known Digital Divide effects including gender, socio-
economic status, and some race categories given sample (Reynolds & Chiu,
forthcoming, JASIST)
Existing Effects findings
 Conditions among factors that influence the learning, as measured
quantitatively, thus far:
 Intrinsic motivational disposition among students predicts successful
knowledge outcomes in Globaloria (Reynolds & Chiu, 2013, ICLS)
 Self-reported uses of the learning management system features in a
survey (Reynolds & Baik, 2013, ASIST)
Current Study: By What Mechanisms?  Globaloria
Student Learning Processes and Information Uses
DBR research agenda with multiple strands of data collection
underway (survey, observational video cases, screen capture, trace data,
content analysis of student artifacts and communications, LMS features and
natural experiments).
THIS STUDY’S RQs:
(1) What tasks are central to students’ participation in this social
constructivist context?
(2) How are students engaging in inquiry and using resources to address
emergent, in situ questions and solve problems?
(3) Which resources appear to help most?
(4) In what ways might these findings contribute to learning sciences
debates on social constructivism/critiques around structure?
(5) In what ways might these findings contribute to information science
theory-building on guided inquiry, Task, CIS, learning outcomes and
sense-making?
Methods, Dataset 1: (See Slides 46-47 for details)
Google Analytics page read data for a sample of students who
participated in the Globaloria Game Design Program, during the
2012/2013 school year.
NOTE: Google Analytics data available at SCHOOL level only. [Limitations]
Methods, Dataset 2:
During Spring semester 2013 (a week in March and a week in
May), we followed 2 teams per grade in grades 6-8 during site
visits at one school in AUSTIN, TX, collected observational
video data, for a total of 6 team case studies. Filmed,
qualitatively coded, categorized, analyzed, cross-tabbed the
frequencies.
Methods, Dataset 3:
Content analysis of student game quality, ranked by team,
school (inter-coder reliable analysis across 4 dimensions)
Profile
Project
Team
Google Analytics LMS Pages Reported in Findings:
Curriculum (Intro, Wiki Tools, Units 1-4,
Actionscript Tutorials); Social Media Pages (Profile,
Project, Team)
Results, Dataset 1: Descriptive data on resource uses
[in the aggregate – all 21 schools]
Social media pages:
• Students appear to engage more so with their profile pages
than the more productivity-oriented project pages or team
pages.
• Schools varied substantially in their extent of use of these
pages.
Curriculum unit pages:
• Students appear to engage more so with earlier curriculum
topics (Intro, Wiki Tools, Unit 1, Unit 2) than later (Units 3, 4,
Actionscript Tutorials).
• A small number of schools spike in their pvs to latter curricular
topics later in the school year
Standardized PVs for Schools with Higher Avg. Game
Evaluation Scores
Standardized PVs for Schools with Lower Avg. Game
Evaluation Scores
Standardized PVs for Schools with a 0 Game Evaluation
Score (No final games to evaluate)
Results, Dataset 1: Apparent relationships between
resource uses and outcomes
• Schools with lower game design quality averages and
schools creating 0 games appear to visit profile pages more
frequently than project and team pages, whereas,
• Schools with higher game design quality averages appear to
visit the team pages to a greater extent, and also more so as
the program proceeds over time.
• Curriculum PVs: Higher performing schools also appear to
view Unit 2, Unit 3, Unit 4 and Actionscript curriculum more so
than those with lower game evaluation scores, and utilize
latter units in the sequence increasingly as the school year
progresses.
Engaging in teamwork with the wiki and using its resources seem to
contribute to learning. But how are students using these resources? [lots of
hidden phenomena in school-level aggregates]
Results, Dataset 2: Variables of interest
• Task: What was student engaged in?
• Collaborative Information Seeking Modality:
Permutations of collaborative modality and information
source
• Inquiry incident outcomes: Was each incident complete
and resolved, or did it remain in limbo?
Results,
Dataset 2:
Descriptive
Findings, Task, CIS
Modality, Outcome
TABLE 1. Coding scheme and N of incidents for emergent collaborative information seeking (CIS) variables
Task Driving Inquiry CIS Modality: Solo, team, tchr, by resource Outcome
Graphic design and
drawing of digital images
93 Teammates helping each other with their
own expertise
109 Successful solution
found
151
Advanced programming
functions
62 Solo inquiry using wider Internet 38 Task still in progress 103
Basic programming
functions
52 Teacher helping student with their own
expertise
36 No solution found,
problem left in limbo
33
Game subject
development
39 Teammates helping each other with wider
internet
34 Not sure/ researcher
unable to discern
14
Version control and file
mgmt
15 Solo inquiry using wiki LMS 31 No solution found, task
abandoned
8
How to search 12 Teammates helping each other using wiki 25 Problem solution found
and tested but failed
6
Character animation 10 Help seeking from classmates with their
own expertise
14 Problem solution found
but rejected
5
Music 10 Teacher initiates help for student 13
Not sure 10 Teacher helping student using wiki 11
Sharing of formal
feedback
8 Teacher helping student with wider
internet
4
Review of other games 5 Help seeking from classmates using wiki 2
Basic computer functions 4 Classmates helping each other with wider
internet
2
Results, Dataset 2: Cross-tabs, Frequency Distributions
across 2 Variables…
Task X CIS Modality:
• Appears that for easier graphic design tasks, the students
consulted each other and the internet more so, whereas for
more complex programming, students made attempts to
utilize informational wiki resources to a greater extent.
CIS Modality x Inquiry Incident Outcome
• When students relied on their peers’ expertise (most
prevalent), the results had more failed inquiry attempts
than wiki-based LMS resource uses.
Wiki can be a good source, for those who actually can/do use
it…
Nested Cross-tabs: Which Tasks and CIS Modalities
Reflect Higher Unresolved Rates? Those Highlighted
N Row Labels Resolved Unresolved In Progress
186 Graphdesdra
w
57% 10% 32%
2 alonewiki 0% 0% 0%
54 aloneint 59% 7% 33%
78 teamself 56% 10% 33%
6 teamwik 0% 0% 100%
38 teamint 58% 16% 26%
4 nteamself 100% 0% 0%
4 teachself 100% 0% 0%
156 Progbasic 38% 8% 46%
30 alonewiki 10% 0% 60%
69 teamself 52% 9% 39%
21 teamwik 43% 0% 43%
3 teamint 0% 0% 100%
6 nteamself 0% 50% 50%
3 nteamwik 100% 0% 0%
18 teachself 50% 17% 33%
6 teacherinitiat
ed
0% 0% 100%
N Row Labels Resolved Unresolved In Progress
248 Progadv 26% 32% 39%
56 alonewiki 7% 21% 64%
40 teamself 60% 30% 10%
28 teamwik 14% 14% 71%
16 nteamself 0% 75% 25%
4 nteamwik 0% 0% 100%
4 nteamint 0% 100% 0%
44 teachself 36% 27% 36%
40 teachwik 30% 40% 20%
8 teachint 50% 50% 0%
8 teacherinitiat
ed
0% 50% 50%
195 Gamesubdev 67% 10% 23%
35 aloneint 100% 0% 0%
75 teamself 67% 7% 27%
40 teamint 75% 25% 0%
15 teachself 33% 33% 33%
10 teachint 100% 0% 0%
20 teacherinitiat
ed
0% 0% 100%
Results, Dataset 2: Cross-tabs, Frequency Distributions
across 2 Variables…
For the most complex task of Advanced Programming. . .
• Higher proportions of unresolved incidents than for other tasks, across
several CIS modalities
• When you separate out information-seeking by Task, CIS Modality and
Outcomes, we see that these 3 variables appear related, such that in
guided discovery based settings like this:
• For the more complex task of advanced programming, the more likely
students are to seek online information resources in the LMS than for
other tasks
• For the more complex task of advanced programming, the more likely the
inquiry incidents are to be unresolved.
• Role of educators’ expertise needs enhancement. . .
• And/or role of students’ information literacy (thus, more IL instruction) . . .
Implications
Important to note:
• Tasks and game quality variables are local to this Constructionist learning
intervention, but may be applicable to others teaching game design / digital
literacy
• CIS modality categories that emerged (collaboration type, melded with
information resource) may be applicable to many other informational and
educational settings . . .
• Inquiry outcome categories that emerged (successful / failed inquiry
attempts) also may be applicable to many other informational and
educational settings . . .
These are in-progress results!
Currently reviewing literature on Task, CIS and “sense-making” and
considering theoretical linkages.
Broad Summary of Findings
Is this an “effective” intervention?
• Digital literacy and digital divide positive effects (Reynolds & Chiu, forthcoming)
• School standardized tests positive effects (Edvantia research reports)
• Globaloria is autonomy-supportive and appears more conducive for
intrinsically motivated students (Reynolds & Chiu, 2012)
Information resource use research on Globaloria indicates:
To the extent that more advanced game programming is a program learning
objective, then it appears the program needs more structure for this. . .
Teachers, System, Information Literacy
Next Steps
Need to deploy and test improvements in the following areas:
• Teacher expertise (professional development)
• LMS system design refinements in the more advanced game
programming topics
o More research needed on what programming functions pose greatest
challenges, how to design better
• Offer more information literacy instruction
o Student question formulation, resource review and search, synthesis,
problem-solving, application
Future Questions invited
• What “levels of analysis” exert greater effects on learning
outcomes in complex, social constructivist “social learning
systems”?
• Design of information system/curricular features, scope,
sequencing?
• Teacher qualities / pedagogical practices;
• Student qualities / collaborative information seeking
practices (Individual; Team)
• Multi-level analysis modeling with colleague, Dr. Ming Ming Chiu
Rebecca.reynolds@gmail.com
Rutgers University website
http://comminfo.rutgers.edu/directory/rbreynol/index.html
Thanks to IMLS!
Thanks to my partners!
Globaloria.org
Worldwideworkshop.org
Thank you!
Extra Slides from Slidebank….
Methods, Dataset 1, additional information
Google Analytics page read data for a sample of students who
participated in the Globaloria Game Design Program, during the
2012/2013 school year.
• 708 students in grades 6-12
• 31 rural West Virginia public schools participated for a full
school year, in Globaloria.
• Omitting single semester and mixed group schools, and thus
the findings below represent the 21 full year schools only, for
a total of 455 students
• These students created a total of 340 individual hidden object
games, and 93 team games.
• NOTE: Google Analytics data available at SCHOOL level
only.
Methods, Dataset 1 (additional information)
Part 1. Descriptive Page Views
Aggregate page view findings by school, observe variation
Part 2. Relationship of Resource Uses (PVs) to Learning
Outcomes
Comparative analysis by school based on descriptive data on
resource uses across time
• Sort process data by outcomes:
• High, Mid, Low performing groups
• Visually observe patterns
• Build rationale for new hypotheses based on apparent
observable differences
Prevalence of E-Learning in the U.S.
1Evergreen Education Group (2013); 2NCES (2011)
 25 states have state virtual schools operating in 2013-2014.1
 29 states and Washington, DC have statewide full-time online schools operating in 2013-
14 1
 An estimated 1,816,400 enrollments seen in distance-education courses in K-12 school
districts in 2009-2010, almost all of which were online courses. 74% of these
enrollments were in high schools. Online courses with the highest level of enrollment
fall under the categories of credit recovery (62%), dual enrollment (47%), and advanced
placement (29%).2
 This enrollment estimate does not include students attending most full-time online
schools — approximately 200,000 full-time students in 2009-2010. As of 2012-2013, the
number of students has grown to 310,000. 1
 Single and multi-district blended and online programs are the largest and fastest-
growing segment of online and blended learning. 1
 Top reasons school districts make online learning opportunities available to their
students are to offer courses not otherwise available, and provide opportunities for
students to recover course credits. Credit recovery is especially important in urban
environments where 81% of schools indicate this is an issue.2
Blended Learning Models
1. Traditional Schools, But With Online Options
• Online Lab Model: Instruction delivered by online teachers through a digital learning
platform but within a brick-and-mortar lab environment.
• Self-Blend Model: Students take one or multiple online courses outside the school
facility to supplement schooling.
2. Blended Schools
• Rotation Model: Fixed schedule rotating students between online learning and
traditional classroom learning. The face-to-face teacher accountable for both the
online and in-classroom work.
• Flex Model: Most of the instruction is delivered by an online platform with face-to-
face teachers available for on-site support. Teachers provide tutoring sessions and
small group sessions.
Blended Learning Models
3. More Virtual Than Traditional
• Online Driver Model: Students receive all primary instruction online (through an
online platform and from an online teacher) with occasional face-to-face check-ins.
• On/Off-Site Rotations: Students receive most primary instruction online, but also
come on-site on a scheduled part-time basis.
4. Lab Programs
• Full-time programs at a site attached to a school, including on-site teachers or
paraprofessionals, online instructors, subject-area
5. Supplemental Programs:
• Serve students in a blended setting for one to several courses, while they take the
rest of their curriculum in a traditional face-to-face environment.
1
2
3
4
Contemporary Learning Ability
(CLA)
Practices Representing Each CLA & How They Are Articulated/Integrated in Globaloria
Invention, creation and completion
of a digital project stemming from
an original idea
Brainstorming and developing game and simulation ideas and storylines using Web2.0 tools
Writing an original game narrative and a proposal to explain it
Generating creative ideas for designs to express the subject of the game and the user experience
Planning/programming/completing a game demo that illustrates the original game design and
functionality
Developing knowledge of the game's domain or topic through game invention/creation/research
Project planning, project
management, teamwork (e.g., role-
taking,
task delegation),
problem-solving
Coordinating and managing the design/creation/programming of game elements
Managing the project’s execution by creating/organizing a wiki and by sharing project assets and
progress updates
Managing team work by defining and assigning team roles/coordinating tasks/executing roles
Project troubleshooting for self and others
Gaining leadership experience through the project management of all game production elements
Publishing and distribution of
self-created digital media artifacts
to an audience and/or community
of peers
Creating a wiki profile page and project pages
Integrating and publishing text/video/photos/audio/programming code/animations/digital designs
on wiki pages
Posting completed assignments/game design iteration and assets/notes and reflections about
projects to wiki
Developing a blog
Giving and getting feedback
about project through
social interaction,
participation, exchange
Collaborating by using Web2.0 tools such as posting to wikis/blogs/open source help
forums/instant messaging
Exchanging/sharing feedback and resources by posting information/links/source code questions/
answers
Reading and commenting on others’ blogs and wiki pages
Presenting final digital projects for others both virtually in game galleries and in person in live
game demonstrations
5
6 Figure 1. Globaloria learning objectives: Promote development of six contemporary learning abilities (CLAs)
Inquiry, information-seeking,
agentive
use of resources (human and
text/digital content), to support
the artifact’s topic/message,
and design/execution
Searching the Web for answers and help on specific issues related to programming games
Searching and finding resources on MyGLife.org network, website, and wiki
Searching the Web for new Flash design, animation and programming resources
Searching for information in support of the game’s educational subject and storyline
Surfing, experimentation and
play with existing Web
applications and tools
Surfing to MyGLife.org starter kit site and other game sites and playing games online
Keeping track of and bookmarking surfing results that are relevant to projects
Browsing Web2.0 content sites such as YouTube, Flickr, blogs, Google tools
Constructionist, knowledge-building conditions supporting development
of 6-CLAs (Reynolds & Harel Caperton 2009; Reynolds & Hmelo-Silver,
2013)
Learning Management System
as Information System
Learning Management System
as Information System

More Related Content

PDF
Trusted Learning Analytics Research Program
PPTX
Driving technology use through learning design
PDF
Computational Participation: Towards a National Education Policy in Uruguay ...
PPTX
E-Learning in Maths - Research, practical tips and discussion
PDF
The stem innovation equation nsta
PDF
Who are you and makes you special?
PDF
Computing curriculum design workshop
PDF
Nli0447
Trusted Learning Analytics Research Program
Driving technology use through learning design
Computational Participation: Towards a National Education Policy in Uruguay ...
E-Learning in Maths - Research, practical tips and discussion
The stem innovation equation nsta
Who are you and makes you special?
Computing curriculum design workshop
Nli0447

What's hot (19)

PPTX
Strijker & Fisser (2019-06-26) A new curriculum for the netherlands including...
PPTX
Developing tools to facilitate integrated reflection
PPTX
Classroom of the futurev3
PPTX
SCONUL Seven Pillars model
PDF
2016-04-06 research seminar
PPTX
UPDATE: "Australian Curriculum, Technologies - September 2014
PDF
Dsfghf
PPT
Rc ss-137-3 net
PPTX
The Internet in the Classroom Professional Development
PPTX
Digital Game-based Learning for Early Childhood
PPTX
The 21st century digital learner and Integrating 21st Century Skills
PDF
Game-Based-Learning-white-paper
PDF
Scaling Informal Learning and Meaning Making at the Workplace
PPT
Pedagogical Design Tools: Planning for Learning with Purpose
PPTX
Analytic and strategic challenges of serious games
PDF
12 Creative Techniques for every-day Teaching
PPTX
Student Opportunities- Year 7 to 10 WA Curriculum: Digital Technologies
PPT
Avatars & The Young Conference
PPSX
Pedagogical patterns for learning programming by mistakes (presentation) (1)
Strijker & Fisser (2019-06-26) A new curriculum for the netherlands including...
Developing tools to facilitate integrated reflection
Classroom of the futurev3
SCONUL Seven Pillars model
2016-04-06 research seminar
UPDATE: "Australian Curriculum, Technologies - September 2014
Dsfghf
Rc ss-137-3 net
The Internet in the Classroom Professional Development
Digital Game-based Learning for Early Childhood
The 21st century digital learner and Integrating 21st Century Skills
Game-Based-Learning-white-paper
Scaling Informal Learning and Meaning Making at the Workplace
Pedagogical Design Tools: Planning for Learning with Purpose
Analytic and strategic challenges of serious games
12 Creative Techniques for every-day Teaching
Student Opportunities- Year 7 to 10 WA Curriculum: Digital Technologies
Avatars & The Young Conference
Pedagogical patterns for learning programming by mistakes (presentation) (1)
Ad

Viewers also liked (14)

PPTX
Inquiry and Resource Use Strategies that Emerge Among Middle Schoolers in a G...
PPTX
Asist 2013 panelpresentation_social media_final
PPTX
Rebecca Reynolds Research Agenda
PPTX
Challenges in Defining, Designing, and Measuring “Digital Literacy” Developm...
PPTX
ALA 2015 Invited Research Talk: Youth Collaborative Information Practices Dur...
PPT
Supporting integration through incidental learning
PPTX
Collaborative Information Seeking in the Wild: Student-Initiated Solutions a...
PPT
Feedback to learners & discipline
PDF
Secondary%20 handbook
PPT
Educational Objectives
PPTX
Using Learning Analytics Data to Understand Project-Based Learning
PPTX
AERA 2013 Conference Presentation: Digital Divide and Globaloria
PPTX
Aera2013knowledgebuilding final
PDF
Program Review for Integrated Planning
Inquiry and Resource Use Strategies that Emerge Among Middle Schoolers in a G...
Asist 2013 panelpresentation_social media_final
Rebecca Reynolds Research Agenda
Challenges in Defining, Designing, and Measuring “Digital Literacy” Developm...
ALA 2015 Invited Research Talk: Youth Collaborative Information Practices Dur...
Supporting integration through incidental learning
Collaborative Information Seeking in the Wild: Student-Initiated Solutions a...
Feedback to learners & discipline
Secondary%20 handbook
Educational Objectives
Using Learning Analytics Data to Understand Project-Based Learning
AERA 2013 Conference Presentation: Digital Divide and Globaloria
Aera2013knowledgebuilding final
Program Review for Integrated Planning
Ad

Similar to UT-Austin Guest Lecture, ""Patterns and Outcomes of Youth Engagement in Collaborative Information Seeking with Varied Resources During Guided Discovery-Based Learning." (20)

DOCX
Educ 118 lesson5-17 Outline
PPT
Constructivisim
PPTX
Designing eLearning: Art Science or Witchcraft?
PPTX
Educational Priorities for the 21st Century
PDF
Faculty Technology Day 2014 Breakout Session on The History and Future of Edu...
PPTX
Education in the Digital Age
PDF
Future Focused Learning
PPTX
Design thinking in wild learning
PPTX
Meda5400_Week1
PPT
Educational design and innovative pedagogies for open and online teaching and...
PPT
Educational design and innovative pedagogies for open and online teaching and...
PDF
Rethinking Pedagogy for a Digital Age Designing for 21St Century Learning 2nd...
PPTX
Changing Paradigm in Interactive Learning System Design
PPTX
Augmenting the reality of education for the 21st century. - Richard Lewington...
PPTX
Augmenting the reality of education for the 21st century by Richard Lewington
PPTX
Transforming Educational Practice
PDF
Crafting Hackerspaces with Moodle and Mahara: The Potential of Creation based...
PPTX
Presentación eng esmirna
PPT
Empowering Students and Educators with New Media Literacies Necessary to Part...
Educ 118 lesson5-17 Outline
Constructivisim
Designing eLearning: Art Science or Witchcraft?
Educational Priorities for the 21st Century
Faculty Technology Day 2014 Breakout Session on The History and Future of Edu...
Education in the Digital Age
Future Focused Learning
Design thinking in wild learning
Meda5400_Week1
Educational design and innovative pedagogies for open and online teaching and...
Educational design and innovative pedagogies for open and online teaching and...
Rethinking Pedagogy for a Digital Age Designing for 21St Century Learning 2nd...
Changing Paradigm in Interactive Learning System Design
Augmenting the reality of education for the 21st century. - Richard Lewington...
Augmenting the reality of education for the 21st century by Richard Lewington
Transforming Educational Practice
Crafting Hackerspaces with Moodle and Mahara: The Potential of Creation based...
Presentación eng esmirna
Empowering Students and Educators with New Media Literacies Necessary to Part...

UT-Austin Guest Lecture, ""Patterns and Outcomes of Youth Engagement in Collaborative Information Seeking with Varied Resources During Guided Discovery-Based Learning."

  • 1. Patterns and Outcomes ofYouth Engagement in Collaborative Information Seeking withVaried Resources During Guided Discovery-Based Learning. UT-Austin Guest Lecture Rebecca Reynolds, Assistant Professor School of Communication & Information Library and Information Science Rutgers University
  • 2. • Theoretical debates in the learning and information sciences around learning, structure and agency • Current landscape of educational technology at this present historical moment (map technological terrain; proliferation into K-12, without research evidence). • Research findings from Globaloria, a game design learning innovation that includes pilot locations in East Austin, TX • Intersections of learning sciences, information science theory in light of findings, and questions invited. Agenda for Today’s Talk
  • 3. Socio-technical systems / social informatics perspectives and technology determinism (understanding reciprocal interrelationships between humans and machines in both design and application; shaping both the technical and the social conditions of work, such that efficiency and humanity do not contradict each other) Learning sciences, social constructivism, Constructionism, design-based research (LS developed as a reaction against the random control trial as the gold standard research methodology in evaluation and ed policymaking, and the standardized testing paradigm as the deterministic driver for curriculum development instead of vice versa) Both share roots in social-constructivist theory; both foster more human / community-centered design of innovation Parallels Between Socio-technical Systems Perspectives and the Goals of the “Learning Sciences”
  • 4. To the extent that “constructivist” teaching pedagogies facilitate exploration and “discovery-based” learning… Along come Kirschner, Sweller & Clark (2006): Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching (Cited by 2759) • Staunch objection to constructivist learning on basis of cognitive load research • If educators / designers have a learning objective in a knowledge domain they are trying to meet, asking learners to search for their own resources and make sense of the information they find is de-motivating and frustrating; detracts from learning, rather than contributing. Setting Up a Straw Man Argument Against Constructivism in Education . . . . The Issue of Cognitive Load
  • 5. • Objection criticized for over-simplifying “constructivism” and minimizing role of intervention design, scaffolding and guidance present in such interventions (e.g., Hmelo-Silver, Duncan, Chinn, 2007, cited by 808) • Rich research evidence for effectiveness of PBL, IPjBL & IBL…(e.g., scholars such as Hmelo-Silver, Martin, Kapur & Kinzer, Blumenfeld, Eccles, Kuhlthau, Eisenberg, Chu) • We also have a history of positive results in Constructionist contexts regarding student computational thinking, engagement, affect, meta-cognition outcomes (e.g., Harel & Papert, 1991; Harel, 1991; Kafai, 1995; Bruckman & Resnick, 1995; Kafai & Resnick, 1996; Urrea, 2001, 2002; Cavallo, 2004; Kafai & Ching, 2004; Kafai, 2006; Peppler, Kafai & Chiu, 2007; Klopfer, 2008; Reynolds, 2008). “Constructivist” learning interventions come in many shapes and sizes. . . Counter-arguments to Kirschner, Sweller, Clark
  • 6. What About Learning in the Context of Information Seeking? Inquiry-based learning “in the wild” tends to be less-structured, occurring more so in informal, learner-driven information search contexts without a human guide (unless one is lucky enough to have a personal librarian!). While learning during information seeking is often the underlying goal of search (explicit or implicit), if we are to believe Kirschner, Sweller & Clark, • Open ended, unstructured exploration and resource use without an expert guide is not optimal way to learn things • If someone really wants to learn material, better suited to do so in guided, mentored, scaffolded, problem-based scenarios designed by experts. . . (schools/teachers) • Informal/Formal . . . Extent of structure are factors Any information scientist will advocate there is an appropriate role for independent research in the knowledge creation process  Key Factor in Success: Learner Expertise (Novices vs. Experts)
  • 7. EducationalTechnologies as Information Systems. . . Mapping theTerrain • While information search systems tend to be more generic, archival, multi-use, targeted towards experts…. • Educational technologies as information environments tend to be more specific to formal learning settings involving lessons and discrete learning objectives, and users with lesser extents of expertise/domain knowledge (e.g., youth). What comes to mind?  Hardware (Computers/laptops/tablets/mobiles; Smartboards; robotics; other devices)  Software and web services:  MS Office and related multimedia productivity tools  Learning management systems  Games  Simulations  Mobile apps  Game design platforms (Unity; Flash; HTML5, etc.)
  • 8. EducationalTechnologies as Information Systems. . .  Some software and web services (LMS’s, games, sims) have been designed specifically to teach a particular knowledge domain. . . The learning objective drives the design; design is emergent from the learning objective.  Others are more generic . . . Multi-purpose . . . Educator populates the content [e.g., course shells]  Still others provide “canned” curriculum . . . Publishing company products distributing existing print material e.g., text book content, in electronic format (not much of a departure in pedagogy per se; transformation of print to digital content, with multiple choice tests added on to the end). Contexts of E-Learning Implementation: Varying… informal (e.g., after-school) vs. formal (during school time, for credit and a grade); includes line distance education, blended e- learning, fusion environments in which instructor leads in FTF classes, leveraging online environments/experiences/content. . . Role of educator varies across. A few quick examples…
  • 9. Games designed around key learning objectives (e.g., GLS games) • .
  • 10. Simulations in the school subject domains. . . • .
  • 14. Pre-Packaged, Standards-Driven Curriculum Content from Existing Publishers, Delivered in IncreasinglyComplex Modalities and Online Distribution Channel Models for K-12
  • 15. Research Paradigms, Challenges IES: Effects data on research effectiveness WhatWorks: “Blended learning” 12 results Clearinghouses: So useful in theory, so challenging to SUSTAIN
  • 16. Research Paradigms, Challenges Learning sciences scholarship: Smaller-N studies • While yielding many necessary innovations, SO FAR most of these smaller N programs featuring “ideal” features don’t scale. More generically designed LMS Solutions: • Scaling, but relatively little research evidence base for effectiveness • Schools are diving in, making do, based on sales claims. The diffusion of K–12 blended learning is far outpacing the availability of useful research
  • 17. So MuchVariation. . . How do we know what is effective? • Considering Kirschner et al’s critiques  Design innovations for specific knowledge domains that are refined closely to meet explicit learning objectives would be higher quality. • And yet. . . LMS shell systems proliferate, and perhaps… a lowest common denominator of innovation. • To what extent do less-guided systems support learning? • Role of research . . . .
  • 18. My Research Focus: Issues of structure and agency in the context of educational technology design and learner engagement. Globaloria has served as a rich and complex test case in which to investigate these questions. . .
  • 19. Where does Globaloria fit in the theoretical / design features landscape?  Design-based research-driven learning innovation developed by PhD researchers with many iterations of R & D. “Guided Discovery” . . .  Learning objectives: Introductory CS principles, digital literacy, 6-CLAs, through game design for MS and HS students [not in US core curriculum]  Pragmatic, semi-structured: Many teachers are not yet domain experts in CS/digital literacy, but the need for teaching children in this NOW is clear. . .[industry job market; live/livelihood skills Scholarship on CS Education, computational thinking]  Globaloria LMS has pre-populated game design tutorials, innovations in content delivery, scope & sequencing (rather than mere digitization of existing print textbook materials). Theoretically driven by Constructionism.  Guide AND autonomy-supportive: Electives; teachers w varying levels of expertise; does require self-driven learning and discovery but within a closed system that offers many rich resources in an organized sequence
  • 20. INTERVENTION: Guided discovery-based game design program and curriculum offered by the WorldWide Workshop. MS, HS teachers and students gain experience and expertise in a range of agentive digital practices.
  • 21. Domains of Learning and Expertise • Constructionist digital literacy (skills needed in knowledge economy => 6-CLAs) • Computational thinking through game design in Flash and programming in Actionscript • Core curricular subject matter: o When game subjects are linked to core curriculum and students deepen knowledge about topic through online research and design • STEM career interests: Technology & Engineering; Computer Science • Motivation, Affect, Attitudes, Life Choices, New Possibilities and Horizons
  • 22. E-Learning Environment as Information System
  • 23. Inquiry, Collaboration, Creation during a Game Design Course My research investigates MS and HS student engagement in collaborative information-seeking behavior, within a pilot game design learning program involving a wiki-based LMS as a “coordinating representation” and productive social media platform.  Design affordances / constraints of the environment  Student inquiry and collaborative processes  Learning outcomes: successes, struggles / challenges • This testbed environment and program is richly and deeply integrated into the schools in which it is being piloted (students engage daily, for credit and a grade, for a full year +) • Findings may be generalizable to wider phenomena of shell LMS proliferation.... Which are much *less* structured, technologically.
  • 24. Broad,Top-Level Research Question Is Globaloria the best way of teaching these domains of knowledge to middle schoolers and high schoolers?  Not many existing programs yet. . . Against what can we compare?  Flexibility/Adaptability of curriculum = Some non-uniformity  Learn what we can about this implementation, share, compare when possible
  • 25. Existing Effects findings  Globaloria participation increases science, social studies, and reading WESTEST standardized test scores, compared with matched case non- participants in West Virginia  Quasi-experimental research with match-case controls [Chadwick & Gore (2010) , Chadwick & Gore (2011) , Ho, Gore & Chadwick (2012) , Ho, Gore & Chadwick (2013)  Globaloria participation increases student engagement and self-efficacy in the “6 Contemporary Learning Abilities” (a framework of 6 dimensions of digital expertise specified in Reynolds & Harel Caperton (2009), Harel Caperton (2010), and tested in non-experimental pre/post design by Reynolds (2011, 2013).  Globaloria attenuates known Digital Divide effects including gender, socio- economic status, and some race categories given sample (Reynolds & Chiu, forthcoming, JASIST)
  • 26. Existing Effects findings  Conditions among factors that influence the learning, as measured quantitatively, thus far:  Intrinsic motivational disposition among students predicts successful knowledge outcomes in Globaloria (Reynolds & Chiu, 2013, ICLS)  Self-reported uses of the learning management system features in a survey (Reynolds & Baik, 2013, ASIST)
  • 27. Current Study: By What Mechanisms?  Globaloria Student Learning Processes and Information Uses DBR research agenda with multiple strands of data collection underway (survey, observational video cases, screen capture, trace data, content analysis of student artifacts and communications, LMS features and natural experiments). THIS STUDY’S RQs: (1) What tasks are central to students’ participation in this social constructivist context? (2) How are students engaging in inquiry and using resources to address emergent, in situ questions and solve problems? (3) Which resources appear to help most? (4) In what ways might these findings contribute to learning sciences debates on social constructivism/critiques around structure? (5) In what ways might these findings contribute to information science theory-building on guided inquiry, Task, CIS, learning outcomes and sense-making?
  • 28. Methods, Dataset 1: (See Slides 46-47 for details) Google Analytics page read data for a sample of students who participated in the Globaloria Game Design Program, during the 2012/2013 school year. NOTE: Google Analytics data available at SCHOOL level only. [Limitations] Methods, Dataset 2: During Spring semester 2013 (a week in March and a week in May), we followed 2 teams per grade in grades 6-8 during site visits at one school in AUSTIN, TX, collected observational video data, for a total of 6 team case studies. Filmed, qualitatively coded, categorized, analyzed, cross-tabbed the frequencies. Methods, Dataset 3: Content analysis of student game quality, ranked by team, school (inter-coder reliable analysis across 4 dimensions)
  • 29. Profile Project Team Google Analytics LMS Pages Reported in Findings: Curriculum (Intro, Wiki Tools, Units 1-4, Actionscript Tutorials); Social Media Pages (Profile, Project, Team)
  • 30. Results, Dataset 1: Descriptive data on resource uses [in the aggregate – all 21 schools] Social media pages: • Students appear to engage more so with their profile pages than the more productivity-oriented project pages or team pages. • Schools varied substantially in their extent of use of these pages. Curriculum unit pages: • Students appear to engage more so with earlier curriculum topics (Intro, Wiki Tools, Unit 1, Unit 2) than later (Units 3, 4, Actionscript Tutorials). • A small number of schools spike in their pvs to latter curricular topics later in the school year
  • 31. Standardized PVs for Schools with Higher Avg. Game Evaluation Scores
  • 32. Standardized PVs for Schools with Lower Avg. Game Evaluation Scores
  • 33. Standardized PVs for Schools with a 0 Game Evaluation Score (No final games to evaluate)
  • 34. Results, Dataset 1: Apparent relationships between resource uses and outcomes • Schools with lower game design quality averages and schools creating 0 games appear to visit profile pages more frequently than project and team pages, whereas, • Schools with higher game design quality averages appear to visit the team pages to a greater extent, and also more so as the program proceeds over time. • Curriculum PVs: Higher performing schools also appear to view Unit 2, Unit 3, Unit 4 and Actionscript curriculum more so than those with lower game evaluation scores, and utilize latter units in the sequence increasingly as the school year progresses. Engaging in teamwork with the wiki and using its resources seem to contribute to learning. But how are students using these resources? [lots of hidden phenomena in school-level aggregates]
  • 35. Results, Dataset 2: Variables of interest • Task: What was student engaged in? • Collaborative Information Seeking Modality: Permutations of collaborative modality and information source • Inquiry incident outcomes: Was each incident complete and resolved, or did it remain in limbo?
  • 36. Results, Dataset 2: Descriptive Findings, Task, CIS Modality, Outcome TABLE 1. Coding scheme and N of incidents for emergent collaborative information seeking (CIS) variables Task Driving Inquiry CIS Modality: Solo, team, tchr, by resource Outcome Graphic design and drawing of digital images 93 Teammates helping each other with their own expertise 109 Successful solution found 151 Advanced programming functions 62 Solo inquiry using wider Internet 38 Task still in progress 103 Basic programming functions 52 Teacher helping student with their own expertise 36 No solution found, problem left in limbo 33 Game subject development 39 Teammates helping each other with wider internet 34 Not sure/ researcher unable to discern 14 Version control and file mgmt 15 Solo inquiry using wiki LMS 31 No solution found, task abandoned 8 How to search 12 Teammates helping each other using wiki 25 Problem solution found and tested but failed 6 Character animation 10 Help seeking from classmates with their own expertise 14 Problem solution found but rejected 5 Music 10 Teacher initiates help for student 13 Not sure 10 Teacher helping student using wiki 11 Sharing of formal feedback 8 Teacher helping student with wider internet 4 Review of other games 5 Help seeking from classmates using wiki 2 Basic computer functions 4 Classmates helping each other with wider internet 2
  • 37. Results, Dataset 2: Cross-tabs, Frequency Distributions across 2 Variables… Task X CIS Modality: • Appears that for easier graphic design tasks, the students consulted each other and the internet more so, whereas for more complex programming, students made attempts to utilize informational wiki resources to a greater extent. CIS Modality x Inquiry Incident Outcome • When students relied on their peers’ expertise (most prevalent), the results had more failed inquiry attempts than wiki-based LMS resource uses. Wiki can be a good source, for those who actually can/do use it…
  • 38. Nested Cross-tabs: Which Tasks and CIS Modalities Reflect Higher Unresolved Rates? Those Highlighted N Row Labels Resolved Unresolved In Progress 186 Graphdesdra w 57% 10% 32% 2 alonewiki 0% 0% 0% 54 aloneint 59% 7% 33% 78 teamself 56% 10% 33% 6 teamwik 0% 0% 100% 38 teamint 58% 16% 26% 4 nteamself 100% 0% 0% 4 teachself 100% 0% 0% 156 Progbasic 38% 8% 46% 30 alonewiki 10% 0% 60% 69 teamself 52% 9% 39% 21 teamwik 43% 0% 43% 3 teamint 0% 0% 100% 6 nteamself 0% 50% 50% 3 nteamwik 100% 0% 0% 18 teachself 50% 17% 33% 6 teacherinitiat ed 0% 0% 100% N Row Labels Resolved Unresolved In Progress 248 Progadv 26% 32% 39% 56 alonewiki 7% 21% 64% 40 teamself 60% 30% 10% 28 teamwik 14% 14% 71% 16 nteamself 0% 75% 25% 4 nteamwik 0% 0% 100% 4 nteamint 0% 100% 0% 44 teachself 36% 27% 36% 40 teachwik 30% 40% 20% 8 teachint 50% 50% 0% 8 teacherinitiat ed 0% 50% 50% 195 Gamesubdev 67% 10% 23% 35 aloneint 100% 0% 0% 75 teamself 67% 7% 27% 40 teamint 75% 25% 0% 15 teachself 33% 33% 33% 10 teachint 100% 0% 0% 20 teacherinitiat ed 0% 0% 100%
  • 39. Results, Dataset 2: Cross-tabs, Frequency Distributions across 2 Variables… For the most complex task of Advanced Programming. . . • Higher proportions of unresolved incidents than for other tasks, across several CIS modalities • When you separate out information-seeking by Task, CIS Modality and Outcomes, we see that these 3 variables appear related, such that in guided discovery based settings like this: • For the more complex task of advanced programming, the more likely students are to seek online information resources in the LMS than for other tasks • For the more complex task of advanced programming, the more likely the inquiry incidents are to be unresolved. • Role of educators’ expertise needs enhancement. . . • And/or role of students’ information literacy (thus, more IL instruction) . . .
  • 40. Implications Important to note: • Tasks and game quality variables are local to this Constructionist learning intervention, but may be applicable to others teaching game design / digital literacy • CIS modality categories that emerged (collaboration type, melded with information resource) may be applicable to many other informational and educational settings . . . • Inquiry outcome categories that emerged (successful / failed inquiry attempts) also may be applicable to many other informational and educational settings . . . These are in-progress results! Currently reviewing literature on Task, CIS and “sense-making” and considering theoretical linkages.
  • 41. Broad Summary of Findings Is this an “effective” intervention? • Digital literacy and digital divide positive effects (Reynolds & Chiu, forthcoming) • School standardized tests positive effects (Edvantia research reports) • Globaloria is autonomy-supportive and appears more conducive for intrinsically motivated students (Reynolds & Chiu, 2012) Information resource use research on Globaloria indicates: To the extent that more advanced game programming is a program learning objective, then it appears the program needs more structure for this. . . Teachers, System, Information Literacy
  • 42. Next Steps Need to deploy and test improvements in the following areas: • Teacher expertise (professional development) • LMS system design refinements in the more advanced game programming topics o More research needed on what programming functions pose greatest challenges, how to design better • Offer more information literacy instruction o Student question formulation, resource review and search, synthesis, problem-solving, application
  • 43. Future Questions invited • What “levels of analysis” exert greater effects on learning outcomes in complex, social constructivist “social learning systems”? • Design of information system/curricular features, scope, sequencing? • Teacher qualities / pedagogical practices; • Student qualities / collaborative information seeking practices (Individual; Team) • Multi-level analysis modeling with colleague, Dr. Ming Ming Chiu
  • 45. Extra Slides from Slidebank….
  • 46. Methods, Dataset 1, additional information Google Analytics page read data for a sample of students who participated in the Globaloria Game Design Program, during the 2012/2013 school year. • 708 students in grades 6-12 • 31 rural West Virginia public schools participated for a full school year, in Globaloria. • Omitting single semester and mixed group schools, and thus the findings below represent the 21 full year schools only, for a total of 455 students • These students created a total of 340 individual hidden object games, and 93 team games. • NOTE: Google Analytics data available at SCHOOL level only.
  • 47. Methods, Dataset 1 (additional information) Part 1. Descriptive Page Views Aggregate page view findings by school, observe variation Part 2. Relationship of Resource Uses (PVs) to Learning Outcomes Comparative analysis by school based on descriptive data on resource uses across time • Sort process data by outcomes: • High, Mid, Low performing groups • Visually observe patterns • Build rationale for new hypotheses based on apparent observable differences
  • 48. Prevalence of E-Learning in the U.S. 1Evergreen Education Group (2013); 2NCES (2011)  25 states have state virtual schools operating in 2013-2014.1  29 states and Washington, DC have statewide full-time online schools operating in 2013- 14 1  An estimated 1,816,400 enrollments seen in distance-education courses in K-12 school districts in 2009-2010, almost all of which were online courses. 74% of these enrollments were in high schools. Online courses with the highest level of enrollment fall under the categories of credit recovery (62%), dual enrollment (47%), and advanced placement (29%).2  This enrollment estimate does not include students attending most full-time online schools — approximately 200,000 full-time students in 2009-2010. As of 2012-2013, the number of students has grown to 310,000. 1  Single and multi-district blended and online programs are the largest and fastest- growing segment of online and blended learning. 1  Top reasons school districts make online learning opportunities available to their students are to offer courses not otherwise available, and provide opportunities for students to recover course credits. Credit recovery is especially important in urban environments where 81% of schools indicate this is an issue.2
  • 49. Blended Learning Models 1. Traditional Schools, But With Online Options • Online Lab Model: Instruction delivered by online teachers through a digital learning platform but within a brick-and-mortar lab environment. • Self-Blend Model: Students take one or multiple online courses outside the school facility to supplement schooling. 2. Blended Schools • Rotation Model: Fixed schedule rotating students between online learning and traditional classroom learning. The face-to-face teacher accountable for both the online and in-classroom work. • Flex Model: Most of the instruction is delivered by an online platform with face-to- face teachers available for on-site support. Teachers provide tutoring sessions and small group sessions.
  • 50. Blended Learning Models 3. More Virtual Than Traditional • Online Driver Model: Students receive all primary instruction online (through an online platform and from an online teacher) with occasional face-to-face check-ins. • On/Off-Site Rotations: Students receive most primary instruction online, but also come on-site on a scheduled part-time basis. 4. Lab Programs • Full-time programs at a site attached to a school, including on-site teachers or paraprofessionals, online instructors, subject-area 5. Supplemental Programs: • Serve students in a blended setting for one to several courses, while they take the rest of their curriculum in a traditional face-to-face environment.
  • 51. 1 2 3 4 Contemporary Learning Ability (CLA) Practices Representing Each CLA & How They Are Articulated/Integrated in Globaloria Invention, creation and completion of a digital project stemming from an original idea Brainstorming and developing game and simulation ideas and storylines using Web2.0 tools Writing an original game narrative and a proposal to explain it Generating creative ideas for designs to express the subject of the game and the user experience Planning/programming/completing a game demo that illustrates the original game design and functionality Developing knowledge of the game's domain or topic through game invention/creation/research Project planning, project management, teamwork (e.g., role- taking, task delegation), problem-solving Coordinating and managing the design/creation/programming of game elements Managing the project’s execution by creating/organizing a wiki and by sharing project assets and progress updates Managing team work by defining and assigning team roles/coordinating tasks/executing roles Project troubleshooting for self and others Gaining leadership experience through the project management of all game production elements Publishing and distribution of self-created digital media artifacts to an audience and/or community of peers Creating a wiki profile page and project pages Integrating and publishing text/video/photos/audio/programming code/animations/digital designs on wiki pages Posting completed assignments/game design iteration and assets/notes and reflections about projects to wiki Developing a blog Giving and getting feedback about project through social interaction, participation, exchange Collaborating by using Web2.0 tools such as posting to wikis/blogs/open source help forums/instant messaging Exchanging/sharing feedback and resources by posting information/links/source code questions/ answers Reading and commenting on others’ blogs and wiki pages Presenting final digital projects for others both virtually in game galleries and in person in live game demonstrations
  • 52. 5 6 Figure 1. Globaloria learning objectives: Promote development of six contemporary learning abilities (CLAs) Inquiry, information-seeking, agentive use of resources (human and text/digital content), to support the artifact’s topic/message, and design/execution Searching the Web for answers and help on specific issues related to programming games Searching and finding resources on MyGLife.org network, website, and wiki Searching the Web for new Flash design, animation and programming resources Searching for information in support of the game’s educational subject and storyline Surfing, experimentation and play with existing Web applications and tools Surfing to MyGLife.org starter kit site and other game sites and playing games online Keeping track of and bookmarking surfing results that are relevant to projects Browsing Web2.0 content sites such as YouTube, Flickr, blogs, Google tools
  • 53. Constructionist, knowledge-building conditions supporting development of 6-CLAs (Reynolds & Harel Caperton 2009; Reynolds & Hmelo-Silver, 2013)
  • 54. Learning Management System as Information System
  • 55. Learning Management System as Information System