Challenges in Defining, Designing, and Measuring “Digital Literacy” Development in Learners: A Proposed Framework
Download as PPTX, PDF1 like469 views
The document explores the complexities in defining, designing, and measuring digital literacy development in learners, proposing a framework that distinguishes between naturalistic and prescriptive contexts. It highlights the importance of socio-economic factors in accessing digital skills and knowledge, the need for effective instructional design grounded in theoretical frameworks, and the impacts of interventions like Globaloria in enhancing contemporary learning abilities among students. Furthermore, it presents empirical findings that show how engagement in school-based digital activities influences students' digital literacy practices at home.
![Definitions and Operationalizations Vary
• Example 2, scholarship: Barron (2004) defines
technological fluency as learners’ ability “to express
themselves creatively, to reformulate knowledge, and to
synthesize new information. [The concept] entails a process
of lifelong learning in which individuals continually apply what
they know to adapt to change and acquire more knowledge to
be more effective at applying technology to their work and
personal lives.” Operationalization: students’ self -
reported frequency of participation in the following
twelve constructive activities](https://image.slidesharecdn.com/aera2014final-140603144753-phpapp01/85/Challenges-in-Defining-Designing-and-Measuring-Digital-Literacy-Development-in-Learners-A-Proposed-Framework-5-320.jpg)
Challenges in Defining, Designing, and Measuring “Digital Literacy” Development in Learners: A Proposed Framework
- 1. Challenges in Defining, Designing, and Measuring “Digital Literacy” Development in Learners: A Proposed Framework AERA, PHL, April 2014 Rebecca Reynolds, Assistant Professor School of Communication & Information Dept. of Library and Information Science Rutgers University John Wolf, Ph.D., Instructor, NJIT
- 2. “Digital Literacy” in Naturalistic vs. Prescriptive Contexts Naturalistic Phenomena in General Population • Digital Divide, Levels 1 and 2 (Hargittai & DiMaggio, 2002; Barzilai- Nahon, 2006) • Important cultural, social and political activities occur in online environments, and participation in such activities requires digital skills and knowledge (Mossberger, Tolbert, and McNeal, 2007; Jenkins, 2009, Hobbs, 2010; Horrigan, 2011) • More sophisticated forms of content creation, participatory engagement, digital knowledge and cultural benefits have been associated with privilege: higher socio-economic status and level of education (Pew, 2007; Hargittai and Walejko, 2008) • Connects digital literacy to capacity for social mobility (Hargittai, 2010)
- 3. “Digital Literacy” in Naturalistic vs. Prescriptive Contexts Prescriptive Instructional Contexts • Strand 2 of the U.S. FCC’s NBP of 2010 focuses on awareness raising and digital literacy efforts, esp. in schools • NETP of 2010: Optimizing learning and instructional design in US schools with technology-integrated project / problem- based learning interventions. • Prepare learners for 21st century work cultures • Role-taking, epistemic learning • Learning (engage & empower), Assessment (measure what matters), Teaching (prepare & connect), Infrastructure (access & enable), Productivity (redesign & transform) • AASL’s Standards for 21st Century Learner and ISTE National Education Technology Standards for students and teachers: Discuss student-level dispositions and latest versions bring in creativity and productivity
- 4. Definitions and Operationalizations Vary • Example 1, scholarship: Web-use skill (Hargittai,2005): “a user’s ability to locate content on the web effectively and efficiently” with no definition offered in her updated work in 2009; operationalizes digital literacy as a user’s familiarity with terminology reflecting certain technology concepts (JPEG, frames, preference settings, newsgroups, PDF, refresh/ reload, advanced search, weblog, bookmark, bookmarklet, spyware, bcc (on e-mail), blog, tagging, tabbed browsing, RSS, wiki, malware, social bookmarking, pod-casting, phishing, web feeds, firewall, cache, widget, favorites, torrent)
- 5. Definitions and Operationalizations Vary • Example 2, scholarship: Barron (2004) defines technological fluency as learners’ ability “to express themselves creatively, to reformulate knowledge, and to synthesize new information. [The concept] entails a process of lifelong learning in which individuals continually apply what they know to adapt to change and acquire more knowledge to be more effective at applying technology to their work and personal lives.” Operationalization: students’ self - reported frequency of participation in the following twelve constructive activities
- 6. Definitions and Operationalizations Vary Example 3, scholarship: Eshet & Aviram • Skeptical hypothesis 2 centers on technology situatedness of the definition one uses of digital literacy. • The definer must prioritize particular technology contexts of the moment, and any definition reflects the norms and values of the definer in assigning important to given technologies • “Clash of civilizations” ”Old, modern, rationalist, linear, conceptual, book-based culture of Western societies is CLASHING with new emerging post modern, branching, multi- media based reproduction oriented culture that has been developed in the last 20 years largely due to the spread of technology”
- 7. Definitions and Operationalizations Vary Example 1, policy: AASL Standards: “The definition of information literacy has become more complex as resources and technologies have changed. Information literacy has progressed from the simple definition of using reference resources to find information. Multiple literacies, including digital, visual, textual, and technological, have now joined information literacy as crucial skills for this century.”
- 8. Definitions and Operationalizations Vary Example 2, policy: NETS Standards
- 9. Definitions and Operationalizations Vary Example 3, scholarship: Eshet & Aviram • Conservative hypothesis in most studies rests on assumption that digital literacy is a set of distinct skills that we can measure in the population. • Theoretical questions (are skills distinct or are they really something else, like intelligence or motivation?) • Operation-oriented questions (how to measure) • Didactic questions (can they be taught?) • Development and design questions (how are they best taught?) • Skeptical hypothesis 1: No such thing as “digital literacy;” it is orthogonal with individual level capacities: • Motivation / Curiosity, Learning styles, Creativity, Intelligences, Capacities, Personality types • This hypothesis places the individual and his/her expertise at forefront
- 10. Critical Issues • Reynolds (2008): Research and policy definitions hold significant limitations for advancement of research because of the definitional variation, technology determinedness of operationalizations resulting in a need for constant update, and in policy, limited structural, material means for realizing ends specified (e.g., time, curricula, budgets). • Yet, given Digital Divide, and learning opportunities technology poses, the need for intervention clearly remains. • To design, implement and study processes and effects of interventions that inculcate “digital literacy” in learners requires clearly defining learning objectives and outcomes for the intervention, and developing the appropriate pedagogy. • Eshet & Aviram’s skeptical hypotheses advise (a) a focus on individual expertise, and (b) problematization of technology situatedness
- 11. Critical Issues • To take it 1 step further, a learning sciences approach to designing digital literacy intervention requires theoretical grounding of the instructional design principles
- 12. This Study • The definition and framework for digital literacy that follows emerged out of an existing learning innovation called Globaloria that applies social cognitive theory (e.g., Vygotsky) and Constructionism (e.g., Papert & Harel, 1991) in its design. • Reynolds & Harel Caperton 2009 outline the theoretical linkages between G’s instructional design principles and Constructionism • Reynolds & Hmelo-Silver (2013) identify conceptual linkages among Globaloria’s design, Constructionism and the knowledge building literature of Scardamalia & Bereiter. • The definition of “Contemporary Learning Abilities” we propose is student centered, and clearly distinguishes between the abilities and the technology contexts out of which they may emerge.
- 13. 1 2 3 4 Contemporary Learning Ability (CLA) Practices Representing Each CLA & How They Are Articulated/Integrated in Globaloria Invention, creation and completion of a digital project stemming from an original idea Brainstorming and developing game and simulation ideas and storylines using Web2.0 tools Writing an original game narrative and a proposal to explain it Generating creative ideas for designs to express the subject of the game and the user experience Planning/programming/completing a game demo that illustrates the original game design and functionality Developing knowledge of the game's domain or topic through game invention/creation/research Project planning, project management, teamwork (e.g., role- taking, task delegation), problem-solving Coordinating and managing the design/creation/programming of game elements Managing the project’s execution by creating/organizing a wiki and by sharing project assets and progress updates Managing team work by defining and assigning team roles/coordinating tasks/executing roles Project troubleshooting for self and others Gaining leadership experience through the project management of all game production elements Publishing and distribution of self-created digital media artifacts to an audience and/or community of peers Creating a wiki profile page and project pages Integrating and publishing text/video/photos/audio/programming code/animations/digital designs on wiki pages Posting completed assignments/game design iteration and assets/notes and reflections about projects to wiki Developing a blog Giving and getting feedback about project through social interaction, participation, exchange Collaborating by using Web2.0 tools such as posting to wikis/blogs/open source help forums/instant messaging Exchanging/sharing feedback and resources by posting information/links/source code questions/ answers Reading and commenting on others’ blogs and wiki pages Presenting final digital projects for others both virtually in game galleries and in person in live game demonstrations
- 14. 5 6 Figure 1. Globaloria learning objectives: Promote development of six contemporary learning abilities (CLAs) Inquiry, information-seeking, agentive use of resources (human and text/digital content), to support the artifact’s topic/message, and design/execution Searching the Web for answers and help on specific issues related to programming games Searching and finding resources on MyGLife.org network, website, and wiki Searching the Web for new Flash design, animation and programming resources Searching for information in support of the game’s educational subject and storyline Surfing, experimentation and play with existing Web applications and tools Surfing to MyGLife.org starter kit site and other game sites and playing games online Keeping track of and bookmarking surfing results that are relevant to projects Browsing Web2.0 content sites such as YouTube, Flickr, blogs, Google tools
- 15. Constructionist, knowledge-building conditions supporting development of 6-CLAs (Reynolds & Harel Caperton 2009; Reynolds & Hmelo-Silver, 2013)
- 16. This Study Building evidence base for the six dimensions of the CLAs Research Questions and Hypotheses RQ1: What is the empirical relationship among the six proposed dimensions of contemporary learning? RQ2: How does the introduction of Globaloria intervention in school affect frequency of engagement in CLA-based activities and practices in non-school settings? H1: The intervention will have a larger effect on frequency of engagement for the more constructionist dimensions of contemporary learning.
- 17. INTERVENTION: Guided discovery-based game design program and curriculum offered by the WorldWide Workshop. MS, HS teachers and students gain experience and expertise in a range of agentive digital practices.
- 18. Learning Management System as Information System
- 19. Learning Management System as Information System
- 20. Methods Non-experimental pre/post survey design, 2011/2012 school year of Globaloria • Out of 1,356 middle and high school student participants, a total of 1,063 completed the pre-survey (78.4%), and approximately 670 completed the post-survey (49.4%). • Sample includes pre/post only students: 282 middle school and 781 high school students participated. • The pre-intervention sample was 29.0% female and 71.0% male.
- 21. Methods Operationalization of Digital Literacy: • Self-reported frequency of engagement in twenty different Globaloria activities, mapped to the CLAs • Improvements: Definition and operationalization of digital literacy are theoretically grounded in the social cognitivist, Constructionist and knowledge-building inspired instructional design features • That is . . . The chosen technology activities included in our operationalization of digital literacy reflect the theory that inspired the instructional technology context’s design About this, we are explicit.
- 22. Results • Factor analysis, reliability analysis indicate that the given survey items for each factor hang together as 6 factors (note, all CLAs had 3 or more items, except the Publish factor (CLA 3) which had only 1 item). • Bivariate correlational analyses indicate that at school, pre- program results indicated the strongest apparent correlations among the more constructionist dimensions of contemporary learning (CLAs 1-3) • Repeated measures ANOVAs indicate that students increased in their frequency of engagement in all CLA dimensions @School from pre- to post-program except Surfing (CLA 6) • Repeated measures ANOVAs indicate that students increased in their frequency of engagement in CLA dimensions 1,2, 3 and 5 (Create, Manage, Publish, Research) @Home from pre- to post- program
- 23. Regression Model Results • We ran multiple regression models investigating how change in program engagement in all 6 CLA dimensions @School contributed to change in the 6 dimensions @Home. For instance: • Does change in Creation @School contribute to change in Creation @Home? • Does change in Research @School contribute to change in Creation @Home?
- 25. Regression Model Results • Neither Gender nor proxy for SES (parent education) predicted change in engagement @Home (very interesting in light of digital divide research; Globaloria factors may mitigate digital divide factors; Reynolds & Chiu, 2013 in fact bear this out) • Engagement @School led to an increase in engagement @Home, even though no homework.
- 26. Results • Factors predicting change in Create @ Home: • Change in Create @ School • Change in Socialize @ School • Change in Surf @ School • Factors predicting change in Manage @ Home: • Change in Manage @ School • Change in Socialize @ School • Change in Surf @ School • Factors predicting change in Publish @ Home: • Change in Socialize @ School • Change in Surf @ School • Factors predicting change in Socialize @ Home: • Change in Socialize @ School • Change in Surf @ School • Factors predicting change in Research @ Home: • Change in Research @ School • Change in Surf @ School • Factors predicting change in Surf @ Home: • Change in Research @ School • Change in Surf @ School
- 27. Discussion and Implications • Claim that the practices are contributing to each other in this social cognitivist learning experience are supported by the findings • Surfing, experimentation, play, along with social engagement in the context drive transfer of engagement to home (measure of motivation) • For the more effortful, creative work (Creation, Publish), experiencing those practices @School is influencing students to engage at home, too. • Contribution: • Study shows how when Digital Literacy expertise domains, technology contexts and instructional design features are more clearly articulated and identified, it creates opportunities to explicate factors of influence
- 28. Discussion and Implications • This general learning objectives approach of digital literacy conceptualization can support improvements in more pragmatic and executable educational policy recommendations, instructional design, measurement and assessment
- 29. Rebecca.reynolds@gmail.com Rutgers University website http://comminfo.rutgers.edu/directory/rbreynol/index.html Thanks to IMLS! Thanks to my partners! Globaloria.org Worldwideworkshop.org Thank you!