Evaluating digital learnign objects
0 likes101 views
1) Digital learning objects range from simple individual lessons and media to complex professionally developed courseware. It is important for educators to evaluate the level of sophistication, completeness, and rigor of digital resources. 2) There are tens of millions of digital learning objects available both freely and commercially. Evaluating these resources thoroughly requires understanding their design, content coverage, and ability to promote learning. 3) A framework is proposed for evaluating digital learning objects based on their technical design, instructional completeness, and ability to promote rigorous learning. Educators can use this framework at different levels of depth depending on their evaluation needs and time constraints.
![18 19the Learning Counsel: GOING DIGITAL the Learning Counsel: GOING DIGITAL
Rigor Scale: one point per each (1)
1. Citations.
2. Comprehension difficulty level as in reading
(lexile) or mathematics level ( ), if applicable.
3. Fidelity. (The instruct inherent in the learning
object delivers its concept in the way it should
be to achieve learning with the software, or in
a way that is a true assessment if it is a test.)
4. Individual object, for its type, has an appropri-
ate scope – not too broad or too narrow
5. Elicits higher levels of thinking and analyzing
in the learner.
6. Elicits creativity in thinking and responding.
7. Uses an appropriate user interface with proper
navigation.
8. Provides an overall positive user experience.
9. Has appropriate Actions * (one point per
each of audio enhancement, live chat, live
video, annotating, accessibility, social gaming,
spell checkers, spatial-temporal reasoning,
dynamic definitions, probeware viewing,
digital social interactions, sketching, touch
enabled, storied).
10. Has appropriate Aesthetics*(characters,
voice, virtual reality, animations, visual
advantages, video embedding, avatars).
11. Has appropriate Controls*(dynamic
curation, plagiarism checking, interoper
able, project- based, student personalization,
navigation).
12. Has appropriate Individualizations *
(multiple languages, collections, work
product curation.)
13. Has appropriate Instructs * (favoriting,
administrative personalization, second
screen, projection, portability, feedback,
programming practice, inference, clip
embedding, standards alignment/attainment).
14. Has appropriate Mechanisms * (machine
learning, intervention, training, project
mastery, distance live-lab, authorware,
terminology, interactive queries, real-time
attention data/neuro-determinism, device
input, intelligent learning engines, gesture
controlled data, collaboration-ware,
assembly, practice micro-games, social
experimentation, formative assessment,
pre-assessment, coding, game-based earning,
gaming rewards, manipulative-object
interplay, summative assessment, artificial
intelligence, recall mechanized [selection or
matching from memory], dimensional).
15. Uses appropriate linearity, as in the presenta-
tion of one concept or one question or one
answer for any one True/False, fill-out field,
or buttons.
16. Uses appropriate aggregation tools, as in
the way shopping-cart technology allows
learners to collect and build artifacts into a
portfolio that demonstrates understanding,
if appropriate.
17. Sets and meets objectives.
* References the “71 Characteristics of Digital
Curriculum” Special Report, available online at
LearningCounsel.com
This new Digital Curriculum Evaluation Matrix
is the start of how to look with a bit more sophis-
tication in mind using our eyes and ears and
sense of learning engagement—at digital learn-
ing objects. Within Knowstory, the ability to
rate objects as a broad community for “Fit,”
“Worked,” and “Easy,” has the promise of creating
a coordinated national, and possibly interna-
tional, sorting of the good from the unworkable
learning objects, the poorly crafted from the
finely wrought, the biased from the unbiased.
Next – an Orientation to tactics in going fully digital.
To speed along without major problems, here are questions to find your own answers that will help you and
your schools work out quality digital curriculum transitions.
1. What is happening as learning objects become consumerized – built expressly for the
individual? What will this mean for professionally developed digital courseware?
2 How many digital learning objects you have? Are teachers aware of what other
teachers have?
3. What is your overall digital transition strategy to provide for digital devices that use
curriculum that takes a lot of bandwidth, like video streaming? (Consider the devices, student
access when not at school, what’s allowed on the networks, filtering, and security and privacy.)
4. Will you be using a Learning Management System? One, none, or many?
5. What can your networks handle? What if every student brings multiple devices and then
crowds into three rooms instead of twelve for some project work – can those three rooms
handle the load?
6. What sorts of curriculum things could be services you can have other organizations do
including technology support? Language pathology help via distance teaching? Foreign language
courses? Homework hot-lines?
7. What sort of curriculum “mix” do you want by grade? All discrete, granular objects, or more
systematized? Why? What are the repercussions of that?
8. What is the pedagogical use of any one tactic – concentrated in whole group teaching, project-based
learning, individualized learning, or…?
8. Are the tactics extensible to other teachers in that subject or grade?
9. Are the tactics subject-specific as a learning object, or is it a “Tool App” that is a cross-subject asset
for learning?
10. Are you carefully avoiding just “niftiness” (digital things that are nifty but may have little rigor) to focus
on substantive digital learning object tactics?
Orientation Tactics
“With digital learning, the network has never been more important
to improving student outcomes. Would an Indy racer perform
better on a race track or a dirt road? You need a race track for your
students.” — Richard Nedwich, Director of Education, Ruckus Wireless](https://image.slidesharecdn.com/20200825-evaluatingdigitallearnignobjects-200825155314/85/Evaluating-digital-learnign-objects-7-320.jpg)
Evaluating digital learnign objects
- 1. 6 7the Learning Counsel: GOING DIGITAL the Learning Counsel: GOING DIGITAL Evaluating Digital Curriculum as We Speed Beyond Strategy and Into Tactics by LeiLani Cauthen Discernment between curriculum materials has always been the job of educators. Today it is important to discern whether digital learning objects are granular or systematized. Where are they on a scale of curated free document-type lesson plans and home-built learning objects via build-curriculum-yourself customization software (authorware) versus professionally coded courseware? Does either one have all the elements of rigor? Here we will delve deeper into how to think about digital resources and how leaders and teachers alike can create a method to evaluate all their options. See earlier Special Report at learningcounsel.com for a complete review of digital curriculum characteristics and definitions of terms. Analytics Screen ManipulativesInterfacing Machine Learning Dynam ic Curation Anim ations Gam ing Chunking Avatars Second-Screen Gam ing Mastery The 71 Characteristics of Curriculum Research & Context on the Shift to Digital Curriculum SPECIAL REPORT Digital
- 2. 8 9the Learning Counsel: GOING DIGITAL the Learning Counsel: GOING DIGITAL Digital learning objects are simply off-the-charts in numbers. The hunting around, and then inspection process, is a nationwide undertaking being played out in every school and by every teacher across the land. The Digital Scene The power and depth of digital curriculum and its ability to reach and engage students continues to evolve. There are tens of millions of digital learning objects including massive numbers of items that are in the free and open education resources sites. There have always been seemingly tens of thousands of books and options, but now in the digital arena, the number of companies seems to have massively expanded. Now the trend is to build in virtual reality languages. The hunting around for digital learning objects, and the inspection process, is a nationwide undertaking being played out in every school and by every teacher across the land. There is hope in the construction of Knowstory.com to solve many of these current issues. It will provide a place to create a personal inventory against requirements, standards, lesson-time-used, and more. It will also make it possible to share these inventories with others who might want to copy parts of it to use in their own teaching or learning. All this will help bring order to the great sifting and refining of all these digital things. Educational leaders want to differentiate various digital learning objects. They also want to have some stability and workability in their digital resources. A new problem of digital continuity of operation is at hand since files and lesson plan storage are not the same as handing a substitute or new teacher a textbook that has already been in use. The complexity of operation for digital stewardship is an important new consideration of leaders. Digital Object Sophistication What is essential for educators to think with is the degree to which an object is digitally fashioned. How digitally sophisticated is it? As content started to go digital, the initial learning objects were static bits of knowledge. Examples of such content include links to sites and excerpts from books, loose videos out on Youtube, and unsophisticated databases of text-based information. As network speed and capacity improved, these objects evolved to be more sophisticated, such as full courseware with embedded video and virtual reality. Many of the free Open Educational Resources (OER) came out of greater academic works that were “atomized” into little bits that could be mixed and matched up with lessons as needed. Think about ripping up a whole textbook into chapters and the chapters into the individual concepts inside each one and you get a picture of how many objects live now on the open Internet. Initially, a lot of the work of curating these “chunks” of knowledge into a meaningful scope and sequence for learning occurred manually by educators for consumption by students. They must be meta-tagged and filed and the path-to-finding- them well traveled. Even after all that work is done they may still be disassociated from larger contexts of learning, like how textbooks were planned works with defined scope and sequence. Highly Designed Digital Courseware This scale shows a progression of low-value technical objects up to the more sophisticated objects in terms of the complex- ity of their programming. At the height of the scale is software that is known as “courseware” and requires very little in the way of instructional planning, because it is the plan all-inclusive within a gaming type environment of scope and sequenced learn- ing. A “single-loop animation” is a function like those in PowerPoint or Prezi where any user can do a single-loop action layered on any object to allow it to “swoop in” or “appear” in a timed interval after the slide is already up. A “collection” is often multiple eBooks or videos or objects with a website or App that serves them all to learners by assignment or by user choice. Feedback is something software does to give users the ability to input answers or receive answers on text, pictures, or video, which is a usual function inside learning management systems. Coding is something done to use a multiplicity of computer code mechanisms to craft a whole learning environment or a simple app to do a function – but by a coder. The range of characteristics in between this and the highest-level of digital object, designed digital courseware, is a wide arena of coding characteristics defined in our “71 Characteristics” Special Report.
- 3. 10 11the Learning Counsel: GOING DIGITAL the Learning Counsel: GOING DIGITAL Chunks & Lesson Planning vs. Courseware Compared to courseware organized into a sequen- tial progression of chapter-type elements with quizzes at the end, all the various chunks of video and text are non-sequential little bits of knowl- edge that require the work of contextualizing from teachers to make them into something meaning- ful. That curation and sequencing is called digital lesson planning, and while the objects them- selves may be copyrighted or open-licensed, in many instances the lesson plan framework around them is automatically the intellectual copyright of the School or District, and not the teacher’s. Although this is rarely known by most teachers. There are still arguments that could be had about whether the act of creating a framework around other licensed works is itself a “work” to be copyrighted or simply a compilation effort, like collecting movies of a genre together in a refer- ence library and critiquing and comparing them one against another in a blog about the whole genre. The ideological difference between this sort of library-plus-blog and a lesson plan framework is minor, and the Learning Counsel predicts the future copyright issues around what text or fram- ing was uniquely created versus simply curated will be numerous, because the mixing-and-matching is a growing new area of art for digital teaching. Since many times teachers never used the entire textbook for a subject in the past, the chunk- ing idea is great in some respects. It allows a certain freedom, but it is more work than buying the equivalent of a textbook, or the new digital textbook that has been re-mod- ernized into the “full courseware” idea. One courseware provider, Dreambox, has this to say on their website about courseware: Courseware has programming aspects that deliver the most appropriate type of object in sequence for learning, the same things teachers do that is critical for good digital lessons, and adding the work of truly individualizing that object so that it’s the right one for that specific student user. Courseware is simply log-in-and-go-through-the-lessons, pre-built and structured in some instances to offer “levels” much like games do, so that faster or slower students follow routes unique to them, or are looped back into multiple avenues of remedial learning. Courseware “Our innovative technology captures every interaction a student makes while working within and between lessons. As it dynamically adapts and individualizes instruction in real time, it provides millions of learning paths tailored to each student’s unique needs. It is a ground-breaking, student-driven learning environment that leverages gaming fundamentals to inspire and empower students to build 21st century thinking and problem solving skills, master key concepts, increase achievement, and boost long-lasting confidence in learning.” — Dreambox Learning (http://www.dreambox.com/company) Choose with a Consumer Attitude The ability to distinguish if a digital learning object is “flat” (for example, a text document that does not do anything—the lowest level of object) from an entire program of courses that are animated and contain internal logic, requires a different mindset. In fact, the consideration would need to be more than an academic attitude – it would require a consumer attitude and a designer view about the user interface and user experience. Such an attitude would guarantee a higher level of user appreciation and therefore have a heavy impact on learning. But beware! Just because it’s glamorous does not mean it teaches. Early software was more glitz than substance. Consumers are drawn to appeals promised by software that are usually very simple. A game like Candy Crush allows a player to do exactly that, crush digital candy. It is also only doing that. Courseware is changing this over-simplification of appeal with a consumer appeal to accomplishment, which is an implied appeal in game Apps but usually not emphasized, because it is only a game, so getting a high score is not an actual achievement level in life like finishing a course or graduating. The more sophisticated courseware is a game-world of options and sometimes allows teachers to assign or “lever” learners into certain routes within it. Whether an object is just one concept or a collection of concepts incorporated into courseware is an issue for educators to come to grips with by actually looking inside and experiencing all the routes of courseware. The same is not as true for direct learning courseware consumers who have typically had experiences with e-commerce and gaming worlds online, which have already raised their level of expectation for learning objects. They expect the software to navigate them. Educators typically do not expect this, but the K12 market is now smaller than the open consumer market for learning. Therefore, we see publishers creating awesome digital courseware and massive collections of digital objects that have suggested meta-tags for student reading levels and more with the navigation consumers expect. When schools use courseware, it is an adjustment for the teacher who must learn two things: 1. How to use it. The full breadth of the courseware and how it is structuring learning, where it allows the teacher to interject and what sorts of analytics it provides, and, 2. How to supplement it. How the use of the courseware will replace old pedagogy and allow the teacher to use it as core while the teacher now flips the learning and focuses on supplemental activities, projects, and some lecturing. This is one of the real revolutions in digital. However, it is no different than the days of textbooks. Textbooks had to be considered for how to use and how to supplement, but in the early days of digital publishing, the learning objects were the dead opposite – supplemental and not core. Teachers already know how to do this well. What’s also interesting about courseware is that the consumer/learner can see it fit itself to them as they use it. This is the same way that our intelligent assistants like Siri or Cortana on our phones work. It is also the true definition of “personalization” in the software world. For consumers and high numbers of screen-learning students (in-class activity or out of a school), curriculum software products need to be highly- systematized, and courses that contain preset programs that can be personalized will be expected. An everyone-is-the-same experience and a whole lot of odd navigational quirks that cause large amounts of user questions will fail, Digital Courseware – TenMarks, an Amazon Company Discrete Digital Objects – Overdrive, New York State Education Deptartment & others
- 4. 12 13the Learning Counsel: GOING DIGITAL the Learning Counsel: GOING DIGITAL as some publishers experienced in their early forays into online sales and customer support. Again, when learning objects are only discrete concepts, they present more work for teachers to curate and frame. However, and this is one of the more interesting things to consider, when students themselves curate, discrete learning objects that are not systematized represent a rebirth of discovery as part of the learning process. Allowing them to do that is the tricky part if the school’s system is built to only allow teacher discovery, manipulation, and assignment of learning objects, and the student is still a passive receiver-only of the pre-determined set of objects. Optimum Digital Teachers tend to do better with optimally aggregated digital learning objects, full courseware, for at least 60% of their materials. These should be used for a maximum of 40% of lesson time used in screen learning with the rest being active discussion, lab, and other activities, projects, and the like. Full courseware or collections that function autonomously give teachers back more of their much- needed classroom time to individualize learning and give their attention to students. The promise of the emerging industry of digital courseware, both for- profit and free, can return teachers to greater time spent on student interaction and the “human side” to balance the screen learning. Evaluating Digital Learning Objects Evaluating any digital learning object is complex because, like textbooks, it can’t be done based on just its title or marketing claims. It must be done with each of the concepts inside the soft- ware, and sometimes there are a large number in any one learning object. Unfortunately, being a “software guru” and knowing what you are looking at in terms of how well something is designed for interface and rigor, is not some- thing that any research or ratings firm other than the Learning Counsel has a handle on. Merriam-Webster’s defines the noun “concept” as the “idea of what something is or how it works.” When that definition is used for evalua- tion, the digital learning object is either a single concept or can be partitioned into multiple con- cepts. Evaluators are then analyzing objects in the same way that developing software is done – by each concept being coded individually. To do this you must look at and validate each part of the digital learning object by experiencing it. The Super-Hurried Evaluation Evaluating digital learning objects by concept is arguably the best way to evaluate what the worth of a digital object is, because it can be broken down into three things: 1. Technical and design sophistication By looking at the application, there is clearly a single focus on each web page with a practice question for fill out that is clear to see, centered, without any distraction, and it is offering feedback of how much time is ticking off per each practice question. It could perhaps use more stylizing with a digital character guide appealing to different age sets, like Mickey Mouse greeting software guests as they enter some Disney Interactive lessons. 2. Completeness of knowledge coverage (actual instruction) The software is dynamically giving practice but does not appear to give lesson context for what Past Tense means as a lesson – it is therefore most probably something to classify as supplemental and not core curriculum. 3. The combination of the first two in the digital realm that results in the ability to cause learning, a.k.a. “rigor.” The software appears rigorous from the definition of being right in each practice question and the design is functional and not distractive. So, in summary, this one concept of all the IXL Learning lesson concepts appears to be 1) technical design appropriate, 2) supplemental instructional (practice), and 3) rigorous – at least upon cursory review of that one lesson within the total number of lessons in their website collection. People are already generally using these simple 1-2-3 steps of concept evaluation when looking at digital learning objects. Design + instruction + rigorousness are the major points and comprise the “super hurried” evaluation. Scoring an object across these three most import- ant things in detail requires a sort of “matrix” to evaluate, and that is what we are presenting to you here in this article. Regular Hurry - Overall Evaluation Using Level 1 of the Matrix If you’re in a regular hurry, you can use the comprehensive new Digital Curriculum Evaluation Matrix to do a rapid evaluation using just the “Overall” points. You would only get into using the entire matrix if it’s a major adoption. Teachers or committees of curriculum experts can make rapid analysis just on the Overall points. Schools can figure out their own specific number score for digital learning objects for overall value using the Overall section of the Matrix. Highly sophisticated programs can take a short amount of time to evaluate by looking at each characteristic of the software and experiencing it from a user point of view. Using the list of just the items in “Level 1 – Overall” anyone can make a quick mental calculation of those things for that object. These are the listed below in numbers 1 – 7. This ideology of how you would take a quick-look or go deeply into inspecting any piece of software for what it is and does as a learning object is a work-in-progress. Courseware 60% Discrete Digital Objects 40% Screen Learning 40% Projects | Activities 60% Example: Concept: Identifying Past Tense in Language Arts – 1st Grade. (Example from IXL Learning, see: https://www.ixl.com/ela/grade-1/ form-and-use-the-regular-past-tense)
- 5. 14 15the Learning Counsel: GOING DIGITAL the Learning Counsel: GOING DIGITAL Level 1 - Quality of the Knowledge Object At the simplest levels, the most critical things for any evaluation are the things that have always been critical for instructional design, but now with a digital twist. Note, we suggest a scale of 1-10 for each of the areas and all of the 71 Characteristics. The highest score, then, would be 78. 1. Quality knowledge concepts or artifacts (a fancy word for things, like pictures, video, and primary sources that have been scanned in – still usually a picture). Watch it, this one is more subjective and opinionated than the others. 1-10 Value per concept. It would be wise to number how many concepts there actually are in the learning object. Having to evaluate all of them if there are hundreds or even thousands, so a random sampling is usual. 2. Taking one or more of the concepts, individually, next in importance in instructional design is a reasoned scope that takes advantage of the things that are of high quality when the content is digital: visualizations, animations, scaffolding of information, content levering, pre- and post-use formative assessments, audio, and all other enhancements (see the “71 Characteristics,” which is soon to be updated to slightly more in number) that are useful for the chosen subject and topic. 1-71 Value. 3. Next, in-sequence delivery of that concept through the digital mechanisms that support mastery of what is being taught. Once mastery occurs, the learner can then be exposed to a multiplicity of in-sequence con- cepts building on the original topic. These extensions of the original topic can go in various directions in ways that are themselves instructional because they are non-linear and serve to create more learning. 1-10 Value. Digital Curriculum Evaluation Matrix Level 3—Academic RigorLevel 1—Overall Level 2—Digital Learning Object Concept Contextualized Courses, eBooks Applet, Apps Full Grade Course Courseware Collection Aggregated>DisaggregatedAggregated>Disaggregated • Citations • Comprehension Level • Fidelity • Scope for Type • Thinking & Analyzing • Creativity • User Interface • User Experience • Actions • Aesthetics • Controls • Individualizations • Instructs • Mechanisms • Linearity • Aggregating • Objectives • Quality • Scope • In-Sequence • Fit • Privacy/Security • Cross-indexing • Lesson-Time-Used Digitized Discrete Objects Granular/Atomized Single Concepts up to context/ framework aggregation with authorware around those, and further up to single courses and eBooks. Digital Professional Applets or apps including Short Form Games up to Long Form Games; Full Grade Courses; Courseware & Collections —with or without Intelligent Learning Engines, virtual reality, Augmented reality, etc. Regular Hurry—Overall Evaluation Least Hurried—Serious Full Evaluation Super-Hurried Evaluation 4. Fit (alignment) with requirements and standards. 1-10 Value. 5. Privacy and security of student data. Note that this is an entire subject of its own and best left to the experts for right now. The Learning Counsel will be releasing important national data and analysis points on this that can be asked by non-techies in a pinch in the near future. In the meantime, we suggest you ask of any software what data it takes that might potentially compromise student privacy and see what the publisher’s terms of service include. 1-10 Value. 6. Beyond those, and what makes digital delivery unique, is a cross-indexing of the concept with other concepts in a greater ecosystem of knowledge. This might be as low as simple meta-tagging or correct filing. If the object doesn’t have this quality, it will be hard to find again. 1-10 Value. 7. Lesson Time Used (LTU). Total LTU is a single numerical value per year. You can figure out LTU by first asking approximately how much time it takes to learn or do one learning concept in any learning object, or the average time of that concept if the object has many different features and functions. This is most like one chapter in a textbook or one lesson or section or quiz or one book (a book being usually themed around one major concept.) Apps and courseware might break down into different concepts as lessons. These are typically individual concepts attempting to be taught. What’s important about this is that it is very hard for teachers to assign a digital object to be completed by a learner if they have no idea the time involved to complete it. One concept might take 15 minutes for a learner, or more. Evaluating digital learning objects by concept is arguably the best way to evaluate what the worth of a digital object is, because it can be broken down by design, instruction and rigor.
- 6. 16 17the Learning Counsel: GOING DIGITAL the Learning Counsel: GOING DIGITAL 2. Digital Professional – Developed by designers and programmers with testing for the educa- tion market. Again, this doesn’t necessarily mean it’s totally awesome or even more rigorous, just that it is professionally built by someone who is a code-writer. Within those two categories are the granular, dis- aggregated objects at the top of the scale and the more aggregated objects at the bottom. Schools may need to decide to target some ratio in their mix of all objects of professional versus non-pro- fessional to lessen the expense and inefficiencies of custom creation and upkeep. This is especially true if the discrete learning objects are found via links for their use in lesson plans and the links break. They may also want to have a care as to how much is purely granular versus systematized and aggregated objects. Potentially, paid professional industry work can more efficiently spread the cost of creation and upkeep across many schools and keep accurate track of what is working. At any level, each object can contain various digital characteristics. “Scoring” in this area is Non- Professional versus Professional (developer) and the quality and quantity of added characteristics in the software. In addition, how granular versus aggregated is another area of scoring. The question of what a school or teacher scores an object is a question of what percentage of their time they want on building all home-grown materials versus professionally built. Schools may have signed pledges to be fully free and open education resources so their score model would be weighted in favor of more granular because that is more typical to the free resource world. It may also be more typical to the older grades where discovery of what object could be a part of the learning design. As a comment on the all free open education resources, it is important to note that originally the train of thought for focusing attention on free and open resources was very pointed at money savings. That was when schools couldn’t conceive of buying all their computing devices and high-value digital courseware. That is just as it should be, of course, but presently the market has shifted, and more schools are realizing they can achieve full or nearly full coverage models of high-value digital curriculum with a mix of their own materials or outside courses if they stop buying paper books and shift over. The Learning Counsel’s 2016 Survey showed that fully 80% of schools have not yet reorganized budgets away from paper purchase into digital purchase. Other considerations on the technology side of what is it? include, the deeply technical ideas that would require the ability to inspect inside the actual program code for syntax, data structure, control structure, and how well it’s all documented, organized, and labeled. Whether any digital learning object “works” and is “easy or difficult” in terms of user experience should be evaluation points of the technical sophistication of the user interface/user experience where it intersects with the academic rigor of the concept. Level 3 - Academic Rigor The following list can be used with a point system to evaluate the rigor of digital learning objects. It would be used in combination with the other two levels of the scale already noted, and is adaptable for teacher or school needs. It combines the concept of academic rigor with the digital qualities that are instrumental in delivering that rigor. It should be used against every individual concept or minor aggregation within a digital object, as in one section of courseware teaching something like addition and subtraction of fractions. Educational leaders want to differentiate various digital learning objects. They also want to have some stability and workability in their digital resources. The basic thing to ask is “What number of concepts are covered?” The second basic thing to ask is “How much time do the concepts take on average?” Then multiply those two numbers to get the total. Later on, you may want to find out how much time is actually being used of total LTU by any one user for courseware that has been purchased, because you are not sure it is getting enough use to warrant the spend. Least Hurried — Serious Full Evaluation Using Levels 2 & 3 of the Matrix A full evaluation looks at all of Level 1 after using Level 2 and 3. So you start with “What is it?” which is Level 2, then move through each part of Level 3, grading as you go, and end with an overall evaluation of Level 1. Level 2 – Object Scale This is the “what is it?” level. You must understand the quality of the digital object you are looking at. This is not “quality” as in being better necessarily, but in terms of its having a quality that can be defined. The two major quality categories are: 1. Digitized by a non-Developer– meaning usually teacher-made with support from the school or district. This could be build-curric- ulum-yourself software (authorware) that uses pre-built parts of development to make a lesson or a course and be pretty cool, but the object is not professionally coded by some developer. LTU CALCULATION Concepts x 10 Minutes Average = 500 minutes or (divided by 60 minutes in an hour) = 8.33 Hours Examples of Lesson Time Used Example 1: A digital collection of books with 3,000 total books has been purchased. Each book typically is one major concept, particularly for fiction. Non-fiction might have multiple concepts, such as a textbook. In this example, we’re using one (1) as the number of concepts per book and we are going to say that one book takes ten and a half hours, on average, to read. 3000 (total number of books) x 10.5 (average time) = 31,500. This means the total Lesson Time Used (LTU) is 31,500 hours. A digital book collection is like a whole library, so this LTU calculation should not be a surprise. Example 2: An App has been purchased which covers math for both addition and subtraction, two major concepts. To learn the concept of addition in the App takes 2 hours of in-App screen learning. To learn the concept of subtraction takes 3 hours of in-App screen learning. With one concept (addition) taking 2 hours, and the other (subtraction) one taking 3 hours, the total is 5 hours of lessons in that App that teaches two concepts. The total Lesson Time Used (LTU) is 5 hours.
- 7. 18 19the Learning Counsel: GOING DIGITAL the Learning Counsel: GOING DIGITAL Rigor Scale: one point per each (1) 1. Citations. 2. Comprehension difficulty level as in reading (lexile) or mathematics level ( ), if applicable. 3. Fidelity. (The instruct inherent in the learning object delivers its concept in the way it should be to achieve learning with the software, or in a way that is a true assessment if it is a test.) 4. Individual object, for its type, has an appropri- ate scope – not too broad or too narrow 5. Elicits higher levels of thinking and analyzing in the learner. 6. Elicits creativity in thinking and responding. 7. Uses an appropriate user interface with proper navigation. 8. Provides an overall positive user experience. 9. Has appropriate Actions * (one point per each of audio enhancement, live chat, live video, annotating, accessibility, social gaming, spell checkers, spatial-temporal reasoning, dynamic definitions, probeware viewing, digital social interactions, sketching, touch enabled, storied). 10. Has appropriate Aesthetics*(characters, voice, virtual reality, animations, visual advantages, video embedding, avatars). 11. Has appropriate Controls*(dynamic curation, plagiarism checking, interoper able, project- based, student personalization, navigation). 12. Has appropriate Individualizations * (multiple languages, collections, work product curation.) 13. Has appropriate Instructs * (favoriting, administrative personalization, second screen, projection, portability, feedback, programming practice, inference, clip embedding, standards alignment/attainment). 14. Has appropriate Mechanisms * (machine learning, intervention, training, project mastery, distance live-lab, authorware, terminology, interactive queries, real-time attention data/neuro-determinism, device input, intelligent learning engines, gesture controlled data, collaboration-ware, assembly, practice micro-games, social experimentation, formative assessment, pre-assessment, coding, game-based earning, gaming rewards, manipulative-object interplay, summative assessment, artificial intelligence, recall mechanized [selection or matching from memory], dimensional). 15. Uses appropriate linearity, as in the presenta- tion of one concept or one question or one answer for any one True/False, fill-out field, or buttons. 16. Uses appropriate aggregation tools, as in the way shopping-cart technology allows learners to collect and build artifacts into a portfolio that demonstrates understanding, if appropriate. 17. Sets and meets objectives. * References the “71 Characteristics of Digital Curriculum” Special Report, available online at LearningCounsel.com This new Digital Curriculum Evaluation Matrix is the start of how to look with a bit more sophis- tication in mind using our eyes and ears and sense of learning engagement—at digital learn- ing objects. Within Knowstory, the ability to rate objects as a broad community for “Fit,” “Worked,” and “Easy,” has the promise of creating a coordinated national, and possibly interna- tional, sorting of the good from the unworkable learning objects, the poorly crafted from the finely wrought, the biased from the unbiased. Next – an Orientation to tactics in going fully digital. To speed along without major problems, here are questions to find your own answers that will help you and your schools work out quality digital curriculum transitions. 1. What is happening as learning objects become consumerized – built expressly for the individual? What will this mean for professionally developed digital courseware? 2 How many digital learning objects you have? Are teachers aware of what other teachers have? 3. What is your overall digital transition strategy to provide for digital devices that use curriculum that takes a lot of bandwidth, like video streaming? (Consider the devices, student access when not at school, what’s allowed on the networks, filtering, and security and privacy.) 4. Will you be using a Learning Management System? One, none, or many? 5. What can your networks handle? What if every student brings multiple devices and then crowds into three rooms instead of twelve for some project work – can those three rooms handle the load? 6. What sorts of curriculum things could be services you can have other organizations do including technology support? Language pathology help via distance teaching? Foreign language courses? Homework hot-lines? 7. What sort of curriculum “mix” do you want by grade? All discrete, granular objects, or more systematized? Why? What are the repercussions of that? 8. What is the pedagogical use of any one tactic – concentrated in whole group teaching, project-based learning, individualized learning, or…? 8. Are the tactics extensible to other teachers in that subject or grade? 9. Are the tactics subject-specific as a learning object, or is it a “Tool App” that is a cross-subject asset for learning? 10. Are you carefully avoiding just “niftiness” (digital things that are nifty but may have little rigor) to focus on substantive digital learning object tactics? Orientation Tactics “With digital learning, the network has never been more important to improving student outcomes. Would an Indy racer perform better on a race track or a dirt road? You need a race track for your students.” — Richard Nedwich, Director of Education, Ruckus Wireless
- 8. 20 the Learning Counsel: GOING DIGITAL Definitions Key terms as we speed into digital curriculum tactics Concepts are single discrete knowledge objects, and could be called just “content” or “resource,” but are more accurately a concept. These are artifacts like photos or videos. Also called “granular” and “atomized” bits of knowledge. Examples include Word documents, PowerPoints, notes, short videos, pictures, etc. Contextualized concepts are lectures, stories, articles, lesson plans, or assessments/tests and are usually meta-tagged. These could be elements made with authorware of various types including animated objects. May have minor goals or objectives. A part of this category are systems that are contextualizers, or Learning Management Systems (LMS) that frame and hold such objects in repositories. Some LMS’s contain analytics on use of objects and connected grading, but are not by definition creating object internal intelligence adaptations – the objects themselves are not coded with algorithms. However, they could give single-sign-on into independent professionally coded adaptive learning courseware. Courses and eBooks contain multiple concepts. This category includes non-coded courses made inside LMS’s and eBooks, which can use non-programmers to author them. Such objects are often bundled with directions for activities. These have short or mid-range goals or objectives to achieve, and include defined scope and usually sequence. Apps are whole programs that perform small, short-form (Applet) operations, or large operations like long-form learning games. Courses that are professional coded are composed of a set of lessons and goals to achieve that takes longer than a single 50-minute class period. Professional courses have a defined scope and a workable sequence towards learning goals. Some courses include intelligent learning engines so they can be adaptive to individual learners and provide analytics back to users and instructors. Courseware is like courses except it may have a much wider scope with multiple subjects and topics. Once logged in, the student uses the navigation to go from lesson to lesson without further interjection by an instructor. However, the courseware may give some levers to the instructor to gate or ungate progress (as in hitting a digital lever that allows freedom to surge forward to new lessons), and at times provides analytics to the students and teacher. Collection(s) are typically many eBooks, lectures, courses, Apps, or videos that have a management framework around them. This may include other functionality like testing for the entrance level of the learner. Note: This list of terms does not include “Tool Apps” which are usually single-function like note-taking, or authorware, like Adobe Illustrator, or technical software, like computer-aided-design programs (CAD), none of which have knowledge concepts already embedded. OPEN A FREE ACCOUNT TODAY. Visit KnowStory.com CALL US TO JOIN THE PREMIUM INVENSTORY BETA GROUP. 888-611-7709x27 A social way to build the story of your knowledge inventory. Full Beta Release January 2017 nce upon a time, your district or school had an imaginary invenstory of all your digital learning objects. Oh, the wonders of being able to cost analyze your curriculum map! All the teachers of the land purchased, found or made their own digital learning objects. They were 100% perfect at curation, filing and tracking. Each teacher had every subject covered completely with digital invenstory and kept it orderly in browser bookmarks and a learning management system. No teachers ever left, and new ones came fully ready for anything. Sigh. Still, they sometimes dreamed of sharing their invenstory with other teachers in far off places on the internet. Then came Knowstory! A place where teachers and schools come together for a magical global view of that institution and all associated individual’s objects. Together at last. The End o P.S. Plus a lot of sharing with others not in your school, and eventually a marketplace and more and more function to become the most super-awesome thing ever for education. You’ve got to be in on this.