The evaluation test_description[1]
- 1. The Evaluation Test EDTECH 505 Who likes tests? Right – almost no one. I’ve found that graduate students in particular (even if they are teachers who gives tests frequently) get quite nervous about actually *taking* tests. It's normal for anyone to be concerned about not knowing how to answer questions, receiving poor grades, etc. All this results in a relatively high level of test anxiety (although this has a certain usefulness despite the discomfort it causes). For the “evaluation test,” for EDTECH 505, my goal is not to trick you or to see how many minute details from the text you can recall. I am not particularly concerned, either, with your ability to correctly identify dozens of definitions of glossary words. The assessment for your reading of the course text, The ABCs of Evaluation (Boulmetis & Dutwin, 2005), could take many forms – ranging from multiple choice items, to short-answer questions, to essay-length questions. While each of these types of assessments has its place in the learning process, I have developed a more compelling way to assess your knowledge. If you have taken a class with me previously, you will recognize the format. This is what I call a “visual metaphor” exam. Let me explain. By “visual metaphor,” I mean that you have to take certain concepts (identified by me) and find, for each concept, some kind of visual representation of it (in this case, I mean a still picture or even something you create). To be a true metaphor, the picture is not an exact representation of the topic, but instead a representation of it that reminds you of the concept. For example, I recently gave a presentation and I wanted to talk about trustworthiness of the study on a slide. Rather than fill the slide with text (blah!), I used a picture of a wild bird eating out of a person’s hand (I love Flickr.com!!!) The pictures, though, are not the only part of the test. There will be some actual words that you have to write. These words are limited, though. I ask for the limited text for two reasons: 1) I do not want to read essays and few, if any, want to write them, and 2) (more importantly) a student who writes a compact, precise statement in her or his own words is much more likely to remember it. The Nitty Gritty Your visual exam has 33 precisely slides. These are listed in tabular format (below). Each one is based on a things you have read or defined over the past few weeks. On each “text” slide, you will have three areas of text: 1. Across the top will be the slide title (these are provided on the table below). 2. In the middle is your ~ 40 word (36-44 is the acceptable range) statement that explains the concept. 3. At the bottom (in a slightly smaller font) is the URL that points to the picture you’ve used OR the words “Created by author” if it is a picture from your own collection. NO text will appear on the visual slide unless the text is somehow part of the graphic (as it might be if you draw an evaluation model), and no pictures or distracting graphics should appear on the text slides.
- 2. Submission Logistics You can create your visual exam in any way you would like, but I will only evaluate it by using GoogleDocs. Whatever you create, then, must be something I can see by ONLY using GoogleDocs. If you simply upload a presentation, which causes me to have to open PowerPoint or some other program, then you will fail the exam on a technicality (which counts heavily toward any ‘do-over’). If you are unsure how to use the presentation tool in GoogleDocs, then there’s no better time to learn it! Actually, I am assuming you already know how to use one presentation software or another, and the skills readily transfer. If you are still lost, then I encourage you to use either A) the Help documentation, B) Tutorials on YouTube or on another site, or C) Your peer’s knowledge. Notice that I did not say, “Contact me,” as I do not give tutorials on productivity software or websites (Why not? Because I figure that our graduate students, who are studying educational technology, should be able to figure it out on their own. The encouragement of these kind of skills is part of my ‘hidden curriculum”). Some people create a presentation directly from within the GoogleDocs’s presentation tool, while others use PowerPoint or the OpenOffice version of it, and then upload it to GoogleDocs. If you upload, you have to allow GoogleDocs to convert the presentation into GoogleDoc format. If you choose this option, be careful. If you make something too fancy on your laptop/desktop, as sometimes there is a disconnect between advanced features and the web-based version. After creating the GoogleDoc presentation, you must put it in the EDTECH 505 folder that you created at the beginning of the course and shared with me. That way, I can simply go to your folder and see the presentation there. You do NOT need to “publish” the presentation so that it can be viewed by anyone on the web. Simply put it in the shared folder, and I’ll see it there. There is NOTHING to upload via Moodle regarding this test. The presentation will be in the folder that already bears your name, so the file name does not need your own. Instead, it can simply be “505 Reading Test.”
- 3. Slide Content Table Slides do not have to be numbered via text that you insert. Slide # Slide Title Text Content 1 Title slide Student name, Semester, and Class Section (a picture can be used here if you’d like – either of yourself or one that is particularly interesting/attractive ). 2 Reflection Slide You can write enough text to fill this slide if you would like, but I want you to respond to this prompt, “If you were to to consider the most important things you've learned about evaluation over the past few weeks, what would those lessons be? Why do you find these aspects to be important? Importantly, how are they related to you job (or future job) as an educational technologist?” There is no wrong or right answer here, but you should write a thoughtful response that has some substance to it. Do not simply repost a blog entry – though you are free to use thoughts from your previous entries as inspiration. Single sentences or incoherent thoughts will have points deducted. PLEASE do not use a font face any smaller than “12”. Merci. 3 (picture slide) A visual representation of (AVRO) the idea, “Why we should evaluate” 4 Rationale for Evaluation Your answer to the question “Why we should evaluate.” 5 (picture slide) AVRO the relationship of evaluation to educational technology 6 Evaluation & EdTech Your explanation of the relationship between evaluation and educational technology. 7 (picture slide) AVRO the difference between research and evaluation 8 Research & Evaluation Your explanation of the difference between evaluation and research. 9 (picture slide) AVRO of efficiency, efficacy, and impact (or just one of these is fine) 10 Efficiency, Efficacy, & Impact Your explanation of the difference between efficiency, efficacy, and impact. 11 (picture slide) AVRO of the impact of evaluation on decision making 12 Impact of Evaluation Your explanation of the impact evaluation has on decision making 13 (picture slide) AVRO one evaluation model (here, you should create a visual representation of the model). 14 (Model #1 name) Your explanation of the purpose of the first selected model 15 (picture slide) AVRO a second evaluation model (here, you should create a
- 4. visual representation of the model). 16 (Model #2 name) Your explanation of the purpose of the second selected model 17 (picture slide) AVRO quantitative vs. qualitative data 18 Quantitative vs. Qualitative Your explanation of the difference between qualitative and quantitative data 19 (picture slide) AVRO nominal, ordinal, interval & ratio data (so, four pictures). 20 Levels of Data Your explanation of the differences between these types of data (you are allowed to use up to 36 - 60 words on this slide only). 21 (picture slide) AVRO four different kinds of data collection instruments (again, four pictures) 22 Data Instruments Your identification of the four instruments you placed on slide #21. (this will be very simple). 23 (picture slide) AVRO formative and summative evaluations (two pictures) 24 Formative vs. Summative Your explanation of the difference between formative and summative evaluation. 25 (picture slide) AVRO population vs. sample 26 Samples and Populations Your explanation of the difference between a population and a sample. 27 (picture slide) AVRO validity vs. reliability 28 Validity and Reliability Your explanation of the difference between validity and reliability 29 (picture slide) AVRO independent and dependent variables 30 Independent & Dependent Your explanation of the difference between independent and dependent variables 31 (picture slide) AVRO criterion-referenced vs. norm-referenced tests 32 Criterion- vs. Norm- Your explanation of the difference between criterion- referenced referenced and norm-referenced tests 33 Activity Evaluation Answer this question: Please provide your feedback on this particular kind of “test.” What is your opinion about the use of visual metaphor to reinforce your learning?
- 5. Grading Rubric CATEGORY OUTSTANDING SATISFACTORY BELOW (9.3-10) (8 – 9.2) EXPECTATION (< 8) Submission of test In GoogleDocs (10) Not in GoogleDocs (0) Slides in correct order Yes (10) No (0) Reflection slide Solid writing; great Good writing, sufficient Contains either poor depth; obvious depth. Does a decent writing, surface-level connection to EdTech. job at reflection but reflection, lack of No more than one slide. perhaps lacks a quality connections, incorrect No “wasted words” (ex., of reflectiveness or formatting, etc. “I think, I believe, I feel, connection that would etc.) . Font is be “outstanding.” Font is appropriate size. appropriate size. Picture Slides Overall incredible Good pictures or Fuzzy pictures, pictures pictures or graphics that graphics … the depth of or graphics that lack make me say, WOW! metaphor may be meaning, overall The visual metaphor is somehow lacking OR it pictures that are just interesting. Not seems to be completely silly or random, etc. necessarily obvious, but unrelated (tantamount connected. to a “random” picture”) Text slides The title appears at the Title and photo credit The slide title is missing top. The 40 word (+/- 4 appear on all slides. The or incorrect; the text words) explanation is 40 word (+/- 4 words) description of the idea succinct and correct. explanation is good – or concept is incorrect The photo credit perhaps something is or missing; the photo appears at the bottom incorrect or phrased credit it missing on a oddly. number of slides (-1 for each slide where the credit is missing) Instructor's discretion Up to 5 points overall 55 points total possible
- 6. Frequent Questions 1. Can I use more than one picture on my picture slide? a. Yes, but not more than four. 2. I would rather just write an essay. Can I? a. No. 3. I would rather just take a 100 question multiple choice test. Can I? a. If you design it, vet it, and provide the answer key, I’ll think about it. 4. Can my pictures simply be silly things that I find wherever and put into the middle of slide. I will be sure that they are lossy and are of generally poor quality and hard to see. a. NO!! The picture(s) you select for each idea should fill the slide. They should be of good quality and should fill the slide itself (you do not need the highest resolution photo to do this, necessarily). You should try to find pictures that have Creative Commons licensing. 5. If I upload something incorrectly or if you cannot open it, do I automatically get a zero? a. While I'd be sorely tempted to do so, you will not get a zero. However, you will not be eligible for full credit once things get corrected. If I find a submission error, I will let you know, and you'll have 5 minutes to fix. Ok, longer than that.. but not as long as a week. REFERENCES Boulmetis, J., & Dutwin, P. (2005). The ABCs of evaluation (2nd Ed.). San Francisco, CA: Jossey-Bass