The Development and Usability Evaluation of a Standards-Based Grading Tool for Faculty
Download as PPTX, PDF2 likes1,131 views
The document summarizes a study that developed and evaluated a standards-based grading tool called RealGrade for faculty at Sultan Qaboos University in Oman. The tool aimed to streamline grading practices and facilitate standards-based assessment. Researchers conducted interviews and focus groups to determine faculty needs, then developed RealGrade to automate grading, enable qualitative assessment, and generate reports. 116 faculty tested the tool and completed a questionnaire finding RealGrade effective, efficient, and satisfactory overall, with more computer-experienced and less experienced instructors viewing it most positively. Interviews provided additional insights. Researchers concluded RealGrade supports standards-based assessment but report cards need improvement and more research is needed to address instructor differences.
The Development and Usability Evaluation of a Standards-Based Grading Tool for Faculty
- 1. The Development and Usability Evaluation of a Standards-Based Grading Tool for FacultyAlaa Sadik, Sultan Qaboos University (OMAN)
- 2. IntroductionGrades are the common method for reporting student performance across universities.Today, universities tend to use standards-based assessment approach to assess student performance.
- 3. IntroductionSultan Qaboos University(SQU):a cross-cultural organization, brings together hundreds of instructors from around the world.the largest academic community, consists of 9 colleges,awide variance with regard to grading practices.
- 4. Problem of the StudyMany instructors believe that:Preparing, scoring, grading, and reporting student academic performance are each extremely exhausting and time-consuming tasks.It is difficult to perform standards-based assessment according to the university quality assurance policy and requirements.
- 5. The NeedTherefore,the need was emerged to: Use/develop a standards-based assessment tool that allows for more accurate and relevant grading and reporting.
- 6. Research QuestionsHow effective (useful) is the assessment tool in documenting, grading and reporting student performance?How efficient (easy to use) is the assessment tool as perceived by users?What is the overall satisfaction of users toward the use of the assessment tool?What individual difference variables influence instructors’ perceived usability of the assessment tool?
- 7. Purpose of the StudyThe main purpose of this study was to design, develop and evaluate a standards-based assessment tool for instructors at SQU.
- 8. MethodDevelopment of the assessment tool A review of the literature A series of interviews and focus groups were conducted, to determine the gap between the existing grading practices of faculty and those that are needed for the assessment tool.
- 9. MethodThe assessment tool shouldSupport the University grading scale and generate students’ grades automatically; Enable qualitative assessment of student performance;Attach content and performance standards to grade sheets; Have a built-in e-mail function for communicating grades with students;Facilitate total point and percent weight of scores and assignments; andGenerate course statistics for distributions, correlations and variances.
- 10. Method
- 11. Method
- 12. Method
- 13. Method
- 14. MethodSample116 faculty members from 4 colleges at SQU.
- 15. MethodProceduresParticipants were asked to perform a series of specific tasks in RealGrade, each of which had several subtasks.
- 16. MethodInstrumentsGrading tool usability questionnaireTo solicit participants’ perceptions toward the effectiveness, efficiency and acceptance of the tool. Instructor interviewsTo gain a thorough understanding of the use of RealGrade and to provide rich detail and insights into instructors’ experiences.
- 17. Results: QuestionnaireEffectivenessParticipants strongly agreed or agreed (m=3.93) that RealGrade : Is effective in facilitating the process of standards-based assessment, Has the tools needed to assess student performance and communicate grades with students, and Increases their productivity.
- 18. Results: QuestionnaireEfficiencyParticipants indicated that (m=3.34): Learning to use RealGrade takes a short time, The user interface menus and dialogue boxes are favorable, Weighting, assessing, and managing student assignments based on performance standards are very easy tasks.
- 19. Results: QuestionnaireSatisfactionThe majority of participants (m=4.19) reported that they liked and felt comfortable with RealGrade as a tool to assess student performance.
- 21. Results: Questionnaire Participants with greater degrees of computer experience had higher perceptions of effectiveness, efficiency, and satisfaction with RealGrade. Participants who had less than five years of teaching experience found RealGrade more effective, efficient and satisfactory than those who had more than five or ten years of experience.
- 23. Discussion and ConclusionsDifferences among instructors are vital to the eventual acceptance and implementation of the standards-based assessment tool.Further research is needed to develop report card to help instructors and students understand the standards-based assessment information included and to make it more comprehensible.
- 24. Thank YouMore informationwww.alaasadik.net/realgradeContact alaasadik@gmail.com