Session2.10 Mirabaha

Editor's Notes

• #2: Jenn teaches students biology both in an online and a hybrid format; Chris teaches students mathematics (both developmental level and college algebra). Each has been teaching in the online environment for several years. Chris is certified as a Quality Matters Reviewer, and Jenn expects to complete her training and receive her certification for Quality Matters this coming spring. CCBC has been revising its evaluation process for faculty members for the past two years. During this process, we discovered that there was no formal observation form or procedure for supervisors to use in order to formally observe online faculty members in their “classroom” environment. Since there are an increasing number of online classes being offered, supervisors need to be able to observe their faculty members in the online classroom environment as part of their comprehensive evaluation for contract purposes, as well as to ensure that the online courses are providing the same high quality educational experiences that exist in the face-to-face environment. Another consideration was the fact that several supervisors have neither taught in an online environment, nor have they taken any online courses. So, they are unfamiliar with that learning environment and its demands. As supervisors, we recognize the need for a form that will help to guide supervisors as they evaluate online faculty members in their course environment. In this presentation, we are providing our discoveries, considerations, and a rough draft of a possible form for documenting a formal observation of an online faculty member within their course environment.
• #3: An increasing number of students have schedules that make attending traditional classes on-site difficult to do. Those students have turned to online courses for “just in time” content delivery so they have the flexibility to do “attend class” at any time, day or evening, seven days per week. Originally, students would take one or two courses online. Now, students can complete most, if not all, of their coursework online. A survey by Harris (2006) found that students are now more informed and comfortable with the online instructional environment., and that students expect more frequent and timely communication with their instructors. In today’s society, people communicate through email, text messaging and synchronous chat sessions on computers, cell phones, pagers, etc. So, they are adept and accustomed to instantaneous feedback. Also, use a different language to communicate through those mediums. So, most students are comfortable using these tools used in their online classes.
• #4: Traditional assessment techniques generally include examinations that are administered in the classroom. Online courses, too, might require proctored examinations, especially within the realm of mathematics and science. However, students taking online classes do so because they cannot (or do not want to) take exams on a specific day and/or time. A growing number of online students expect their instructors to provide online evaluation instruments (exams) that they can access and complete from a remote (off-campus) location at a time that is convenient to their schedule. Although several online instructors have not incorporated remote online testing into their course structure, several others have embraced online testing, with the blessing of their institutions. Students expect their instructors to provide immediate feedback. That is, online students expect replies to their emails almost immediately (within hours rather than the next day). They expect that synchronous chat sessions can go on indefinitely (no time limitations). In short, they expect their instructor to be available for them 24 hours per day, seven days per week. Such expectations are unrealistic; however, instructors need to set limitations and adhere to the restrictions they set forth to the students. Otherwise, the faculty member could “burn out” very quickly. Written communication skills appear to be adversely affected, as evidenced by email, discussion board and chat room communications. Students are using abbreviations and acronyms to communicate within the text of their postings. Increasingly, there is no evidence of an attempt to correct grammatical or spelling errors within such communications.
• #5: With more and more students taking online courses in various formats (strictly online, hybrid, etc.), higher education institutions are having to offer increasing numbers of courses to meet the demand. Therefore, colleges must apply the same standards to their online offerings with regard to instruction as their on-site counterparts. Upholding the integrity of the course is paramount for any college that is hoping to maintain a good reputation and a strong presence within the higher education community. Online course offerings must be well-organized, structured, and contain the same degree of rigor as their on-site counterparts. The quality of instruction depends largely on the quality of the instructor. Online classes require faculty members to provide students with all of the instruction and support afforded to on-site students. But, they must go beyond the traditional classroom expectations in order to provide the same high quality instruction in the online class as is already present in the on-site class. Therefore, online instructors must be dedicated and successful classroom instructors who are willing to put a lot of time and effort into each online class they teach. A set of well-defined expectations for both the students and the instructors are needed in order to meet these goals.
• #6: Quality assurance of on-site classes is achieved through periodic course and instructor evaluations. Students are evaluated through exams, papers, projects, speeches, etc. On-site instructors are evaluated periodically through classroom observations (in addition to grade monitoring and student evaluations of their instructors). Recognizing the importance of these classroom visits, The Community College of Baltimore County (CCBC) requires that all of its teaching faculty members be observed in the classroom in accordance with a mandated schedule based on longevity and type of teaching contract: CCBC observes faculty on one year contracts once per year Faculty on three year contracts are observed in the second year of the contract Faculty on five year contracts (or those with tenure) are evaluated in the fourth year of the contract or every four years Adjunct faculty are evaluated in the first semester of teaching. Additional observations are made once per academic year. To date, CCBC has no defined standard method in place for evaluating online instruction, even though a full-time faculty member may teach up to 60% of their semester teaching load in the online environment. Supervisors need to be able to evaluate the teaching effectiveness of online instructors in order to ensure that high quality instruction is maintained.
• #7: There are several rubrics that have been designed to evaluate the structure of the online class environment. One high-quality rubric is called the “Rubric for Online Instruction.” It was developed by The Committee for Online Instruction at California State University at Chico in 2002, and implemented at the college during the Spring 2003 semester. They based their rubric on three resources: “WebCT Exemplary Course Rubric” by Maisie Caines and David Graf Bloom’s Taxonomy “7 Good Teaching Practices in Undergraduate Education” by Arthur W. Chickering and Zelda F. Gamson Source: http://www.csuchico.edu/celt/roi/index.shtml
• #8: Another well-designed rubric is “Quality Matters: Inter-Institutional Quality Assurance in Online Instruction.” This rubric was developed through a three-year FIPSE grant, awarded to MarylandOnline, Inc., that was awarded in 2003, but has been self-supporting through since 2006. It provides a vehicle for certifying online courses, and their various components. It is a nationally recognized program, which provides both for-fee evaluation services and evaluator training programs. Source: http://www.qualitymatters.org/
• #9: The Rubric for Instruction and Quality Matters are both designed to evaluate the course structure, but they do not provide a tool for supervisors to evaluate an instructor’s “classroom” performance. At CCBC, supervisors are able to access online courses for the purpose of evaluating instructors if they request and are granted access to the course by the online instructor. However, there is no official observation policy nor is there an official observation form available at the present time. In fact, currently there is no practice in place whereby online course instructors are officially observed within that environment. As supervisors who also teach online courses, we believe that there should be a mechanism in place that provides for online instructor observations. Wanting to make such observation meaningful yet in line with on-site classroom observations and evaluations, we considered several types of evaluatory tools. Self-assessment of the course would be helpful since knowing how the faculty member perceives their performance in the academic environment affects teacher effectiveness. Also, if the instructor and the supervisor are using the same instrument to evaluate the instruction, a comparison and a productive discourse can occur. The supervisor would be given Teaching Assistant’s access level to the class so that they would have the ability to observe the instructor in several areas without disturbing the course proceedings. To understand the effectiveness of the course, the supervisor has to approach the course from the student’s perspective, as well.
• #10: These evaluations would have to carry the same weight as a face-to-face observation evaluation. There are advantages to online observations that need to be pointed out: These observations should occur over a prescribed number days. Although this number should be regarded as consistent for all online course observations, there might be some cases whereby the supervisor extends the observation period for an extra day or two. Online observations provide an extensive overview of the instructor within the course environment. The supervisor can observe the amount of time the instructor actually works within the course environment, can observe comments on assignments. Observation forms usually serve as a guideline, or checklist, for supervisors to follow. In an online observation or evaluation tool, a checklist will serve as a tool for focusing on special characteristics of online courses.
• #11: Introductory documents often provide the initial formal contact between the student and the instructor. This is extremely important since the written form constitutes the bulk of the communications throughout the course. As with any communication taking place online, all documentation must be clearly written using a language that students are able to comprehend. There are several items that should be included in every course’s information documentation. The starting point should be clearly designated. Many instructors refer to this as the “Getting Started” page because it provides the students with a place to begin. A detailed syllabus should be provided so that the student will have an outline of topics, procedures and expectations of the instructor for the course. “Frequently Asked Questions” or FAQs provide a resource that the students may refer to throughout the course. Documents that contain the important course dates (deadlines for assignments, test dates, important college-wide dates, etc.) should be clearly stated in several places that are accessible to the students (e.g. calendar, syllabus, and separate appropriately titled documents). Navigating the various components within the course environment must be simplistic and follow a clear, logical structure. All courses must be organized in such a way that the students are able to readily find all of the information needed in order to be successful in the class.
• #12: All documentation provided in the course should be user-friendly. That is, everything should be clearly and accurately marked, updated whenever necessary (all links to webpages are active, accurate, and current), and located in several logical places throughout the site. Documents should be posted in several different formats whenever possible: Microsoft Office – the most common is MS Word documents; these are best published in Word ’97 – 2003 Document format (.doc) because MS Word 2007 documents (.docx) cannot be read by earlier versions of MS Word PDF – files that may be read by Acrobat Reader or some other program that reads such files RTF (Rich Text Format) – text format that most, if not all, word processing programs can read HTML – the format that is interpreted by a web browser; this is the best format for posting files that are to be opened within the course itself (as opposed to downloaded and printed) All plug-ins and drivers should be readily available, or links to their downloads easily accessible. Pop-up announcements should be used to get the students’ attention whenever an important announcement or a due date reminder are published.
• #13: Observations and instructor evaluations of teaching performance are conducted knowing that classes vary from one instructor to the next in just about every aspect. However, every on-site class is judged on the same set of merits. Online environments are no different than their in-class counterparts in many respects. Content will vary greatly, not only between the different disciplines and courses by their very nature. It is also true that the class structure and delivery might differ from one instructor to the next, even in cloned (replicated) courses. This is the case because of different teaching methods that are based on the pedagogical philosophy of the instructor. So, each class must be observed by the supervisor with this in mind. Oftentimes the supervisor who is evaluating the faculty member’s performance is not trained in that discipline. Yet, that supervisor is expected to observe and evaluate each faculty member within their academic division. This is possible because all courses have certain components that should be present in all of them. All courses, regardless of delivery method should have clearly stated objectives (these are usually found in the syllabus, but they might also be included within the textbook or posted assignments). Every online course should have course notes posted so that the students can see what the instructor’s ideas are with regard to the content. All assignments should have clear and well-defined textbook assignments (if there is a textbook assigned for the course). If there are scheduled chat sessions, the transcripts of those sessions should be posted in a timely manner for all students to reference, using a universally accessible format.
• #14: Assessment is an important part of any college course. The first consideration any online faculty member must make is how those assessments will be administered. Students in online courses often assume that homework assignments, quizzes, tests, etc., will be administered online. This is not necessarily true. Many faculty members insist that at least some of their assessments be completed in a proctored environment to keep the integrity of the instrument and its administration intact. There are several electronic assessment software packages available for use. One such package that interfaces well with course management environments is Respondus. Book publishers provide assessment tools both as a stand-alone product and as part of their e-book site’s components. Assessment items are available in several formats, so that questions can be graded by the computer, the instructor, or a combination of the two. Also, items may be used as constructed by the publisher, imported from outside sources, and/or developed from scratch by the instructor. Instruments may be designed for self-assessment or instructor-graded. Oftentimes students want to have practice exams to help them focus their studies or to help alleviate test anxiety. For online courses, there is a wide variety of assessment techniques and tools available for use. Regardless of the methods of assessment used, faculty members must make sure that their assessments evaluate the stated course objectives, using appropriate and reasonable instruments to accomplish the course goals.
• #15: The online environment software tracks (keeps an electronic record of) every aspect of course access. Appropriateness and timeliness of faculty responses and postings can be verified for the entire course, which is not feasible in the face-to-face environment. Up-to-the-minute tracking of instructor activity and communication is observable by the evaluator as long as they have access to the online course. This enables the supervisor to accurately document the instructor’s time spent in the class on a daily basis, noting response times for emails and discussion board items. Also, the supervisor can verify that the instructor is posting grades, calendar items, announcements, etc., in a timely manner. Another advantage of this feature is that the instructor can provide documentation of their time spent in the course and on various activities within the course for their annual professional summary reports.
• #16: In contrast to the traditional classroom environment, the online course environment must be closely monitored since changes that affect the course environment, content delivery, and communications can occur at any time. Care must be taken to ensure that the class environment and tools (email, discussion board, chat rooms, etc.) are used solely for course-related activities. This avoids inappropriate use of the technology and the problems such abuse would cause for the students, instructor, and school. The links to online sites must be verified constantly since a website can be deactivated or re-directed at any time without prior notice. Since there are a variety of software packages with the same purpose are available for consumers and because new versions (updates) of existing programs are constantly produced, plug-ins and other software needs that are necessary in order to access each of the course components must be provided for students. Along those lines, several students with disabilities choose to take courses in the online environment to maximize their learning potential in an environment that they feel more comfortable working within. Thus, the online courses must be ADA compliant. This might involve closed captioning, voice interpretations, etc., as specified by their prescribed (and documented) accommodation requirements.
• #17: The resources listed here and on the next couple of slides are resources that discuss the various aspects of online courses and sound pedagogical practices for the online class environment.
• #18: In conclusion, although online courses are becoming increasingly popular for students taking college-level courses, many supervisors have neither taught nor taken courses in the online environment. Those supervisors are now being faced with having to evaluate an instructor who teaches more than 50% of their course load in the online environment. To date, there have been no formalized mechanisms put in place to observe and evaluate online faculty members within that environment to date. At The Community College of Baltimore County, the need for such an evaluation tool has become apparent to supervisors in recent semesters. There are course evaluation programs and projects available for evaluating the structure and pedagogy of the course, but nothing for the evaluation of the instructor’s performance within that class environment.
• #19: Dr. Kilbourne and Ms. Mirbaha are both supervisors in the CCBC School of Mathematics and Science, in addition to teaching courses in their field in the online environment. Based on sound online pedagogy and their observations, they have developed a checklist of components that they believe an online faculty observation/evaluation tool for supervisors should contain. Using these checklist items, they have modified an existing in-class observation form to serve as a starting point for evaluating online course instructors that they might evaluate in their supervisory role at CCBC.
• #20: Please contact us if you have any questions or comments.
• #21: Thank you!