Evaluation of online courses
- 1. This session will use a wiki page entitled evaluatingacoi. wikispaces.com Take time before we begin to log-on or create an account at Wikispaces.com Directions are included in your handouts.
- 2. Theoretical Framework Define teaching, cognitive, and social presence using the Community of Inquiry (CoI) theoretical framework, and consider the dynamic interconnectedness and the need for balance of each presence (Garrison, Anderson, & Archer, 2000). Survey Data Examine the CoI survey and data on students’ perceptions of the degree of each aspect of presence and expressed satisfaction. Consider additional survey items which provide enhanced student feedback. Course Improvement Propose elements of course design which target different aspects of presence with the goal of balancing perceptions of each type of presence and increasing satisfaction.
- 3. “collaborativeconstructivist… learning experience through the development of three interdependent elements – social, cognitive and teaching presence” From CoI webpage http://communitiesofinquiry.co m/model Figure adapted from (Garrison, Anderson, & Archer, 2000)
- 4. “The reality is that the CoI theoretical framework with its collaborative constructivist perspective is essentially incompatible with traditional distance education approaches that value independence and autonomy over collaborative discourse in purposeful Communities of Inquiry” (Garrison, 2012, p. 251).
- 5. Teaching Presence (TP) is . . . “the design, facilitation, and direction of cognitive and social processes for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes” (Anderson, Rourke, Garrison, & Archer, 2001, p. 5). Figure adapted from (Garrison, Anderson, & Archer, 2000)
- 6. Discourse Facilitation Defining boundaries and ideas Focusing the conversation on point or allowing exploration of topic Motivating with attention to students’ efficacy Clarifying ideas and shared experiences Instructional Design & Organization Setting curriculum, standards, targets, scaffolding Creating and implementing course calendar Managing netiquette Providing feedback and direction Direct Instruction Leading discussion, summarizing ideas in play Questioning and probing Injecting of new knowledge Designing methods, activities Selecting media or tools Providing models Offering direction for technical support (Garrison, 2007; van Shie, 2008, retrieved from http://communitiesofinquiry.com/sites/communityofinquiry.com/files/concept-map.gif)
- 7. Social Presence (SP) is . . . “the ability of participants to identify with the group or course of study, communicate purposefully in a trusting environment, and develop personal and affective relationships progressively by way of projecting their individual personalities” (Garrison, 2011, p. 34). Figure adapted from (Garrison, Anderson, & Archer, 2000)
- 8. Open Communication Continuing threads Quoting and referring to each others’ statements when developing one’s own responses Complimenting and agreeing Affective Expression Using humor, text styles, emoticons to express feelings Expressing statements of personal growth or thought processes without risk Group Cohesion Encouraging and including others Using names and greetings Expressing feelings Using plural pronouns Expressing a sense of safety to speak without fear of unfair treatment by any member of community Working together toward a goal; mentioning the goal in terms of each other’s participation and valued contributions Garrison, 2007; van Shie, 2008, retrieved from http://communitiesofinquiry.com/sites/communityofinquiry.com/files/concept-map.gif
- 9. Cognitive Presence (CP) is . . . “the extent to which learners are able to construct and confirm meaning through sustained reflection and discourse” (Garrison, Anderson, & Archer, 2001, p. 11). Figure adapted from (Garrison, Anderson, & Archer, 2000)
- 10. Four Stages of the Practical Inquiry Model (Akoyol & Garrison, 2011) Exploration Triggering Event Puzzling about a topic Recognizing the problem but not yet working Integration Resolution Exchanging information in discussion Connecting ideas Testing or defending solutions Recognizing divergent views Offering suggestions or brainstorming Finding point of convergence Offering tentative conclusions or solutions Applying new ideas to problem Applying knowledge to other settings
- 11. Cognitive In his seminal work on communities of learners, Dewey warned against the dangers of allowing any one aspect of presence to overshadow another aspect. He wrote of the necessity to maintain balance between all groups of the community. (Tolu, 2013) Teaching Social “The CoI is a generic theoretical framework that must be viewed as a means to study collaborative constructivist educational transactions – be they in online, blended, or face-to-face environments. The validation of this framework would also suggest that it can be used as a rubric to test for functioning communities of inquiry” (Garrison, 2012, p. 251, bolding not in the original text). The degree of each aspect of presence shifts over time during the administration of a course; social presence may be stronger at first in a blended design (Akyol & Garrison, 2008, 2011b).
- 12. Ten years of research demonstrated that teaching presence is “a significant determinant of student satisfaction, perceived learning, and sense of community” (Garrison, Anderson, & Archer, 2010, p. 7). “Despite a CoI’s learner-centered, socially rich experience, learners demand instructional presence and sharing of expertise from their instructors” (Shea, Hayes, Smith, Vickers, Bidjerano, & Picket, 2012).
- 13. Percentage Very/Dissatisfied Students After employing strategies to improve teaching presence, the percentage of positive student perceptions of teaching presence in the course increased, and negative responses decreased. Improving Teaching Presence 25.00 20.33 20.00 15.50 15.00 13.33 12.00 10.00 12.25 11.67 11.00 8.67 5.00 1.00 0.00 1 Teaching Presence Average Series1 Fall 20.33 Class #1 Series2 Winter 12.00 Class #2 Series3 Spring Class #3 8.67 2 Social Presence Average 13.33 3 Cognitive Presence Average 15.50 11.67 12.25 1.00 11.00 Three Small Enrollment Course Designs Adjusted with Targeted Improvements to Teaching Presence Over a Period of Three Terms (N = 25-35) (CoI survey retrieved from http://communitiesofinquiry.com/methodology )
- 14. (Bonk & Zhang, 2008; Lehman & Conceição, 2010; Stavredes, 2011) 1. Scavenger hunt: Create a synchronous or asynchronous scavenger hunt to orient students within the learning management system 2. Course calendar: Develop, distribute, and post a complete and thorough course calendar with specific directions, deadlines, and objectives/rubrics 3. Chronological template: Chronologically organize study materials and assignments; include TED talks, MOOC segments, and podcasts (consider a program design template) 4. Scaffolding strategies: Construct “soft” scaffolded assignments with lessening support 5. Clear directions: post audio, video AND text-based directions and technical support 6. Clear expectations: Visibly post minimum forum expectations and samples (take care to post samples that will not impair creative response to assignments) 7. Prompt feedback: Provide text feedback, audio feedback and combinations of audio and written feedback (Ice, Curtis, Philips, & Wells, 2007) 8. Coffee-shop talk: Create an “open” forum for off-topic conversations or help-needed question and answer
- 15. 9. Options: Provide choices 10. Chunks: Present instruction and assignments in segments 11. Goals: Require students to set goals and track progress 12. Communication: Create clarification/muddy point forum 13. Application: Assign authentic case studies (later have students create their own case studies to share and to resolve) 14. Tension: Assign or take minority or unpopular view in discussions (may use surveys to determine minority view) 15. Reflection: Ask students to reflect on and connect new learning to life situations (Dennen & Bonk, 2007)
- 16. Phases of Engagement Model (Conrad & Donaldson, 2012) Manage large groups of students with one-on-one interaction on ice-breakers and finding commonalities Suggestion to Enhance Teaching Presence: Apply the Phases of Engagement Model Help pairs share and move to consensus on content-specific tasks including reflection Guide or assist small 3-5 person contract-based teams to complete tasks that build real world skills Act as fellow learner & guide while students take more responsibility in learner-initiated/led tasks Empower learners to reflect, recognize personal growth, and apply content knowledge for their own career or daily needs
- 17. Choice of Assignments: Address Same Objective Template: Introduce new skill OR begin a task Webliography Wiki: Glossary of terms, ideas , & links Pro/Con Wiki Debate: Defend both sides Muddiest Point or Vitals Check: Wiki or DB Forum Case Studies: Resolve Using Groups Wiki Pictionary: Describe/Define and Collaborate (Low-Risk Activity)
- 18. Social presence is critical if learning occurs because of dialog and collaboration. Students must be present, feel present, engage with others in a safe environment for exploring and applying new knowledge (Garrison, 2011). In years past, researchers established that it is possible to be feel and to project oneself socially in a virtual world (Gunawardina, Lowe, & Anderson, 1997; Rovai 2002). Perceptions of social presence are linked to student satisfaction and perceived learning outcomes (Richardson & Swan, 2003; Swan, Garrison, & Richardson, 2009). High levels of social presence were most necessary at the beginning of a course, but as the group bonds, the need for social presence lessens, and teaching and cognition are more in the foreground (Akyol & Garrison, 2008).
- 19. Improving Social Presence After employing SP improvement strategies, the percentage of positive student perceptions of SP increased, and negative responses decreased. Percentage Very/Dissatisfied Students 30.00 24.67 25.00 21.50 20.00 17.33 15.00 13.67 11.50 10.67 10.00 5.00 5.00 4.50 3.67 0.00 1 2 Social Presence Average Series1 Fall Class #1 17.33 24.67 21.50 Series2#2 Winter Class 10.67 13.67 11.50 Series3#3 Spring Class 3.67 5.00 4.50 Teaching Presence Average 3 Cognitive Presence Average Three Small Improvement Course Designs Adjusted with Targeted Improvements to Social Presence Over a Period of Three Terms (N=25-35) (CoI Survey retrieved from http://communitiesofinquiry.com/methodology )
- 20. When students respond with dissatisfaction about measures of social presence, they are saying that they did not interactive effectively or feel comfortable online. Send a Welcome E-mail to Class Greetings, and welcome to our ABCD123 course. Join us for a scavenger hunt . . . Class Roster Wiki: SelfIntroduction with Photo ) Try Brookfield’s Discussion Techniques with the CoI framework Face2Face or Synchronously (Armstrong & Thornton, 2012) Post Rules: Discourse Netiquette
- 21. Social and teaching presence create the path to engagement and reflection necessary for higher levels of cognition (Garrison, 2007; Swan, Garrison, & Richardson, 2009, bolding not in the original text). “70% of the variance in the online students’ levels of cognitive presence, a multivariate measure of learning, can be modeled based on their reports of their instructors’ skills in fostering teaching presence and their own abilities to establish a sense of social presence” (Shea & Bidjerano, 2009). Difficulties noted in attaining higher levels of cognition on discussion forums were attributed to types of questions posed (Garrison & Arbaugh, 2007; Garrison & Cleveland-Innes, 2005, bolding not in the original text). Students proceed to higher levels of cognition when given a specific problem to resolve or provided with directives that demand integration and resolution, operation at top cognitive levels (Shea & Bidjerano, 2008, bolding not in the original text).
- 22. Improving Cognitive Presence 16.00 14.67 Percentage Very/Dissatisfied Students Cognitive presence may be the most important aspect of a class, being as the goal of teaching is learning, but it may be the most difficult to establish partly due to its reliance on teaching and social presence 14.00 12.00 10.00 9.00 9.50 9.00 8.00 6.67 6.00 4.33 4.00 2.00 0.00 0.00 1 Teaching Presence Average Series1 14.67 Class #1 Fall Social Presence Average 2 3 Cognitive Presence Average 11.00 11.50 Series2 Class #2 Winter 9.00 9.00 9.50 Series3 6.67 4.33 0.00 Class #3 Spring (Arbaugh 2007; Garrison, & ClevelandInnes, 2005). 11.50 11.00 Three Small Enrollment Course Designs Adjusted with Targeted Improvements to Cognitive Presence Over a Period of Three Terms (N= 25-35) (CoI Survey retrieved from http://communitiesofinquiry.com/methodology )
- 23. Suggestions to Improve Cognitive Presence 1. Repeated practice: Provide several opportunities for practice on a desired objective: Increase the number of wikis in the course to improve social presence and collaboration. See (West & West, 2009) 2. Multiple demonstrations and sample target papers: Improved the process for Peer Review and Case Study responses using a combination of tasks: DB (exploration) > Wiki (integration) > Journal (reflection) 3. Varied and connected discussion tasks: Structured, scaffolded, role play, debate mixed methods in discussions (Darabi, Arrastia, Nelson, Cornille, & Liang, 2011) 4. Posed divergent or minority points of view: Assigned or argued in discussion, wiki, & journal topics to challenge critical thinking or reflective responses (Dennen & Bonk, 2007) 5. Assigned reflection: Various activities for Journals & Wiki for personal and shared reflection (Conrad & Donaldson, 2012)
- 24. Suggestion to Improve Cognitive Presence Employ Combinations of Interactive Tools in a Series of Related Assignments Journal or Blog Design Features Discussion Board Wiki Author centered reflection, author-owned, static, linear, comments at the end, static , chronological, single post Topic centered and comment driven threads and postings, static, collaborative, linear Document centered, collaborative creation, constantly changing edited by entire group; all interactive; nonlinear Reading responses, notes, reflections on topics, summaries, evaluations, rough drafts of projects Self-introductions, role play, exploration, case studies, debate, sharing of ideas & some media Creating documents, projects, sharing media or resources, building pages, analysis, brainstorming Graphic Illustration Possible Uses
- 25. Collaboration Activity Participants are grouped in manageable-sized groups, preferably 3-5. Groups will address one of the forms of presence: Teaching, Cognitive, or Social. Each group will find its own previously created and labeled Wikispace.com page on ImprovingaCoI. To build the wiki page, each group will describe activities which could enhance the aspect of presence that they have been assigned. After the conference ends, your presenter will edit and combine all entries to form one wiki page from the conference which will be accessible to you and editable by you for as long as Wiki Go to Improvingacoi.wikispaces.com Login (make an account) Find your group page. Click on it to open it for editing. Add ideas for activities to consider. Troubles? Check the How to Use Wikispaces Handout in your packet of materials.
- 26. Contribute ideas that you have regarding activities/practices that could improve Teaching Presence to your group wiki page. Instructional Design Strategies Facilitative Discourse Techniques Direct Instructional Tools
- 27. Contribute ideas you have regarding activities/practices that could improve Social Presence to your group wiki page. Affective Expression Open Communication Group Cohesion
- 28. Contribute ideas that you have regarding activities/practices that could improve Cognitive Presence to your group wiki page. Triggering Exploration Integration Resolution
- 29. Learning Presence “The findings from this study underscore the importance of quality of teaching presence and social presence, but also the fact that rankings of cognitive gains are likely to depend on the characteristics the learner bring to the learning environment when quality of teaching or the quality of social interaction are low or inadequate” (Shea & Bidjerano, 2012, p. 326) Structuring environment Managing time and tasks Seeking help Self-evaluating performance Teaching Presence Cognitive Presence Setting goals Social Presence Shea & Bidjerano, 2010, 2012
- 30. Current research: the construct of metacognition may be related to the aspect of cognitive presence in a CoI Knowledge of Cognition Monitoring of Cognition See the metacognition survey items which have been recently introduced for research development in the handouts for the session. Cognitive Presence Regulation of Cognition Adapted from (Garrison & Akyol, 2013; see also Akyol & Garrison, 2011a, 2011b)
- 31. New Research on CoI Survey Items: Emotional Presence Emotions could act on all aspects of presence. Novice students must learn to act in a virtual environment; instructors who are sensitive to emotional presence present interventions to allow students to adjust and learn (Cleveland-Innes & Campbell, 2012). Emotional Presence Emotional Presence: “the outward expression of emotion, affect, and feeling by individuals and among individuals in a community of inquiry, as they relate to and interact with the learning technology, course content, students, and the instructor” (Cleveland-Innes & Campbell, 2012, p. 283).
- 32. References Akyol, Z., & Garrison, D. R. (2008). The development of a community of inquiry over time in an online course: Understanding the progression and integration of social, cognitive, and teaching presence. Journal of Asynchronous Learning Networks, 12(3-4), 3-22. Akyol, Z., & Garrison, D. R. (2011a). Assessing metacognition in an online community of inquiry. The Internet and Higher Education, 14(3), 183-190. Akyol, Z., & Garrison, D. R. (2011b). Understanding cognitive presence in an online and blended community of inquiry: Assessing outcomes and processes for deep approaches to learning. British Journal of Educational Technology 42(2), 233-250. Arbaugh, J. B. (2007). An empirical verification of the community of inquiry framework. Journal of Asynchronous Learning Network, 11(1), 73-85. Armstrong, A., & Thornton, N. (2012). Incorporating Brookfield's discussion techniques synchronously into asynchronous online courses. Quarterly Review of Distance Education, 13(1), 1-9. Bonk, C. J., & Zhang, K. (2008) Empowering online learning: 100+ activities for reading, reflecting, displaying, and doing. San Francisco, CA: Jossey-Bass, Wiley. Caspi, A., & Blau, I. (2008). Social presence in online discussion groups: Testing three conceptions and their relations to perceived learning. Social Psychology of Education, 11(2), 323-346. Cleveland-Innes, M. & Campbell, P. (2012). Emotional presence, learning, and the online learning envi-ronment. The International Review of Research in Open and Distance Learning, 13(4), 269-92. Community of Inquiry Survey (2011). Retrieved at http://www.communitiesofinquiry.com/ Conrad, R. M., & Donaldson, J. A. (2012). Continuing to engage the online learner: More activities and resources for creative instruction. San Francisco, CA: Jossey-Bass, Wiley. Darabi, A., Arrastia, M. C., Nelson, D. W., Cornille, T., & Liang, X. (2011). Cognitive presence in asynchronous online learning: A comparison of four discussion strategies. Journal of Computer Assisted Learning, 27(1), 216-227. Dennen, V. P., & Bonk, C. J. (2007). We’ll leave the lightp on for you: Keeping learners motivated in online courses. In L. A. Tomei (Ed.). Online and distance learning: Concepts, methodologies, tools, and applications (pp. 64-76). Hershey, PA: Idea Group Global. Dewey, J. (1933). How we think: A restatement of the relation of reflective thinking to the educative process. Boston, MA: D. C. Heath. Garrison, D. R. (2007). Online community of inquiry review: Social, cognitive, and teaching presence issues. Journal of Asynchronous Learning Networks, 11(1), 61-72. Garrison, D. R. (2011). E-Learning in the 21st century: A framework for research and practice (2nd ed.). London: Routledge/Taylor and Francis. Garrison, D. R. (2012). Article review: Social presence within the community of inquiry framework. International Review of Research in Open & Distance Learning, 13(1), 250-253. Garrison, D. R., & Akyol, Z. (2013). Toward the development of a metacognition construct for communities of inquiry. Internet and Higher Education 17(2), 84–89. Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2–3), 87–105. Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence, and computer conferencing in distance education. American Journal of Distance Education, 15(1), 7–23. Garrison, D. R., Anderson, T., & Archer, W. (2010). The first decade of the community of inquiry framework: A retrospective. Internet and Higher Education 13(1-2), 5–9. Garrison, D.R. & Arbaugh, J.B. (2007). Researching the community of inquiry framework: Review, Issues, and Future Directions. The Internet and Higher Education, 10(3), 157-172. Garrison, D. R. & Cleveland-Innes, M. (2005). Facilitating cognitive presence in online learning: Interaction is not enough. American Journal of Distance Education, 19(3), 133-148. Garrison, D. R., Cleveland-Innes, M., & Fung, T. (2010). Exploring causal relationships among cognitive, social and teaching presence: student perceptions of the community of inquiry framework. The Internet and Higher Education, 13(1–2), 31–36. Gunawardena, C., Lowe, C. A., & Anderson, T. (1997). Analysis of a global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17(4), 397-431. Ice, P., Swan, K., Diaz, S., Kupczynski, L., & Swan-Dagen, A. (2010). An analysis of students’ perceptions of the value and efficacy of instructors’ auditory and text-based feedback modalities across multiple conceptual levels. Journal of Educational Computing Research, 43(1), 113-134. Lehman, R. M., & Conceição, S. C. O. (2010). Creating a sense of presence in online teaching. San Francisco, CA: Jossey-Bass. Richardson, J. C., & Swan, K. (2003). Examining social presence in online courses in relation to students’ perceived learning and satisfaction. Journal of Asynchronous Learning Networks, 7(1), 68-88. Richardson, W. (2010). Blogs, Wikis, Podcasts. Thousand Oaks, CA: Corwin/Sage. Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2001). Assessing social presence in asynchronous textbased computer conferencing. Journal of Distance Education, 14(2), 51-70. Rovai, A. P. (2002). Development of an instrument to measure classroom community. The Internet and Higher Education, 5(3), 197-211. Shea, P., & Bidjerano, T. (2008). Measures of quality in online education: An investigation of the community of inquiry model and the net generation. Journal of Educational Computing Research, 39(4), 339-361. Shea, P., & Bidjerano, (2009). Community of inquiry as a theoretical framework to foster "epistemic engagement" and "cognitive presence" in online education. Computers in Education, 52(3), 543-553. Shea, P., & Bidjerano, T. (2010). Learning presence: Towards a theory of self-efficacy, self-regulation, and the development of a communities of inquiry in online and blended learning environments. Computers and Education 55(4), 1721-1731. Shea, P., & Bidjerano, T. (2012). Learning presence as a moderator in the community of inquiry model. Computers & Education 59(2), 316-326. Shea, P., Hayes, S., Smith, S. U., Vickers, J., Bidjerano, T., & Picket, A. (2012). Learning presence: Additional research on a new conceptual element within the community of inquiry (CoI) framework. Internet and Higher Education, 15(2), 89–95. Stavredes, T. (2011). Effective online teaching: Foundations and strategies for student success. San Francisco, CA: Jossey-Bass. Swan, K., Garrison, D. R., & Richardson, J. (2009). A constructivist approach to online learning: The community of inquiry framework. In C. R. Payne (Ed.). Information technology and constructivism in higher education: Progressive learning frameworks (pp. 43-57). Hershey, PA: IGI Global. Swan, K., Shea, P., Richardson, J., Ice, P., Garrison, D. R., Cleveland-Innes, M., & Arbaugh, J. B. (2008). Validating a measurement tool of presence in online communities of inquiry. E-Mentor, 2(24), 1-12. Tolu, A. T. (2013). Creating effective communities of inquiry in online courses. Procedia: Social and Behavioral Sciences, 70(1), 1049-1055. Van Shie, J. (2008). Community of inquiry visual. Community of Inquiry. Retrieved from http://communitiesofinquiry.com West, J. A. and West, M. L. (2009). Using wikis for online collaboration. San Francisco, CA: Jossey-Bass, Wiley.