Computer Science Education: Tools and Data
0 likes26 views
This document discusses various forms of "smart content" that can be used in computer science education to engage students in meaningful learning activities through interaction. It describes different types of smart content including interactive coding problems, program visualizations, and worked examples. It also discusses frameworks for providing levels of support, feedback, and assessment for problems, examples, and coding activities. Finally, it discusses how adaptive learning systems can utilize student data and smart content to provide personalized navigation support, recommendations, and engagement.
Computer Science Education: Tools and Data
- 1. Computer Science Education: Tools and Data Peter Brusilovsky
- 2. • Online learning system should engage student in meaningful learning activities – Interact with worked examples – Explore a simulation – Work on solving problems – Receive feedback Image credit: http://merchandisingblog.inspire.ca/find-the-hidden-treasure/ People Learn through Activities
- 3. “Smart Content” in CS Education • Many domains use of “static” content (text, images, videos) and tested with simple MCQ • CS Educators developed a variety of different of “smart content” – interactive, dynamic, provides feedback • ITiCSE 2014 Working Group revied SLC: – Program visualization – Coding problems with automatic assessment – Problem-solving support with “tutors” Brusilovsky, P., Edwards, S., Kumar, A., Malmi, L., et al. (2014) Increasing Adoption of Smart Learning Content for Computer Science Education. In: Working Group Reports of the 2014 on Innovation & Technology in Computer Science Education Conference, Uppsala, Sweden, ACM, pp. 31-57.
- 4. Smart Content: Forms • Problems – Learning by doing – Mastering domain knowledge • Worked-out examples – Demonstrating how to solve problems in a domain – Step-by-step, with explanations – Acquiring knowledge • Expertise reversal – Reversal in the relative effectiveness of instructional methods as levels of learner knowledge in a domain change – Worked examples more efficient on early stages, problems should be preferred in later stages
- 5. Smart Content: Knowledge • Syntax – Semantics – Pragmatics (of a language) • Program interpretation knowledge – How various constructs and programs work? – Notional Machine • Program construction knowledge – How to construct programs that achieve specific goals?
- 6. A Simple Guide to SLC Worked examples Problems Comprehension (behavior, tracing) Program animation Program Tracing Demo Code tracing problems Tracing ITS Construction Annotated code Codecasts Parson’s problems Coding problems • Demo: PAWS Lab Sandbox • http://adapt2.sis.pitt.edu/kt • Log in: adl02 (adl03, adl04, adl05…) • Password – same as log in
- 8. Levels of Support for Coding Problems • Simple test-based analysis – PCRS (Toronto), CodeWorkout (VT) • Intelligent analysis with feedback – Plan-based: ELM-ART – Constraint-based: SQL-Tutor – Data-driven (code matching) • Interactive support with hints – Trace-based (CMU Python Tutor, Rivers)
- 9. Coding Problems: CodeWorkout Edwards, S. H. and Murali, K. P. (2017) CodeWorkout: Short Programming Exercises with Built-in Data Collection. In: Proceedings of 2017 Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE'17, ACM Press, pp. 188-193.
- 11. Parson’s Problems: Aalto U. Version Helminen, J., Ihantola, P., Karavirta, V., and Malmi, L. (2012) How do students solve parsons programming problems?: an analysis of interaction traces. In: Proceedings of the ninth annual international conference on International computing education research, ICER '12, Auckland, New Zealand, Sptember 9-11, 2012, pp. 119–126.
- 12. Code Tracing Problems: QuizJET Hsiao, I.-H., Sosnovsky, S., and Brusilovsky, P. (2009) Extending parameterized problem-tracing questions for Java with personalized guidance. In: Proceedings of 14th Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE'2009, Paris, France, July 6-8, 2009, ACM Press, pp. 392.
- 13. Levels of Support for Tracing Problems • Correctness check + Correct answer • Diagnosing misconception • Step-by-step tracing (inner loop) • Tracing Tutor (personalization + hints) If “final resul” problem could not be solved, a step-by-step code tracing could be offered: Risha, Z., Barria-Pineda, J., Akhuseyinoglu, K., and Brusilovsky, P. (2021) Stepwise Help and Scaffolding for Java Code Tracing Problems With an Interactive Trace Table. In: Proceedings of 21st Koli Calling International Conference on Computing Education Research, ACM, pp. 1-10.
- 14. Table Tracing Tutor (Huang) Huang, Y., Brusilovsky, P., Guerra, J., Koedinger, K., and Schunn, C. (2023) Supporting skill integration in an intelligent tutoring system for code tracing. Journal of Computer Assisted Learning 39 (2), 477-500.
- 15. Worked Examples for Comprehension • Run-a-code (final result only) • Stepwise execution (intermediate results) -> • Step-by-step program visualization Stepwise execusion for SQL in DBQA: Akhuseyinoglu, K., Hardt, R., Barria- Pineda, J., Brusilovsky, P., Pollari-Malmi, K., Sirkiä, T., and Malmi, L. (2022) A Study of Worked Examples for SQL Programming. In: Proceedings of 2022 Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE’22, Dublin, Ireland, July 8-13, 2022, ACM Press, pp. 82-88.
- 16. Code Visualization: jsvee Sirkiä, T. (2018) Jsvee & Kelmu: Creating and tailoring program animations for computing education. Journal of Software: Evolution and Process 30 (2).
- 17. Code Visualization: Python Tutor Guo, P. (2013) Online python tutor: embeddable web-based program visualization for cs education. In: Proceedings of the 44th ACM technical symposium on Computer Science Education (SIGCSE 2016), March 2-5, 2016, ACM Press, pp. 579–584.
- 18. Worked Examples for Construction • A commented code or textbook presentation • Explorable example (WebEx, PCX) • A “codecast” or “screencast” A CodeCast. From Sharrock, R., Hamonic, E., Hiron, M., and Carlier, S. (2017) CODECAST: An Innovative Technology to Facilitate Teaching and Learning Computer Programming in a C Language Online Course. In: Proceedings of Proceedings of the Fourth (2017) ACM Conference on Learningat Scale, Cambridge, Massachusetts, USA, ACM, pp. 147-148.
- 19. Explorable Code Examples: PCX Hosseini, R., Akhuseyinoglu, K., Brusilovsky, P., Malmi, L., Pollari-Malmi, K., Schunn, C., and Sirkiä, T. (2020) Improving Engagement in Program Construction Examples for Learning Python Programming. International Journal of Artificial Intelligence in Education 30 (2), 299-336.
- 20. The Problem of Engagement • Great free content and top teachers are not enough to engage students • Peter Norvig: Motivation and engagement are key problems for MOOCs • A lot of great practice content – Works perfectly in lab studies, great gains – Released to students free use to enhance learning – No impact – students do not use it
- 21. Engagement and Interaction • Learning from tracing and visualization examples is passive, even with learner control over execution or which lines to explore • Combine demonstration with questions – ask to predict some steps – and assess – Ask fill-in the missing line – and assess – Ask to provide explanation - and assess – Could be adaptive – choosing where to ask question based on learner knowledge • Smooth transition between examples and problems – faded scaffoling
- 22. Example-Based Challenges in PCX Hosseini, R., Akhuseyinoglu, K., Brusilovsky, P., Malmi, L., Pollari-Malmi, K., Schunn, C., and Sirkiä, T. (2020) Improving Engagement in Program Construction Examples for Learning Python Programming. International Journal of Artificial Intelligence in Education 30 (2), 299-336.
- 23. Using Smart Content • Homeworks and assessment • Labs • Content-focused practice systems – Codeworkouts, Python Tutor • Interactive textbooks and courses – OLI, Runestone • Practice Systems • Adaptive textbook and practice systems
- 24. Smart Content in a Textbook: Runestone https://runestone.academy/ns/books/published/fopp/index.html
- 25. • Assessment-based • Same for all • Not enough doing • Weak feedback loop • High threshold – From reading to complex problems – Many are not ready Why Practice Systems? Labs/Homework Don’t Cover the Need
- 26. • Each student need different activities (kind, amount, order) • An adaptive learning system could use data about each student (knowledge, goals, …) to guide them to the next most relevant activity Why Adaptive Practice? People Learn Differently
- 27. Recipes for Personalized Engaging Practice • Adaptive navigation support • Open learner modeling • Social comparison • Knowledge/opportunity visualization • Content recommendation – Proactive – Remedial – Explainable
- 28. Personalized Navigation Support for Activity Selection with Rich-OLM Guerra, J., C. Schunn, S. Bull, J. Barria-Pineda and P. Brusilovsky (2018). Navigation support in complex open learner models: assessing visual design alternatives. New Review of Hypermedia and Multimedia 24(3): 160-192.
- 29. Mousing over this activity Concepts in the selected activity are highlighted This gauge estimates the how much you can learn in the selected activity. You will probably learn more in activities that have more new concepts Guerra-Hollstein, J., Barria-Pineda, J., Schunn, C., Bull, S., and Brusilovsky, P. (2017) Fine-Grained Open Learner Models: Complexity Versus Support. In: Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization, Bratislava, Slovakia, ACM, pp. 41-49.
- 30. Content Recommendation • Stronger guidance than Adaptive Navigation Support • Proactive recommendations – Expand knowledge • Remedial recommendation – Address problems • Explanations – Visual explanations with OLM – Text-based explanations
- 32. Remedial Recommendations Remedial visual explanations Related concepts highlighted Knowledge estimates as bar-chart Recent success rate as bar-color Warning sign on “struggled” concepts 33 Barria-Pineda, J., Akhuseyinoglu, K., and Brusilovsky, P. (2019) Explaining Need-based Educational Recommendations Using Interactive Open Learner Models. Proceedings of International Workshop on Transparent Personalization Methods based on Heterogeneous Personal Data, ExHUM at the 27th ACM Conference On User Modelling, Adaptation And Personalization, UMAP '19, Larnaca, Cyprus
- 33. Introducing SPLICE • Second-Round NSF Infrastructure project • Support the CS Education community by supplying an infrastructure CER • development and broader re-use of innovative learning content that is instrumented for rich data collection; • formats and tools for analysis of learner data; and • best practices to make large collections of learner data and associated analytics available to researchers in the CSE, data science, and learner science communities. • https://cssplice.org/
- 34. SPLICE: Smart Content Challenges • Make smart content reusable – SLC itemcould be connected to any “host system” with keeping learner data and progress • Lightweight protocols • LTI – Standards for representing an SLC item to be saved and uploaded to SLC systems • PEML • Provide data collection options for content – xAPI, Caliper
- 35. SPLICE Learning Data Challenges • Increase the volume and variety of datasets in DataShop – https://pslcdatashop.web.cmu.edu/ • Standard Data representation – Code submission standard ProgSnap2 https://cssplice.github.io/progsnap2/ – Clickstream standard • Reusable data processing and analysis approaches