From heterogeneous activities to unified analytics dashboards
0 likes214 views
The document discusses integrating heterogeneous educational activities into a unified analytics dashboard to enhance engagement in software engineering courses. It outlines various activity types, including offline activities, quizzes, and games, while addressing challenges like user ID unification and dynamic activity management. Future work involves validating the system, improving the interface, and addressing educational setting challenges such as GDPR compliance.
![Improve students engagement in software engineering
Integrating four types of activities into unified dashboards:
● Generic results from offline activities (in this case, Kahoot! [3]) (Offline
activity), reported as CSV files with a simple structure.
● Interactive quizzes (via ARSNOVA.click) (NodeJS), where students
compete for quick and correct answers.
● A formal-specification game (FormalZ) (Java), where students build
specifications that correctly reject invalid inputs and, for valid inputs, correctly
describe their outputs.
● A unit-test writing game (Code Defenders) (Java), where students
compete to build tests that detect all bugs, and introduce bugs not detected
by current tests.
Code Defenders FormalZ](https://image.slidesharecdn.com/fromheterogeneousactivitiestounifiedanalyticsdashboards-190409123650/85/From-heterogeneous-activities-to-unified-analytics-dashboards-3-320.jpg)
From heterogeneous activities to unified analytics dashboards
- 1. From heterogeneous activities to unified analytics dashboards Iván Martínez-Ortiz, Iván Pérez-Colado, Dan Cristian Rotaru, Manuel Freire, Baltasar Fernández-Manjón
- 2. Teachers and gamified activities in the class ● Teachers: increasing number of gamified activities that they can use. ● A typical analytics dashboard feeds from a single system ○ complex to integrate different independent activities into a single dashboard ● Multiple activities integrated together? ○ Instead of switching dashboards: teachers expect both: ■ class overview dashboards that integrate all class activities ■ per-student dashboards with all activities by a single student ○ Activities are often grouped together in a hierarchy -> aggregation. ● Tools not originally designed with analytics? -> offline activities
- 3. Improve students engagement in software engineering Integrating four types of activities into unified dashboards: ● Generic results from offline activities (in this case, Kahoot! [3]) (Offline activity), reported as CSV files with a simple structure. ● Interactive quizzes (via ARSNOVA.click) (NodeJS), where students compete for quick and correct answers. ● A formal-specification game (FormalZ) (Java), where students build specifications that correctly reject invalid inputs and, for valid inputs, correctly describe their outputs. ● A unit-test writing game (Code Defenders) (Java), where students compete to build tests that detect all bugs, and introduce bugs not detected by current tests. Code Defenders FormalZ
- 4. Game Learning Analytics Import CSV Tracker TrackerTracker Code Defenders FormalZ Offline Activity Realtime Long Term
- 5. DESIGN AND PROOF OF CONCEPT ● Reuse Open-source pre-existing analytics platform -> EU H2020 RAGE ○ xAPI standard -> trackers ○ Two credentials: ■ The user who plays ■ The activity being performed ● Different issues: ○ Unifying User-IDs ○ Managing Activity Trees ○ Offline Activities ○ Updating Activity Trees ○ Informing Activities Context of Use ○ Estimating Implementation Effort Session of gameplay
- 6. Unifying User-IDs in IMPRESS ● Common user-IDs -> Traces belonging to the same player ● Several of the participating institutions do not have institutional LMSs ● Use of single-sign-on (SSO) highly important -> Simple SAML. Code Defenders FormalZ SSO
- 7. IMPRESS: Managing Activity Trees ● IMPRESS require to structure different game activities into a larger activity ● When building dashboards: ○ Add and remove activities. ○ Weights of sub-activities. ● Activity Rollup: Second iteration from Beaconing H2020 Project 1. Storage: Formula and parameters to be used during rollup + actual tree structure. 2. Evaluation: Child activities completion -> Progress and success tracking using formula 3. Display of results ● Activity manager -> dynamically modify the activity tree.
- 8. Managing Activity Trees Config Result parent.variable1 = (Child1.variable1 * 0.5) + (Child2.variable1 * 2)
- 9. Integrating Offline Activities ● Purpose: generate similar traces to tracker-enabled activities. ● Upload the activity results (CSV files) -> public and well-documented API. ○ Each line corresponds to an xAPI-SG trace.
- 10. Activity Trees Diagram of the rollup process, where root activities update their variables based on the values present in their child activities.
- 11. Updating Activity Trees ● If the activity tree + variables are modified -> Inconsistent results ● “Reload all traces” button -> Discards all results + regenerate using new config Results Raw tracesAnalytics Framework AnalysisQueue Traces Clears Reload all traces
- 12. Updating Activity Trees: How rollup works? ● All traces have a target activity -> Unique Activity Identifiers ● Same type activity to be used multiple times ● Traces are re-enqueued changing the activity target with parent identifier ● Traces are rolled-up until the root node ● Weighted variables change values in a child activity ↪ synthetic rollup trace is added to the queue
- 13. Updating Activity Trees: How rollup works?
- 14. Informing Activities of their Context of Use ● Different versions of an activity -> different parts of a given activity tree ○ Same quiz developing tool -> multiple quizzes ● Example: Two URLs from the same game, different activity: ○ https://quiz.server/q?activityid=1234&quiz=unit-test ○ https://quiz.server/q?activityid=5678&quiz=formal-spec ● Require additional support from activity-server side ○ Parse the activity-ID -> include it in all analytics traces generated. Quiz Server URL Parser formal-spec Traces Analytics Fram AnQueue Activity 5678
- 15. CONCLUSIONS ● Addressed integration of multiple heterogeneous activities into a unified analytics dashboard to improve a software engineering course. ○ Feasible but still complex ● We consider it to be easily generalizable to many other cases and integrations ○ Integrate tracker components or uploading of their activity logs ○ Configure the server to indicate ■ Who participates ■ What the activity hierarchy looks like ■ How it should be presented and aggregated for analytics purposes
- 16. FUTURE WORK ● To validate the activity manager and the overall integration of the component activities ○ performing several experiments with actual students from the participating institutions ● Improve and simplify the interface to simplify operation and increase understanding ● Deal with different non technical issues in different educational settings (e.g. GDPR) ● Partial simplified integration ○ Full integration with Formal Z ○ Improving dashboard provided by Code Defender