An Adaptive Assessment System with Knowledge Representation and Visual Feedback

September 28, 2012
Villach, Austria

An Adaptive Assessment System
with Knowledge Representation and Visual Feedback

Speaker: Joaquim Fernando Silva joaquim.silva@fe.up.pt
Francisco Restivo fjr@fe.up.pt

1

Agenda
• Goal Research

• Research Problem

• Research Hypothesis & Question

• Adaptive Assessment Systems

• General Architecture

• Solution Approach

• Conclusions and Ongoing Work

September, 28 2012 J. F. Silva & F. Restivo ICL 2012 Villach Austria 2

Goal Research
• Regulate the education process.

• Design a technological artefact for formative
adaptive assessment.

• Effectively monitor student’s learning
achievements.

• Get student’s involvement through visual
feedback.

Goal Research
• Use assessment as individual/class awareness.
• Deliver an effective feedback assessment tool
for a teaching-learning system.


Research Problem
• Assessment missed opportunity in education.
• Students are not motivated or engaged in
learning activities.
• Teachers aspire to find the average student.

• Students with difficulties :
– make small gains in school performance.
– academic achievement gap widens year after
year.

September 28, 2012 J. F. Silva & F. Restivo ICL 2012 Villach Austria 5

Adaptive Assessment Systems
• Different approaches:
– Presentation (color schemes, layouts,
functionalities);
– Curriculum sequencing; My Focus
– Problem-solving;
– Evidence based (knowledge, skills, attitudes).
• Several processes:
– Tests (item based);
– Questions (curriculum and rule based). My Focus


Research Hypothesis
Students desire immediate results during the
learning process and quantitative measures to
differentiate themselves.
Research Question
How to use technology, for formative
assessment, in formal learning contexts to
individually infer which knowledge each student
is acquiring and identify the learning strengths
and weaknesses.


General Architecture
Modules:
• Domain
Supports the domain/concepts structure. Teacher

• Activities
Defines, classifies and stores activities
for assessment tests. Activities Module Domain Module
Pedagogical

• Pedagogical Module

Defines the instructional method and the
multiple concepts/ activities sequences.
• Student Environment
Module
Feedback
Module
Student Module

Stores student’s profile and log
information when doing the tests.
• Feedback
Transmits, using text and graphics, the Student
test results and gives hints to students.
• Environment
Web/desktop/game/mobile based.


Solution Approach
Adaptive assessment
– Static dimension based on a structured domain
knowledge;
– Dynamic dimension based on student’s performance
and curriculum sequence;
– Modelling student behaviour (age, sex, learning style,
reaction towards failure, future commitment) within
assessment patterns.
– Centred in the first three lower Bloom’s Taxonomies
(remembering, understanding and applying).


Solution Approach
Visual Feedback
•Static for the knowledge
domain, using a Treemap
graphic;

•Dynamic for the student
performance, using a time-
oriented graphic
(PeopleGarden).


Solution Approach
Use Case *
Adaptive Assessment System

«extends» «uses» *
Create Class Class Choose Class

Teacher: -End37
«extends»
-End40

Creates class;
*
* -End33 *
*
* -End36
-End28
-End34 -End35 -End32
-End30
-End38 Choose Domain -End39
-End22
*

Choose domain;
-End24
-End26 *
*
* * * *
* -End27
Teacher n

Authors, classifies activities;
Register Student n
-End21 *
* -End29

Structure concepts; -End23
Profile
*

Defines rules and abilities.
* -End31

Global Reports
-End25 *

+End2
Author Activities

Student: -End4
*
«uses» -End8

Register and fill profile;
Classify Activities * -End10
«uses»
-End3
+End1
-End5
-End5 *
-End5 -End6 -End7
-End9 *

Choose class and domain;
«uses» -End11
** On Line Assessment Student 1 -End12
* Structure Concepts
* Test *
* * *
*
«uses» Student 2
*

Defines global reports;
-End6
«uses»
Teacher n Define Rules
*

Do online assessment tests;
«uses» Student n
-End6

Define Abilities *
*

Visualizes feedback results.
-End18
-End15
* * * -End17
-End19 -End20
*
Feedback Results
-End16 Student 1
-End14 -End13 ** *
Student 2

Teacher n Student n


Solution Approach
Proof of Concept
 Experimental work in a high school context;

 Assessment artefact in Web and mobile based
environments;

 A test pilot school experiment with multiple domains
(disciplines) and school levels (years);

 Online survey for qualitative measuring the tool
impact in terms of interest, changing attitudes towards
learning and improving knowledge acquisition.

 September 28, 2012 J. F. Silva & F. Restivo ICL 2012 Villach Austria 12

Some Conclusions
Teacher
Provides knowledge domain structure as
individual basis, difficulty in sharing;
Slow changes in teaching patterns;
Student
 Appealing interest in visual representations;
 Emotional and intuitive thinkers;
 On-demand for personalised feedback;
 Media interaction provides deeper insight;
 Social dimension reaches individual learning
awareness.


Ongoing Work
• Complete the evaluation experiment in the High
School ;
• Inquiry teachers and students about the tool
usefulness;
• Design the interactive visual feedback;
• Qualitative analysis on changing teacher’s
attitude and student’s impact of visual
assessment feedback.


Thank you for listening.
Any questions are welcome.

Faculty of Engineering at University of Porto
PORTUGAL
Joaquim Fernando Silva joaquim.silva@fe.up.pt

Soon more information available at:
September 28 2012 J. F. Silva & F. Restivo ICL 2012 Villach Austria 15

An Adaptive Assessment System with Knowledge Representation and Visual Feedback

More Related Content

Similar to An Adaptive Assessment System with Knowledge Representation and Visual Feedback (20)

More from Joaquim Silva (10)

Recently uploaded (20)

An Adaptive Assessment System with Knowledge Representation and Visual Feedback