Tips for Creating and Delivering Effective Mobile Assessments

Tips for Creating and Delivering
Effective Mobile Assessments
Ivan Forward
ivan.forward@questionmark.co.uk

Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

What will we cover today?
 Tips for mobile and embedded assessment
 Example applications of mobile assessment
 Embedded assessment


Growth in Internet Connected Mobile
Devices: 41% from 2009 to 2010

Source: ComScore

Slide 3

Mobile delivery enables new possibilities for
observational assessment and mobile exam rooms

Surveys and course evaluations: Sometimes less is
more

Embedded assessments can add value and provide
useful metrics on social/informal learning

Think small: Screens, input devices and bandwidth

To Flash, or not to Flash? Be aware of what
technologies your participants’ devices support

Test and test again: Try before you deliver!

Slide 4

Slide 5 Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Inexpensive, very portable Internet-
connected mobile devices


What are the potential applications?

Deliver to the
Observational Mobile exam
device in your
assessments rooms
hand


Observational assessment vs.
‘traditional’ quiz, test or exam
Traditional assessment
• Participant is presented with
questions for direct response

Observational assessment
• Observer is presented with
questions to ‘rate’ participant
according to pre-determined
criteria

Slide 9

What are Observational Assessments?
 Observational assessments provide a mechanism
for an ‘observer’ to assess a participant
 For example, a participant might be observed and
rated on…
 Performance of a specific task or procedure
 Knowledge acquisition and application
 Demonstration of skills/abilities that ‘traditional’
assessment may be less reliable in measuring

Slide 10

Typical workflow for “assessor”
Assessor logs in

Selects assessment

Selects participant to be observed

Answers/submits responses to questions

Results/feedback reported to stakeholders

Slide 11

Mobile Exam Rooms
 Use mobile devices
 Greater flexibility in where you deliver
assessments
 Assessments can be conducted ‘on location’

Slide 13

“Securing” an iPad…
 Consider if a standard browser
presents security concerns
 For example, Questionmark provides
an app that prevents candidates
from easily getting to other URLs
 Consider if access to “home
button” (which allows task-
switching) presents security
concerns
 Hardware solutions are available to
enable use of iPad in a “kiosk” mode

Slide 14

Consider the “stakes” and if
monitoring/proctoring is appropriate
g
Levels of Monitoring

High
Medium Stakes
Stakes Exams
Low Exams
Stakes Tests
Diagnostic
Tests

Formative Quizzes
Course Evaluations Stakes of Assessment
Risk of cheating; content security

Quizzes on mobile devices
Low stakes and openly available

Often appropriate for mobile devices

Enables retrieval practice to reduce
forgetting

Provides the participant with corrective
feedback for revision and learning

Encourages self-directed and independent
learners


When is feedback useful?
 Dealing with difficult questions
Helps participant understand and learn from the
experience

 Correcting misconceptions
Provide guidance when there is evidence of a
fundamental misunderstanding of a topic

Slide 18

Surveys and course evaluations
 Boost response rate by encouraging participants
to access via mobile delivery – “on the spot”


True for most surveys;
Especially true for mobile surveys
 Consider your objectives:
 Keep your surveys brief to help increase
response rate
 Put your most important questions at the
start of the survey
 Include only one concept per question
 Group related questions together
 Avoid relying on open-ended
questions (e.g. ‘essay’)

Slide 21

Authoring the Survey –
questions focused on:
- Demographics
- Instructor
- Course Materials
- Facilities

Slide 22

Course Evaluation Analytics

Slide 23

Screen Furniture
 Ensure your form buttons and other navigation
controls (a/k/a “screen furniture”) work with low
screen resolutions

Slide 25

Input
 Provide short URLs when hyperlinking not possible
 e.g. writing the URL on the board
 Consider using a QR Code linked to an assessment

Slide 26

Images
 Optimise size and compression of larger image files to
minimise bandwidth requirements
 Detail on images should be kept to a minimum to
ensure readability on small screens


Be aware of what technologies your participants’ devices
support


Be cautious in use of Flash

 Helps minimise bandwidth requirements
 Strengthens cross-device compatibility
 Flash will not work on Apple iPhones, iPads or iPod
touch devices
 Flash is compatible with some mobile platforms such as
Android phones (and tablets), Windows Mobile,
Blackberry (later this year)

Slide 29

Test your content!
 Test your assessment content…
 With different browsers
 On as many different devices as practical
 Emulators can help understand how content will appear
on the ‘small screen’ -- but don’t always give a true
“user experience”
 Setting up a mobile exam room?
 Test delivery environments when possible – particularly
for signal/wifi strength
 Many free online bandwidth tests are available


The same technology that can accommodate small-screen
mobile devices can also help make it easier to embed
iFrame delivered assessments in your learning content


Technologies used to support
social/informal learning initiatives

Portals

Social Networking

Wikis Forums

Blogs

Slide 33

Embed in a Wiki

Embedded in a
Wiki

Slide 34

Embedding in Blogs, Portals, Wiki’s

For Learners For Instructors

 Focus the learners’ attention  Easily use assessments
 Practice memory recall throughout the learning
 Identify what was learned process
 Provide feedback to assist the  Provide seamless user
learner to: experience
 Correct misconceptions  Centralised management
 Instill confidence of assessment content
 Point to relevant material and results

Slide 35

Blended Delivery
g
Levels of Monitoring

High
Medium Stakes
Stakes Exams
Low Exams
Stakes Tests
Diagnostic
Tests

Formative Quizzes
Course Evaluations
Stakes of Assessment

Access to your Assessments
 Mobile delivery makes it easier for
participants to access
assessments:
 Use mobile Apps
 Place links in content
(PDFs, HTML Web Pages, etc)

 Embed within a Portal
 For employees, partners and
customers


Autosensing Device and Browser

Questionmark Packages the
Assessment for Device/Browser/iframe

Sends
Senses
Device/Browser
HTML

Auto-Size
senses pixel size of window

Slide 39

Benefits
Author once Deliver to any
device or context

Schedule once

Single results set

Slide 40

Apps for Delivery
 Can sometimes offer more control,
convenience than built-in browser
 Apps available for
 Android
 Apple iOS


Mobile delivery enables new possibilities for
observational assessment and mobile exam rooms

Surveys and course evaluations: Sometimes less is
more

Embedded assessments can add value and provide
useful metrics on social/informal learning

Think small: Screens, input devices and bandwidth

To Flash, or not to Flash? Be aware of what
technologies your participants’ devices support

Test and test again: Try before you deliver!

Slide 43

Tips for Creating and Delivering Effective Mobile Assessments

More Related Content

Similar to Tips for Creating and Delivering Effective Mobile Assessments (20)

More from Kenji Lamb (20)

Recently uploaded (20)

Tips for Creating and Delivering Effective Mobile Assessments