Waycottand thompsonascilite2010 slideshare

Transforming assessment in higher education:
A participatory approach to the development of a
good practice framework for assessing student
learning through social web technologies

Jenny Waycott, Kathleen Gray & Celia Thompson,
The University of Melbourne

Judithe Sheard & Rosemary Clerehan, Monash University

Joan Richardson & Margaret Hamilton, RMIT University

Outline of presentation

• The project – what, who
• Issues in the literature
• Project aims and phases:

1. Documenting Web 2.0 assessment practices
2. Identifying principles of good practice
3. Field-testing guidelines / improving practice
4. Producing and sharing resources

2

About the project

ALTC-funded priorities project:
Web 2.0 authoring tools in higher education
learning and teaching: new directions for
assessment and academic integrity.

Project team

Jenny Waycott (project manager), Faculty of Medicine, Dentistry and Health
Sciences, University of Melbourne.
Celia Thompson, School of Languages and Linguistics, University of Melbourne.
Margaret Hamilton, School of Computer Science and IT, RMIT University.
Joan Richardson, School of Business Information Technology, RMIT University.
Kathleen Gray (project leader), Faculty of Medicine / Department of Information
Systems, University of Melbourne.
Rosemary Clerehan, Faculty of Medicine, Nursing and Health Sciences, Monash
University.
Judithe Sheard, Faculty of Information Technology, Monash University.

Web 2.0 for learning, teaching & assessment in
higher education?

O’Reilly & Battelle “One of the fundamental ideas underlying
(2009, p. 2) Web 2.0 [is] that successful network
applications are systems for harnessing
collective intelligence ... a large group of
O’Reilly, T., & Battelle, J. (2009). Web
Squared: Web 2.0 Five Years On.
people can create a collective work
Special Report for the Web 2.0
Summit, 20-22 October , San Francisco
whose value far exceeds that provided
CA.
http://assets.en.oreilly.com/1/event/2
by any of the individual participants”
8/web2009_websquared-
whitepaper.pdf

higher education?
Kakutani “jump to the summary, the video clip, the
sound bite — never mind if context and nuance are lost
(2010, in the process; never mind if it’s our emotions, more
paras 13-14) than our sense of reason, that are engaged; never mind
if statements haven’t been properly vetted and
sourced”
“tweet and text one another during plays and
Kakutani, M. (2010, 17
movies, forming judgments before seeing the arc of the
March). Texts without entire work”
context. [Book review].
New York Times. “power-search for nuggets of information that
http://www.nytimes.co
m/2010/03/21/books/ might support their theses, saving them the time of
21mash.html?ref=book
s
wading through stacks of material that might prove
marginal but that might have also prompted them to
reconsider or refine their original thinking”

higher education?
• Social web activities can be substantially
different from assessment tasks students and
lecturers are used to.
• Much has been written about pedagogical
affordances of social web technologies.
• What about assessment?

Project aims
Participatory approach to supporting good practice in
assessment of students’ social web (Web 2.0) activities:

1. Documenting how academics are assessing students’ Web 2.0
activities:
 Survey and interview teaching academics (September 2009)
 Advisory group and national roundtable (November 2009)
 17 case studies in learning and teaching settings (February to June
2010)
 Watch this space...

1. Documenting Web 2.0 assessment practices

• Online survey:
– 64 Australian academics who have assessed
students’ Web 2.0 activities

• Follow up interviews with 22 respondents
– further exploration of issues around Web 2.0
assessment.

Documenting Web 2.0 assessment practices

Field of Study Number of respondents

16
Humanities / Society & Culture

15
Education

11
Information Technology

9
Medicine & Health

6
Management & Commerce

3
Other

Type of Web 2.0 activity Number of responses

Wiki writing 32

Blogging/microblogging 31

Social networking 17

Audio/video podcasting 16

Virtual world activities 12

Social bookmarking 11


Number of students Number of responses
enrolled in subject

Less than 50 21

50-100 10

101-200 9

More than 200 7

69% undergraduate and 31% postgraduate subjects

How much the assignment is Number of responses
worth

01-10% 7
11-20% 11
21-30% 9
31-40% 6
41-50% 9
51-60% 2
61-70% 0
71-80% 3
81-90% 2
91-100% 4

Intended learning outcomes Number of
responses

Generic or graduate skills or attributes 35

Specialised knowledge or skills required in a
29
discipline or profession

Foundation knowledge or skills preparatory to
28
a discipline or profession

What staff have said about Web 2.0 and assessment ...

Open publishing

It’s not unusual for the musician or his manager or
someone to make a comment on the blog and to
correct misinformation or thank them for an opinion
or whatever and I think that is a really important
lesson for [students] to learn that whatever they
write they’re writing for an audience and if they’re
writing for more than an audience of one that has
implications


Informal writing / communication styles

it’s not a formal writing exercise, the idea is to let them
express their thoughts, reflections, interests in the
different topics rather than focusing on good
grammar and formal sentence structure, which I
think tends to constrain a lot of essays.


Personal identity and experience

There a process that goes into them finding their
different voices, how to share appropriately, how to
write with authority. A lot of them say ‘but I’m just a
student’.


Co-authoring content

Students found it challenging to co-create content and
collaborate with other students
How do you mark assignments when students can
change/overwrite each other’s work! Many students
who contributed early found that their work was
completely lost. How do you manage this process of
overwriting and still contributing to the same
content?


Designing, managing, marking, reviewing the assignment

[There is a lot of] work involved in setting it up and
making sure all the students know how to do it. If you
ask them to write an essay they just go off and write
it, you don’t have to spend the first three weeks of
the course teaching them about essays

What staff have said about Web 2.0 and assessment...


I found the bottom third of the class had difficulty
thinking about what to post on when it was left
completely up to them. ... This time around I’ll try
giving them a specific topic each week that they can
discuss



The assessor is not assessing a written document,
they’re assessing a page which ... is a whole labyrinth
of choices and connections, so they’ve got to actually
work their way through


Protecting students

I tell the students over and over again, that it is on the
WWW, it’s not associated with the university, be
careful what you put up there, make sure you are
comfortable with this.


Protecting students

I certainly do what I can to protect [students]. I
wouldn’t publish critical comments on their blogs, I
don’t let other students know which ones I think are
good, bad or indifferent. ... I protect their privacy to
that extent.


• International advisory group: 30 members
• National roundtable:
– participants included academics from diverse
disciplines, educational developers, and students.
– Discussions aimed to gather recommendations for
good practice guidelines
• Proceedings available at:
http://web2assessmentroundtable.pbworks.com

What would good practice look like?
Affordances
Affordances checklist ... • Open publishing
• Communication styles and
What is an appropriate fit texts
between what assessment • Personal identity and
is trying to achieve and experience
what Web 2.0 can do? • Co-creation, collaboration,
crowd-sourcing
• Content management

Processes
Processes checklist ...

How do teachers use Web 2.0 Design
to support student, self- and
organisational learning
Review Implement
throughout the cycle of
activities involved in the
assignment?

Feedback Mark

Policies
Policies checklist ... • disability
• access to IT services or
How can assessment using equipment
Web 2.0 be made safe and fair • appropriate conduct
for students and staff? • identity and privacy
• academic honesty and
integrity
• special consideration
• moral rights and copyright

Cinema Studies / Criminal Law
17 case studies: Blogging
Cultural Studies / Media Studies
Draft guidelines Social
pilot-tested Education
bookmarking
in 17 subjects
Social networking Languages
at 5 universities
Video sharing Business / Economics
in Victoria
during Semester 1, Photo sharing Communication Design
2010
Virtual worlds Languages
Accounting / Education
Wiki writing Information Technology
Languages / Science
Combined Information Management
Web 2.0 tools Information Technology

Field-testing guidelines / improving practice

Case studies involved...
• Introductory workshops
• Meetings with researchers, class observations
• Examples of marked student work,
assessment artefacts, etc.
• Focus groups
– Staff reflecting on experience
– Students’ perspective on using Web 2.0 for
assessment in HE

Staff reflections
“I think that being involved in something like this gives you an
opportunity to step back and look at what you’re doing,
through an outsider’s lens. In many ways our discussions with
you and some of the questions you were asking gave us
something to think about. There was perhaps a little bit of
tinkering that could be done around the edges. I think that
was really useful for us. We tend to use our intuition a lot with
this type of thing. To actually have to sit down and explain to
someone and justify what we’re doing and why we think it’s
working – it reinforced for us that we were on the right track.”

Staff reflections
“I separated the technology from the assessment and I think the
basics around good assessment practice are still the same
principles. What new technology does is just make you review
things and think about it differently in terms of
implementation ... We wanted to review our process because,
you know, we’re putting ourselves out there. We’re still not
there, but we’re putting ourselves out there to trial things
[and] that leads to review”

Staff reflections
“[I found] tension between wanting to promote the benefits of
Web 2.0 in terms of innovative and flexible modes of engaging
the students and I think sometimes when it comes to
assessment too much innovation and flexibility can cause real
problems or it certainly did in our students [...] Students really
want maximum clarity. They really want to be told how many
entries they’ve got to do, when they’ve got to do them and
this kind of thing.”


• Watch this space:

http://web2assessment.blogspot.com/

• Bookmarks:
www.citeulike.org/tag/assessment20

Publications and activities

Gray, K., Thompson, C., Clerehan, R., Sheard, J., & Hamilton, M. (2008). Web
2.0 authorship: Issues of referencing and citation for academic integrity.
The Internet and Higher Education, 11(2), 112-118.
Gray, K., Thompson, C., Sheard, J., Clerehan, R., & Hamilton, M. (2010).
Students as web 2.0 authors: Implications for assessment design and
conduct. Australasian Journal of Educational Technology, 26(1), 105-122.

Webinar: www.transformingassessment.com/events_26_may_2010.php
Workshops 2010-11 @ HERDSA, ATN Assessment, ASCILITE, Global Learn Asia
Pacific
AJET Special Issue on Assessing Students’ Web 2.0 Activities in Higher
Education: http://www.ascilite.org.au/ajet/about/special-issues/assess-
students-web2-2011.html

Acknowledgements
Project Advisory Group
• Matthew Allen, Bill Anderson, Greg Battye, Robyn Benson, Tracey Bretag, Jenny Buckworth,
Denise Chalmers, Geoffrey Crisp, Leitha Delves, Bobby Elliott, Jacqui Ewart, Glenn Finger, Tom
Franklin, Merrilyn Goos, Scott Grant, Ashley Holmes, Christopher Hughes, David Jones, Marj
Kibby, Adrian Kirkwood, Mark Lee, Catherine McLoughlin, Beverley Oliver, Kaz Ross, Alison
Ruth, Royce Sadler, Mary Simpson, Arthur Winzenried, Katina Zammit, Lynette Zeeng.

Project Reference Group
• Michael Abulencia, Robyn Benson, John Benwell, Marsha Berry, Marilys Guillemin, Laura
Harris, Deborah Jones, Gregor Kennedy, Shaun Khoo, George Kotsanas, Lauren O’Dwyer,
Jason Patten, Emma Read, Julianne Reid, Gordon Sanson, Cristina Varsavsky.

Project Pilot-testing Group
• Matthew Absolom, Anne Davies, Cathy Farrell, Scott Grant, Terry Hallahan, Michael
Henderson, John Hurst, Ramon Lobato, Warren McKeown, Michael Nott, Kerry Pantzopoulos,
Michele Ruyters, Michael Smith, Sandra Smith, Robyn Spence-Brown, Elizabeth Stewart, John
Terrell, Jenny Weight, Lynette Zeeng

ALTC Support for this project has been provided by the Australian Learning and Teaching
Council Ltd. (www.altc.edu.au), an initiative of the Australian Government Department of
Education, Employment and Workplace Relations. The views expressed in this presentation
do not necessarily reflect the views of the Australian Learning and Teaching Council, or the
views of individual contributors apart from the project team.

Waycottand thompsonascilite2010 slideshare

More Related Content

What's hot (9)

Similar to Waycottand thompsonascilite2010 slideshare (20)

Recently uploaded (20)

Waycottand thompsonascilite2010 slideshare