CUbRIK Tutorial at ICWE 2013: part 2 - Introduction to Games with a Purpose

AN INTRODUCTION
TO
HUMAN COMPUTATION
&
GAMES WITH A PURPOSE
ALESSANDRO BOZZON
DELFT UNIVERSITY OF TECHNOLOGY
LUCA GALLI
POLITECNICO DI MILANO

ABOUT THE TUTORIAL
•  Crowdsourcing, Human Computation, and GWAPs are hot topics
•  “Human Computation” => more than 3000 papers
•  400 in 2013
•  “Crowd Sourcing” => more than 36000 papers
•  4800 in 2013
•  “Games With A Purpose” => more than 1400 papers
•  162 in 2013
•  This short tutorial is necessarily shallow, but
•  Concrete Examples
•  Lot of references and links
•  An outlook on the future
•  Slides and additional materials available
•  http://hcgwap.blogspot.com
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
2

ABOUT THE SPEAKERS
ALESSANDRO BOZZON
Assistant Professor - TU Delft
http://www.alessandrobozzon.com
a.bozzon@tudelft.nl
LUCA GALLI
Ph.D. Student - Politecnico di Milano
http://www.lucagalli.me
lgalli@elet.polimi.it
3
•  RESEARCH BACKGROUND AND INTERESTS
•  Web Data Management
•  Crowdsourcing and Human Computation
•  Game Design
•  Web Engineering and Model Driven Development

AGENDA
4

AGENDA
•  PART 1 => CrowdSourcing and Human Computation
•  Introduction
•  Design of Human Computation Tasks
•  Frameworks For/With Human Computation
•  The Future of Human Computation
•  PART 2 => Games With a Purpose
•  Play vs Games
•  Introduction to Game Design
•  Games with a Purpose Design Guidelines
5

PART 2
AN INTRODUCTION TO
6

7
Something really bad is
going to happen…

8
Or not?

9
MAGIC CIRCLE
“All play moves and has its being within a playground marked off beforehand materially or
ideally, deliberately or as a matter of course… The arena, the card-table, the magic circle,
the temple, the stage, the screen, the tennis court, the court of justice, etc., are all in form
and function play-grounds, i.e., forbidden spots, isolated, hedged round, hallowed, within
which special rules obtain. All are temporary worlds within the ordinary world, dedicated
to the performance of an act apart.”
Johan Huizinga, “Homo Ludens”, (1938-1950)

WHAT IS PLAY?
10
Autonomy:
play
is
a
voluntary
ac2vity.

Safety:
during
play
there
are
radically
reduced
serious

consequences
in
what
we
do

A<unement:
being
interested
in
the
same
things

temporarily
while
playing
together

Explora2on:
the
possibility
to
experiment
and
try
out

new

things

Mastery:
the
will
to
improve
one’s
own
skills

WHICH ARE THE
BENEFITS?
11
“Play
is
an
enjoyable
ac/vity
that
allows
us
to
train
and
test
our

skills
in
a
safe
environment.”

Fosters
crea2vity,
ﬂexibility,
and
learning

An2dote
to
loneliness,
isola2on,
anxiety,
and
depression

Teaches
perseverance

Hints
on
how
to
cooperate
with
others

Increases
energy
and
prevents
burnout

Stuart Brown, “Play: How it Shapes the Brain,
Opens the Imagination, and Invigorates the
Soul”, 2009.

GAMES VS PLAY
12

Games
are
dis/nguished
from
play

–
Play
is
free-‐form

–
Games
are
rule-‐based

A
game
is
a
closed,
formal
system
that

·∙
Engages
players
in
structured
conﬂict
and

·∙
Resolves
its
uncertainty
in
an
unequal

outcome.

Fullerton, T.; Swain, C. & Hoffman, S.
Game Design Workshop: A playcentric approach
to creating innovative games, 2008

CLASSIC GAME DEFINITION
13
1.  Rules

•
Games
are
rule-‐based.

2.
Variable,
quan/fiable
outcome

•
Games
have
variable,
quan/fiable
outcomes.

3.
Valoriza/on
of
outcome

•
The
different
poten/al
outcomes
of
the
game

are
assigned
different
values,
some
posi/ve

and
some
nega/ve

CLASSIC GAME DEFINITION
14

4.
Player
effort

•
The
player
exerts
effort
in
order
to
influence
the
outcome

(games
are
challenging).

5.
Player
aVached
to
outcome

•
The
player
is
emo/onally
aVached
to
the
outcome
of
the

game
in
the
sense
that
a
player
will
be
winner
and

“happy”
in
case
of
a
posi/ve
outcome,
but
a
loser
and

“unhappy”
in
case
of
a
nega/ve
outcome.

6.
Nego/able
consequences

•
The
same
game
[set
of
rules]
can
be
played

with
or
without
real-‐life
consequences

Juul,J.“Half-Real:VideoGamesBetweenReal
RulesandFictionalWorlds”,2005

INCENTIVES
15
Money, Passion, Ease of Participation,
Altruism, Appeal to one’s own
knowledge, Reputation, Fun
Organisciak, Piotr. Why Bother?: Examining the Motivations of Users
in Large-scale Crowd-powered Online Initiatives. Diss. University of
Alberta, 2010.

INCENTIVES
16

Intrinsic
mo2va2on
is
deﬁned
as
doing
an
ac2vity
for

its
inherent
sa2sfac2ons
rather
than
for
some
separable

consequence.

Extrinsic
mo2va2on
is
deﬁned
as
doing
an
ac2vity
to
a<ain

some
separable
outcome

Deci, Edward L., and Richard M. Ryan. Self‐Determination. ,1985.

17

18

19

Games
with
a
Purpose
(GWAPs)
are
digital
games
that
generate

useful
data
as
a
by-‐product
of
play.
[vA06,LvA09]

The
design
of
a
GWAP
requires
to
create
a
game
so
that
its

structure
encourages
computa2on,
correctness
of
the
output
and

players
reten2on.

WHAT THEY ARE USED FOR
20
Krause,M.&Smeddinck,J.(2011b).HumanComputation
Games:ASurvey.In:Proceedingsofthe19thEuropean
SignalProcessingConference(EUSIPCO-2011).
Intui2ve
Decisions
Aesthe2c
Judgment

Contextual
Reasoning
Embodiment
Issues

ARE THEY WORKING?
21
ESP Game
Purposes:
Let players determine the
contents of images by
submitting meaningful labels
they can agree on.
Results:
A total of 13,630 people
played the game during the
test phase, generating
1,271,451 labels for
293,760 different images.
Luis von Ahn and Laura Dabbish. 2004.
Labeling images with a computer game.

ARE THEY WORKING?
22
Foldit
Purposes:
The objective of the game is
to fold the structure of
selected proteins to the best
of the player's ability, using
various tools provided within
the game to discover new
structural configurations.
Results:
In 2011, players of Foldit helped to decipher the crystal structure of
an AIDS-causing monkey virus. While the puzzle was available to play
for a period of three weeks, players produced an accurate 3D model of
the enzyme in just ten days. The problem of how to configure the
structure of the enzyme had stumped scientists for 15 years.
Khatib, F.; Dimaio, F.; Cooper, et al. (2011). "Crystal structure of a monomeric
retroviral protease solved by protein folding game players"

GAMIFICATION
23

“The
use
of
game
design
techniques
and
game

mechanics
to
enhance
non-‐game
contexts”

S. Deterding, M. Sicart, L. Nacke, K. O’Hara, and D. Dixon,
“Gamification. Using game-design elements in non-gaming contexts”

Actually
ques/onable,
we
will
see
why

Sebastian Deterding Miguel Sicart

WHAT IS USED FOR
24

IS IT WORKING?
25
Samsung Nation
Purposes:
Samsung Nation, a social
loyalty program that lets
users earn badges for
activities as writing reviews
and watching videos and
compete for rewards.
Results:
500% increase in
customers product reviews
66% increase in site
visitors
30% increase in comments
reduced marketing costs
reduced product support
costs

IS IT WORKING?
26
MTV Italia MyChart
Purposes:
User generated video chart
based on various “game
dynamics” like avatars,
points and leaderboards to
drive users from Facebook
to TV and to loyalize existing
TV audience.
Results:
More than 500.000 votes in
less then three months
More than 150.000 videos
viewed

COMMONALITIES AND
DIFFERENCES
27
One
or
more
objec2ves
or
tasks
that
users
have

to
accomplish.

A
gaming
experience,
deﬁning
challenges
to

overcome
and
rewards
for
their
solu2on.

One
or
more
players,
the
users
of
the

applica2on,
who
are
proﬁled
and
monitored
in

their
ac2vi2es.

But...

28
INTRODUCTION TO
GAME DESIGN

29
Mancala, 7th
century AD

30
Unreal
Tournament 3,
Epic Games,
2007

31
Turn
based
boardgame
vs
Real
/me
ac/on
shooter

Handmade
physical
board
vs
Personal
Computer

Public
domain
rules
vs
Copyrighted

1 - PLAYERS
32
Number
of
players

Roles
of
the
players

Interac2ons
among
players
and
the
game:

Single
Player/Mul2
Single

Unilateral/Mul2lateral
Compe22on

Coopera2ve

Team
compe22on

2 - OBJECTIVES
33

Objec2ves:
speciﬁc
goals
for
the
player
to
accomplish

Capture:
capture
or
destroy
a
resource

Chase:
catch
an
opponent
or
elude
one

Race:
reach
a
goal
before
the
other
players

Alignment:
arrange
game
objects
in
a
spa/al
or
conceptual

conﬁgura/on

RescueEscape:
bring
a
unit
to
safety

Construc2on:
build,
maintain,
manage
objects

Explora2on:
explore
the
environment
where
the
game
take

place

Solu2on:
solve
a
problem
or
puzzle
before
(or
more

accurately)
the
compe//on
or
certain
constraints

Outwit:
gain
and
use
knowledge
to
defeat
other
players

2 – OBJECTIVES: EXAMPLES
34
Alignment:
arrange
game
objects
in
a
spa2al
or

conceptual
conﬁgura2on

Tetris, Alexey Pajitnov, 1984 Puzzle Bobble, Taito
Corporation, 1994

2 – OBJECTIVES: EXAMPLES
35
Solu2on:
solve
a
problem
or
puzzle
before
(or

more
accurately)
than
the
compe22on
or

following
certain
constraints

Connect Four, Milton Bradley,
1974
Professor Layton and the
Miracle Mask, Level-5, 2011

3 - PROCEDURES
36
Procedures:
the
methods
of
play
and
the
ac/ons
that

players
can
take
to
achieve
the
game
objec/ves.
They

are
used
to
deﬁne
who
does
what,
where
and
how.

Typically
a
game
is
composed
of:

Star%ng
ac%on:
How
to
put
a
game
into
play.

Progression
of
ac%on:
Ongoing
procedures
a_er

the
star/ng
ac/on.

Special
ac%ons:
Available
condi/onal
to
other

elements
or
game
state.

Resolving
ac%ons:
Bring
gameplay
to
an
end.

3 – PROCEDURES: EXAMPLE
37
Star%ng
ac%on:
Choose
a
player

to
go
ﬁrst.
Each
player
chooses
a
color:

red
or
black.

Progression
of
ac%on:
On
each

turn,
a
player
drops
one
colored

checker
down
any
of
the
slots
in

the
top
of
the
grid.

Resolving
ac%ons:
The
play
alternates

un/l
one
of
the
players
gets
four

checkers
of
one
color
in
a
row.
The
row

can
be
horizontal,
ver/cal,
or
diagonal.

Connect Four

4 - RULES
38
• 
Define
Objects

• 
Restrict
Ac/ons

• 
Determine
Effects
(ECA
rules)

Chess:
A
player
cannot
move
her
king
into
check.

Poker:
A
straight
is
five
consecu8vely
ranked

cards;
a
straight
flush
is
five
consecu8vely
ranked

cards
of
the
same
suit.

WarCrad
II:
To
create
knight
units,
a
player
must
have

upgraded
to
a
keep
and
built
a
stable.

5 - RESOURCES
39
Resources:
valuable
objects
that
can
help
the
players
to
achieve

their
goals
but
are
scarce
in
the
system.

Lives:
number
of
“trials”
that
can
be
aVempted
to
reach
a
goal

Units:
mul/ple
objects
that
has
to
be
managed
instead
of
lives

Health:
represent
the
status
of
loss
or
near
loss
of
lives
and
units

Points:
numerical
value
that
represents
a
measure
of
the
skill
and

progression
of
a
player

Ac2ons:
number
of
possible
dis/nct
choices
that
a
player
can

make
within
a
deﬁned
/meframe

Power-‐ups:
object
that
gives
a
boost
to
the
player

Items:
used
by
the
player
to
accomplish
an
objec/ve,
made

scarce
by
the
system

Turns:
the
number
of
game
phases
within
which
a
player
must

accomplish
the
objec/ves

Time:
restricts
player
ac/ons
or
phases
in
periods
of
/me.

5 – RESOURCES: EXAMPLE
40
Card Hunter, Blue Manchu Pty Ltd, TBR
Time

Points

Ac/ons

6 - CONFLICTS
41
Conflicts:
emerges
from
the
players
trying
to

accomplish
the
goals
of
the
game
within
its
rules
and

boundaries,
since
procedures
and
rules
tend
to
deter

players
from
accomplishing
goals
directly
or
make

players
work
against
each
other.

The
most
common
conflicts
are
generated
by:

Obstacles,
objects
or
rules
that
limit
the
freedom
of

the
players

Opponents,
since
they
are
usually
trying
to
achieve
an

objec/ve
faster
than
us

Meaningful
Choices,
players
have
to
make
choices

that
will
influence
the
outcome
of
the
game

7 - BOUNDARIES
42
Boundaries are what
separate the game from
everything that is not the
game and defines the
physical or virtual
scopes in which the
game is performed.
Example: Football would
not be the same game if
the boundaries of the
football field were not
defined.

8 - OUTCOME
43
Outcome:
the
outcome
of
the
game
(if

present)
is
uncertain,
since
it
is
not

possible
to
predict
the
results
ahead.

If
an
outcome
can
be
obtained
it
has
to

be
quan/fiable
with
respect
to
the

defined
goals.

It
is
different
than
the
objec/ve
since
all

the
player
can
reach
the
same
objec/ve.

44
Derived by modeling the guidelines and best practices found in:
Fullerton, T.; Swain, C. & Hoffman, S.
Game Design Workshop: A playcentric
approach to creating innovative games,
2008
Crowford, C. The Art of Computer
Game Design, 1984
GAME DEVELOPMENT PROCESS

45

GWAPS AS SERIOUS GAMES
46
BenSawyer,PeterSmith:SeriousGames
Taxonomy,2008

47
Katharina Siorpaes and Martin Hepp. 2008. OntoGame:
weaving the semantic web by online games.

GAMES WITH A PURPOSE:
CURRENT ISSUES
48
●  “Ad-‐Hoc”
Design

●  Engagement
and
reten/on
mechanisms.

●  Mapping
from
task
to
game
mechanics

●  Valida/on
techniques
and
strategies

E.G.
Real
user
comments
about
OntoGalaxy

49
1)
PURPOSE

2)
TASK
DEFINITION

3)
GAME

MECHANICS

DESIGN

4)
INSTANCE

MATCHING

5)
OUTPUT

VALIDATION

6)
PERFORMANCE

EVALUATION

7)
ENGAGEMENT

SRATEGY

GWAPS DIMENSIONS

GWAP DEVELOPMENT
PROCESS
50

REQUIREMENTS
SPECIFICATION
51
uTask
Task
Description
Input Objects
Output Objects
Execution Interface
Operations Output Validation

INPUT OBJECTS
52
Adapted from“Mathematical Modeling of social games, IrwinKing, SIAG2009”
An input object is an object with data t23e and a set of
ﬁnite a67ibutes denoted as
tex9, images, audio seg=ents, video seg=ents, other
unst7@ct@red data, st7@ct@red data
where the data t23e is the media t23e presented by
Each a67ibute has a relationship to a set of
values .
Each is an object with its own data t23e that represent the
metadata of

DEFINING
OPERATIONS
53
uTask
Task
Description
Input Objects
Output Objects
Execution Interface

OPERATION TYPES
54
They
fall
in
two
broad
categories:

Genera/ve
tasks

Decision
tasks

A
possible
(non-‐exhaus/ve)
list
of
human
computa/on

tasks
may
include:

• 
Object
Recogni/on/Iden/ﬁca/on/Detec/on

• 
Sor/ng
(Clustering/Ordering)

• 
Natural
Language
Processing

• 
State
Space
Explora/on

• 
Content
Genera/on/Submission

• 
User
preference/opinion
elicita/on

ENSURE OUTPUT QUALITY
55
uTask
Task
Description
Input Objects
Output Objects
Execution Interface

Twofold
purpose:
ensure
output
correctness
and

counter
player
collusion.

OUTPUT VALIDATION
56
Player
Tes2ng:
compare
the
output
provided
by
the

user
with
known
correct
outputs

Repe22on:collect
more
than
one
solu/on
for
the

same
input
object,
then
aggregate
the
solu/ons

Taboo
Output:
limit
the
possible
solu/ons
that
can

be
submiVed

Random
Pairing:
pairing
players
at
random

TASK EXECUTION
57
uTask
Task
Description
Input Objects
Output Objects
Execution Interface

TASK TO GAME
MECHANICS MATCHING
58
Does
a
game
with
game

mechanics
similar
to
the
task

exists?

If
so,
integrate
the
task
within

the
game

If
not,
custom
game
mechanics

has
to
be
implemented.

Clear
and
Transparent
game

mechanics

SOLUTION MECHANICS
59
Pa<ern
Recogni2on

Wikipedia
stands
to
knowledge
as

BoardGameGeek

stands
to
games.

Benng/Wagering

Line-‐Drawing

Tile-‐Placement

Pa<ern
Building

Memory

Hand
Management

Ac2on
Programming

Auc2on/Bidding

Partnership

Possible,
meaningful
game
mechanics:

Take
Inspira/on!

TILE PLACEMENT
60
Tile Placement games feature placing a piece to
score points, based on adjacent pieces or pieces in
the same group/cluster, taking into consideration
non-spatial properties like color, "feature
completion", cluster size etc.

PATTERN RECOGNITION APPLIED TO
GWAPS
61
Kawrykow A, Roumanis G, Kam A, Kwak D, Leung C, et al. (2012) Phylo: A Citizen
Science Approach for Improving Multiple Sequence Alignment
Purpose:
Used
to

arrange
the
sequences

of
D.N.A,
R.N.A
or

proteins
to
iden/fy

regions
of
similarity

Game
Mechanic:
align
the
sequence
contained
in
each
row
in

order
to
obtain
the
greatest
number
of
columns
with
matching

colors.

LINE DRAWING
62
Games that make use of this mechanics
involve drawing drawing of lines in one
way or another.

LINE DRAWING
APPLIED TO GWAPS
63
Purpose:
Segment
images

Game
Mechanic:
draw
the
shapes
of
objects
in
a
provided
image

in
order
to
make
the
other
players
guess
the
underlying
object.

MEMORY
64
Games that use the Memory mechanic
require players to recall previous game
events or information in order to reach
an objective.

MEMORY APPLIED TO
GWAPS
65
Purpose:
Find
similar
Images

Game
Mechanic:
Exploit
the
visual
memory
of

players
to
ﬁnd
images
which
are
similar.

PARTNERSHIP
66
Games
with
partnerships
oﬀer
players
a
set
of
rules
for

alliances
and
teams.
Partners
are
o_en
able
to
win
as
a

team,
or
penal/es
are
enforced
for
not
respec/ng
alliances.

PARTNERSHIP
APPLIED TO GWAPS
67
Purpose:
Collect
“common-‐sense”
fact
for

speciﬁc
words.

Game
Mechanic:
provide
hints
to
the
partner

in
order
to
let
him
guess
the
secret
word

GWAPS VALIDATION
MECHANICS
68
Bad
solu/ons
due
to:

incompetence

mischief
behavior

plain
mistakes

Find
or
adapt
game

mechanics
in
order

to
validate
the
results.

GWAP FORMAL MODEL
69
Based on Mathematical Modeling of social games, Irwin King, SIAG2009
is the set of uTask that are associated with the
problem, with the uTask defined as we have
previously seen.
is the answer domain; it contains the set of
possible cer9ain answers fi (based on the input
D) if the out3ut is contained within a defined
domain, or empt2 if the answers are provided by
the players
is a f@nction that deter=ines if a par9icular
out3ut is a valid solution to the problem
is the GWAP Domain, that maps the input to
the out3ut of a GWAP

MULTIPLAYER: INPUT AGREEMENT
70
Same
or
different
input

Based
on
each
other’s
hints,
player

must
agree
if
they
are
dealing
with
the

same
input

Given
a
correct
answer

set
for
the
problem

players
have
a
set
of

poten/al
outputs

which
cardinality
influence
the

probability
of
agreeing
upon
the

input

Results Submission:
One Trial
Suggested for:
Subjective Information
Same
or
different

input,
same
roles

INPUT AGREEMENT: TAG A TUNE
71
Same
Input:

Audio
file

Two
different

shared
sets
of

descrip/ons,

one
for
each

player

Result:
same

input
or

different
input

Results

Submission:

One
Trial

MULTIPLAYER: OUTPUT AGREEMENT
72
Same
Input

Agreeing
on
common
output
without

communica/on

Same
Input,

Same
Roles.

Given
a
correct

answer
set
for
the

problem,

players
have
a
set
of

poten/al
outputs

the
probability
to
obtain

an
agreement
depends

on
the
shared
output

Results
Submission:

Mul/ple
Trials

Suggested
for:

Objec/ve
Informa/on

OUTPUT AGREEMENT: ESP GAME
73
Same
Input:

Image

List
of
possible

guesses,
hidden

to
the
other

players

Result:
Meaningful tag
for the provided
image
Results

Submission:

Mul/ple

Trials

MULTIPLAYER: INVERSION PROBLEM
74
One
players
knows
the
whole
problem

and
gives
hints
to
the
other
players.

The
others
try
to
unveil
the
secrets

based
on
the
hints.

Input
to
just
one
player

the
output
of
one

player
is
the
input

for
the
others

The
correct
answer
set
for

the
problem
is
unknown

The
probability
of
guessing

depends
on
the
cardinality

of
the
hints

Result:

User
generated
content,

solu/ons
for
complex
data

structures

Results
Submission:

Mul/ple
Trials

INVERSION PROBLEM: IMAGE
LABELING GAME
75
Describer:
Image
Seekers:
Textual
description
given by the
describer
Results
Submission:
Multiple trials
Results:
Meaningful
descriptions
for the image

INSUFFICIENT PLAYERS:
PRERECORDED GAMES
76
Fake players based on the logs of previous matches
Pros: The gwap can be played anytime
Cons:
The actions in the game has to be modeled and
stored
Inversion problem games can be difficult to simulate
Requires a bootstrapping phase to acquire the initial
traces

SINGLE PLAYER: ALGORITHMIC
EVALUATION
77
Automa/c
algorithm
able
to
par2ally
assess
the

quality
of
the
provided
results.

For
each
instance
of
the
game
the
input
is

provided
to
just
one
player

The
correct
answer
set
for
the
problem
is
deﬁned

The
probability
to
obtain
meaningful

results
is
applica/on
speciﬁc,based

on
the
algorithm
that
is
used
to

make
the
evalua/on

ALGORITHMIC EVALUATION: TYPE
ATTACK
78
Jovian,
L.T.;
Amprimo,
O.,
"OCR
Correc8on
via
Human

Computa8onal
Game,"
System
Sciences
(HICSS),
2011
44th

Hawaii
Interna8onal
Conference
on
,
vol.,
no.,
pp.1,10,
4-‐7

Jan.
2011

Improving
the
results

of
OCR
in
digitalizing

ar/cles
old

newspapers.

Human
players
transcribe
snippet
of
text
and
their

performance
is
measured
by
comparing
the
number
of

similar
words
in
the
player’s
output
and
the
text
from

the
library’s
OCR
transcrip/on.

HYBRID
79
Combines
the
previous
approaches

Input
data
based
on
previous
results
or
user
generated
content

Inputs
to
one
or
more
players
with

same
or
diﬀerent
roles

Answer
set
depending
on
the
nature
of

the
problem
or
provided
by
a
player

The
probability
of
obtaining
a

solu/on
depends
on
the

combina/on
of
the
strategies

HYBRID: SKETCHNESS
80
Input:

One
image
to
a
player

Hints
to
the
others

Segments
traced
by

one
player,
possible

tags
submiVed
by

the
others

Result:

Meaningful
tag

+
segmenta/on

Results
Submission:

Mul/ple
Trials

INSTANCE MATCHING
81
How to assign tasks (instances) to players?
• Random
Strategy:
assign
a
random
task

Pros:
Easy
to
implement

Cons:
Does
not
take
into
considera/on
player
skills

• Past
History:
assign
a
task
based
on
past
performances

Pros:
Can
improve
the
quality

Cons:
Keeping
track
of
gaming
history

Performance
measures
must
be
defined

• Selec2ve
Assignment:
assign
a
specific
task
based
on

skills
and
past
performances

Pros:
Can
improve
the
quality

Cons:
Same
as
Past
History

Needs
profiling
of
the
player
to
assess
skills

PERFORMANCE EVALUTION
82
Luis von Ahn and Laura Dabbish. 2008. Designing games with a
purpose. Commun. ACM 51, 8
Throughput = average number of problem instances
solved per human hour
ALP = average (across all people who play the game)
overall amount of time the game will be played by an
individual player
Expected contribution = throughput multiplied by
ALP.

ENGAGEMENT STRATEGY
83
Core activity must be already engaging for the user.
Gamification involves defining clear and definitive objectives
that last even past the gaming experience, and are either set
by, or negotiated between the user and the game along with a
progressive path of short and intermediate goals leading to it.
Onboarding
Retention
Virality
Social Dimensions
Additional Mechanics

POINTS
84
Points
or
Player
Scores
are
a
numerical
value
that
represents
a

measure
of
the
skill
of
a
player.

• 
Immediate
and
las/ng
feedback

• 
External
display
of
progression

• 
May
determine
the
win
state

• Connec/on
between
progress
in
the
game
and
rewards

Galli, L., Fraternali, P. “Achievement
Systems Explained“ SGSC2012,
Singapore
Werbach, K. & Hunter, D.
For the Win: How Game Thinking
Can Revolutionize Your Business
Wharton Digital Press, 2012

LEADERBOARDS
85
A
Leaderboard
is
an
ordered
list
of
players
based
on
the
scores

they
have
obtained
in
a
speciﬁc
game
or
system.

• 
Relates
the
performance
of
a
player
to
the
others

• 
Fosters
compe//on
and
par/cipa/on

• 
Risky:
May
be
demo/va/ng.

Singapore
Werbach, K. & Hunter, D.
For the Win: How Game Thinking
Can Revolutionize Your Business
Wharton Digital Press, 2012

ACHIEVEMENTS AND BADGES
86
An
Achievement
is
a
set
of
tasks,
defined
by
a
designer,
for
the

player
to
fulfill
so
to
achieve
a
milestone
and
track
the

progress
in
a
system.

A
Badge
is
an
ar/fact
associated
to
the
comple/on
of
an

achievement
and
given
to
a
player
a_er
its
comple/on,
or,
in

gaming
terms,
a_er
“unlocking
the
achievement”.

• 
Define
goals

• 
Onboarding
tool

• 
Visual
markers
for
reputa/on,

• 
Provide
las/ng
rewards

Singapore

THIS IS JUST A GLIMPSE OF
WHAT GAMIFICATION IS...
87

EXAMPLE: ZAMZEE GAMIFICATION
88
Personalization Rewards
Challenges
Social Status

CASE STUDY: FASHION TREND MINING
89
Problem statement: segment fashion images for mining trends based
on visual features of garments (e.g. color and texture)
Use case: identifying trends in collections of images of people
and garments
Applications: retrieving similar garments, inspect clothing trends
in image collections, analyzing trends change in the years
Color
descriptors
Texture
descriptors
coarse
(sub-)image
similarity

THE FASHION TREND MINING PIPELINE
90
Male, 24 Female, 22Female?, ??

REQUIREMENTS SPECIFICATION
91
Task Description: Recognize if a particular
garment is present within a picture or define a
new one and outline its contours.
Input Objects: A fashion image, an optional
tag defining the garment to identify.
Aggregation Strategy: assign a value of 0 to
each pixel outside the contour and 1 to each
pixel contained within the contour, sum all the
contribution and apply a threshold based on the
number of players.
Output Data: For each submitted task the game
has to provide the contour of the garment within
the image (Polyline) and a tag defining the
garment that has been segmented

GWAP DESIGN
92
Solution Mechanics:
Pattern Recognition
Established genre:
Draw and Guess
Inversion Problem Mechanic
PLAYERS:Number >=2
Multilateral Competition
Two different roles:
Sketcher: has to draw the contours
of the stated garment
Guesser: has to guess the garment
drawn by the sketcher

PLAYER ROLE: SKETCHER
93
●  The only player to see the
low confidence image
●  “May” be asked to provide a
tag for the image
●  Is asked to draw the contour
of the object for which the
tag is provided within the
allotted time
●  Goal of the Sketcher is to
let the other players guess
the tag within a time slot
without providing
any other hints than the
contour

PLAYER ROLE: GUESSER
94
●  Any other player in the game
●  His/Her goal is to guess the
object for which the Sketcher
has provided the contour
●  Not allowed to draw on the
whiteboard, just to type
in the chat box the probable
answer as fast as possible
●  Scoring:
●  Sketcher: 10 pts + 1 for each guesser
●  Guesser: 10 pts to the first, then
decreasing down to five

FRAMEWORKS: WEB GAMES
95

FRAMEWORKS: GAMIFICATION
96
Google Play Game
Services
Mozilla Open Badges
User Infuser

THE CUBRIK PROJECT
97
●  CUbRIK is a research project
financed by the European Union
●  Goals:
●  Advance the architecture of
multimedia search
●  Exploit the human
contribution in multimedia
search
●  Use open-source
components provided by
the community
●  Start up a search business
ecosystem
●  http://www.cubrikproject.eu/

CUbRIK Tutorial at ICWE 2013: part 2 - Introduction to Games with a Purpose

More Related Content

What's hot (19)

Viewers also liked (18)

Similar to CUbRIK Tutorial at ICWE 2013: part 2 - Introduction to Games with a Purpose (20)

More from CUbRIK Project (12)

Recently uploaded (20)

CUbRIK Tutorial at ICWE 2013: part 2 - Introduction to Games with a Purpose