Open Source IAT - SPSP 2013

An Open Source IAT for
Online Data Collection
Winter Mason
Stevens Institute of Technology

Implicit Association Test (IAT)
 The Implicit Association Test (IAT; Greenwald, McGhee, &
Schwartz, 1998) was devised as a way to measure implicit
attitudes towards categories of objects or people.
 Since then it has been used extensively in the field
(McConnell & Leibold, 2001; Conrey, et al., 2003; Nosek, 2005,
2007)
 Including online applications (c.f., Project Implicit:
http://projectimplicit.com).
 However, until now, the means to run the IAT has required
commercial software such as e-Prime for laboratory studies
and has not been widely and freely available for online
research.

Open Source software
 Open-source software is software that has been developed by
and/or shared with a community of developers
 Firefox, Linux, R, etc., etc.

 Based on the idea that sharing resources reduces unnecessary
duplication of effort and increases social efficiency

 This idea should be applied to all aspects of science
 Sharing methods and materials reduces duplication of effort and
facilitates replication
 Sharing data allows transparency and reproducibility
 Also see openscienceframework.org

Requirements
 Experimenter:  Has been successfully tested
 A web server that runs PHP with
v4.0 or greater  Mac OS 10.5, 10.6,
 Windows XP, Vista
 Ubuntu 11.
 Participant:
 A web browser that allows  It has also been successfully
Javascript tested with
 Mozilla’s Firefox
 Google Chrome
 Internet Explorer 8 & 9.

A list of available IAT
templates to modify or
make active

The currently active IAT;
Participants will complete
this version when they go
to the main IAT page.

Choose whether
participants are
presented with the
results of their IAT at
the end of the test.

Select the category
you want to edit

This is the name of the
category seen by the
participants.

This is the name of the
category that is stored
in the output data
file.

Choose whether the
items in the category
will be text or images.

With images, you can
see a preview of the
image next to the file
name.

…and update the
thumbnail image with
this button.

With text items, the
thumbnails and
“update images”
button are hidden.

Delete an item with
the „x‟ button

Once you have made
all of your changes,
don‟t forget to save!

If you select another
template or create a
new template,
unsaved changed will
be lost!

To change the IAT
participants see,
select the template
and press this button.

Create a new template with
this button.

Make sure you‟ve
added items for all 4
categories, and don‟t
forget to save!

Enter a unique identifier for the
participant (a random string is pre-
filled). The output data file will be
associated with this ID. You can
also use this to match with a
Turker‟s submitted HIT.

The name of the
active template is
shown here.

The category labels for
Category A and B are shown
in the first trial

The templates are stored in a folder with the same
name as the template, and image files are stored
in the “img” folder inside the corresponding
template folder.

The category labels for
Category 1 and 2 are shown
in the second trial

If the participant gets it wrong,
the X appears until the correct
key is pressed.

Output
Trial Round Category Item Errors RT
0 1 F 4 0 560
0 2 M 3 0 432
0 3 F 0 1 913
… … … … … …

The output is stored in the “output” folder in the template‟s folder as
a comma-delimited file. The name of the file follows the pattern
[Template Name]-[Subject ID]-YYYY-MM-DD-HH-mm. For example,
Race-JohnDoe-2013-01-22-18-52.csv.

Output
0 1 F 4 0 560
0 2 M 3 0 432
0 3 F 0 1 913
… … … … … …

There are 7 Trials in the standard IAT

Output
0 1 F 4 0 560
0 2 M 3 0 432
0 3 F 0 1 913
… … … … … …

There are 20 or 40 round per trial,
depending on whether it is a
practice trial or not

Output
0 1 F 4 0 560
0 2 M 3 0 432
0 3 F 0 1 913
… … … … … …

The character(s) used here to
indicate the category are defined in
the template.

Output
0 1 F 4 0 560
0 2 M 3 0 432
0 3 F 0 1 913
… … … … … …

The item number refers to which
particular item was presented. The
order of item presentation is
randomized. The number indicates
its order in the template (starting
from zero).

Output
0 1 F 4 0 560
0 2 M 3 0 432
0 3 F 0 1 913
… … … … … …

This is the number of errors the
participant made on that round.

Output
0 1 F 4 0 560
0 2 M 3 0 432
0 3 F 0 1 913
… … … … … …

This is the participant‟s response
time in milliseconds for that item on
that round. The final IAT score is
determined by these reaction times.

Validation
 The standard deviation of
system error was 29.4ms, with
fewer than 4% larger than 50ms
and the largest error recorded
at 167ms.

 The typical standard deviation
of a user's reaction time is
approximately 300ms, as
reported in Greenwald, et. al
(1998, 2003) and observed in
the replications reported here.

 The error introduced by the
system is an order of
magnitude less than user
error.

Replicating prior work
 63 participants recruited from Amazon‟s Mechanical Turk

 46.7% female

 median age of 29 (mean age 31.8)

 median household annual income less than $30k

 median education of a Bachelor’s Degree

 55.5% from India, 38% from the U.S., and the remainder were
from Pakistan, Lithuania, Algeria, and Sweden.

Flowers-Insects IAT
Congruent first Incongruent first
(Flowers-Good / Insects-Bad) (Flowers-Bad / Insects-Good)

Results
Congruent Incongruent
IAT t-value p d'
(ms) (ms)
Flowers-Insects
727.52 1003.12 26.342 ~0.00 0.65
(Congruent first)
Flowers-Insects
750.88 850.85 10.3841 ~0.00 0.26
(Incongruent first)

Race 771.1 903.88 12.8621 ~0.00 0.32

Future extensions
 Allow easy duplication of templates in experimenter
interface

 Data storage in a database instead of text files

 Automatic aggregation and display of results in
experimenter interface

 Suggestions? Requests?

Thank You!
http://github.com/winteram/IAT

Open Source IAT - SPSP 2013

More Related Content

Viewers also liked (18)

Similar to Open Source IAT - SPSP 2013 (10)

Open Source IAT - SPSP 2013

Editor's Notes