2000 - CSCW - Conversation Trees and Threaded Chats

50
Conversation Trees and Threaded Chats
Marc Smith & JJ Cadiz
Collaboration & Multimedia Group
Microsoft Research
One Microsoft Way
Redmond, WA 98052 USA
+1 425 936 6896
{masmith, jjcadiz}@microsoft.com
Byron Burkhalter
University of California, Los Angeles
2201 Hershey Hall
610 Charles E. Young Drive
Los Angeles, CA 90095-1551
+1 408 445 9779
burkhalt@ucla.edu
ABSTRACT
Chat programs and instant messaging services are
increasingly popular among Internet users. However, basic
issues with the interfaces and data structures of most forms
of chat limit their utility for use in formal interactions (like
group meetings) and decision-making tasks. In this paper,
we discuss Threaded Text Chat, a program designed to
address some of the deficiencies of current chat programs.
Standard forms of chat introduce ambiguity into interaction
in a number of ways, most profoundly by rupturing
connections between turns and replies. Threaded Chat
presents a solution to this problem by supporting the basic
turn-taking structure of human conversation. While the
solution introduces interface design challenges of its own,
usability studies show that users’ patterns of interaction in
Threaded Chat are equally effective, but different (and
possibly more efficient) than standard chat programs.
Keywords
Chat programs, turn-taking, conversation, computer
mediated communication, synchronous communication,
persistent conversation, human computer human interaction
INTRODUCTION
Chat is an old and increasingly popular form of computer-
mediated communication. Commercial on-line service
providers like America Online and non-commercial
networks like Internet Relay Chat provide a myriad of chat
rooms filled by millions of people daily. Instant messaging
programs from AOL, ICQ, Yahoo, and MSN are becoming
increasingly popular. 430 million instant messages are
exchanged each day on the AOL network, and 330 million
are exchanged on ICQ [12]. This form of communication is
likely to increase as cell phones and wireless handheld
computers make mobile messaging even more prevalent:
wherever cell phone short message system (SMS) service is
available, its use is rising dramatically. Chat is here to stay.
Although these chat programs are popular for informal
interaction, several companies are now bringing chat to the
business world [2]. However, chat has not evolved much in
the past twenty years and remains poorly suited for holding
complex discussions. Innovations in chat have mostly
ignored this problem. There have been a number of chat
systems released by commercial Internet software
companies that have integrated a variety of 2D and 3D
graphical representations with standard chat [1, 4, 5, 14,
23]. However, few have altered the way chat organizes
people’s exchanges of messages in a positive way, making
chat even less easy to comprehend. The recent explosion of
“Instant Messages” and “Buddy Lists” has not changed the
underlying structure of chat either.
In this paper, we will discuss the core problems we see in
chat and describe the ways this guided our design of
Threaded Chat. In addition we report the results from a lab
study that tested the usability of Threaded Chat in contrast
to standard forms of chat with eighteen small groups
engaged in a decision-making task. We discuss the
challenges raised by the design of Threaded Chat and
suggest future directions for improvement of systems to
support persistent computer-mediated interaction.
COMPARING CHAT AND SPOKEN CONVERSATION
Chat may be a form of computer-mediated communication
that closely resembles spoken interaction, but in contrast to
spoken interaction, chat is poor at managing interruptions,
organizing turn-taking, conveying comprehension, and
resolving floor control conflicts. Studies of chat from a
variety of fields (including sociology, communication,
CSCW, HCI, and linguistics) share a focus on the
challenges and ambiguities chat introduces into the normal
mechanisms of social interaction.
Conversation Analysis (CA)—the sociological study of the
structures of ordinary face-to-face and spoken interaction—
is of particular value when seeking ways to improve chat.
CA’s study of naturally occurring conversation reveals that
people use a suite of fine tuned, ordinary techniques for
maintaining coherent and understandable spoken
conversations. Spoken conversations have turn and
response structures governed by a set of simple rules that
organize how groups of people exchange turns of talk.
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
CSCW’00, December 2-6, 2000, Philadelphia, PA.
Copyright 2000 ACM 1-58113-222-0/00/0012…$5.00.
97

51
Sacks et. al. [18] argue that turns are valuable commodities
that require an orderly allocation system:
For socially organized activities,
the presence of ‘turns’ suggest an
economy, with turns for something
being valued—and with means for
allocating them, which affect their
relative distribution, as in
economies.
Using simple turn-taking rules, people are able to sustain
spoken conversations across a wide variety of topics where
there is almost always one party talking at a time.
Interruptions and overlaps do occur but are brief, and
transitions between speakers commonly occur without gap
or overlap [18]. In contrast, in its most common form, chat
organizes turns in order of their arrival at a central server,
not in the order of turn and response in which they were
constructed. This undermines the techniques people use for
organizing coherent conversations [9]. The result is an
inclination for confusing exchanges of short messages in
ambiguous order. This makes chat a poor decision-making
tool and knowledge store and reduces its value for meetings
and presentations of detailed ideas.
Computer-mediated conversation has the potential to
transform the constraints of the economy of spoken
interaction in more positive ways. Our inability to listen to
two or more people speaking at the same time for very long
limits the number of possible turns available in any spoken
conversation. In contrast, chat may be less restricted than
spoken communication since more than one person may
construct a message at the same time, and reading can be
quicker than listening. Nonetheless, turn-taking systems for
spoken discussions allow for more coherent and productive
conversations than standard chat programs. Thus, the
properties of spoken conversation systems offer guidance
for the design of text chat.
CA directs attention towards improving the turn-taking
system used in the exchange of chat messages. Threaded
Chat presents a possible solution to this problem by
supporting a synchronous form of the turn-taking structure
found in asynchronous threaded discussion boards like
Usenet. Systems like Usenet and a vast number of
discussion boards on web sites allow for the creation of
extensive discussion trees composed of message (“post” or
“article”) turns and responses linked together. These
systems have predominantly been used as a form of
asynchronous interaction in which delays of hours or days
between turns and responses are common. While these
systems suffer from problems of their own [13], discussions
of complex ideas can be developed over time with
responses clearly linked to the messages they are in reply
to. In Threaded Chat, we have modified this structure to
make it more accommodating to both synchronous and
asynchronous use.
FIVE CORE PROBLEMS WITH TEXT CHAT
Research rooted in the sociological study of conversation
has identified and addressed some of the major issues with
standard chat programs [7, 8, 9, 15, 21]. These findings
lead us to identify five main flaws in existing chat systems:
Lack of links between people and what they say
Chat programs present each participant’s messages in a way
that makes it hard to differentiate speakers. The high
turnover of participants in many chat rooms further
aggravates this problem. Many systems address this issue
in one form or another. Some chat clients provide ways of
associating a color or font with particular people. More
recently, systems have focused on awareness of presence of
people in the room [3, 20], representations of the timing of
the conversation [21], and improved visualization of
conversations [20].
No visibility of listening-in-progress
In chat, participants do not receive moment-by-moment
information about the reaction of those who are listening to
them. This means that turns cannot be altered as they
unfold, increasing the likelihood that they will be
misunderstood or taken in the wrong way. Without
indications of listening, chat systems loose a great deal of
their sense of social presence.
Some experimental systems have addressed this issue.
Erickson et al.’s Babble [4] addressed this problem by
presenting a “social proxy”, a graphic design that
represented the activity of people with the application. This
allowed people to have an intuitive sense of who was
recently active but lacked the granularity to present
reactions to turns-in-progress.
Lack of visibility of turns-in-progress
Chat systems only transmit turns when users press the
ENTER key. While some systems do transmit messages
keystroke-by-keystroke (i.e. the Unix program “Talk”) most
do not. As Garcia notes, the result is that the process of
message production is separate from message transmission.
Chat is not truly synchronous: it has a sporadic rhythm in
which fully formed turns pop out in a single moment
instead unfolding in real time. Chat lacks the “mutual
availability of utterances-in-production” [8].
In contrast, the moment-by-moment surveillance of others
in spoken conversations allows people to be highly
sensitive to small variations in timing. For example, when
declining an invitation or disagreeing with another’s
assessment, people will often slightly delay the beginning of
their turn. The delay projects a dispreferred response [15]
(a response the listener would not like), allowing the
original assessor to downgrade or alter the assessment in
order to maintain agreement. People are able to connect
turns so quickly and assess the gaps between them because
speakers project where their turns are heading and listeners
recognize those projections as the talk unfolds.
98

52
Delays in chat resulting from typing difficulty or the other
user leaving the room can easily be misinterpreted as a
dispreferred response. Furthermore, delays encourage users
to type additional turns (which may modify their initial turn
or start a new topic of conversation) instead of waiting.
Garcia [8] found that timing and sequencing distortions
introduced by standard chat systems meant that a significant
portion of chat turns were used to clear up confusion caused
by prior turns.
Vronay’s Flow Chat [21] explicitly presented the stream of
time and the resulting interleaving of turns of chat. Flow
Chat placed each user’s text on a separate vertically stacked
parallel track. While text entered by the user was not
displayed until the turn was completed a colored band was
extruded from the right side of the display to indicate when
the user began typing and how long they had been
composing the message. Once entered the text was
displayed in the color bar, which then continued to slide
towards the left of the display on its track. While this
clarified the sequential ordering of turns, it did not provide
any other way to indicate a link between two turns. In large
groups this means that links between turns separated by
many tracks were difficult to associate.
Viegas and Donath’s Chat Circles [20] approach this from a
different direction. Chat Circles presents each user as a
colored circle that expands with the amount of text entered
by the user. Circles then slowly shrink in size as the text
fades. The timing of turns is thus visible and turns-in-
progress are presented as expansions in the size of the
circle. This view of the conversation lacks a historical
component as turns evaporate over time. As a result the
application has an alternative historical view, which
visualizes the conversation along a vertical time line cross
marked with lines indicating the timing and size of each
user’s turn. This is in many ways an alternate form of
Vronay’s Flow Chat that shares its limitations.
Microsoft’s MSN Messenger is one commercial product
that partially addresses the problem of seeing turns-in-
progress. When others are typing, “[name] is typing a
message” appears at the bottom of the window. Although
this alleviates some of the problem by providing a binary
indicator of typing, it does not entirely solve the problem
because users cannot see exactly what others are typing
until the ENTER key is pressed.
Lack of control over turn positioning
Much of the work in conversation coordination relates to
shaping a turn’s meaning based on its location. However,
the techniques used to accomplish this in spoken interaction
are undermined in chat conversations [9]. Standard forms
of chat position turns based only on the time that the
ENTER key is pressed, which often ruptures the links
between turns and their replies. “Participants in QS-CMC
cannot assume that their attempts to be a ‘first poster’ will
result in the message they are typing being placed adjacent
to its intended referent,” writes Garcia [8].
In standard forms of chat, ownership of the floor is only
known when a turn is completed, at which point a race
begins to finish one’s own thought, which is newly fitted to
the recently emerged turn. This twisted set of
conversational rules has two ramifications: first, one can
only begin to fit a “next” turn after the last turn has been
displayed in its entirety, and second, there is a preference
for short turns because one must press the return key in
order to secure the floor. Therefore, extended turns, which
can allow more complex material to be discussed, are much
less frequent.
For example, consider the following chat interaction:
1 Larry: boy do we need to work on our
interview skills....
2 James: who's conducting the
interviews, anyway?
3 Scott: Yes
4 James: okay...
5 Larry: All of us
Notice that James and Scott are entering both turns
simultaneously. Each turn is fitted to Larry’s initial turn.
Although Scott’s turn “Yes” appears immediately after
James’ turn “who’s conducting the interviews, anyway?” it
obviously does not fit as the next turn. Similarly, Larry’s
turn “all of us” follows but does not fit the prior turn of
“okay…”. The only way users can make sense of the turn is
to scroll up and find a candidate “prior turn.” That people
can do this is itself interesting, but the procedure is time
consuming (and the conversation continues while this is
done). The result is that transcripts of chat conversations
are often confusing and demand significant effort to read.
Babble [4] addresses this by designing for an expectation of
slower interaction rates than typically found in chat. The
slower rate allows users to have greater certainty that their
turn will occupy the position it was crafted for. As a result,
short expressions of concurrence (ex. “I agree”, and “yes”)
are possible and meaningful. Sequencing problems do
sometime occur, however, and are likely to increase if
Babble is used more synchronously.
Lack of useful recordings and social context
Chat rooms are social spaces that never develop a social
history [3]. In practice, most chat rooms are not publicly
persistent: their content evaporates as soon as it scrolls out
of each user’s history buffer. This lack of persistence
means that most chat spaces do not accrete a social history.
Groups do use other media (for example, web pages) to
create durable artifacts of their interaction, but the chat
room itself does not change as a result of the activity within
it. Even if logs are maintained, as noted above, the
resulting transcript is often nearly unintelligible.
This usually is less of a problem during the conversation
than several days or months later when one tries to review
chat logs. For instance, when a chat conversation occurs, if
two turns appear within a tenth of a second of each other, it
is probably clear to an attentive participant that the second
99

53
turn was not intended to be a reply to the first. However,
timing cues are missing from most history logs. Thus,
ruptured and jumbled turn sequences make the conversation
log ambiguous and unreliable as records. (This problem
can be addressed by including timestamps with chat logs,
but reconstructing the events of a chat room using
timestamps is tedious.)
This has two implications. Having no useful recordings of
chat conversations is a significant obstacle in workgroups
and business environments, particularly when used in
decision-making processes. It also means that chat
programs demand full immersion to remain comprehensible
to their users. When users look away or try to maintain
peripheral awareness many find it difficult to catch up with
conversations.
THREADED CHAT
Threaded Chat addresses the problems of confusing history
logs, lack of social history, and the rupture of turn
sequences in standard chat rooms. Threaded Chat departs
from traditional chat in a number of ways by bridging the
gap between threaded asynchronous discussions and
synchronous chats. The Threaded Chat user interface is
displayed in Figure 1. All chat turns are structured as a
tree, similar to the Microsoft Windows Explorer interface to
the file system on a computer’s hard disk, and similar to an
idea proposed by Herring as a way to make chat
conversations more coherent [9]. The key element of this
structure is that turns are organized into turn and response
structures called threads that can grow to any size. Thus,
proper use of Threaded Chat eliminates the possibility of
ruptured sequences of turns: turns are linked directly to the
turn they are intended to respond to. Even if a turn is
Figure 1: The Threaded Chat user interface. Users chat in the top portion of the window while participation information
is displayed at the bottom.
100

54
misplaced, it can be dragged and dropped to the correct
location. Turns can also be edited or deleted.
To chat, users click on the turn they want to respond to and
being typing. Pressing return completes the turn. When a
user begins to enter text, their name and a placeholder
message (“Entering Text”) appears to all other users. When
the return key is pressed, the entire message becomes
visible to everyone.
As turns are entered, they are displayed to other users in a
bold font. Over time, the font fades to gray so that most
recently added turns stand out clearly. This feature is
especially important since Threaded Chat does not structure
turns in order of arrival (a point we return to below). Turns
are unbolded and marked as read when clicked on, replied
to, or cursored over with the arrow keys. As a turn is
replied to, the count of the number of replies and unread
turns beneath it are displayed.
Selecting the room node at the top of the chat room and
entering text creates a new top-level thread, which is
highlighted with a colored background. Top-level turns are
typically the major topics of conversations, thus they are
distinguished from other turns.
The tree structure of Threaded Chat provides users with the
ability to collapse any branch of the conversation if they no
longer wish to pay attention to it. For example, users may
collapse discussion branches that no longer concern them or
that have come to a conclusion and are no longer pertinent.
If additional turns are added to a collapsed thread the count
of unread child turns is incremented.
The bottom of the Threaded Chat window contains
information about the conversation participants.
Information includes time of entry, number of entries
(labeled “sessions”), and time of exit. Basic statistics about
the number and types of turns are also displayed. These
statistics persist from session to session, and users remain in
the list even when they are not present (although they are
marked as not currently active). This information is useful
for providing a sense of history and context for the room.
Threaded Chat automatically labels turns that are likely to
be questions or answers. If a question mark is found in the
text of a turn, the turn is tagged with a “Q”. All replies to
questions are tagged with an “A”. Numbers of question and
answer turns are tracked in the social accounting pane.
Turns can be edited, deleted, or dragged and dropped to
different places in the tree. Although this can be a helpful
feature, it also raises the possibility of abuse. Thus, each
Threaded Chat turn has permission properties based on an
extension of the Unix user/group/world model. These
permissions are accessed by right clicking on a turn and
allow a turn’s author to determine who can see the turn,
reply to it, delete it, and extend the turn’s permissions.
Only authors of a turn can edit the turn’s text.
Turn authors are also the only people who can modify a
turn’s permissions, although owners of turns higher up in
the tree may override the rights. For example, if person B
replies to person A and specifies a set of rights on the reply,
person A could override person B’s permissions by
specifying rights on the original turn. Users retain the
power to override permissions of the turns that are replies
to their turns, including the power to delete or move the
entire thread to another location. The first person to start a
big conversational branch wields significant power over it.
Using Threaded Chat’s permissions, it is possible to have a
private chat in the middle of a public room, or to have a
public discussion with a select group without possibility of
interruption from others. It also means that users can enter
a turn and determine who may see and reply to the text.
TESTING THREADED CHAT
Given that Threaded Chat is designed to address some of
the key problems with standard chat, we conducted a user
test to see if the design was successful.
Specifically, because proper use of Threaded Chat
guarantees that turns will always be placed in their intended
context, we expect that:
• Threaded Chat will support better topical
coherence than standard chat. Users should be
able to maintain coherent sequences of
conversational turns more easily.
• Transcripts of a Threaded Chat discussion will be
easier to comprehend than transcripts from a
standard chat room.
• Turns will be longer than those found in standard
chat rooms. People will know that they do not
have to rush to get their turn in before someone
else types a message, thus Threaded Chat turns
should use more words and/or characters.
• Fewer repair statements will be made. As a result,
we expect participants will produce fewer turns
using Threaded Chat when compared to plain chat.
• There should be a more balanced level of
participation among people using Threaded Chat.
With standard chat programs, people frequently
abandon their turns when they are not able to
finish before others enter turns that change the
conversation context. Threaded Chat guarantees
context for a turn, thus there’s no reason to
abandon turns. We believe this effect will be
especially true for slower typists.
Experimental Methodology
70 participants were recruited for a lab study to test
Threaded Chat. Participants were grouped into eighteen
teams of four; however, due to no-shows, eleven groups had
only three people. Participants received a free Microsoft
software product for their time.
All participants had used a chat program at least once in the
past year, were comfortable with typing, had graduated
101

55
from high school, and were at least intermediate computer
users. Participants were recruited such that the pool was
diverse in terms of age, gender, and occupation. The pool
had 38 men and 32 women. The average age was 39 with a
standard deviation of 10.
Participants used three different chat programs for this
study. Threaded Chat and a “standard” chat program were
used. In addition, this study was combined with a study of
another experimental chat program, LeadLine. (A report of
the LeadLine study is available in these proceedings [6]).
The order in which the three chat programs were used was
counterbalanced to minimize order effects.
Participants were told they were employees for the same
company and had recently interviewed three candidates for
one job opening. Their task was to chat with each other for
20 minutes and then, as a group, rank the candidates in
order of hiring preference. This task was repeated three
times, each time using a different chat program, a different
set of candidates, and a different job position. In each case,
participants were given unique information about the
candidates so no single participant could correctly rank the
candidates without chatting with other group members.
When using the Threaded Chat program each group started
with a room populated with six initial threads:
Introductions
Review the qualifications for this position
Discuss candidate #1
Final decision: Who should we hire?
Although these threads were made available as guides for
the discussion (similar to a agenda for a business meeting),
users could (and did) ignore them if they wished.
USER STUDY RESULTS
After each session, participants answered a variety of
questions about their reactions to the chat program they
used. On all the measures except one, Threaded Chat was
rated significantly worse than the regular chat program
(Table 1). To a certain extent, this was not surprising given
the early stage of the prototype: some basic user interface
issues had not yet been resolved (for example, lines that
were longer than the screen width did not automatically
wrap to the next line). However, the core concept of
chatting with threads was functional, thus these data
indicate that participants did not think highly of Threaded
Chat. Reasons for this reaction are explored further in the
participant comments section.
Task Performance
Despite the lower subjective ratings Threaded Chat
received in contrast to standard chat, the study showed that
users quickly adapted to the new interface. Performance on
the hiring task did not differ significantly between
programs. Each hiring task was designed such that there
was a correct solution, and each set of candidate rankings
was assigned a score relative to its distance from the correct
solution. The highest possible score for each task was 5
points. Threaded Chat groups had an average score of 3.7
while plain chat groups had an average score of 3.9. This
difference was not found to be significant, even when
taking into account various demographic variables such as
typing speed, level of education, and experience with chat
programs.
Levels of Participation
Even though scores on the task were equivalent for each
chat program used, Threaded Chat did affect the processes
used by groups to reach their decisions.
Groups that used Threaded Chat took fewer turns than in
the regular chat program. Threaded Chat rooms had an
average of 21.7 turns, while the regular chat rooms had an
average of 34.7 turns, which is a significant difference
(t(25.5) = 3.7; p = 0.001). In a regression equation
controlling for various demographic variables, the use of
the Threaded Chat program was the strongest predictor of
the number of turns taken, accounting for 28% of the
variance (t = -5.2, p < .001).
Of course, it could be hypothesized that fewer turns were
taken in Threaded Chat because people took longer turns.
However, this was not the case. The average standard chat
turn was 7.3 words long while the average Threaded Chat
turn was 7.6 words long, which is not a significant
difference (t(3020) = -1.3; p = 0.205).
First sessions only All sessions
Threaded
Chat (n=6)
Standard
Chat (n=6)
p t / df
Threaded Chat
(n=19)
Standard
Chat (n=19)
Level of satisfaction (2 questions) 4.0 5.3 0.001 4.5 / 9.7 3.9 5.7
Perceived quality of discussion (7 questions) 4.5 5.2 0.010 3.4 / 7.8 3.9 5.7
Perception of usability of program (3 questions) 4.3 5.9 0.001 6.2 / 9.9 3.6 6.3
Perceived quality of decision process (5 questions) 5.5 5.0 0.345 1.0 / 9.4 4.8 5.8
Table 1: Results from post-session surveys. Possible scores ranged from 1 to 7 where higher scores are better. On the
left are data examining only the first sessions (where no order effect is possible). On the right are data from all sessions.
Significance statistics are not reported for the data from all sessions due to the possibility of order effects.
102

56
It is possible that Threaded Chat reduced the ambiguity
introduced by standard chat, thus allowing people to enter
fewer, more coherent turns whose meaning was partially
derived from their parent turn. While the frictions imposed
by the Threaded Chat interface may have simply been a
drag on the speed of participation, the equivalent scores on
the task show that Threaded Chat users were equally able to
complete their task using fewer turns.
Balance of Participation
We also examined the question of whether there was a more
equal level of participation among group members in the
different types of rooms. We used the standard deviation of
number of turns taken by the people in each room as a
measure of equal participation. If everyone in a chat room
took the same number of turns, then a group would have a
standard deviation of zero.
Threaded Chat rooms had a standard deviation of 2.9 while
standard chat room had a standard deviation of 3.9, which
is a significant difference (t(33.7) = 2.7; p = 0.01). Thus,
there was a more balanced level of participation in
Threaded Chat rooms.
The next logical question is whether the more balanced
level of participation in Threaded Chat was due to slower
typists generating more turns, faster typists generating
fewer turns, or a combination of both. Based on typing
speed, we split all the participants at the median into two
groups. Table 2 displays the number of turns that were
made by fast and slow typists in each type of chat room.
These data show that all typists made fewer turns, but the
effect was greater for faster typists.
There are two possible explanations for this phenomenon.
First, the method of interacting with Threaded Chat may
significantly diminish the advantage that fast typists
normally have in standard chat rooms. In a standard chat
room, the interaction loop is: read new text, type, press
return, read new next, type, press return, and so on. With
Threaded Chat, reading new text takes longer because it
appears in various locations. In addition, the interaction
loop is: read new text, take hand off keyboard, move
pointer to turn to reply to, click, type, press return. Thus,
typing speed may no longer be the most important variable
for rapidly generating turns in Threaded Chat room (which
could be verified with a GOMS model [11]).
Second, fast typists may not need to generate more turns. If
Threaded Chat significantly reduces confusion resulting
from ruptured sequences of turns, then the large number of
repair turns documented by Garcia [8] are no longer
necessary. However, when we attempted to code turns as
being repairs or not, we found very few repair turns in both
standard chat and Threaded Chat rooms (only one or two
repairs were found in the initial eight rooms examined).
When reading the transcripts, it occurred to us that the low
number of repairs may have resulted from the exclusive
focus on the single task: The conversation was typically
composed of only one conceptual thread whereas normal
chat rooms are substantially multi-threaded. Thus, we may
not have designed a task in which participants could benefit
considerably from Threaded Chat.
Use of Thread Structures
Although groups focused on only one conceptual task, they
still made extensive use of the thread structures supported
by Threaded Chat. The average turn was 3.79 turns deep,
the deepest one growing to a depth of 21 turns, with an
average depth of 10 turns. Although they are reproduced
too small to be read, the overviews of rooms created in
three of the trials illustrate the range of variation in the
ways threading was used (Figure 2).
Examination of the room logs shows that participants
expected themselves and others to properly place turns,
something that did not occur at all in standard chat rooms.
Thus, a norm for topical coherence was supported:
User A: Hi Glen how are you doing
User A: Hi Glen how do you feel about Joyce
and her abilities?
User B: Oops. Should have said that down here.
Forgot to click on the question.
Participants used the thread structure to create extended
turns by replying to themselves in a series of turns. This
was not the intent of the feature, however it was frequently
employed:
User C: These are my two choices also and in that
order
User C: I think we have to eliminate Joyce
because of her lack or visual graphic arts
level
User C: My final is Steve, Linda and then
Joyce.
PARTICIPANT COMMENTS
Although groups using Threaded Chat were able to
complete the task as well as groups using standard chat,
data from the subjective ratings table indicate that Threaded
Chat still has room for improvement. In this section, we
examine participant comments to discuss this issue as well
as ways in which using Threaded Chat differed from using
standard chat.
First, user feedback identified that having no single point of
focus was the most significant problem for Threaded Chat.
Chat
program
Mean Std Dev Median
Standard 31.2 13.9 28
Threaded 20.5 7.2 19.5
Slow
typists
Total 26.1 12.3 24
Standard 36.7 16.2 32
Threaded 21.8 8.4 20
Fast
typists
Total 29.3 14.9 25
Standard 34.1 15.1 30
Threaded 21.1 7.8 20
All
Typists
Total 27.6 13.6 24
Table 2: The number of turns taken by slow and fast
typists in the different chat rooms.
103

57
“…confusing… knowing where to click” (User 1,
Threaded Chat trial)
“…unclear where to place comments” (2, TC)
“I was looking all over the screen to see what was being
said next” (3, TC)
“Difficult to follow the discussion… Had to scroll too
much” (4, TC)
“Difficult to track new messages” (4, TC)
“…cursor seemed to jump around.” (29, TC)
“I had to move my eyes all over the screen…” (41, TC)
Standard chat has a single point of focus for new material,
which allows users to fix their gaze at the bottom of the
screen to guarantee not missing anything new. But in
Threaded Chat, new material can blossom anywhere in the
conversation tree, requiring frequent scanning to search for
new turns.
Users had suggestions for addressing this problem:
“Add some sound so you know when you have new
incoming text” (5, TC)
The most frequently requested feature was color-coding for
each user to help differentiate speakers.
“Make each person’s text a different color” (5, TC)
Despite these problems, users reported that Threaded Chat
helped them sustain their conversations more than they
could in standard chat, which supported their decision
making process.
“For the task… this was the strongest tool.” (43, TC)
“I liked being able to follow a discussion thread. It
helped keep the idea linear.” (4, TC)
“It was easier to answer questions and reread posted
information.” (9, TC)
“Organizing the information into conversations helped
in the decision making process.” (21, TC)
“I think it’s easier to choose and see what you’re
responding to with threaded chat.” (22, TC)
“…as we used it, we got better, it did allow us to stick to
certain subjects.” (35, TC)
“…easy to review what has been said about a certain
candidate without having to scroll around looking for
it.” (39, TC)
“I liked being able to manage the screen – close up
sections and reopen them. I liked being able to edit
messages after they were sent, and move them to other
more appropriate sections. It was much different from
the noisy conversation style of the plain chat…” (53, TC)
In contrast, when they used the non-threaded versions of
chat they reported difficulties managing the conversation:
“…difficult to follow conversation threads when
multiples are going.” (45, Lead Line trial)
“topics got intermixed…that was confusing.” (53, LL)
“if you are not paying attention, you get lost easily” (25,
Standard chat trial)
Given the nature of the task, many users felt that access to
the history log was essential and that it was poorly
organized in standard forms of chat.
“I wanted at times to review what was said earlier in the
discussion, but that line had scrolled off the screen and it
would have been difficult to keep up with the current
discussion and review previous comments.” (1, LL)
“…to remember the chat, you have to keep paper notes,
the threaded chat was easier in that aspect…” (17, LL)
-1: z #2 TChat User Study
-1: Introductions
10: Debi here
8: Hey there
7: This is a test for me
9: test
9: ok
-1: Review the qualifications for this position
10: Prepares and organizes documents. Is KEY comunications contact. Coordinates meetings, schedules,etc
8: Hmmm... Covers the basics with documents, main person for interoffice communication for managers and tea
m...
10: What do you have on the job description? I listed my major points.
8: Do we all have the same job description? I think we do.. the interviews are just a tad different...
10: don't know about that one, but it sounds like you are correct.
7: Anna has good communication, math and numbers are good, and excellent graphics abilities
7: It sounds as though Anna hits a majority of the key points we are looking for
9: web developing is not a MUST in this position, but thisjob is the central hub of the office
7: Mark has great conflict resolution skills which is important as the main interface for managers and team
members
9: Anna has the people skills, do we all agree on that?
7: I agree on her people skills
9: [Entering Text...]
-1: Discuss candidate #1
10: Does anyone have anything on Anna's organizational skills? She looks really good from my notes--but that pa
rt is missing.
8: Her decision making skills are good.
8: Her level of composure is good
10: D ...
7: I only have here communication skills as very good
8: That may be enough.. this position warrants strong organization and communication.. Anna seems to be
a good fit.
9: anna has the people skills
8: According to my notes, her organizational skills are excellent. Superb. Unmatched.
9: i agree
8: Probably a good candidate. :)
10: I show "fair" on the creativity meter. Excellent verbal communications. Good visual and graphic skills.
7: Mark has good leadership and people skills
9: mark's decision making skills are so/so, and this position needs someone who is confident
7: I show his leadership and people skills to be very good. That is interesting to what you saw
10: U ...
10: Sorry, team. I crashed! Unless someone can convince me otherwise, I am ready to stop viewing Mark as a viab
le applicant.
8: Notes on Mark Williams
8: Web Development: Good
8: Math and Number Skills: Fair
8: Very good with desktop processing and computing...
8: Me thinks Mark is not a good candidate for this position.
10: I agree
9: i agree..he can go make coffee with Steve
10: I'm picky about my coffee!
10: I show that Emily is good with numbers, but her "composure" (that word again) skills are only fair. People
skills are excellent.
7: Emily may make good decisions, but if she gets people really upset in doing so, we can't have that
7: I only show emily having excellent decision making skills and not the people skills
8: It doesn't sound like Emily is a people person.
10: My notes show she is an excellent people person. But she could still be easily upset.
9: Emily has the tech skills, but her communication is not that great
9: how can her "comosure" be fair and communications be excellent? : )
9: oops..composure
10: All I can think of is she get rattled when things aren't going smoothly. Which in my line of wo
rk is never!
8: I have excellent comosure.. :) I learned it when I was a child!
7: E ...
8: My notes show Emily as: Very good leadership skills, and excellent creativity. BUT, her conflict resolution
skills are only 'fair'.
8: Emily doesn't seem to be a good fit!
7: I agree that she could be a runner up, but not a candidate for the position
-1: Final decision - Who should we hire?
7: Base on what we see so far, how do you guys feel about the candidates for ranking??
9: the job need someone with eyes in the back of thier head, and 6arms and can keep up with it all
10: I'd have to go with Anna too. She does look good. Based on my experience with Admins, the conflict reso
lution skill alone are enough to get her the job.
7: Anna does have very good communication skills which are key in this position.
10: I'm ready. Anna, Lisa, Steve.
9: anna looks good
7: I am not sure between Anna and Emily
9: here here..i totaly agree
9: time to vote?
7: I think we can start voting
9: anna, emily
9: Lisa? did i interview for this job? : )
10: Sorry, don't know where that came from!! Anna, Emily, and Steve.
8: Yes you did, but your typing skills were 'fair' ;0
9: np : ), but not considering Mark
10: I must need that coffee that Steve and Mark are making!
9: i'm going to tell u are picking on me Robert, MS wont invite you back hehe
7: I think we should Hire Anna
10: What Eric said here. I am afraid to type again.
7: I think it would be Anna, Emily, then Mark
8: Your official Vote?
8: Anna, Emily, Mark
7: Anna, Emily, then Mark
9: anna, emily,mark
-1: Introductions
6: hello world
6: follow the conversation
6: next line
6: Modify the text
4: hello
4: this is differnt
6: responding to Larry
4: of which ever discussion you wish to add to
6: reply
4: wild
4: cascading chats
6: next subject
5: Sounds like you might need some good people skills
6: click on each thread
5: this is different
5: S ...
6: Qualifications for this position:
6: I feel that people skills are the #1 requirement
4: So far, I'm for either Sharon or Daniel
6: agreed
6: We're looking for someone that we can groom for store management
4: right, that is why I think Sharon would be a good candidate. Her writing skills can be improved
with encouragement
4: unfortunately, we don't have any indication of Daniels' math skills
6: do we now about sharon's?
6: Know
4: for this resean, without further knowledge, I'm inclined to vote for Sharon
5: Sharon has good conflict resolution skills which is good management material
6: agreed
6: Okay, so @ this point, my list looks like: Sharon, Daniel, Eric
6: Thoughts?
4: That my thoughts as well
6: Scott?
6: Larry: am I missing scott's posts?
4: Scott? Your thoughts?
6: Okay, so we're in agreement on candidate #1
6: I've got my list for # 2 and 3, what about you guys?
5: S ...
6: thoughts, scott?
6: how about creating a single thread for the discussion
5: I also think tha Sharon 's visual/ graphic arts talents will be a benefit our company
5: Yes that is my thought also
4: good, I move to hire Sharon
6: so, let's give her an offer for a 90-day 'test' period
4: Ok with me
5: Yes
5: I second the motion.
6: Preferences for candidates # 2 and 3?
4: If other positions become available, as floor clerk
5: Eric should be our second choice
6: can you explain your thinking?
4: who's thinking James
6: both of you, regarding your choice for candidate #2
4: Eric is last choice, math skills I consider a requisit of teh job
6: For someone we're grooming for store management, conflict resolution and decisio
n-making skills are top priority
6: I agree, Daniel sounds better than Eric @ this point
4: But I still think Sharon is teh best for the job of Assistant Store mane
gement, with her leadership skills, she can keep the other motivated
6: So #1, 2, and 3 is is?
4: Sure
4: bye
6: seeya!
4: I agree
5: Then Daniel for the third
5: Ok
5: Daniel's Math skills aren't the best
4: His math skills aren't noted
6: right
5: We need someone with good math skills
5: What do you think about that James
5: I agree larry
5: So gentlemen we agree that it is going to be #1, #2, #3
6: Sounds good to me!
5: Scott agree's
5: goodbye gentlemen
6: seeya!
-1: Final Decision - Who should we hire?
6: New Topic
-1: Introductions
13: This is weird - just weird
11: Really?
12: yes...I understand the concept, but would prefer IM
11: Yes huh?
12: instant messenger
12: or tin cans...
13: :-) Agreed
12: I have to agree...
13: Also, not to make it more confusing - but maybe we should start here with "general" what we want stuff
13: I want someone who can COMMUNICATE clearly - not like this app
11: That's a start...
12: I'll third that motion...
13: Who has ANYTHING about communicating clearly?
13: I have nothingabout communication
12: all I have is composure ratings
11: Steve has good people skills
13: He also has excellent development skills
11: [EnteriI show very organizational skills here , maybe he can fix this chat appn
g Text...]
12: Hey, if you were gonna vote now...who would it be?
13: Get down to it... I like that
12: Joyce?
13: Well, Linda is out on my sheet - maybe that's a start
12: linda looks good on my sheet....maybe my sheet had too many beers
13: If we can get rid of one that will make the end decision easier
13: let's get rid of Linda...
12: l ...
11: I'd go w/Joyce.. if had to vote now,,,
13: I'd pick Steve at this point
12: all I know is, if this was my company, I wouldn't be discussing things like
this....I'd be broke..
11: [Entering Text...]
11: Steve shows no web skills on my sheet but Joyce has excellent Web d
evelopement on my sheet
13: Steve shows excellent web on my sheet
12: I vote for Steve...
13: I second that
11: Agree snail mail is as fast, maybe faxing would be easier
13: Anything would be easier than this
12: a ...
12: This makes rotary dialing look easy
13: Well, I guess we start here
12: she doesn't seem graphically strong enough for this position...
11: Yes gues so?
11: Who Joyce?
13: Yes Joyce is my Candidate #1
11: Joyce is taking the lead...
12: Steve doesn't seem technical enough for this position
13: I want someone who can communicate clearly-which is more than I'm able to do right now
11: Joyce ??
11: That's for sure who's on first??
13: I show candidate #2 is Steve - is that what you show?
13: I think this is candidate #2 area????
11: Yes looks that way,,, guess we are supposed to chat about ea. in theier own area???
13: that's what I take from this but this chat progam is waaaaay confusing and hard to
watch
11: I have Steve as two here
13: So this "thread" should be stuff about Steve I guess
12: I
12: I could see how this would work well in 'theory'.....but it's just too confusin
11: Yes guess so toooooo confusing.
11: Time for the next chat program this is too wierd.
12: I have to read all of it again just to see what you're talking abou
t at that particular moment
11: Yes could you imagine more than the three of us???
13: NO!!
11: Scaryy thought...
12: how is Steve's communication rated on your sheets?
13: Nothing about communication on my sheet for Steve
12: all it says is that she(Linda) has excellent composure...which could mean that she'll be st
anding there smiling as the place falls apart...
11: LOL,,,
11: Good people skills on mine
11: Agree, more desktop oriented
11: Gota say this is more confusing way to communicate
13: I'm rather lost myself
13: Okay guys - let's chat here about Steve and see if we can get anywhere
11: Sure I'm lost
11: Joyce looks good, j
12: She seems much better...
-1: Final Decision - Who should we hire?
13: Final decision time guys - time is almost up
12: Steve!!! Steve!!!Steve!!!1
13: Second!
11: [Entering Agree, I did not see any web stuff on my sheet..Text...]
13: We have a winner - Steve gets the job!
11: Yahoo!!!,,,
11: His first jod will be to fix this... LOL
13: :-)
12: yay for Steve...booo for threaded chat
13: Agreed - I like the "idea" but the practice is horrible
12: is this the right thread?
11: Don't know???
13: This is just too crazy for me
12: I've gotten more out of Pakistani cab drivers
Total turns: 87
Answer turns: 17
Normal turns: 59
Question turns: 10
Max thread depth: 7
Total turns: 82
Answer turns: 6
Normal turns: 64
Question turns: 11
Max thread depth: 16
Total turns: 95
Answer turns: 15
Normal turns: 62
Question turns: 17
Max thread depth: 16
Figure 2. Variations in the thread structures generated in user trials
104

58
CONCLUDING REMARKS & FUTURE RESEARCH
Chat has been used for years, but its basic interface remains
relatively unchanged. Threaded Chat addresses some of the
problems with the standard chat interface, especially as they
pertain to business applications.
Our user test of Threaded Chat demonstrated that people
could easily adapt to its interface. Although their subjective
ratings of the system were low, they were able to complete
the task just as well as they could with standard chat.
However we found many areas in which Threaded Chat
could be improved.
Improving the usability of Threaded Chat requires further
research to overcome significant obstacles. How can the
interface present new material recently added to distant
branches of the tree be presented? How can the interface
minimize the jumpy quality of the display as the turns are
added at various locations throughout the tree? We are
investigating the application of a “thread ruler” that would
allow users to reorder threads and provide “limits” that
manage the real estate devoted to each threads. In
combination with automatic branch control that closed up
the oldest turns in each thread branch, the problems
associated with attention management may be significantly
mitigated. Other directions for development include the
creation of alternative views of chat that highlight other
dimensions of the conversation, such as its temporal order.
In addition, further research questions still exist. The effect
of Threaded Chat on the number of repair statements could
not be answered by this study, and we also have not tested
the readability of the resulting Threaded Chat transcripts.
ACKNOWLEDGMENTS
Special thanks to Jyotindra Tandel for implementing
Threaded Chat. Shelly Farnham and Debbie McGhee were
very helpful with the planning and execution of this study.
We would like to thank Ken Bachman, Megan Hazen, and
the rest of the Microsoft Usability Test Coordination team
for recruiting and scheduling the participants for this study.
We would also like to thank Doug Tutmarc and the
Microsoft Usability Setup Operations Support team for
their assistance with the use of the Microsoft usability labs.
REFERENCES
1. ActiveWorlds, http://www.activeworlds.com
2. Derfler, F. Instant Messaging: Chat Goes to Work. PC
Magazine. 19(2), January 18, 2000, 82-101.
3. Donath, J., Karahalios, K., Viegas, F. Visualizing
Conversation. Proceedings of HICSS-32, Maui, HI,
January 5-8, 1999.
4. Erickson, T., Smith, D., Kellogg, W., Laff, M.,
Richards, J., and Bradner, E. Socially translucent
systems: Social proxies, persistent conversation, and the
design of babble. Proceedings of the 1999 ACM
Conference on Human Factors in Computing (CHI 99).
5. Everquest, http://www.everquest.com
6. Farnham, S. D., Chesley, H. McGhee, D., & Kawal, R.
Structured On-line Interactions: Improving the
Decision-making of Small Discussion Groups. In
Proceedings of CSCW 2000, Philadelphia, December
2000.
7. Garcia, A., and Jacobs, J. The eyes of the beholder:
Understanding the Turn Taking System in Quasi-
Synchronous Computer Mediated Communication.
Research on Language and Social Interaction, Vol. 32,
No. 4, 1999, 337-367.
8. Garcia, A., and Jacobs, J. The Interactional
Organization of Computer Mediated Communication in
the College Classroom. Qualitative Sociology, Vol. 21,
No. 3, 1998, 299-317
9. Herring, S. Interactional Coherence in CMC. Journal of
Computer-Mediated Communication, Thomas Erickson
(ed.), Vol. 4, No. 4, June 1999.
10.Hill, W., and Hollan, J. History Enriched Data Objects:
Prototypes and Policy Issues, The Information Society,
Volume 10, pp. 139-145.
11.John, B., and Kieras, D. The GOMS family of analysis
techniques: comparison and contrast. ACM
Transactions on Computer-Human Interaction, 3, 4
(December 1996), pp. 320 – 351.
12.Kobielus, J. Instant Messaging. The Burton Group
Network Strategy Report, 1. October 1999.
13.Kollock, P., Smith, M. Managing the Virtual
Commons: Cooperation and Conflict in Computer
Communities. Computer-Mediated Communication, S.
Herring (ed.). Amsterdam: John Benjamins, 1996.
14.Microsoft V-Chat, http://www.microsoft.com
15.Pomerantz, A. Agreeing and Disagreeing with
assessments: some features of preferred/dispreferred
turn shapes. In John Heritage and J. Maxwell Atkinson
(eds.), Structures of Social Action: Studies in
Conversation Analysis. Cambridge. 1984. 57-101.
16.Reid, E. Electronic Chat: Social Issues on Internet
Relay Chat. Media Information Australia, 67, 1993, 62-
79
17.Rintel, S., Pittam, J., Strangers in a Strange Land:
Interaction Management on Internet Relay Chat, Human
Communication Research. Vol. 23, No. 4, 1997, 507-
534
18.Sacks, H., Schegloff, E., and Jefferson, G. A simplest
systematics for the organization of turn-taking for
conversation. Language, 50(4). 1974.
19.The Palace, http://www.thepalace.com
20.Viegas, F. and Donath, J. Chat Circles. Proceedings of
the 1999 ACM Conference on Human Factors in
Computing (CHI 99).
21.Vronay, D., Smith, M., and Drucker, S. Alternative
Interfaces for Chat. Proceedings of the 12th
Annual
ACM Symposium on User Interface Software and
Technology (UIST 99).
22.Wiemann, J., and Knapp, M. Turn-Taking in
Conversations, Journal of Communication, 25, 1975,
75-92
23.WorldsAway, http://www.worldsaway.com
105

2000 - CSCW - Conversation Trees and Threaded Chats

More Related Content

What's hot (20)

Viewers also liked (6)

Similar to 2000 - CSCW - Conversation Trees and Threaded Chats (20)

More from Marc Smith (20)

Recently uploaded (20)

2000 - CSCW - Conversation Trees and Threaded Chats