![2010 CRC PhD Student Conference
Issues and techniques for collaborative music making
on multi-touch surfaces
Anna Xambó
a.xambo@open.ac.uk
Supervisors Robin Laney
Department/Institute Department of Computing
Status Visiting research student (4 months)
Probation viva -
Starting date -
A range of applications exist for collaborative music making on multi-touch
surfaces. Some of them have been highly successful, but currently there is no
systematic way of designing them, to maximize collaboration for a particular
user group. We are specially interested in applications that will engage
novices and experts. Traditionally the challenge in collaborative music
instruments is to satisfy the needs of both [1]. For that purpose, we developed
a collaborative music making prototype for multi-touch surfaces and evaluated
its creative engagement.
Applications for musical multi-touch surfaces are not new. A pioneering work
is the ReacTable [2, 3], which allows a group of people to share control of a
modular synthesizer by manipulating physical objects on a round table. Iwai’s
Composition on the Table [4] allows users to create music and visuals by
interacting with four tables which display switches, dials, turntables and
sliders. Stereotronic Multi-Synth Orchestra [5] uses a multi-touch interface
based on a concentric sequencer where notes can be placed. What is less
addressed is the evaluation of creative engagement in these applications.
There are numerous theoretical accounts of the nature of emotional
engagements with art and artefacts. Current models are based on a
pragmatist view, which conceptualises the aesthetic and affective value of an
object as lying not in the object itself, but in an individual’s or a group’s rich
set of interactions with it [6, 7]. In the context of pleasurable creative
engagement and the collective composition of music, Bryan-Kinns et al. [8]
see attunement to others’ contributions as the central principle of creative
engagement. The phenomena of personal full immersion in an activity, also
known as ’fow’ [7], has been extended to groups as means of heightening
group productivity [9].
Our approach is, frst, to study the issues and techniques of multi-user
instruments and multi-touch applications in general, second, to design a
simple application in an initial attempt to clearly analyse some of these issues,
and third, to evaluate its creative engagement. For that purpose, a prototype
was built which allowed groups of up to four users to express themselves in
collaborative music making using pre-composed materials. By
keeping the prototype minimal, we were able to investigate the essential
aspects of engaging interaction.
Case studies were video recorded and analysed using two techniques derived
from Grounded Theory (GT) and Content Analysis (CA). For the GT, which is
a qualitative research method employed in the social sciences that derives
theoretical explanations from the data without having hypotheses in mind [10],
we adopted an open coding strategy of identifying key moments of the video
Page 118 of 125](https://image.slidesharecdn.com/xambo-100528095548-phpapp02/85/Xambo-1-320.jpg)
![2010 CRC PhD Student Conference
interactions; grouping the codes by concepts and generating general
explanations from the categorization of the concepts. Given that this approach
is based on creative interpretation, we added more evidence by
complementing GT with CA. Content Analysis (CA) is defned by Holsti (1969)
as ”any technique for making inferences by objectively and systematically
identifying specifed characteristics of messages” [10]. This defnition includes
content analysis of text, videos, music or drawings. There are varied
approaches to CA using quantitative, qualitative or both techniques. Our
approach is derived from ethnographic content analysis or qualitative content
analysis [11], an approach to documents that emphasises the role of the
investigator in the construction of the meaning of texts. We took same steps
as in the open coding, but in the frst step we used instead structured codes to
help us identify key points of the video-recorded interactions.
The case study protocol was the following: The users were expected to
perform three musical tasks of different character as well as an informal
discussion in order to generate suffcient data to analyse several aspects of
behaviours using the prototype. A questionnaire was also conducted and
evaluated. The main focus of the analysis was on the evaluation of the
collaborative interactions enabled by the prototype. The questions we wanted
to address were:
1. What were the modes participants found to collaborate with one
another;
2. What were the diffculties that participants encountered and the extent
to which they found the exercise engaging;
3. What was the degree of satisfaction at the end result.
From transcription of the video speech and behaviours, and then the process
of open coding, we identifed the following concepts: collaboration, musical
aesthetics, learning process and system design. After that, we analysed the
same data using the nomenclature chosen from two existing theoretical
frameworks. The frst one is a general framework of tangible social interaction
which includes the concepts of tangible manipulation, spatial interaction,
embodied facilitation or expressive representation [12]. The second one is
focused on the engagement between participants in music collaboration,
which considers the following features: mutual awareness, shared and
consistent representations, mutual modifability and annotation [8]. We found
that some of the content analysed was already discussed in the open coding
process, which provides consistency. Data was also collected using a
questionnaire, which was designed to probe such issues as how aware each
participant had been of other instruments; the diffculty of the tasks, and how
much they felt they had enjoyed and concentrated on them; and the extent to
which they considered they had operated as a team and felt part of a
collaborative process. Responses were recorded using numerical scores, but
the questionnaire also asked for qualitative feedback on how participants
organised themselves as a group and the nature of any rules they created.
We also recorded anonymously the participants age, gender, previous
experience, love of music, and the instrument they had been allocated on the
table.
Within a user-centered design approach of active participation of users in the
process of designing the prototype, the most prominent aspects that have
emerged as enhancements of multi-touch applications in music collaboration
are:
• Responsiveness. The responsiveness determines the perceived
emotiveness. This parameter should be adequately related to the
Page 119 of 125](https://image.slidesharecdn.com/xambo-100528095548-phpapp02/85/Xambo-2-320.jpg)
![2010 CRC PhD Student Conference
application performance in terms of time and computer resources used.
A consistent audiovisual feedback will enhance the perceived response
of the application.
• Shared vs. individual controls. Both shared and individual spaces are
needed. Shared features would strength mutual awareness and mutual
modifability. Individual spaces would strength personal opinion,
musical identity and musical expression.
The fndings of this study help us understand engagement in music
collaboration. Qualitative video analysis and the questionnaires provide
indication of participants having mutual engaging interaction in terms of being
engaged with the music collaboratively produced and also being engaged with
others in the activity. High degree of satisfaction at the end result is evidenced
mostly by the gestural mode. The evidence found of participants exchanging
ideas constantly indicates that the prototype strongly facilitates conversation,
which, as noted earlier, is important in terms of group productivity.
In the future, we are interested in how many, and what type of, affordances
such applications should offer in order to maximise engagement. We are also
interested in validating our evaluation method. To that end, there is scope to
improve the responsiveness of the prototype and to redesign the distribution
of shared and individual controls. Furthermore, there is a plan to add
individual continuous controls for sound parameter modifcations in order to
encourage a process-oriented composition. The mutual experience might be
enhanced and collaboration deepened, by adding common controls, as well. A
balance between adding more features and keeping simplicity must be kept in
order to attract both novices and experts alike.
[1] T. Blaine and S. Fels, “Collaborative musical experiences for novices,”
Journal of New Music Research, vol. 32, no. 4, pp. 411–428, 2003.
[2] S. Jordà, M. Kaltenbrunner, G. Geiger, and R. Bencina, “The reacTable*,”
in Proceedings of the International Computer Music Conference (ICMC 2005),
(Barcelona, Spain), 2005.
[3] S. Jordà, G. Geiger, M. Alonso, and M. Kaltenbrunner, “The reacTable:
Exploring the synergy between live music performance and tabletop tangible
interfaces,” in TEI ’07: Proceedings of the 1st international conference on
Tangible and embedded interaction, (New York, NY, USA), pp. 139–146, ACM,
2007.
[4] T. Iwai, “Composition on the table,” in International Conference on
Computer Graphics and Interactive Techniques, SIGGRAPH: ACM Special
Interest Group on Computer Graphics and Interactive Techniques, ACM,
1999.
[5] http://www.fashionbuddha.com/, 15/3/2010.
[6] M. Blythe and M. Hassenzahl, The semantics of fun: differentiating
enjoyable experiences, pp. 91–100. Norwell, MA, USA: Kluwer Academic
Publishers, 2004.
[7] M. Csikszentmihalyi, Beyond Boredom and Anxiety: Experiencing Flow in
Work and Play. Jossey-Bass, 1975.
Page 120 of 125](https://image.slidesharecdn.com/xambo-100528095548-phpapp02/85/Xambo-3-320.jpg)
![2010 CRC PhD Student Conference
[8] N. Bryan-Kinns and F. Hamilton, “Identifying mutual engagement,”
Behaviour and Information Technology, 2009.
[9] K. Sawyer, Group Genius: The Creative Power of Collaboration. Basic
Books, 2007.
[10] J. Lazar, J. Feng, and H. Hochheiser, Research Methods in Human-
Computer Interaction. Wiley, 2010.
[11] D. L. Altheide, “Ethnographic content analysis,” Qualitative Sociology, vol.
10, pp. 65–77, 1987.
[12] E. Hornecker and J. Buur, “Getting a grip on tangible interaction: A
framework on physical space and social interaction,” in CHI ’06: Proceedings
of the SIGCHI conference on Human Factors in computing systems, (New
York, NY, USA), pp. 437–446, ACM Press, 2006.
Page 121 of 125](https://image.slidesharecdn.com/xambo-100528095548-phpapp02/85/Xambo-4-320.jpg)
Xambo
- 1. 2010 CRC PhD Student Conference Issues and techniques for collaborative music making on multi-touch surfaces Anna Xambó a.xambo@open.ac.uk Supervisors Robin Laney Department/Institute Department of Computing Status Visiting research student (4 months) Probation viva - Starting date - A range of applications exist for collaborative music making on multi-touch surfaces. Some of them have been highly successful, but currently there is no systematic way of designing them, to maximize collaboration for a particular user group. We are specially interested in applications that will engage novices and experts. Traditionally the challenge in collaborative music instruments is to satisfy the needs of both [1]. For that purpose, we developed a collaborative music making prototype for multi-touch surfaces and evaluated its creative engagement. Applications for musical multi-touch surfaces are not new. A pioneering work is the ReacTable [2, 3], which allows a group of people to share control of a modular synthesizer by manipulating physical objects on a round table. Iwai’s Composition on the Table [4] allows users to create music and visuals by interacting with four tables which display switches, dials, turntables and sliders. Stereotronic Multi-Synth Orchestra [5] uses a multi-touch interface based on a concentric sequencer where notes can be placed. What is less addressed is the evaluation of creative engagement in these applications. There are numerous theoretical accounts of the nature of emotional engagements with art and artefacts. Current models are based on a pragmatist view, which conceptualises the aesthetic and affective value of an object as lying not in the object itself, but in an individual’s or a group’s rich set of interactions with it [6, 7]. In the context of pleasurable creative engagement and the collective composition of music, Bryan-Kinns et al. [8] see attunement to others’ contributions as the central principle of creative engagement. The phenomena of personal full immersion in an activity, also known as ’fow’ [7], has been extended to groups as means of heightening group productivity [9]. Our approach is, frst, to study the issues and techniques of multi-user instruments and multi-touch applications in general, second, to design a simple application in an initial attempt to clearly analyse some of these issues, and third, to evaluate its creative engagement. For that purpose, a prototype was built which allowed groups of up to four users to express themselves in collaborative music making using pre-composed materials. By keeping the prototype minimal, we were able to investigate the essential aspects of engaging interaction. Case studies were video recorded and analysed using two techniques derived from Grounded Theory (GT) and Content Analysis (CA). For the GT, which is a qualitative research method employed in the social sciences that derives theoretical explanations from the data without having hypotheses in mind [10], we adopted an open coding strategy of identifying key moments of the video Page 118 of 125
- 2. 2010 CRC PhD Student Conference interactions; grouping the codes by concepts and generating general explanations from the categorization of the concepts. Given that this approach is based on creative interpretation, we added more evidence by complementing GT with CA. Content Analysis (CA) is defned by Holsti (1969) as ”any technique for making inferences by objectively and systematically identifying specifed characteristics of messages” [10]. This defnition includes content analysis of text, videos, music or drawings. There are varied approaches to CA using quantitative, qualitative or both techniques. Our approach is derived from ethnographic content analysis or qualitative content analysis [11], an approach to documents that emphasises the role of the investigator in the construction of the meaning of texts. We took same steps as in the open coding, but in the frst step we used instead structured codes to help us identify key points of the video-recorded interactions. The case study protocol was the following: The users were expected to perform three musical tasks of different character as well as an informal discussion in order to generate suffcient data to analyse several aspects of behaviours using the prototype. A questionnaire was also conducted and evaluated. The main focus of the analysis was on the evaluation of the collaborative interactions enabled by the prototype. The questions we wanted to address were: 1. What were the modes participants found to collaborate with one another; 2. What were the diffculties that participants encountered and the extent to which they found the exercise engaging; 3. What was the degree of satisfaction at the end result. From transcription of the video speech and behaviours, and then the process of open coding, we identifed the following concepts: collaboration, musical aesthetics, learning process and system design. After that, we analysed the same data using the nomenclature chosen from two existing theoretical frameworks. The frst one is a general framework of tangible social interaction which includes the concepts of tangible manipulation, spatial interaction, embodied facilitation or expressive representation [12]. The second one is focused on the engagement between participants in music collaboration, which considers the following features: mutual awareness, shared and consistent representations, mutual modifability and annotation [8]. We found that some of the content analysed was already discussed in the open coding process, which provides consistency. Data was also collected using a questionnaire, which was designed to probe such issues as how aware each participant had been of other instruments; the diffculty of the tasks, and how much they felt they had enjoyed and concentrated on them; and the extent to which they considered they had operated as a team and felt part of a collaborative process. Responses were recorded using numerical scores, but the questionnaire also asked for qualitative feedback on how participants organised themselves as a group and the nature of any rules they created. We also recorded anonymously the participants age, gender, previous experience, love of music, and the instrument they had been allocated on the table. Within a user-centered design approach of active participation of users in the process of designing the prototype, the most prominent aspects that have emerged as enhancements of multi-touch applications in music collaboration are: • Responsiveness. The responsiveness determines the perceived emotiveness. This parameter should be adequately related to the Page 119 of 125
- 3. 2010 CRC PhD Student Conference application performance in terms of time and computer resources used. A consistent audiovisual feedback will enhance the perceived response of the application. • Shared vs. individual controls. Both shared and individual spaces are needed. Shared features would strength mutual awareness and mutual modifability. Individual spaces would strength personal opinion, musical identity and musical expression. The fndings of this study help us understand engagement in music collaboration. Qualitative video analysis and the questionnaires provide indication of participants having mutual engaging interaction in terms of being engaged with the music collaboratively produced and also being engaged with others in the activity. High degree of satisfaction at the end result is evidenced mostly by the gestural mode. The evidence found of participants exchanging ideas constantly indicates that the prototype strongly facilitates conversation, which, as noted earlier, is important in terms of group productivity. In the future, we are interested in how many, and what type of, affordances such applications should offer in order to maximise engagement. We are also interested in validating our evaluation method. To that end, there is scope to improve the responsiveness of the prototype and to redesign the distribution of shared and individual controls. Furthermore, there is a plan to add individual continuous controls for sound parameter modifcations in order to encourage a process-oriented composition. The mutual experience might be enhanced and collaboration deepened, by adding common controls, as well. A balance between adding more features and keeping simplicity must be kept in order to attract both novices and experts alike. [1] T. Blaine and S. Fels, “Collaborative musical experiences for novices,” Journal of New Music Research, vol. 32, no. 4, pp. 411–428, 2003. [2] S. Jordà, M. Kaltenbrunner, G. Geiger, and R. Bencina, “The reacTable*,” in Proceedings of the International Computer Music Conference (ICMC 2005), (Barcelona, Spain), 2005. [3] S. Jordà, G. Geiger, M. Alonso, and M. Kaltenbrunner, “The reacTable: Exploring the synergy between live music performance and tabletop tangible interfaces,” in TEI ’07: Proceedings of the 1st international conference on Tangible and embedded interaction, (New York, NY, USA), pp. 139–146, ACM, 2007. [4] T. Iwai, “Composition on the table,” in International Conference on Computer Graphics and Interactive Techniques, SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques, ACM, 1999. [5] http://www.fashionbuddha.com/, 15/3/2010. [6] M. Blythe and M. Hassenzahl, The semantics of fun: differentiating enjoyable experiences, pp. 91–100. Norwell, MA, USA: Kluwer Academic Publishers, 2004. [7] M. Csikszentmihalyi, Beyond Boredom and Anxiety: Experiencing Flow in Work and Play. Jossey-Bass, 1975. Page 120 of 125
- 4. 2010 CRC PhD Student Conference [8] N. Bryan-Kinns and F. Hamilton, “Identifying mutual engagement,” Behaviour and Information Technology, 2009. [9] K. Sawyer, Group Genius: The Creative Power of Collaboration. Basic Books, 2007. [10] J. Lazar, J. Feng, and H. Hochheiser, Research Methods in Human- Computer Interaction. Wiley, 2010. [11] D. L. Altheide, “Ethnographic content analysis,” Qualitative Sociology, vol. 10, pp. 65–77, 1987. [12] E. Hornecker and J. Buur, “Getting a grip on tangible interaction: A framework on physical space and social interaction,” in CHI ’06: Proceedings of the SIGCHI conference on Human Factors in computing systems, (New York, NY, USA), pp. 437–446, ACM Press, 2006. Page 121 of 125