From Networked Music to Behavior Driven Interaction
1 like638 views
The document discusses networked music performance and the challenges of latency and synchronization in remote music collaboration. It highlights research on acoustic properties, digital interfaces, and behavior-driven interactions that enhance musical performances across distances. The author, Álvaro Barbosa, shares insights from various projects and experiments aimed at improving the experience of networked music creation.
![Background
on
Network
Music
Performance
(5) Slow Attack Times
Bregman (1990)
Auditory Scene Analysis
This
experiment
clearly
reveals
that
it
is
harder
to
perceive
the
order
of
overlapping
sound
events
when
their
rise
]me
(aIack)
is
slower.
In
other
words,
overlapping
sounds
with
slower
rise
]mes
are
beIer
perceived
as
synchronous
even
when
their
onsets
are
not
physically
simultaneous.
Slide
#
23
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)](https://image.slidesharecdn.com/talk-nmbdi-120518041520-phpapp02/85/From-Networked-Music-to-Behavior-Driven-Interaction-23-320.jpg)
![Background
on
Network
Music
Performance
(6)
ANTIPHONY
Technique
of
alternate
or
responsive
singing
by
a
choir
in
two
separate
sec@ons.
Each
sec]on
sings
alternate
musical
phrases
Or
POLYCHORAL
ANTIPHONY
(AKA
Vene]an
polychoral
style)
When
two
or
more
groups
of
singers
sing
in
alterna]on.
Origin
in
the
late
Renaissance
and
early
Baroque
Can
this
Technique
improve
network
latency
tolerance?
Slide
#
27
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)](https://image.slidesharecdn.com/talk-nmbdi-120518041520-phpapp02/85/From-Networked-Music-to-Behavior-Driven-Interaction-27-320.jpg)
![Behavior-‐Driven
Interac9on
Used
in
Live
Performances:
Live
Set
at
Galerija
Kapelica
in
Ljubljana,
Slovenia
During
The
EARZOM
Fes]val
2010
Slide
#
39
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)](https://image.slidesharecdn.com/talk-nmbdi-120518041520-phpapp02/85/From-Networked-Music-to-Behavior-Driven-Interaction-39-320.jpg)
![Behavior-‐Driven
Interac9on
Live
Set:
2
Ipads
and
1
Iphone
runing
CS-‐Grain,
Reactable
and
Cur]s;
1
Radio
String
Chimes;
2
Wind
Blowers;
1
E-‐Bow
Slide
#
41
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)](https://image.slidesharecdn.com/talk-nmbdi-120518041520-phpapp02/85/From-Networked-Music-to-Behavior-Driven-Interaction-41-320.jpg)
![Behavior-‐Driven
Interac9on
Audio
Materials:
Recorded
at
the
Antarc]ca
Expedi]onin
January
2012
by
Álvaro
Barbosa
and
Victor
Gama
Slide
#
43
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)](https://image.slidesharecdn.com/talk-nmbdi-120518041520-phpapp02/85/From-Networked-Music-to-Behavior-Driven-Interaction-43-320.jpg)
![Behavior-‐Driven
Interac9on
Photography
Books:
Drieing
Ice;
Antarc]cans
Slide
#
47
/
Author:
Álvaro
Barbosa
(www.abarbosa.org)](https://image.slidesharecdn.com/talk-nmbdi-120518041520-phpapp02/85/From-Networked-Music-to-Behavior-Driven-Interaction-47-320.jpg)
From Networked Music to Behavior Driven Interaction
- 1. From Networked Music to Behavior-‐Driven Interac9on Álvaro Barbosa -‐ www.abarbosa.org Research Center for Science and Technology of the Arts (CITAR) School of Arts, Catholic University Porto – www.artes.ucp.pt
- 2. Álvaro Barbosa h=p://www.abarbosa.org/ Since 1998 – EA, UCP-‐Porto, Portugal hIp://artes.ucp.pt/ Since 2004 – CITAR, UCP-‐Porto, Portugal hIp://artes.ucp.pt/citar/ 2000/2006 – MTG, UPF-‐Barcelona, Spain hIp://mtg.upf.edu/ 2010 – CCRMA, Stanford-‐CA, USA hIp://ccrma.stanford.edu/ Since 2011 – USJ, Macau – China hIp://www.usj.edu.mo/ Slide # 2 / Author: Álvaro Barbosa (www.abarbosa.org)
- 3. Álvaro Barbosa Very early Web Developer in the industry (Caleida & aeiou.pt, 1995-‐1998) Slide # 3 / Author: Álvaro Barbosa (www.abarbosa.org)
- 4. Research / Book • Displaced Soundscapes is a metaphorical description of the way lively generated sounds can be perceived over the Internet." • (Book) Barbosa, A. 2008. “Displaced Soundscapes”" • VDM Verlag (ISBN: 978-3-8364-7154-1)" - Extensive Survey of NM Applications " - Research on Latency and Internet Acoustics" - Models for Networked Music Systems" - Implementation “Public Sound Objects” " Slide # 4 / Author: Álvaro Barbosa (www.abarbosa.org)
- 5. Background on Network Music Performance Performing Music at a Distance Synchronizing Berlin and Hong Kong (1/April/2011 – 8.753 Km) Hong Kong New Music Ensemble & Ensemble Adapter http://hknme.org/hongkongartsblog/?p=1525 Slide # 5 / Author: Álvaro Barbosa (www.abarbosa.org)
- 6. Background on Network Music Performance Performing Music at a Distance Slide # 6 / Author: Álvaro Barbosa (www.abarbosa.org)
- 7. Background on Network Music Performance • Stanford-Mcgill 2002 (4.076 Km) Slide # 7 / Author: Álvaro Barbosa (www.abarbosa.org)
- 8. Background on Network Music Performance • Stanford-Mcgill 2002 – Discovery Channel Slide # 8 / Author: Álvaro Barbosa (www.abarbosa.org)
- 9. Background on Network Music Performance • Stanford-Mcgill 2002 (the real thing) Slide # 9 / Author: Álvaro Barbosa (www.abarbosa.org)
- 10. Background on Network Music Performance Multiple Location – Same Performance ARTECH 2008: Rambouillet (FR) / Casa da Música (PT) / SARC (IR) http://artes.ucp.pt/artech2008/ Slide # 10 / Author: Álvaro Barbosa (www.abarbosa.org)
- 11. Background on Network Music Performance Latency and Networked Music In a vocal conversation it is possible to maintain it even with one-way delays of up to 500 ms. Holub, J., Kastner, M., Tomiska, O. 2007 In order to maintain a synchronized and smooth musical interaction reduces drastically to the order of tens of milliseconds. Schuett, N. 2002 Chafe C., Gurevich M, 2004 Lago, N and Kon, F. 2004 Barbosa, A., Carôt, A. 2005 Chew, E., Sawchuk, A., Tanoue, C., and Zimmermann, R. 2005 Bartlette, C., Headlam, D., Bocko, M., Velikic, G. 2006 Farner, S., Solvang, A., Sæbo, A., Svensson, U. P. 2009 Chafe, C., Cáceres, J-P., Gurevich, M., 2010 Slide # 11 / Author: Álvaro Barbosa (www.abarbosa.org)
- 12. Background on Network Music Performance Latency and Networked Music For the Human ear to perceive the order of two simultaneous sounds, they should not be displaced in time over 20ms (Hirsh, 1959) A difficulty in discerning the order of sounds events ê hard to maintain a synchronous musical interaction. Slide # 12 / Author: Álvaro Barbosa (www.abarbosa.org)
- 13. Background on Network Music Performance • Jitter and delay asymmetry introduce further disruption in Network Communication. • Furthermore, Latency is not a technological condition that can be overcome in the near future. Consider mobile or satellite technology or a communication setup that spans worldwide. Moscow – Santiago (fiber-optic link): Bidirectional Latency= 94,3 ms Unidirectional Latency= 47,15 ms Slide # 13 / Author: Álvaro Barbosa (www.abarbosa.org)
- 14. Background on Network Music Performance Latency and Networked Music The ability to perform music synchronously is strongly dependent on: • The music expressive qualities (Dynamics and Articulation) • The music style (rhythm, melody, harmony) • The music perceptual qualities (pitch, texture, timbre) • The music structure/form • The musician’s experience and practicing strategies • Complementary feed-back modalities (visual, tactile) • The listening conditions Slide # 14 / Author: Álvaro Barbosa (www.abarbosa.org)
- 15. Background on Network Music Performance Barbosa, A. 2003. “Displaced Soundscapes: A Survey of Network Systems for Music and Sonic Art Crea9on” Leonardo Music Journal 13, pp. 53-‐59 -‐ MIT Press, Cambridge; doi:10.1162/096112104322750791). Slide # 15 / Author: Álvaro Barbosa (www.abarbosa.org)
- 16. Background on Network Music Performance Latency and Networked Music Basic Principles - Remote Performance (1) Ensemble Performance Threshold (EPT) (2) Echo Feed-Back (Self Delay) (3) Inverse Proportion to Tempo (4) Reverb and Complementary Modalities (5) Slow Attack Times (6) Antiphony Slide # 16 / Author: Álvaro Barbosa (www.abarbosa.org)
- 17. Background on Network Music Performance (1) Ensemble Performance Threshold (EPT) (CCRMA 2004, USC 2005) Comfort Zone for Expert Musicians EPT Interval for Expert Musicians Comfort Zone for EPT Interval for Untrained Users Untrained Users Almost Extremely Impossible to Difficult to Compensate Unnaturally Difficult to Compensate Low Latency Compensate DELAY (ms) 0 5 10 15 20 25 30 35 40 45 50 75 100 150 Slide # 17 / Author: Álvaro Barbosa (www.abarbosa.org)
- 18. Background on Network Music Performance (2) Echo Feed-Back (USC 2005) Chew, E., Sawchuk, A., Tanoue, C., and Zimmermann, R. 2005 Slide # 18 / Author: Álvaro Barbosa (www.abarbosa.org)
- 19. Background on Network Music Performance (2) Echo Feed-Back (UCP-Porto 2005) Slide # 19 / Author: Álvaro Barbosa (www.abarbosa.org)
- 20. Background on Network Music Performance (3) Inverse Proportion to Tempo (UCP-Porto 2005) Slide # 20 / Author: Álvaro Barbosa (www.abarbosa.org)
- 21. Background on Network Music Performance (3) Inverse Proportion to Tempo (CCRMA/Banff 2006) St. Lawrence String Quartet (over 50ms Delay) Slide # 21 / Author: Álvaro Barbosa (www.abarbosa.org)
- 22. Background on Network Music Performance (4) Reverb and Complementary Modalities Studies on the effect of Reverberation (Farner 2009) The Influence of Delay and Various Acoustic Environments Studies on the effect of Visual Feed-Back Pilot experiments at CCRMA since 2006 Studies on the effect of Haptic Feedback ??? Slide # 22 / Author: Álvaro Barbosa (www.abarbosa.org)
- 23. Background on Network Music Performance (5) Slow Attack Times Bregman (1990) Auditory Scene Analysis This experiment clearly reveals that it is harder to perceive the order of overlapping sound events when their rise ]me (aIack) is slower. In other words, overlapping sounds with slower rise ]mes are beIer perceived as synchronous even when their onsets are not physically simultaneous. Slide # 23 / Author: Álvaro Barbosa (www.abarbosa.org)
- 24. Background on Network Music Performance • Pilot Study Hypothesis • If slow a?ack @mes allow a be?er percep@on of sound simultaneity this might also lead to a be?er ability to perform synchronous musical interac@on • Experimental Setup Slide # 24 / Author: Álvaro Barbosa (www.abarbosa.org)
- 25. Background on Network Music Performance • Rhythmic Performance • 80 BPM (to avoid possible overlaying of subsequent notes) • Slow Tempo => Higher EPT • Range from 25 ms to 110 ms (with steps of 10 ms) • 28 trials for Slow and Fast aIack strokes Slide # 25 / Author: Álvaro Barbosa (www.abarbosa.org)
- 26. Background on Network Music Performance Results How each musician performs in rela@on to the other? The analysis was performed using the soeware tool MATCH: Music Alignment Tool Chest, developed by Simon Dixon (this soeware analyses the alignment of audio files using the OLTW algorithm) Slide # 26 / Author: Álvaro Barbosa (www.abarbosa.org)
- 27. Background on Network Music Performance (6) ANTIPHONY Technique of alternate or responsive singing by a choir in two separate sec@ons. Each sec]on sings alternate musical phrases Or POLYCHORAL ANTIPHONY (AKA Vene]an polychoral style) When two or more groups of singers sing in alterna]on. Origin in the late Renaissance and early Baroque Can this Technique improve network latency tolerance? Slide # 27 / Author: Álvaro Barbosa (www.abarbosa.org)
- 28. Background on Network Music Performance Latency and Networked Music Basic Principles - Shared Sonic Environments (1) Digital Control Interface (software or tangible) (2) Local Synthesis & Transmission of Control Data (3) Peer-To-Peer Communication (4) Latency Adaptive Dynamics (5) Behavior Driven Interaction (Loose Coupling) Slide # 28 / Author: Álvaro Barbosa (www.abarbosa.org)
- 29. Background on Network Music Performance Latency and Networked Music Examples Shared Sonic Environments PUBLIC SOUND OBJECTS THE HORGIE: COLLABORATIVE by Álvaro Barbosa ONLINE SYNTHESIZER by Jorge Herrera Slide # 29 / Author: Álvaro Barbosa (www.abarbosa.org)
- 30. Public Sound Objects Public Sound Objects 2004/2006, MTG (BCN) Slide # 30 / Author: Álvaro Barbosa (www.abarbosa.org)
- 31. Public Sound Objects Local Network of PSOs – CITAR (Porto) commissioned by “Casa da Musica” http://en.wikipedia.org/wiki/Casa_da_musica Slide # 31 / Author: Álvaro Barbosa (www.abarbosa.org)
- 32. Public Sound Objects Slide # 32 / Author: Álvaro Barbosa (www.abarbosa.org)
- 33. Public Sound Objects Slide # 33 / Author: Álvaro Barbosa (www.abarbosa.org)
- 34. Public Sound Objects Slide # 34 / Author: Álvaro Barbosa (www.abarbosa.org)
- 35. Public Sound Objects Permanent Installation Casa da Música (Renaissance Hall) Slide # 35 / Author: Álvaro Barbosa (www.abarbosa.org)
- 36. Behavior-‐Driven Interac9on Small Fish (Fujihata e Furukawa 1999) Slide # 36 / Author: Álvaro Barbosa (www.abarbosa.org)
- 37. Behavior-‐Driven Interac9on Radial String Chimes (1st prototype Developed at CCRMA – Stanford University) Interactive musical device triggered by motion applied to a spinning vinyl record. Users can spin the record, making hanging coffee straws to bounce and pluck twelve radial guitar strings, applied to a round wood board. Slide # 37 / Author: Álvaro Barbosa (www.abarbosa.org)
- 38. Behavior-‐Driven Interac9on Slide # 38 / Author: Álvaro Barbosa (www.abarbosa.org)
- 39. Behavior-‐Driven Interac9on Used in Live Performances: Live Set at Galerija Kapelica in Ljubljana, Slovenia During The EARZOM Fes]val 2010 Slide # 39 / Author: Álvaro Barbosa (www.abarbosa.org)
- 40. Behavior-‐Driven Interac9on Jorney to the Last Fron9er -‐ with Victor Gama – B&W 2012 Slide # 40 / Author: Álvaro Barbosa (www.abarbosa.org)
- 41. Behavior-‐Driven Interac9on Live Set: 2 Ipads and 1 Iphone runing CS-‐Grain, Reactable and Cur]s; 1 Radio String Chimes; 2 Wind Blowers; 1 E-‐Bow Slide # 41 / Author: Álvaro Barbosa (www.abarbosa.org)
- 42. Behavior-‐Driven Interac9on Victor Gama: www.victorgama.org Performed on His own Instruments – Dino, Acrux and Thoa Slide # 42 / Author: Álvaro Barbosa (www.abarbosa.org)
- 43. Behavior-‐Driven Interac9on Audio Materials: Recorded at the Antarc]ca Expedi]onin January 2012 by Álvaro Barbosa and Victor Gama Slide # 43 / Author: Álvaro Barbosa (www.abarbosa.org)
- 44. Behavior-‐Driven Interac9on Field Recordings at the Antarc9ca Peninsula Slide # 44 / Author: Álvaro Barbosa (www.abarbosa.org)
- 45. Behavior-‐Driven Interac9on Video and Photography for Mul9media Perfoamances Slide # 45 / Author: Álvaro Barbosa (www.abarbosa.org)
- 46. Behavior-‐Driven Interac9on Slide # 46 / Author: Álvaro Barbosa (www.abarbosa.org)
- 47. Behavior-‐Driven Interac9on Photography Books: Drieing Ice; Antarc]cans Slide # 47 / Author: Álvaro Barbosa (www.abarbosa.org)
- 48. School of Arts – Porto, Portugal Thank You!!!! alvaro.barbosa@usj.edu.mo abarbosa@porto.ucp.pt | hIp://www.abarbosa.org Slide # 48 / Author: Álvaro Barbosa (www.abarbosa.org)