Emotion-Based Interactive Installation

Emotion-Based Interactive Installation
FranciscoVentura,António Pedro Oliveira and Amílcar Cardoso
University of Coimbra, Portugal

2
Outline
 Introduction
 EDME
 Interactive Installation
 Experiments
 Demonstration
 Conclusion

3
Outline
 Introduction
 EDME
 Experiments
 Demonstration
 Conclusion

Introduction
4
 We prepared an installation to explore the interactive
capabilities of an Emotion-Driven Music Engine (EDME)
 This installation allows participants to experience and
influence the emotional state of a computational system
 The system perceives emotions from the environment by
using a camera and expresses emotions to the environment
with music and images by using, respectively, loudspeakers
and a projector

5
Outline
 Introduction
 EDME
 Experiments
 Demonstration
 Conclusion

EDME
6
 EDME is a computer program that intends to produce
music expressing a desired emotion
 It is composed by an offline stage and a real-time stage
 At the offline stage, pre-composed music is segmented,
emotionally classified and stored in a music base
 At the real-time stage, EDME selects from a music base the
segments that are closer to a desired emotion

EDME
7
 Transformation and sequencing algorithms are applied on
selected segments to form song-like structures
 These song-like structures are then synthesized and given
to the listener
Real-Time
Stage
Offline Stage

8
Outline
 Introduction
 EDME
 Experiments
 Demonstration
 Conclusion

Interactive Installation
 The installation is composed by a computer system that
uses a camera to take pictures of the environment

 The system analyses these pictures to recognize the
number of participants and quantity of movement
System recognizes:
• Number of Participants
• Quantity of Movement
System recognizes:

 These two variables are used to define, respectively, the
values of valence and arousal of the emotional state of the
system
System recognizes:
System recognizes:
Emotional State
of the System
Emotional State
of the System

12
 These two variables are used to define, respectively, the
values of valence and arousal of the emotional state of the
system
 The emotional state influences the expression of music and
images to the environment
System recognizes:
System recognizes:
Emotional State
of the System
Emotional State
of the System
Emotional
expression
(Music & Images)
Emotional
expression
(Music & Images)

13
 The number of participants is determined by the
percentage of change between scenario (image taken when
the system starts) and current image
 Quantity of movement is determined by the change
between current and last image

14
 EDME generates music according to the values of valence
and arousal of the system
 As images are emotionally classified with valence, the
selection of images is only based on this emotional
dimension

15
 The emotional behaviour of the system is controlled by a
multi-agent system

16
Outline
 Introduction
 EDME
 Experiments
 Demonstration
 Conclusion

Experiments
17
 We made a preliminary experiment with students with the
objective of observing the interaction between the system
and participants

Experiments
18
 The second experiment was focused in obtaining
participants feedback via a questionnaire of 7 questions
 Example of questions:
 Music expressed expected emotion?
 Importance of images in the emotional expression?
 System expressed expected emotion?

Experiments
19
 Answers from the questionnaire show us that:
 the system correctly related arousal with the amount of movement
and valence with the number of participants
 the system was successful in the expression of emotions with both
music and images

20
Outline
 Introduction
 EDME
 Experiments
 Demonstration
 Conclusion

21
Outline
 Introduction
 EDME
 Experiments
 Demonstration
 Conclusion

Conclusion
22
 EDME is prepared to be used in interactive contexts
 People experienced and influenced the emotional state of
the system
 Both music and images are relevant in the emotional
expression of the system
 Installation assumes no variation of environmental
conditions; however, it can be adapted to deal with this
situation by changing the scenario

Emotion-Based Interactive Installation

More Related Content

Similar to Emotion-Based Interactive Installation (20)

More from António Oliveira (10)

Recently uploaded (20)

Emotion-Based Interactive Installation