MediaEval 2018: Emotion and theme recognition in music using jamendo
- 1. Emotion and Theme recognition in music using Jamendo A new MediaEval task Alastair Porter, Dmitry Bogdanov, Philip Tovstogan Music Technology Group, Universitat Pompeu Fabra
- 2. Emotion and mood in MIR ● A popular task in Music Information Retrieval ● Relevant for music search and recommendation ● MIREX (Music Information Retrieval Evaluation eXchange) https://www.music-ir.org/mirex/wiki/MIREX_HOME ● Commercial music (cannot be shared outside of contest) ● 5 clusters of similar moods Cluster_1: passionate, rousing, confident,boisterous, rowdy Cluster_2: rollicking, cheerful, fun, sweet, amiable/good natured Cluster_3: literate, poignant, wistful, bittersweet, autumnal, brooding Cluster_4: humorous, silly, campy, quirky, whimsical, witty, wry Cluster_5: aggressive, fiery, tense/anxious, intense, volatile,visceral X. Hu and J. S. Downie, “Exploring mood metadata: Relationships with genre, artist and usage metadata,” in International Conference on Music Information Retrieval, 2007.
- 3. Emotion and mood in MIR ● Presented in MediaEval 2015: http://www.multimediaeval.org/mediaeval2015/emotioninmusic2015/ ○ 1000 Songs, annotated by 10 subjects each ○ Audio samples from Free Music Archive Soleymani, M., Caro, M. N., Schmidt, E. M., Sha, C. Y., & Yang, Y. H. (2013). 1000 songs for emotional analysis of music. In Proceedings of the 2nd ACM international workshop on Crowdsourcing for multimedia (pp. 1-6). ACM.
- 4. Emotion and mood in MIR ● Mood dataset used in AcousticBrainz, prepared by Cyril Laurier http://acousticbrainz.org/datasets/accuracy#mood_acoustic ● acoustic, electronic, aggressive, happy, sad, relaxed, party Laurier, C., Meyers, O., Serra, J., Blech, M., & Herrera, P. (2009). Music mood annotator design and integration. In 7th International Workshop on Content-Based Multimedia Indexing (CBMI'09), pp. 156-161.
- 5. Jamendo ● https://jamendo.org ● Database of music released under creative commons licenses, can be uploaded by anyone ● Metadata added by performers during upload, including instrumentation, vocals, genre, moods, and themes ● Commercial product, “Jamendo Licensing”, to allow musicians to earn from their works
- 6. Jamendo
- 7. Jamendo data ● 50k tracks with annotations, including mood and theme ○ 3.3k happy ○ 2.8k dark ○ 2.3k energetic ○ 2.2k relaxing ○ 2.2k epic ○ 2.0k melodic ○ 2.0k love ○ ...
- 8. Possible approaches ● Label classification (Autotagging) ● Label cluster classification (e.g. MIREX) ● Arousal / Valence regresion (Emotion In Music 2015) ● Arousal / Valence segment classification (ISMIR 2018)
- 9. Mapping tags to emotion space ● 267 mood and theme tags ⇒ 204 tags in Warriner’s list ○ 13,915 English words with Arousal and Valence ratings according to Russell’s model ● Instead of asking for people to listen and annotate song, use existing labels and map it to a space Warriner, A.B., Kuperman, V., & Brysbaert, M. (2013). Norms of valence, arousal, and dominance for 13,915 English lemmas. Behavior Research Methods, 45 Musical Texture and Expressivity Features for Music Emotion Recognition, Renato Panda, Ricardo Malheiro, and Rui Pedro Paiva International Society on Music Information Retrieval Conference 2018
- 10. Mapping tags to emotion space data I (22695)II (2610) III (8443) IV (29301)
- 11. Jamendo data ● Some tracks have tags that fall into multiple quadrants ● 28k out of 50k tracks fall in a single quadrant
- 12. Our open questions ● How many of the possible approaches are a good idea for subtasks? ○ Classification, Clustering, continuous ○ The "Emotion in Music Task" MediaEval task focused on continuous annotations. Is it an issue if we don’t? ● How to deal with skewed data (“all music is happy”) ● In classification, what to do with neutral labels (close to 0) ● Evaluation metrics?
- 13. Thanks