Multimodal analysis1
Download as PPT, PDF1 like596 views
This document provides an overview of different media and modes that can be used for multimodal interactional analysis, including sound, video, photography, and field notes. It discusses analyzing these different media sources to understand how technologies influence relationships and communication. Key things to look for include reciprocity, rituals, defining physical space, and divided consciousness. The document also provides guidance on developing multimodal themes by identifying significant moments across media sources and relating them to literature on multimodal communication. Standard annotation conventions are suggested for transcription.
Multimodal analysis1
- 2. Capturing Different Media 1. Sound 2. Video 3. Photo 4. Observation (Field Notes)
- 3. Media Modes Sound Speech, music, laughter, background noise Video Lighting Ambient lighting in homes, daylight Physical presence Posture, facial and bodily gestures, eye contact, pointing Production elements On screen effects, framing of shots, cutting between scenes, etc. Physical objects Placing objects in field of view, gesturing with objects, Photography Similar to video but without the movement/time aspects Observation Spoken language, physical behaviours, social (field notes) dynamics, printed text/signs, etc.
- 4. Multimodal Activity Observe and record examples of how technologies influence the way that people relate to one another Use one of the following media: o Sound Recording o Observation (Field Notes) l Still Photos l Video Recording
- 5. Things to look out for? Reciprocity in communication coordination of turn taking in conversation ‘gift’ exchange Ritualised behaviour openings and closings Defining the ‘meaning’ of physical space What behaviours are allowed/expected/norms Reinforcement of these norms Divided Consciousness/Absent Presence Dealing with immediate and distant social contact simultaneously
- 6. Feedback Session What modes have been captured? What different stories do they tell? Were there any issues with collecting/using particular media?
- 7. Developing Multimodal Themes Watch the videos/listen to recordings/view images / Identify themes ◦ Observe and identify significant moments that highlight multimodal interaction as an aspect of communication – note the time ◦ Refer to existing literature on multimodal aspects of communication to check what you see and hear – for example Goffman (1959) 3. Watch the videos 3 Describe the multimodal aspects of the videos t Roughly transcribe the video data to clarify significant moments in communication and identify commonalities across videos d Watch the videos e Final transcription in a suitable form to explain themes and subthemes ◦ Include standard annotation conventions
- 8. Multimodal transcription of video data (using my own coding scheme)
- 9. Multimodal Transcription www.sigridnorris.com
- 10. Multimodal transcription – from conversational analysis: (n) pause noted in seconds = joining of words : an extension of the preceding syllable; ; rising or falling intonation for subsequent utterance underlined text indicates spoken with particular vocal energy - a sudden cut off to an utterance; ? a rising tone . a falling tone , a shifting continuous tone bridging utterances.
- 13. References Clark, H. H. (1992). Arenas of language use. Chicago: The University of Chicago Press. Clark, H. H., & Schaefer, E. F. (1989). Contributing to discourse. Cognitive Science, 13, 259–294 Kendon, A., Harris, R. M., & Ritchie Key, M. (1975). Organization of behavior in faceto-face interaction. Paris: Mouton Publishers. Kendon, A. (1990). Conducting interaction: Patterns of behavior in focused encounters.Cambridge: Cambridge University Press. Goffman, E. (1963). Behavior in public places: Notes on the social organization of gatherings. New York: The Free Press. Ten Have, P. (1999). Doing conversation analysis. A practical guide. London: Sage Publications.
Editor's Notes
- #4: Media and modes are not distinct entities