Introduction to Audiovisual Communications
- 1. Audiovisual Communications Introducción a las Comunicaciones Audiovisuales Profesores: Francesc Tarrés, Gabriel Montoro 1
- 2. Audiovisual Communications Introduction •Audiovisual Communications Technologies •Set of technologies for transmitting or storing signals representing speech, audio, image or video information in different formats. •Audiovisual Technologies Impact: •Communications among persons through sound and image is (still today) the media of major social impact. •Classical Media: Films, TV, Radio, Telephone •Networked contents: Production webs, Podcast, Video On-Demand •Contents merchandising: iTunes, Spotify •Personal Communications: Facetime, WeChat •Social Networks: Facebook, Instagram, Flickr, etc. 2
- 4. Audiovisual Communications Historical Evolution •Constantly improving the audiovisual experience •Technologies are ‘determined’ by the characteristics of human auditory and visual systems. •Today almost all AV systems are digital. This course is centered on digital systems. •However, many AV paradigms are understood when looking to the historical evolution of the system. Some KEY historical aspects are introduced to facilitate the understanding of current systems. •Audiovisual Systems are evolutionary. The introduction of new technologies preserves some degree of compatibility with previous systems. 4
- 5. Audiovisual Communications Introduction •Evolution example (Comercial TV) •Black & White Television •Analog Color •Digital Color (Standard Definition) •High Definition •3D •Ultra High Definition •Non-evolutionary attempts have usually failed •Analog High Definition TV in Europe (late 80s, beg. 90s) •Economic efforts for acquiring new technologies is almost constant along years 5
- 6. Audiovisual Communications Future Systems •It is difficult to predict the evolution of AV systems. Past experience show that many promising AV systems have not succeed: HD-MAC, DCC, Laser-disc, Minidisc, etc. •A possible future line prediction could be •Ultra-High Definition (4K and 8K commercially available at low cost) • Cameras, editing software, displays •High Quality Audio Portable systems: Pono •Introduction of Multisense information into the experience • Location, motion, body sensors, weather, light, etc. •Advances in 3D technologies • Displays without glasses (multiuser) • Introduction of holographic technologies • Multicamera - Multiview •Capturing and transmission of otner sensorial signals • Olphative • Tactil 6
- 7. Audiovisual Communications Future Systems •No matter how real systems evolve the main characteristics from the point of view of signal processing will remain the same •The following steps will be necessary 1. Signals must be captured and coded efficiently 2. The coding has to be adapted to the transmission channel or to the storage support. 3. Reproduction technologies must guarantee a quality experience to the audience •The design and technologies of these stages has to be adapted to the psycho-physiology human sensorial system (auditory, visual, olphactive, etc) 7
- 8. Audiovisual Communications Communication Systems Block Diagram 8 Capture Source Coding Multiplexer Encryption & Channel Coding Channel Decoding & Decryption Demux Source Decoding Presentation Storage o Transmission Modulators & Demodulators (QAM, OFDM, PSK) IP data transmission File Downloading DVD-Video, Blu-ray Capture Presentation Signal transducer Source Coding Compression Multiplexing Collect several channels into one stream Encryption and Channel Coding Encription Adapt Data to Media Error protection Microphones Video Cameras Displays Speakers A/D and D/A PCM coding FLAC, APE MP3, AAC TIFF, PNG JPEG, JP2 MPEG-2, H.264 MPEG-TS MPEG- PS AVI, MP4, MKV Block Coding Hamming Reed-Solomon Conv. Coding Trellis LDPC Condtional Access Encription Scrambling
- 9. Audiovisual Communications Digital AV Systems •Advantages •Immunity to noise •Easy reprogramming •Low dependence on external factors (e.g. temperature) •Easy encryption, data protection, Digital Rights Management •Disadvantages •Introduction of Quantization Noise •Bandwidth Increase •Difficulty for managing huge databases (video browsing and retrieval) 9
- 10. Audiovisual Communications Quantization Noise •Example in audio •The number of levels in the A/D determines the quality of the resulting digital signal. •Error in quantizing is equivalent to noise. •2 different criteria for digitalization •Quality: Suppose analog signal is noise free. Determine the number of bits necessary for a human being not being able to distinguish between original analog and the digital version • N bits audio = 16 bits (20 bits, 24 also possible in some systems). •SNR. Suppose the analog signal has a given SNR. Choose the number of bits (number of levels) to have a quantization noise below the analog noise. 10 S NR = 6N + 1.76
- 11. Audiovisual Communications Bandwidth Increase •Example audio (CD audio - 1 channel) •Example video (Standard Definition) 11
- 12. Audiovisual Communications Example in Digital Cinema 12 Bandwidth calculation in Digital Cinema Applications Digital Cinema requires a huge bandwidth to support a full film if no compression is applied
- 13. Audiovisual Communications Browsing and Retrieval •Insert Metadata in Audio and Video Files •Manually •Index data by humans: ex. soccer, TV News, others •Automatic •Find automatic descriptors (low-level) for audio and video data. •Use these low level descriptors to infer semantic contents •Standard MPEG-7 •Other proprietary solutions: •Google Images, Shazam, Midomi, SoundHound, etc. 13
- 14. Audiovisual Communications Digital AV Systems 14 •Advantages •Immunity to noise •Easy reprogramming •Low dependence on external factors (e.g. temperature) •Easy encryption, data protection, Digital Rights Management •Disadvantages •Introduction of Quantization Noise •Bandwidth Increase •Compression: MP3, AAC, MPEG-1, MPEG-2, MPEG4, H.264, H.265 •Difficulty for managing huge databases (video browsing and retrieval) • Metadata description • MPEG-7
- 15. Audiovisual Communications List of Contents •Introduction •Image Processing I: Point Transforms •Image Processing II: Spatial Transforms •Cameras. Principles of geometric optics •Audiovisual Systems: Historical Perspective •Principles of Compression •Still Image Compression: JPEG, JPEG 2000 •Video Compression: MPEG-2, MPEG4, H.264, H.265 •Fundamentals of audiovisual description: MPEG-7 15
- 16. Audiovisual Communications Organization •Theory •2 sessions of 1.5 h every week (Prof. Francesc Tarrés) •Controls and Exams •Arrange meetings: francesc.tarres@upc.edu •Updated slides and textbooks: francesctarres.wordpress.com •Laboratory •1 session of 2 h every week (Prof. Gabriel Montoro) •Lab work and Lab reports •Arrange meetings: gabriel.montoro@upc.edu 16
- 17. Audiovisual Communications Assessment 17 Control (week 4-5) 10% Mid-Term Exam 20% Final Exam 30% Lab work 15% Lab reports 15% Participation 10%