Streaming Media West: Webrtc the future of low latency streaming
6 likes1,366 views
The document discusses the significance and future of WebRTC, highlighting its role in low-latency communication and streaming services. Dr. Alex Gouaillard emphasized WebRTC's adoption by major companies and its advanced capabilities in handling real-time media and resolving bandwidth-related issues. The document also touches on ongoing developments and potential enhancements, including standards for increased security and efficiency.
Streaming Media West: Webrtc the future of low latency streaming
- 1. WebRTC The future (?) champion of low latency Dr Alex Gouaillard, CTO millicast.com
- 2. Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California https://medium.com/spankchain/spankchain-development-update-007-february-27-2018-414247c85445 Real-time? Why? Well, have you ever tried to make love with 5s delay? (NSFW)
- 3. WebRTC ? Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 4. Historically: google gap analysis Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California • I own the net, or at least I have a toll on it • I want everybody to do everything on the net • What are people still using the desktop for they cannot use the net for?
- 5. One-way Media Streaming pipeline Media Capture Screen Display Encoding Decoding Segm entation Fragm entation Reassem bly Encryption Decryption Transport Internet Internet Transport Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 6. One-way Media Streaming pipeline Media Capture Screen Display Encoding Decoding Segm entation Fragm entation Reassem bly Encryption Decryption Transport Internet Internet Transport Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California WebRTC STACK Media Engine WebRTC STACK Media Engine In the Browser UA, No plugin, standard Based on 20 years battle-tested VoIP techs (RTP) latency = ( RTT + 100 ms ) +
- 7. One-way Media Streaming pipeline Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California no need for Eng. Web app is enough
- 8. Google Initiative, to start a standard Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California • 2009 - ON2 acquisitions (libVPx) for 100+ M • 2010 - GIPS acquisition (libjingle => webrtc) 68M • 2011 - Google Announce the WebRTC initiative
- 9. WebRTC Today Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California • Chromecast • Youtube live • Google’s Hangout, meet, allo, duo, • Comcast Xinfinity • Facebook messenger since 2012 • 17 billions video chats in 2017 • messenger is the 2nd most popular iOS app • … https://www.youtube.com/watch?v=n1gahO5S0Bg
- 10. Theoretical Streaming ( with WebRTC ) Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 11. One-way Media Streaming pipeline Media Capture Screen Display Encoding Decoding Segm entation Fragm entation Reassem bly Encryption Decryption Transport Internet Internet Transport Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California Media Engine Media Engine
- 12. Optimisations for pre-recorded content Streaming Media Capture Screen Display Encoding Decoding Segm entation Fragm entation Reassem bly Encryption Decryption Transport Internet Internet Transport Pre-Encoded C ontent D ual-pass encoding Pre-C hunk’ed uploaded C ontent internet cache protected by buffer Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California C an O ptim ise p2p ex: peer5, stream root.io
- 13. Problems / Solutions • Large Bandwidth fluctuations • Heterogeneous (receiving / viewer clients) • Have several resolutions of the original available, and a way to switch between them. • multiple encoders (waste info in low rez) • file chunks => switch in receiving application (slower, longer path bigger chunks, buffers) • WebRTC simulcast => switch in SFU • layered encoder (SVC) => switch in SFU Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 14. Bandwidth Adaptive Media Streaming pipeline Media Capture Screen Display Encoding Decoding Segm entation Fragm entation Reassem bly Encryption Decryption Transport Internet Internet Transport MediaDistributionServer Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California Media Engine Media Engine Media Engine Media Engine
- 15. Bandwidth Adaptive Media Streaming pipeline in practice - the usual MediaCapture Screen Display MediaDistributionServer Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California Media Engine (send) Media Engine (rec) ME (send) Media Engine (send) Media Engine (send) ME (rec) Player Transcoder possibly hosted and distributed e.g. BitMovin Source RTMP HLS
- 16. Bandwidth Adaptive Media Streaming pipeline in practice - webrtc end-to-end MediaCapture Screen Display MediaDistributionServer Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California Media Engine (send) Media Engine (rec) Media Engine (send) Media Engine (send) PlayerSource No storage 1 less enc/dec = 50% load
- 17. Problems / Solutions • Ports, multiplexing, Nat and firewall firewall, transport protocol, … • discovery, handshake, streaming • file-based => “tunnel” over HTTP and use URIs (add delay) • WebRTC => Dynamic transport allocation and NAT traversal with ICE Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 18. Problems / Solutions • Network quality adaptation (jitter and packet loss) • file-based => buffer (Add delay) • WebRTC => use RTP (jitter buffer, NACK, PLI, RTX, RED, FEC) • Small Network bandwidth adaptation • file-based => buffer (Add delay), and file-based (add delay) • WebRTC => bitrate adaptation based on RTP bandwidth evaluation • Multistreams congestion control and multiplexing • file-based => some multiplexing included, no CC • WebRTC => rtcp-mux, BUNDLE, and transport-CC Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 19. Practical Streaming ( with WebRTC ) Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 20. Simulcast, codecs, SVC support Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California 10/2018 10/2018 04/2018 06/2018 xx/2018 Millicast / CoSMo software contributions to the browser vendor code base
- 21. Simulcast, codecs, SVC support Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 22. webrtc.org the default client side stack Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 23. CallStats.io: Network Instrumentation and Stats: Verify Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 24. KITE Interop SE Grid - Browser configs Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 25. WebRTC Testing: Interesting scientific publications NARVAL, A No-Reference Video Quality Tool for Real-Time Communications, Augustin Lemesle, Alexis Marion, Ludovic Roux and Alexandre Gouaillard in Proceedings of Human Vision and Electronic Imaging, Burlingame, California, USA, January 2019 Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 26. 1 OS “Kite” Interop. mode: AppRTC Design Conductor Dispatcher 2 browsers, AppRTC-Test list of N configs Validate Config, against SE Grid Interop. mode: Compute list of tests, i.e. pairs of BrowserObj AppRTC-Test, N^2 pairs of BO Smart Load Balancing of the N^2 pairs, Aggregate results and send to Dashboard 1 2 AppRTC-Test, 1 pair< BO > Run tests and instruments 2 browsers Collect/Compute result Pluggable AppRTC-Test Connect to appRTC URL wait for “connected || completed” or timeout Pluggable Hybrid SE Grid local-Grid Rem-BB Rem-BaaS (SauceLabs) Rem-BaaS (BrowserStack) Rem-VM (AWS) Test Runner Validate Configs Pluggable Viz Dashboard Visualize and Interact with Results Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California Test Runner Test Runner Test Runner
- 27. Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California Comparative Study of WebRTC Open Source SFUs for Video Conferencing, Emmanuel André, Nicolas Le Breton, Augustin Lemesle, Ludovic Roux and Alex. Gouaillard in Proceedings of IIT Real-Time Communications, Illinois Institute of Technology, Chicago, USA, October 2018 WebRTC Testing: Interesting scientific publications
- 28. WebRTC Testing: Interesting scientific publications Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 29. Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California WebRTC Testing: Interesting scientific publications (1/2)
- 30. WebRTC Testing: Streaming: CPU footprint Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 31. WebRTC Testing: Streaming: Bitrate Adaptation Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 32. WebRTC Testing: Bitrate Adaptation Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 33. WebRTC at scale: cascading, not clustering Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California • Vidyo https://vidyo.io/blog/features/vidyo-io-delivers-massive- scalability-maintaining-reliability-quality-cascading-sfus/ • Jitsi: https://webrtchacks.com/sfu-cascading/
- 34. WebRTC at scale: cascading, not clustering Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 35. Conclusion - the future Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California • Webrtc is already used by giants, accumulating billions of minutes of audio and video annually. The streaming industry seems to be slightly late to the game. • The standards are already thinking about Next Version: QUIC, end-to-end encryption, Machine learning, AV1, …..
- 36. Conclusion - the future Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 37. Conclusion - the future Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 38. Conclusion - the future Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California Oct, 31st 2018 first public webrtc call with AV1 Oct, 31st 2018 happy halloween effect.
- 39. Thank You Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 40. The WebRTC stack Media Capture Screen Display Encoding Decoding Raw Media Bitstream Segmentation Fragmentation Reassembly Jitter bufferRTP packet RTCP packetpayload S & R S & R Encryption Decryption SRTP packet TCP / UDP / TLS / DTLS / QUIC packets IPv4,6 packets Transport Internet link layer and below (3) Dynamically allocated through ICE Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 41. The HLS / MPEG-DASH way Media Capture Screen Display Encoding Decoding Raw Media Bitstream Segmentation Fragmentation Reassembly fmp4, MPEG2 TS, CMAF S & R S & R Encryption Decryption AES-128 HTTP(s) => TCP(TLS) packets IPv4,6 packets Transport Internet link layer and below (3) Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California
- 42. The WebRTC stack Media Capture Screen Display Encoding Decoding Raw Media Bitstream Segmentation Fragmentation Reassembly Jitter buffer RTP packet: media transport RTCP packet: Bitrate Adaptation, Bandwidth estimation, Congestion Control payload S & R S & R Encryption Decryption SRTP packet: authenticated packet with encrypted content TCP / UDP / TLS / DTLS / QUIC packets IPv4,6 packets Transport Internet link layer and below (3) All Media Specific features are in RTP, Not in TRANSPORT ! Dr. Alex Gouaillard - @agouaillard - webrtcbydralex.com Streaming Media West - November 14, 2018 - Huntington Beach, California