Introduction to WebRTC
Download as PPTX, PDF8 likes4,654 views
The document provides an introduction to WebRTC, a technology that enables real-time audio and video communications in web applications via simple JavaScript APIs, allowing peer-to-peer connections without plugins. It discusses various aspects of WebRTC including its architecture, APIs like getUserMedia and RTCPeerConnection, and outlines its support across different browsers and devices. Additionally, it highlights real-life applications and use cases of WebRTC, particularly emphasizing Gruveo, a platform that offers easy-to-use video calls.
Introduction to WebRTC
- 1. Introduction to WebRTC ART MATSAK, GRUVEO JANUARY 2016
- 2. Imagine If… • You could add HD video calling to your web app using simple JavaScript APIs • Your users could make calls right in the browser, no plugins • Peer-to-peer • Industry-standard end-to-end encryption for all calls • Sending arbitrary data, too Too good to be true? © 2016 GRUVEO 2
- 3. Try It Right Now 1. Go to www.gruveo.com 2. Enter “test123” for the code 3. Hit “Video call” 4. Go to Gruveo on another device or browser tab, and do the same 5. Enjoy your conversation © 2016 GRUVEO 3
- 4. What Is WebRTC? • Real-Time Communications for the Web • A free, open project hosted at webrtc.org • Provides RTC capabilities for browsers and apps via simple APIs • Idea born in the Google Chrome team in 2009 • Acquired On2 in 2010 and Global IP Solutions (Gips) in 2011 • Gips technology open-sourced in mid-2011 to give start to WebRTC © 2016 GRUVEO 4
- 5. How Simple Is It? Initialization of a WebRTC call in a nutshell: var pc = new RTCPeerConnection(); navigator.getUserMedia({video: true}, function(stream) { pc.addStream(stream); pc.createOffer(function(offer) { pc.setLocalDescription(new RTCSessionDescription(offer), function() { // Send the offer to a signaling server to be forwarded to the peer. }, errorCb); }, errorCb); }); © 2016 GRUVEO 5
- 6. WebRTC Connection Diagram signaling server audio / video STUN/TURN server WebRTC signaling NAT traversal audio / video (if relayed) © 2016 GRUVEO 6
- 7. A Tale of Two APIs • navigator.getUserMedia() (gUM) • Prompts the user to use a video and/or audio device (camera, screensharing, microphone…) • Produces a MediaStream with one or more MediaStreamTrack’s • RTCPeerConnection • Represents a WebRTC peer-to-peer connection • Handles bi-directional streaming of media as well as data • Usually has one or more MediaStream’s attached © 2016 GRUVEO 7
- 8. navigator.getUserMedia(): Accessing User’s Media Devices Example - prompt to use the camera and show its feed in a <video> element: navigator.getUserMedia({ audio: true, video: { width: 1280, height: 720 } }, function(stream) { var video = document.querySelector('video'); video.src = window.URL.createObjectURL(stream); video.onloadedmetadata = function(e) { video.play(); }; }, function(err) { console.log("The following error occurred: " + err.name); } ); © 2016 GRUVEO 8
- 9. getUserMedia(): User Experience • UX is different browser to browser • HTTPS now required for cross-browser compatibility! Requires HTTPS Device selection in dialog Remembers permissions Chrome Yes No Yes Firefox No Yes No Opera No Yes HTTPS only © 2016 GRUVEO 9
- 10. getUserMedia(): Constraints • A MediaStreamConstraints object • Specifies the types of media to request + any requirements for each type • Examples: { audio: true, video: { width: 1280, height: 720 } } { audio: true, video: { width: { min: 1024, ideal: 1280, max: 1920 }, height: { min: 776, ideal: 720, max: 1080 } } } { audio: true, video: { facingMode: "user" } } { audio: true, video: false } navigator.getUserMedia(constraints, successCb, errorCb) © 2016 GRUVEO 10
- 11. getUserMedia(): Constraints – Screen Sharing • To enable screen sharing, user has to: • Install an add-on for your service / domain (Chrome) • White-list your domain in browser preferences (Firefox) • Installation of a Chrome add-on only serves as an explicit confirmation that the user allows screen sharing on your domain • Then, screen sharing is as easy as passing this video constraint: • { video: { mandatory: { chromeMediaSource: "screen" } } } // Chrome • { video: { mediaSource: "screen" } } // Firefox navigator.getUserMedia(constraints, successCb, errorCb) © 2016 GRUVEO 11
- 12. getUserMedia(): Success Callback • Called if user grants access to media device(s) • Gets passed a MediaStream object containing one or more MediaStreamTrack’s (audio / video) • A MediaStream can be attached to a <video> or <audio> element… • ...or to an RTCPeerConnection object to be sent to the other peer navigator.getUserMedia(constraints, successCb, errorCb) © 2016 GRUVEO 12
- 13. getUserMedia(): Error Callback • Called if something went wrong, for example: • User denied access to media devices • Device (e.g. video camera) not found • Attempting to call navigator.getUserMedia() on an HTTP page (Chrome) • Gets passed an object / string with more information about the error navigator.getUserMedia(constraints, successCb, errorCb) © 2016 GRUVEO 13
- 14. RTCPeerConnection: P2P Connection Between Clients signaling server audio / video STUN/TURN server WebRTC signaling NAT traversal audio / video (if relayed) © 2016 GRUVEO 14
- 15. RTCPeerConnection: Connection Establishment, or “Offer-Answer Dance” Create offer Set description as local Send description to callee Set description as remote Create answer Set description as local Send description to callerSet description as remote A’s session description B’s session description Peer A (caller) Peer B (callee) A’s description B’s description © 2016 GRUVEO 15
- 16. RTCPeerConnection: Key Methods • RTCPeerConnection(configuration, constraints) – constructor, returns a new RTCPeerConnection object • .addStream(mediaStream) – attaches a MediaStream as a local audio / video source • .createOffer(successCb, errorCb, constraints) – creates offer session description that gets passed to successCb • .createAnswer(successCb, errorCb, constraints) – creates answer session description, passed to successCb • .setLocalDescription(description, successCb, errorCb) – sets description as local session description • .setRemoteDescription(description, successCb, errorCb) – sets description as remote session description © 2016 GRUVEO 16
- 17. RTCPeerConnection: Session Description Protocol (SDP) • Used for negotiating session capabilities between peers • Session description includes information about media streams, media codecs and their parameters, and more • Example of audio lines in a session description: ... m=audio 54278 RTP/SAVPF 111 103 104 0 8 106 105 13 126 c=IN IP4 180.6.6.6 a=rtcp:54278 IN IP4 180.6.6.6 ... • Multi-line text format allows for mangling when necessary, e.g. to prioritize one codec over another © 2016 GRUVEO 17
- 18. RTCPeerConnection: Signaling • How do peers exchange session descriptions etc. before the P2P connection is established? – Via a signaling server! • Signaling is intentionally left to implementation by WebRTC • Any of these would work: • Long polling • WebSockets + Socket.IO • Pigeon post, telegraph, UPS... • Gruveo uses WebSockets + custom Node.js backend © 2016 GRUVEO 18
- 19. Connection Diagram: Signaling signaling server audio / video STUN/TURN server WebRTC signaling NAT traversal audio / video (if relayed) © 2016 GRUVEO 19
- 20. RTCPeerConnection: ICE • Your localhost IP is 127.0.0.1, internal IP may be 192.168.1.100 and external IP may be 147.232.159.135 • ICE (Interactive Connectivity Establishment) allows for discovering all those IPs to find a way for the remote peer to reach your host • Requires STUN (Session Traversal Utilities for NAT) servers • Think of it as WhatIsMyIP.com for WebRTC • Good news: There are a lot of free public ones • A list of STUN servers is passed to the RTCPeerConnection constructor © 2016 GRUVEO 20
- 21. RTCPeerConnection: ICE Contd. • Applying local session description initiates ICE candidate gathering • Think of ICE candidates as IP-port pairs whereby your host is reachable • An ICE candidate line from a session description: a=candidate:1853887674 1 udp 1845501695 46.2.2.2 36768 typ srflx raddr 192.168.0.197 rport 36768 generation 0 • Peers exchange ICE candidates via signaling: pc.onicecandidate() fires Send candidate to peer pc.addIceCandidate(candidate) A’s ICE candidate Peer A Peer B ICE candidate © 2016 GRUVEO 21
- 22. RTCPeerConnection: TURN • What if a peer-to-peer connection cannot be established due to blocked ports etc.? • You’ll need a relay – WebRTC supports TURN (Traversal Using Relays around NAT) protocol • Things to consider when deploying TURN servers: • Bandwidth charges (AWS is expensive! Azure is, too!) • Minimizing latency – automatic choosing of a TURN server based on latency or geographic distance from peers • A list of TURN servers is also passed to the RTCPeerConnection constructor • TURN servers often serve as STUN, too, hence STUN/TURN server © 2016 GRUVEO 22
- 23. Connection Diagram: STUN/TURN signaling server audio / video STUN/TURN server WebRTC signaling NAT traversal audio / video (if relayed) © 2016 GRUVEO 23
- 24. Not Only Media – WebRTC Data Channels • Enable exchange of arbitrary application data between peers • Think WebSockets, but peer-to-peer • Customizable delivery properties: • Reliable or partially reliable delivery of sent messages • In-order or out-of-order delivery of sent messages • RTCDataChannel objects, created using RTCPeerConnection.createDataChannel() • Some exciting ideas and use cases out there – e.g. Peer5 © 2016 GRUVEO 24
- 25. The WebRTC Protocol Stack • Peer-to-peer connection is established over UDP using ICE, STUN and TURN • We can live without a few lost frames; low latency is more important • DTLS (Datagram Transport Layer Security) is used to secure all data transfers between peers • Unlike TLS, DTLS can be used over UDP • Encryption is WebRTC’s mandatory feature • SRTP (Secure Real-Time Protocol) is used to transport audio and video streams • SCTP (Stream Control Transport Protocol) is used to transport application data © 2016 GRUVEO 25
- 26. The WebRTC Protocol Stack Contd. Network (IP) ICE, STUN, TURN Transport (UDP) SRTP SCTP Session (DTLS) – mandatory RTCDataChannelRTCPeerConnection © 2016 GRUVEO 26 Adapted from High Performance Browser Networking by Ilya Grigorik (O’Reilly, 2013)
- 27. WebRTC Media Codecs • A WebRTC implementation can support any codecs for audio and video, but some are Mandatory to Implement (MTI) • For audio, those are Opus and G.711 • Opus is free, provides excellent quality at majority of bitrates • G.711 is a “granddaddy” included for compatibility with legacy systems • For video, VP8 and H.264 are MTI • H.264 is the industry standard, hardware encoding and decoding well-supported on mobile • VP8 is free but with fewer hardware implementations • Firefox supports both VP8 and H.264 • Chrome and Opera only support VP8 and now VP9, H.264 is underway • Why does it matter? • Software video encoding and decoding on mobile is a performance + battery life hit © 2016 GRUVEO 27
- 28. Where Is WebRTC Supported Today? • Chrome, Firefox and Opera on desktop (Windows, Mac OS and Linux) + Android • That’s right – video calling right from your mobile browser! • IE10, IE11 and Safari on desktop through a plugin – Gruveo supports the Temasys one • Microsoft Edge supports ORTC • A standard related to WebRTC • Interoperability with WebRTC is currently limited (no video, other issues) • No browser on iOS currently supports WebRTC… • …But you can bake WebRTC into a native app on desktop, Android and iOS Desktop Android iOS Chrome ✔ ✔ ✘ Firefox ✔ ✔ ✘ Opera ✔ ✔ ✘ IE10/IE11 ✔* – – Safari ✔* – ✘ Microsoft Edge ✔** – – Native apps ✔ ✔ ✔ * With a plugin ** ORTC © 2016 GRUVEO 28
- 29. A Closer Look at Mobile • A WebRTC web application will work in major Android browsers out of the box • Native is the only way on iOS. A few choices: • Build webrtc.org code for iOS • Not for the faint of heart but doable • Support for h/w H.264 coding is still in the works • OpenWebRTC from Ericsson Research • Supports h/w H.264 coding • Not as production ready? • Third-party SDKs like EasyRTC, OpenTok etc. • On Android, you can also go fully native or leverage WebRTC in a WebView (v36+) © 2016 GRUVEO 29
- 30. WebRTC – Real-Life Challenges • Things change and break quickly. Some recent stuff: • Chrome 47 broke getUserMedia() on HTTP pages in December 2015 • That same month, Firefox 43 stopped supporting an old format of RTCPeerConnection constraints • Things like this are everyday business in WebRTC Land • Using a polyfill is a very good idea • Gruveo uses a variation of adapter.js from Google • testRTC is a promising service for monitoring your WebRTC application for problems © 2016 GRUVEO 30
- 31. How to Stay in the Loop • webrtc.org – The Mothership • www.w3.org/TR/webrtc/ – The RTCPeerConnection spec • Google Chrome Developers channel on YouTube – Chrome’s WebRTC team chimes in with updates here • BlogGeek.me – All things WebRTC; trends and analysis • webrtcHacks.com – Technical discussion around WebRTC • www.webrtc-experiment.com – More WebRTC hacks and code samples © 2016 GRUVEO 31
- 32. What Will You Do with WebRTC? The limit is only your imagination: • Peer5 leverages data channels to provide a serverless P2P CDN • Tellybean allows TV video calling with WebRTC • Ziggeo does asynchronous video recording • Talko powers powered group voice communications for teams • Acquired by Microsoft / Skype in December 2015 • CrankWheel is one-click screen sharing for enterprises • …And, of course, Gruveo – the world’s easiest video calls © 2016 GRUVEO 32
- 33. Gruveo • Agree on a code and talk • No installs, no registration • Works on all major platforms • Launched in 2013, Flash-based • Switched to WebRTC in 2014 • Got VC backing in 2015 • Exciting use cases in telehealth, customer support, online tutoring… © 2016 GRUVEO 33
- 34. We Are Hiring! www.gruveo.com/jobs Currently looking for: • Web Developer (Frontend + Backend) – JavaScript, Node.js… • Mobile Developer – iOS & Android • Customer Development Specialist – Sales, Product Management and/or Product Marketing background © 2016 GRUVEO 34
- 35. Thank You Questions? Art Matsak art@gruveo.com © 2016 GRUVEO 35