WebRTC + Socket.io: building a skype-like video chat with native javascript
30 likes26,868 views
The document outlines the development of a Skype-like video chat application using WebRTC and Socket.IO technologies in native JavaScript. It details the components involved, including media streaming, peer-to-peer data sharing, and the signaling required for establishing connections. Key implementation steps and challenges related to user presence and conversation management are also discussed.
WebRTC + Socket.io: building a skype-like video chat with native javascript
- 1. WEBRTC + SOCKET.IO BUILDING A SKYPE-LIKE VIDEO CHAT WITH NATIVE JAVASCRIPT /MICHELE DI SALVATORE @MIKDISAL Javascript Architect @ Objectway
- 2. Audio and video communication and peer-to-peer data sharing through a web application Native javascript (no plugins) Open source appear.in
- 3. HOW IS BORN Global IP Solutions acquired by Google in 2010 Google open sourced the technologies
- 4. SUPPORTED BROWSERS AND PLATFORMS
- 5. IE AND SAFARI? freeTemasys Plugin
- 7. MEDIASTREAM (AKA GETUSERMEDIA) Acquiring audio and video Can contain multiple 'tracks' var constraints = { video: { mandatory: { minWidth: 640, minHeight: 360 } }; function successCallback(stream) { var video = document.querySelector("video"); video.src = window.URL.createObjectURL(stream); } function errorCallback(error) { console.log("navigator.getUserMedia error: ", error); } Constraints controls the contents of the MediaStream
- 8. RTCPEERCONNECTION Communicating audio and video Signal processing Codec handling Peer to peer communication Security Bandwidth management ...
- 9. RTCDATACHANNEL Communicating arbitrary data Same API as WebSockets Ultra-low latency Unreliable or reliable Secure
- 10. SIGNALING Exchanging 'session description' objects What formats I support, what I want to send Network information for peer-to-peer setup Any message mechanism and protocol
- 11. JSEP JavaScript Session Establishment Protocol
- 12. RTCSESSIONDESCRIPTION EXAMPLE v=0 o=- 7614219274584779017 2 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE audio video a=msid-semantic: WMS m=audio 1 RTP/SAVPF 111 103 104 0 8 107 106 105 13 126 c=IN IP4 0.0.0.0 a=rtcp:1 IN IP4 0.0.0.0 a=ice-ufrag:W2TGCZw2NZHuwlnf a=ice-pwd:xdQEccP40E+P0L5qTyzDgfmW a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level a=mid:audio a=rtcp-mux a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:9c1AHz27dZ9xPI91YNfSlI67/EMkjHHIHORiC a=rtpmap:111 opus/48000/2 ...
- 13. P2P IN REAL WORLD NAT & Firewalls?
- 14. STUN SERVER Gives my public IP address Simple server, cheap to run Data flows peer-to-peer
- 15. STUN SERVER
- 16. TURN SERVER Fallback for p2p failing Data via server Call works everywhere
- 17. STUN + TURN SERVERS
- 18. ICE Interactive Connectivity Establishment A framework to connect peers via UDP, enabling RTCPeerConnection via STUN and TURN server
- 19. WEBRTC SECURITY On Chrome only via https Encryption for media and data Sandboxed
- 20. WEBRTC LIBRARIES & CO. by by by by ... adapter.js webrtc rtc-everywhere contra SimpleWebRTC &Yet RTCMultiConnection Muaz Khan
- 21. RTCMULTICONNECTION A javascript wrapper library supporting approximately all possible peer-to-peer features. WEBRTC EXPERIMENTS Tons of open source experiments based on RTCMultiConnection: webrtc-experiment.com
- 22. WEBSOCKETS Standardized in 2011 Interactive communication session between browser and server Event driven Sending strings and binary data
- 23. WEBSOCKETS BROWSER SUPPORT All and IE10+ ajax polling as fallback
- 24. SOCKET.IO A popular open source library with client and server implementations Server side written in Node and easy to start var server = require('http').createServer(); var io = require('socket.io')(server); io.on('connection', function(socket){ socket.on('hi!', function(data){ socket.emit('a-message', {hello: 'world'}); }); socket.on('disconnect', function(){}); }); server.listen(3000);
- 25. SOCKET.IO CLIENT var socket = io.connect('http://localhost:3000/'); socket.on('connect', function () { socket.emit('hi!'); }); socket.on('a-message', function (data) { console.log(data.hello); });
- 26. GOAL A SKYPE LIKE CHAT
- 27. MAIN ISSUES WebRTC architecture like IRC rooms with shared chat: text, video and data direct chat: only one to one room managed by initiator conversation type can change without agreement: text to video
- 28. IMPLEMENTATION
- 29. STEP 1 UserA is logged in and is connected to the socket server
- 30. STEP 2 The app sends a "presence" event to other users through the socket The server forwards the message only to user's contacts Each present user replies with the same "presence" event
- 32. STEP 3 Each present user replies with the same "presence" event
- 33. STEP 4 When a user receives a "presence" event, it automatically knows that the sender is online Also the server sends the “presence” event to all, when a user socket is disconnected
- 34. STEP 5 UserA starts a conversation and the app creates the webrtc connection
- 35. STEP 6 UserA sends a message via webrtc
- 36. STEP 7 UserB receives a notification
- 37. STEP 8 UserB starts writing a message
- 38. STEP 9 UserA receives the "is typing" info sent via webrtc
- 39. STEP 10 UserA starts a video call sending a request to UserB via socket
- 40. STEP 11 UserB receives the request and accepting starts a video call via webrtc
- 41. STEP 12 The 2 users can video chat via webrtc Each user can close the video at any time or start a text chat
- 42. STEP 13 A user can send text messages also
- 43. STEP 14 A user can start an audio chat in the same way as a video chat
- 45. WHERE IS WEBRTC? Behind this actions chat messages video / audio call "is typing" info
- 46. AND THE OTHERS? via WebSockets presence video / audio initialization all status infos
- 47. THAT'S ALL! THANK YOU ALL /Michele Di Salvatore @MikDiSal