How to Architect your WebRTC application, Alberto Gonzalez and Arin Sime, WebRTC.Ventures
Download as PPTX, PDF0 likes211 views
The document discusses strategies for architecting WebRTC applications, outlining three primary approaches: building to the WebRTC standard, using open source media servers, or utilizing CPaaS (Communications Platform as a Service). It highlights key components, such as STUN/TURN servers, application signaling, and different server architectures like SFUs and MCUs for scaling applications. Additionally, it provides considerations for specific use cases like 1-1 video chats, group chats, and live interactive broadcasting.
How to Architect your WebRTC application, Alberto Gonzalez and Arin Sime, WebRTC.Ventures
- 1. How to Architect your WebRTC application Alberto Gonzalez and Arin Sime TADSummit Nov 10-11, 2021
- 2. @WebRTCventures Alberto Gonzalez, CTO Arin Sime, CEO Agenda • Why it’s not easy to build with WebRTC • Open Source vs CPaaS • SFUs and MCUs • A standard 1-1 app • Group Chat • Live Interactive Broadcasting • Contact Centers How to Architect your WebRTC application!
- 3. WebRTC is not quite this simple… • STUN/TURN servers • Application Signaling • Video codecs • Group chat/scaling • Browser/Mobile Support • Recording @WebRTCventures
- 4. 3 Ways to build your app… 1. To the standard, ie “build your own stack” 2. Open source media servers 3. CPaaS – Communications Platform as a Service @WebRTCventures
- 5. #1 – Building to the WebRTC Standard • Compiling webrtc lib • STUN/TURN servers • Application Signaling • Video/audio codecs • Group chat/scaling • Browser/Mobile Support • Recording/Other Add-on features • Can better utilize capabilities like WebCodecs, WebTransport and control low level details for specific use cases You must build and handle of the following – with great power comes great responsibility! https://webrtc.org/ @WebRTCventures
- 6. #2 – Open Source Media Servers Media Servers will handle: • video/audio details • part or all of the signaling • Possibly STUN, TURN • Scaling capabilities • Could be SFUs or MCUs • Browser/Mobile support But you host/manage: • All infrastructure and updates Media Servers Your cloud servers @WebRTCventures
- 7. #2 – Open Source Media Servers Media Servers Your cloud servers janus.conf.meetecho.com jitsi.org @WebRTCventures Popular examples: Pion.ly mediasoup.org
- 8. #3 – CPaaS – Communications Platforms Your Application servers A CPaaS will handle: • All WebRTC support / updates • Media Servers • STUN/TURN • Web/Mobile Support • Additional features like Recording, SMS, Voice/VOIP, Transcription, etc But you pay according to usage CPaaS @WebRTCventures
- 9. #3 – CPaaS – Communications Platforms Your Application servers CPaaS @WebRTCventures Popular examples:
- 10. It’s all about tradeoffs… WebRTC architecture WebRTC Standard Open Source Media Servers CPaaS Up front cost High Medium Low Ongoing cost Low Low High Technical difficulty High Medium Low Features included Low Medium High @WebRTCventures
- 12. SFUs or MCUs can help scale WebRTC MCU – Multipoint Control Unit • Handles mixing of video/audio streams in a central server so each participant only has one stream to deal with SFU – Selective Forwarding Unit • Each participant only connects to the SFU, but receives unique streams for each participant Either can add features beyond scaling • Recording • Broadcasting • Interface to other services like transcription or VoIP legacy systems @WebRTCventures
- 13. MCU example • Multipoint Control Unit • Central server mixes all audio and video • Each participant only gets one downloaded stream each for audio and video • MCU controls a composited layout of that video for everyone, which can be nice but also introduces latency • Heavy processing required on MCU, but more predictable bandwidth requirements Media Servers offering MCU capability (not a comprehensive list): M C U @WebRTCventures
- 14. SFU example • Selective Forwarding Unit • Routes the correct stream to each user • Still unique streams for each participant (allows for layout changes on user side) • More powerful and more modern option but more complicated implementation • Lower CPU requirements but more variable bandwidth required (based on # of users) • Possible to do end-to-end encryption Media Servers offering SFU capability (not a comprehensive list): S F U @WebRTCventures
- 15. Why not both? Concept from IIT RTC 2020 presentation by Lorenzo Miniero of MeetEcho / Janus, on “Can SFUs and MCUs be friends?” S F U M C U SIP Gateway Recording or Broadcasting Server Mobile Devices @WebRTCventures
- 16. Use Case: A Standard 1-1 Video Chat Application Things to consider • CPaaS or native application • Forget about recording or broadcasting to many • Free is not always better (the free approach might mean worst performance) • Real E2EE (End to End Encryption) out of the box • TURN Overview of this use case • Common for telehealth, intercom, remote control and basic click to call applications • Browsers most tested and supported use case @WebRTCventures
- 17. Use Case: A Group Chat Video Application Things to consider • Again, CPaaS or self hosted • Scalability • Interoperability is more complex • Many videos = Resource intensive • Simulcast and SVC • Do you really need video? Overview of this use case • Common for conferencing solutions like Jitsi, Google Meet, Zoom... • SFU architecture is the most common @WebRTCventures WebRTC SFU User 1 User 2 S F U
- 18. Publishers Use Case: Live Interactive Broadcasting Things to consider • Estimated active publishers/viewer numbers • More expensive than streaming on demand video • Scalability • Simulcast/SVC • Do you really need low latency? Overview of this use case • Required for broadcasting media where latency matters: gaming, payments/betting, sports and other live events @WebRTCventures WebRTC based infrastructure User 1 User 2 Viewers User X
- 19. Use Case: Contact Centers Things to consider • Multiparty video conferencing support? • Integration of multiple channels • Integration with VoIP legacy systems • Recording/voicemail and speech to text • IVRs or voicebots Overview of this use case • Very common use case for medium/large companies that offer support/marketing @WebRTCventures MCU IP-PBX Phone caller User 2 Web Publishers User 1 User 1 Web Web Publishers SFU User 2 Phone caller User 3 IP-PBX User 1 MCU
- 20. Summary Implementation Tips • Decide what you are going to compromise • Understand the 3 main ways to build your WebRTC app • Leverage the latest WebRTC capabilities • Focus on your specific use case needs @WebRTCventures
- 21. Thank you! Learn more about us: https://webrtc.ventures Follow us on Twitter: @WebRTCventures Experts in live video app development for: Telehealth, Broadcasting, Contact Centers, and More! @lbertogon @arinsime
Editor's Notes
- #8: Janus built in C, Jitsi built using Java, MediaSoup built with C++ and Pion uses Go
- #11: Arin’s last slide - “and now Alberto will talk more about media servers and architectural use cases”
- #12: As Arin mentioned, building your own WebRTC based library is complicated. Also, native WebRTC peer to peer connections are a very beautiful idea but complicated and not practical when scaling to more than 3-5 participants or when you want to process video for recording or other advanced functionalities… for that we have existing open source implementations of the WebRTC standard or media servers that will help to scale. In this diagram at the right you can see a representation of how a network would look like without a intermediate media server helping us (a bit messy)..
- #18: Scalability. Harder to scale to dozens of active participants Many videos = Resource intensive (mainly CPU and bandwidth) SVC is a technique that allows encoding a video stream once in multiple layers. The layers in SVC can be subtracted while maintaining the video, reducing its quality with the reduction of each layer (fps, resolution or snr layers) Do you really need video? Some great tools just use audio (twitter spaces, clubhouse, slack huddle). In that case we can optimize a lot performance and reach a much larger audience!
- #19: Do you really need low latency? There are easier to implement alternatives HLS / RTMP with higher latencies
- #20: Multiparty video conferencing support? Like for the 1 to 1. In this case we could directly connect to an IP-PBX and have an integrated solution phone with web. However, this doesn’t scale well… Integration of multiple channels (Omni-channel) Integration with VoIP legacy systems is very likely. Support of video to telephony system is very complicated...challenges with different devices (softphones and hardphones). Differences in the SIP implementation, supported resolutions, RTCP Muxing supported or not, different codecs or in case of H264 different profile level ID that define it. ICE (trickle) support…
- #21: That brings us to the end of the presentation. I’d like to summarize some main tips: What are you willing to compromise? (quality, latency, battery performance…) cost? capabilities: codecs, configuration options like svc, hints for the type of media...
- #22: And with that, here concludes our presentation about architecting your WebRTC application. And if you are interested about this topics visit us or follow us on Twitter! Thank you so much for watching!